Too Many Darn Lists

Discuss macro implementations, ask for macro help (to share your creations, see User Creations, probably either Campaign Frameworks or Drop-in Resources).

Moderators: dorpond, trevor, Azhrei, giliath, jay, Mr.Ice

User avatar
aliasmask
Deity
Posts: 8611
Joined: Tue Nov 10, 2009 6:11 pm
Location: Bay Area

Re: Too Many Darn Lists

Post by aliasmask »

It's funny. I've been working on the same thing for my spell library. I'm in the debugging stages now, but it works like a charm so far**. I use dot notation also known as the path to get/set data.

**Well, the one issue I'm reworking is the fact that having too many properties (600+) really kills the time to set a variable, but getting is instant.

Basically what I do is create a new property for any complex data structures and leave a pointer in the parent structure. So, by default, you can put an object inside an object, but that internal object is replace with a pointer and then given a property. For example:

Code: Select all

[H: obj2 = json.append("",1,2,3)]
[H: obj1 = json.set("{}","obj1",obj2)]
[H: createTable("My Table")]
[H: addTableField("mytable","Object 0",obj1)]


So, this is what happens above:
  • createTable creates a property named h.tData_mytable = {}and a property named h.tIndex_mytable = "My Table"
  • addTableField see the data is an object so it creates a new property h.tData_mytable.object0 and it's index. It then puts in the h.tData_mytable a pointer to the new property.
  • When writing the new object, it notices there is an object inside (an array), so it creates a new property h.tData_mytable.object0.obj1 and it's index

As you can see, if you have an object with many fields that is also an object there will be a whole lot of property names. You can actually override having a new property created when adding an object to an object, but I believe that will compromise the get speed. My philosophy is that tables are read from often and written too seldom. The thing I'm changing now is the index where it will be a single object so the number of properties is not doubled.

You can get an idea from my code on what I'm doing. I included my WIP token. Here's a list of function names:

[spoiler=Function Names]

Code: Select all

cleanVar,
glp,
indexPropName,
makePath,
propName,
slp,
createPointerReference,
getPointerLevel,
getPointerRef,
isPointer,
updatePointerSubLevel,
*getTableJSON,
*removeFromTableList,
addTableNode,
appendTableList,
createTable,
getTableField,
getTableIndex,
getTableIndexes,
getTableJSON,
getTableRaw,
removeFromTableList,
setTableField
[/spoiler]

edit:
Oops.. left my debug code in there, but no debug macro.. I updated the token.
Attachments
lib.tables WIP.rptok
MT version 1.3b70
(54.89 KiB) Downloaded 52 times
Last edited by aliasmask on Tue Jun 29, 2010 1:26 pm, edited 2 times in total.

User avatar
wolph42
Deity
Posts: 9794
Joined: Fri Mar 20, 2009 5:40 am
Location: Netherlands
Contact:

Re: Too Many Darn Lists

Post by wolph42 »

neofax wrote:
wolph42 wrote:
neofax wrote:MapTool tables can hold JSON's as well as tokens and as pointed out, they have pics associated.

and as also has been pointed out: its read only AND moreover its insanely cumbersome to create a list even with my excel tool.


I don't foresee someone changing tavern names that often so read only point is kinda moot. Also, I don't see where making a JSON for a table is harder than for a token.


lol

Well in that case I would suggest to download the attachement in my earlier post, open the excelsheet and copy the big weapons table. Open the framework, run the frame macro, paste the data in the box and hit save. (granted one line need to be added to the macro: [propertyName = jTable])

And now try to get the same table as 254 json rows into a MT table

Use a stopwatch for both and let me know which of the two versions is the quickest 8)

User avatar
biodude
Dragon
Posts: 444
Joined: Sun Jun 15, 2008 2:40 pm
Location: Montréal, QC

Re: Too Many Darn Lists

Post by biodude »

neofax wrote:
wolph42 wrote:
neofax wrote:MapTool tables can hold JSON's as well as tokens and as pointed out, they have pics associated.

and as also has been pointed out: its read only AND moreover its insanely cumbersome to create a list even with my excel tool.

I don't foresee someone changing tavern names that often so read only point is kinda moot. Also, I don't see where making a JSON for a table is harder than for a token.

The problem is that you can not write to a MapTool table directly from Macro Script. This severely limits the ways you can get data in (manually). Sure, you can type / copy/paste data in manually, but there is no way to paste in a block of text, parse it out the way you want, and store the resulting data in a table.

Making a JSON for a table is considerably harder than for a token, as far as I can tell:
Token:
  • [ setProperty( propertyName , jsonObject )]
Table:
  • output JSON string,
  • manually copy,
  • manually open editor for destination table,
  • manually find target row,
  • manually paste JSON string into target row
Unless there's a function to write to tables of which I am unaware: if so, please let me know.

MapTool tables simply do not provide the desired functionality here.

I suppose one could use something like wolph42's excel tool to export a multi-column MT-table in xml format, then import it wholesale. But, I don't like having to rely on external tools too much. This is also a lot of work to change a few things, and not necessarily easy for less-experienced users. The data may not be edited very often, but as soon as someone does want to make changes for their own use: add a few rows, take some out, etc.: if they can't do it easily, the rest of the data, or even the whole tool, may be less valuable to them as a result. Why limit yourself when you don't have to?

Besides, the stuff I'm thinking of has more to do with token-specific data (personal inventory, spell lists, links, etc.). The advantage being that putting the same data on a Lib:token makes it super-easy to import/export and is still globally-accessible in MapTool. Plus, it can easily store multiple columns, and you can build an editing interface you like, and read/write from MT script: all of which are easier to do with token data than MT-table data.
Do you have an idea that would allow all this to be achieved using MT-tables? If so, please let us know, I'd be curious.
"The trouble with communicating is believing you have achieved it"
[ d20 StatBlock Importer ] [ Batch Edit Macros ] [ Canned Speech UI ] [ Lib: Math ]

User avatar
wolph42
Deity
Posts: 9794
Joined: Fri Mar 20, 2009 5:40 am
Location: Netherlands
Contact:

Re: Too Many Darn Lists

Post by wolph42 »

aliasmask wrote:So, this is what happens above:

* createTable creates a property named h.tData_mytable = {}and a property named h.tIndex_mytable = "My Table"
* addTableField see the data is an object so it creates a new property h.tData_mytable.object0 and it's index. It then puts in the h.tData_mytable a pointer to the new property.
* When writing the new object, it notices there is an object inside (an array), so it creates a new property h.tData_mytable.object0.obj1 and it's index

you lost me after "so,..." :shock: I've had a look at your wip token, but i cant make head nor tail from it.
Do you have an example macro? So I can run through it step by step?

User avatar
aliasmask
Deity
Posts: 8611
Joined: Tue Nov 10, 2009 6:11 pm
Location: Bay Area

Re: Too Many Darn Lists

Post by aliasmask »

Sure. I got rid of the debug lines and added a Test macro.

[spoiler=Test Code]

Code: Select all

[H: am.tb.createTable("Test")]
[
H: colors = json.set("{}","primary",json.append("","red","blue","green"),"Secondary Colors",json.append("","magenta","cyan","yellow"),"RGB Values",json.set("{}","red","FF0000","blue","0000FF","green","00FF00"))]
[
H: am.tb.setTableField("test","Colors",colors)]
[
H: input("junk||<html>Most of time is wasted in creating Nodes and pointers, <br>especially when the db gets big.<br><br>CONTINUE...</html>|LABEL|TEXT=FALSE")]
Full table as object:<pre> [json.indent(am.tb.getTableJSON("test"))]</pre><br>
Second level pointer: [am.tb.getTableField("test","Colors")]<br>
First level pointer: [am.tb.getTableField("test.colors","RGB Values")]<br>
Array:
 [am.tb.getTableField("test.colors","Secondary Colors")]<br>
Object: [am.tb.getTableRaw("test.colors.rgbvalues")]<br>
Name of Parent field of object: [am.tb.getTableIndex("test.colors.rgbvalues")] 
[/spoiler]

Code: Select all

Full table as object:
 {"Colors": {
    "primary":     [
        "red",
        "blue",
        "green"
    ],
    "Secondary Colors":     [
        "magenta",
        "cyan",
        "yellow"
    ],
    "RGB Values":     {
        "red": "FF0000",
        "blue": "0000FF",
        "green": "00FF00"
    }
}}
Second level pointer: {json.evaluate(am.tb.glp("h.tData_test.colors"))}
First level pointer: {am.tb.glp("h.tData_test.colors.rgbvalues")}
Array: ["magenta","cyan","yellow"]
Object: {"red":"FF0000","blue":"0000FF","green":"00FF00"}
Name of Parent field of object: RGB Values
Attachments
lib.tables WIP DEMO.rptok
MT version 1.3b70
(55.35 KiB) Downloaded 54 times
Last edited by aliasmask on Tue Jun 29, 2010 4:24 pm, edited 1 time in total.

prestidigitator
Dragon
Posts: 317
Joined: Fri Apr 23, 2010 8:17 pm

Re: Too Many Darn Lists

Post by prestidigitator »

Another nice thing about storing the data in token properties is that you can then simply save the token and attach it in a post for someone to drag-and-drop into their own campaign. Nice convenient packaging.
"He knows not how to know who knows not also how to un-know." --Sir Richard Burton

User avatar
wolph42
Deity
Posts: 9794
Joined: Fri Mar 20, 2009 5:40 am
Location: Netherlands
Contact:

Re: Too Many Darn Lists

Post by wolph42 »

@aliasMask: thnx, ill have a look again.

In the mean time I've been working on my own dablings. I've dug up an old project of mine (also in excel) that randomly creates taverns and creates an GUI interface of the taverns population (i've attached it as well). From that I've dug up the table which I created with some hilarious intent and used it for testing purposes. Attached is the updated campaign file. The tavern names have already been parsed onto the token.

Check it out
Attachments
WFRP Inn Generator V2.11.zip
for the fun of it
(361.61 KiB) Downloaded 53 times
tablesList b5.zip
(128.71 KiB) Downloaded 50 times
Last edited by wolph42 on Wed Jun 30, 2010 3:07 am, edited 1 time in total.

User avatar
biodude
Dragon
Posts: 444
Joined: Sun Jun 15, 2008 2:40 pm
Location: Montréal, QC

Re: Too Many Darn Lists

Post by biodude »

wolph42 wrote:ok i tested your piece of code.

This is what i made of it:


Even faster?
[spoiler=json.fromList ROCKS!]

Code: Select all

<!-- List Parsing XPTL -->
[
h: args  = macro.args]
[
h: Title = json.get(args, "Title")]
[
h: text  = json.get(args, "TextBox")]

[
delim = "%09"]
[
deDelim = decode(delim)]
[
lb = "%0A"]

[
jTable = "{}"]
[
row=""]
[
rowName=""]

[
 rows = json.fromList( text , decode( lb ) )]
[
 nRows = json.length( rows )]

<!--
 first line only: might be more efficient to pull this out and only do it once 
    
(rather than having to evaluate and skip EVERY OTHER TIME THROUGH THE LOOP) -->
    [ line.txt = json.get( rows , 0 )]    <!-- get the 1st line (omitting line break) -->
    [ headerRow = json.fromList( line.txt , deDelim )]    <!-- "%09" if encoded, decode( "%09" ) otherwise, or specified by argument? -->
    [ tableName = json.get( headerRow, 0 )]
    [ nCols = json.length( headerRow )]    <!-- this might only really need to be done with the first line -->
    [ rows = json.remove( rows , 0 )] <!-- drop header row (for convenience) -->

<!--
 all subsequent rows -->
[
h, FOREACH( rowText , rows ), CODE: {
    [ row = json.fromList( rowText , deDelim )]    <!-- "%09" if encoded, decode( "%09" ) otherwise, or specified by argument? -->
    [ jRow = "{}" ]
    [ FOR( i , 0 , nCols ), CODE: {
        [ item   = json.get( row , i )]    <!-- get the 'ith' item (omitting delimiter) -->
        [ column = json.get( headerRow , i )]    <!-- get the 'ith' column name -->

        <!--name of current row / assemble row object-->
        [if( i==): rowName = item ; jRow = json.set( jRow , column, item )]
    }]
    <!--fill the json object-->
    [ jTable = json.set( jTable , rowName , jRow )]
}]

 
[/spoiler]
Who needs fancy regex when you got [wfunc]json.fromList[/wfunc]? ;)
"The trouble with communicating is believing you have achieved it"
[ d20 StatBlock Importer ] [ Batch Edit Macros ] [ Canned Speech UI ] [ Lib: Math ]

User avatar
wolph42
Deity
Posts: 9794
Joined: Fri Mar 20, 2009 5:40 am
Location: Netherlands
Contact:

Re: Too Many Darn Lists

Post by wolph42 »

(I haven't tested it yet)

This makes it at least a lot more readable. I've been working with regex a lot lately but it still makes my head spin when I look at it. So it doesnt delay the lot too much I in favour of this method.

I was already considering using json.fromlist for the innerloop (so changing a complete line straight into an array). For retrieval you only need the header use that to retrieve the position of the item within the array. This will however mean that I can't make use of json.pget anymore and thus more coding work :cry: .

Im also a bit unsure whether this method would increase the retrieval time? (It would very likely decrease the parse function and decrease the size of the stored object by roughly 50%). However as aliasmaks pointed out: retrieval time is key!

User avatar
wolph42
Deity
Posts: 9794
Joined: Fri Mar 20, 2009 5:40 am
Location: Netherlands
Contact:

Re: Too Many Darn Lists

Post by wolph42 »

I did the test: the json.fromlist is one second faster (with the huge table). 40.8s vs 41.9s (i tested both methods consecutively)
And as it makes the code also more readable...: its a winner! :mrgreen:


so.... we have something that actually works, can store tables into tokens, even has a rudimentary retrieval function... where are those lists that others promised :wink: :?:

Short manual:
If you currently want to add a list
1. start the frame (red button)
2. paste table into box
3 give title of table (no extensions and the like e.g. "lstMonsters", just "Monsters")
4 hit save.

To 'run' a list: click on the 'lists (from token tables)' macro (blue button). Works exactly the same as the original 'list' function that retrieves its info from the MT tables section. This function gets its list from the lib:Lists token.

I've already stored some tables in the token (dwarf, tiny and tavern)... and I don't know how to delete them.
:?: How do you remove 'hidden' token properties :?:
Attachments
tablesList b6.cmpgn
The new winner, with new functionality
(104.39 KiB) Downloaded 51 times

User avatar
biodude
Dragon
Posts: 444
Joined: Sun Jun 15, 2008 2:40 pm
Location: Montréal, QC

Re: Too Many Darn Lists

Post by biodude »

wolph42 wrote:I did the test: the json.fromlist is one second faster (with the huge table). 40.8s vs 41.9s (i tested both methods consecutively)
And as it makes the code also more readable...: its a winner! :mrgreen:
Sweet!

wolph42 wrote::?: How do you remove 'hidden' token properties :?:

[wfunc]resetProperty[/wfunc] should work. And, as of b68, it also removes properties from the token (if they are hidden).
Last edited by biodude on Wed Jun 30, 2010 9:03 am, edited 1 time in total.
"The trouble with communicating is believing you have achieved it"
[ d20 StatBlock Importer ] [ Batch Edit Macros ] [ Canned Speech UI ] [ Lib: Math ]

User avatar
biodude
Dragon
Posts: 444
Joined: Sun Jun 15, 2008 2:40 pm
Location: Montréal, QC

Re: Too Many Darn Lists

Post by biodude »

wolph42 wrote:double post - (isnt there a way to delete this message?)

Edit the message: there should be a tiny checkbox above the Subject line labelled "Delete Post:" and "[ Once deleted the post cannot be recovered ]" beside it.
"The trouble with communicating is believing you have achieved it"
[ d20 StatBlock Importer ] [ Batch Edit Macros ] [ Canned Speech UI ] [ Lib: Math ]

User avatar
wolph42
Deity
Posts: 9794
Joined: Fri Mar 20, 2009 5:40 am
Location: Netherlands
Contact:

Re: Too Many Darn Lists

Post by wolph42 »

biodude wrote:
wolph42 wrote:double post - (isnt there a way to delete this message?)

Edit the message: there should be a tiny checkbox above the Subject line labelled "Delete Post:" and "[ Once deleted the post cannot be recovered ]" beside it.

Nope...:
nope.jpg
nope.jpg (45.96 KiB) Viewed 932 times

User avatar
wolph42
Deity
Posts: 9794
Joined: Fri Mar 20, 2009 5:40 am
Location: Netherlands
Contact:

Re: Too Many Darn Lists

Post by wolph42 »

biodude wrote:[wfunc]resetProperty[/wfunc] should work. And, as of b68, it also removes properties from the token (if they are hidden).

ah, I did encounter this function in my search but didn't know (and it doesn't show on the wiki either) that it also deletes the prop.

User avatar
wolph42
Deity
Posts: 9794
Joined: Fri Mar 20, 2009 5:40 am
Location: Netherlands
Contact:

Re: Too Many Darn Lists

Post by wolph42 »

and here the update with the 'delete' function

Edit: I guess its time to create a more dedicated topic on this in the right spot. So I've started this topic
Attachments
tablesList b7.cmpgn
(102.63 KiB) Downloaded 56 times

Post Reply

Return to “Macros”