Temp solution for json.sort issue (b90) [v0.2]

Discuss macro implementations, ask for macro help (to share your creations, see User Creations, probably either Campaign Frameworks or Drop-in Resources).

Moderators: dorpond, trevor, Azhrei, giliath, jay, Mr.Ice

Post Reply
User avatar
wolph42
Winter Wolph
Posts: 9999
Joined: Fri Mar 20, 2009 5:40 am
Location: Netherlands
Contact:

Temp solution for json.sort issue (b90) [v0.2]

Post by wolph42 »

The first *real* bug is discovered in b90 in json.sort, which does not work with key:value pairs.
Here is a workaround that overrides the existing json.sort.

If you want to use it:
1. download the attached lib token.
2. Drop it into your campaign,
3. save and reload.
Thats it.

Note: it has not been thoroughly tested, so check this thread for updates.
Note2: in the rare occasion that more then one sort parameter is given, the 2nd one will be ignored.
So here a working proof of concept(copy paste into chat and hit enter to check it out)

Code: Select all

<pre>
[h:jsonArray    = '[ {name:"Hero", HP:10},{HP:5, name:"Wolf"},{name:"Mage", HP:20},{name:"Troll", HP:15},{name:"Eagle", HP:5} ]']
['json.sort(jsonArray, "a", "name")']

[key        = "name"]
[valList    = ""]

[foreach(item, jsonArray), CODE:{
    [valueArray    = json.get(item, key)]
    [valList    = listAppend(valList, valueArray)]
}]

[valList    = listSort(valList, "A+")]

[tmpArray    = "[]"]

[foreach(valueList, valList), CODE:{
    [found        = 0]
    [while (!found), CODE:{
        [item        = json.get(jsonArray, roll.count)]
        [valueArray    = json.get(item, key)]
        [if(valueList == valueArray), CODE:{
            [found        = 1]
            [tmpArray    = json.append(tmpArray, item)]
        ''
        };{''}]
    ''
    }]
''
}]

[r:maro.return    = tmpArray] 


and here the 'final' version (I'll leave the PoC as its easier to read but ill continue with the one below):

Code: Select all

[h:'<!-- ---------------------- json.sort (array, direction, optional:json.key) WORKAROUND ---------------------- -->']
[r, if(argCount() < 3), CODE:{
    [h,if(argCount() > 1): direction    = arg(1) ; direction = "ascending"]
    [r:macro.return    = oldFunction(arg(0), direction)]
};{
    [h:jsonArray    = arg(0)]
    [h:direction    = arg(1)]
    [h:key            = arg(2)]

    [h:valArray       = "[]"]
    [h,foreach(item, jsonArray): valArray = json.append(valArray, json.get(item, key))]

    [h:valArray       = json.sort(valArray, direction)]
    [h:tmpArray       = "[]"]

    [h,foreach(valueList, valArray), CODE:{
        [found    = 0]
        [while (!found), CODE:{
            [item        = json.get(jsonArray, roll.count)]
            [valueArray    = json.get(item, key)]
            [if(valueList == valueArray), CODE:{
                [tmpArray    = json.append(tmpArray, item)]
                <!-- make sure that when a value exists multiple times that its extracted only once -->
                [jsonArray    = json.remove(jsonArray, roll.count)]
                [found        = 1]
            };{}]
        ''
        }]
    ''
    }]

    [r:maro.return    = tmpArray]
[h:'']
}]  
together with:

Code: Select all

[h:defineFunction("json.sort", "json.sort@lib:fixB90", 0, 1)] 
Test macro
(when the lib is installend, and onCampaignLoad has been executed, you can test the function by copy pasting the following into the chat)

Code: Select all

[h:jsonArray    = '[ {name:"Hero", HP:10},{HP:5, name:"Wolf"},{name:"Mage", HP:20},{name:"Troll", HP:15},{name:"Eagle", HP:5} ]']
<pre>
[r:json.sort(jsonArray, "a", "name")]
[r:json.sort(jsonArray, "d", "HP")]
[r:json.sort("[5,3,7,4,5,6]")]
[r:json.sort("['def','12g','b12g','abc']")]
 
Attachments
libfixB90.rptok
version 0.2
(79.64 KiB) Downloaded 90 times

dakuth
Cave Troll
Posts: 31
Joined: Thu Feb 04, 2010 5:26 am

Re: Temp solution for json.sort issue (b90)

Post by dakuth »

I'd been having a number of problems with my own macros, inventory, and the skill sheet to name a few.

All working now, thank you. (b90, using the Pathfinder framework.)

User avatar
aliasmask
RPTools Team
Posts: 9024
Joined: Tue Nov 10, 2009 6:11 pm
Location: Bay Area

Re: Temp solution for json.sort issue (b90)

Post by aliasmask »

I was thinking there was another way and it will sort for more than one key. Basically loop through array creating a special string that has the key values and index of array, do a listSort, loop through list, get index off of end of special string, rebuild array of objects. I'll see what I can do tomorrow. One trick needed will be to write any numbers as a string with leading 0's. Each of the delimiters should be fairly unique like ### or something you wouldn't expect to see together normally.

So, if the keys were init and dex, the special variable could look like this

00000000002###00000000013###0

which would be 2 init, 13 dex and index of 0 in original array.

User avatar
wolph42
Winter Wolph
Posts: 9999
Joined: Fri Mar 20, 2009 5:40 am
Location: Netherlands
Contact:

Re: Temp solution for json.sort issue (b90)

Post by wolph42 »

aliasmask wrote:I was thinking there was another way and it will sort for more than one key. Basically loop through array creating a special string that has the key values and index of array, do a listSort, loop through list, get index off of end of special string, rebuild array of objects. I'll see what I can do tomorrow. One trick needed will be to write any numbers as a string with leading 0's. Each of the delimiters should be fairly unique like ### or something you wouldn't expect to see together normally.

So, if the keys were init and dex, the special variable could look like this

00000000002###00000000013###0

which would be 2 init, 13 dex and index of 0 in original array.
I thought of that too but I thought that to be too tricky for the following reasons.
1. take the example array:
{name:"Hero", HP:10},{HP:5, name:"Wolf"},{name:"Mage", HP:20},{name:"Troll", HP:15},{name:"Eagle", HP:5}
here you see that name and HP are not always in the same order inside the object.
2. use of weird characters (e.g. ","). Granted with my method that can still go wrong, but the chances are slimmer as I only 'expose' the values to be sorted and not the rest of the objects.
3. not every object might have the same number of key:value pairs, which might screw things up.

your method however has indeed the advantage that you can sort on multiple keys and if done correctly then issue 1 should not be an issue at all.
Come to think of it I guess issue 2 could be omitted as well by leaving it as an array... but don't you need to escape special characters in that case or is that already the case inside the object...?

I would suggest in that case not to use listSort, but json.sort (in combination with oldFunction as im currently doing) this most likely prevents issues with 'weird' characters. That would make it more robust then my current function. Hmm maybe I can make that change as well.

edit: why do you need the leading 0's for numbers ??

User avatar
wolph42
Winter Wolph
Posts: 9999
Joined: Fri Mar 20, 2009 5:40 am
Location: Netherlands
Contact:

Re: Temp solution for json.sort issue (b90)

Post by wolph42 »

yup that works (using json.sort instead of listSort) so updated token. Should be more stable. Version 0.2. (code in OP is also updated)

User avatar
aliasmask
RPTools Team
Posts: 9024
Joined: Tue Nov 10, 2009 6:11 pm
Location: Bay Area

Re: Temp solution for json.sort issue (b90)

Post by aliasmask »

wolph42 wrote:edit: why do you need the leading 0's for numbers ??
Because when sorting numbers 10 would be a lower number than 2 when viewing it as a string. So, changing it to 10 and 02 corrects that. Since you don't know how big the biggest number is, forcing the max digits helps. Something I just thought of, but it still works, negative numbers.

User avatar
wolph42
Winter Wolph
Posts: 9999
Joined: Fri Mar 20, 2009 5:40 am
Location: Netherlands
Contact:

Re: Temp solution for json.sort issue (b90)

Post by wolph42 »

aliasmask wrote:
wolph42 wrote:edit: why do you need the leading 0's for numbers ??
Because when sorting numbers 10 would be a lower number than 2 when viewing it as a string. So, changing it to 10 and 02 corrects that. Since you don't know how big the biggest number is, forcing the max digits helps. Something I just thought of, but it still works, negative numbers.
but if you use json.sort (oldfunction) then it sorts correct anyway, so why bother? Else you could resort to listSort N+/- but with the drawback of potential issues with 'weird' characters.

User avatar
aliasmask
RPTools Team
Posts: 9024
Joined: Tue Nov 10, 2009 6:11 pm
Location: Bay Area

Re: Temp solution for json.sort issue (b90)

Post by aliasmask »

wolph42 wrote:
aliasmask wrote:
wolph42 wrote:edit: why do you need the leading 0's for numbers ??
Because when sorting numbers 10 would be a lower number than 2 when viewing it as a string. So, changing it to 10 and 02 corrects that. Since you don't know how big the biggest number is, forcing the max digits helps. Something I just thought of, but it still works, negative numbers.
but if you use json.sort (oldfunction) then it sorts correct anyway, so why bother? Else you could resort to listSort N+/- but with the drawback of potential issues with 'weird' characters.
I don't think we're talking about the same thing. Yes, using json.sort is better, but I'm talking about what is being sorted and how to track the original index.

User avatar
wolph42
Winter Wolph
Posts: 9999
Joined: Fri Mar 20, 2009 5:40 am
Location: Netherlands
Contact:

Re: Temp solution for json.sort issue (b90) [v0.2]

Post by wolph42 »

aah, yes now I get it . Then we were not thinking of the same thing but close. What i meant is:

[{HP:5, ID:Dragon},{HP:6, ID:Eagle, TMP:12}]
lets say you want to sort on on HP and then ID
you cycle through all items, first you get HP value and then ID value, then you check what remains and add that. So you get:
["5####Dragon","6####Eagle####TMP:12"]
and json.sort that. At first i was thinking of a replace() and simply replace the {,},: and " with some smart regex, but the order inside the object is not guaranteed, and ofcourse then you sort on the first key inside. So I though of rebuilding each object in the correct order and THEN replace() the lot, but thought that to be quite slow, so finally i thought of the above...but that is at least as slow as the rebuild option. Hence the method in the OP.

*You* are talking about just retrieving the parts you want (so HP and ID only) and then add some index at the end so you know where the object was, then sort it, then cycle through it again an based on the index at the end rebuild the original array in the correct order.

Something like that?

User avatar
aliasmask
RPTools Team
Posts: 9024
Joined: Tue Nov 10, 2009 6:11 pm
Location: Bay Area

Re: Temp solution for json.sort issue (b90) [v0.2]

Post by aliasmask »

Yeah, pretty much. This is what I came up with, but I haven't tested it.

Code: Select all

@@ @filterString
[H: object = arg(0)]
[H: keyList = arg(1)]

[H: tempVal = "[]"]
[H, foreach(key,keyList), code: {
   [H: value = json.get(object,key)]
   [H, if(isNumber(value)): tempVal = json.append(tempVal,strformat("n%031.15f",value*1.0)); tempVal = json.append(tempVal,value)]
}]

[H: macro.return = tempVal]

!!
@@ @json.sort
<!-- json.sort(array,direction,key1,...): sortedArray

redefine json.sort to fix bug in b90-beta 
Note: comments inside loop should be removed for stack reasons. Left in for instructional purposes.
-->

<!-- get parameters -->
[H: array = arg(0)]
[H, if(argCount() > 1): direction = arg(1); direction = "a"]

<!-- determine if we need to do special process -->
[H: numKeys = max(0,argCount()-2)]

[H, if(numKeys), code: {
   [H: separator = decode("%09%09")]
   [H: indexIdentifier = "||"]
   [H: tempArray = "[]"]
   [H: resultArray = "[]"]
   <!-- keys used for filter -->
   [H: keyList = json.get(macro.args,2,numKeys-1)]
   [H, foreach(object,array), code: {
      [H: index = roll.count]
      [H: tempVal = "[]"]
      [H: key = "Location"]
      <!-- add values to value array converting numbers to strings -->
      [H: tempVal = am.xx.filterString(object,keyList)]
      <!-- tack the original array index to end of value -->
      [H: tempVal = json.append(tempVal,indexIdentifier+index)]
      <!-- convert value array to single string -->
      [H: tempArray = json.append(tempArray,json.toList(tempVal,separator))]   
   }]
   <!-- sort array -->
   [H: tempArray = oldFunction(tempArray,direction)]
   <!-- rebuild result array getting index from end of value -->
   [H, foreach(item,tempArray): resultArray = json.append(resultArray,json.get(array,listGet(item,1,separator+indexIdentifier)))]
};{
   [H, if(json.type(array) == "OBJECT"), code: {
      <!-- sort a json object by the key names -->
      [H: keys = oldFunction(json.fields(array,"json"),direction)]
      [H: resultArray = "{}"]
      [H, foreach(key,keys): resultArray = json.set(resultArray,key,json.get(array,key))]
   };{   
      <!-- normal array sort -->
      [H: resultArray = oldFunction(array,direction)]
   }]
}]
[H: macro.return = resultArray]

!! 
As a bonus, I tossed in my json object key sort.

edit: Hmm, ran in to nested level limit with foreach and had to create another macro listed as am.xx.filterString(). I tried the '' trick with it, but did not work. Apparently only works with [if:] (unless I did it wrong). But the above seems to work for me.

keywords: amsave sort

User avatar
wolph42
Winter Wolph
Posts: 9999
Joined: Fri Mar 20, 2009 5:40 am
Location: Netherlands
Contact:

Re: Temp solution for json.sort issue (b90) [v0.2]

Post by wolph42 »

" trick, up to now works always with any nest. Check out the op to see where and how. Also, did you use two single quotes? (And not one double)

User avatar
wolph42
Winter Wolph
Posts: 9999
Joined: Fri Mar 20, 2009 5:40 am
Location: Netherlands
Contact:

Re: Temp solution for json.sort issue (b90) [v0.2]

Post by wolph42 »

hence this should work (not tested yet)

Code: Select all

@@ @json.sort
<!-- json.sort(array,direction,key1,...): sortedArray

redefine json.sort to fix bug in b90-beta 
Note: comments inside loop should be removed for stack reasons. Left in for instructional purposes.
-->

<!-- get parameters -->
[H: array = arg(0)]
[H, if(argCount() > 1): direction = arg(1); direction = "a"]

<!-- determine if we need to do special process -->
[H: numKeys = max(0,argCount()-2)]

[H, if(numKeys), code: {
    [H: separator = decode("%09%09")]
    [H: indexIdentifier = "||"]
    [H: tempArray = "[]"]
    [H: resultArray = "[]"]
    <!-- keys used for filter -->
    [H: keyList = json.get(macro.args,2,numKeys-1)]
    [H, foreach(object,array), code: {
        [H: index = roll.count]
        [H: tempVal = "[]"]
        [H: key = "Location"]
        <!-- add values to value array converting numbers to strings -->
        [H: tempVal = "[]"]
        [H, foreach(key,keyList), code: {
            [H: value = json.get(object,key)]
            [H, if(isNumber(value)): tempVal = json.append(tempVal,strformat("n%031.15f",value*1.0)); tempVal = json.append(tempVal,value)]
        ''
        }]
        <!-- tack the original array index to end of value -->
        [H: tempVal = json.append(tempVal,indexIdentifier+index)]
        <!-- convert value array to single string -->
        [H: tempArray = json.append(tempArray,json.toList(tempVal,separator))]    
    ''
    }]
    <!-- sort array -->
    [H: tempArray = oldFunction(tempArray,direction)]
    <!-- rebuild result array getting index from end of value -->
    [H, foreach(item,tempArray): resultArray = json.append(resultArray,json.get(array,listGet(item,1,separator+indexIdentifier)))]
''
};{
    [H, if(json.type(array) == "OBJECT"), code: {
        <!-- sort a json object by the key names -->
        [H: keys = oldFunction(json.fields(array,"json"),direction)]
        [H: resultArray = "{}"]
        [H, foreach(key,keys): resultArray = json.set(resultArray,key,json.get(array,key))]
    ''
    };{    
        <!-- normal array sort -->
        [H: resultArray = oldFunction(array,direction)]
    ''
    }]
''
}]
[H: macro.return = resultArray]

!! 
 

Post Reply

Return to “Macros”