advice needed on combining JSON objects

Discuss macro implementations, ask for macro help (to share your creations, see User Creations, probably either Campaign Frameworks or Drop-in Resources).

Moderators: dorpond, trevor, Azhrei, giliath, jay, Mr.Ice

Post Reply
busterbluth
Cave Troll
Posts: 53
Joined: Sat Aug 17, 2019 3:49 pm

advice needed on combining JSON objects

Post by busterbluth »

Has anyone here built a function in Maptool to combine like JSON objects where their relevant keys are equivalent?

I've been trying to write one and I keep getting lost in the weeds on this. So for items that have key values like Weight, Value, Name, Location etc., if those keys are the same across any number of JSON objects, I want to remove the duplicates from the JSON array and simply sum the Quantity key. My guess is that this has already been written by folks here much smarter than I am.

User avatar
aliasmask
RPTools Team
Posts: 9061
Joined: Tue Nov 10, 2009 6:11 pm
Location: California

Re: advice needed on combining JSON objects

Post by aliasmask »

If you're wanting to add values together, you'll just have to loop through each. So, you'll have a new master json, then you read the other two and merge their values in master.

busterbluth
Cave Troll
Posts: 53
Joined: Sat Aug 17, 2019 3:49 pm

Re: advice needed on combining JSON objects

Post by busterbluth »

yeah, i was afraid of that. It'll take a nested loop.

Will this work?

Code: Select all

	[h, foreach(objItem, arrInventory), CODE: {
		[h, foreach(objSubItem, arrInventory), CODE: {
			-- logic for finding a match --
		}]
	}]
or will i need a 2nd copy of the JSON array?

Code: Select all

	[h, foreach(objItem, arrInventory), CODE: {
		[h, foreach(objSubItem, arrSubInventory), CODE: {
			-- logic for finding a match --
		}]
	}]

User avatar
aliasmask
RPTools Team
Posts: 9061
Joined: Tue Nov 10, 2009 6:11 pm
Location: California

Re: advice needed on combining JSON objects

Post by aliasmask »

Well, if you have 2 arrays of objects you're wanting to merge, then you can do a normal loop the size of the largest. Or you could just loop through one and make the other the master and modify it. Either way, they should have a unique identifier of some sort so you know they are to be merged. Then when merging you'll need another loop for the object keys.

Lets say your idenifier is "name".

Code: Select all

[H: largeArray = getProperty("whatever")]]
[H: smallArray = getProperty("whateveragain")]

[H: masterArray = smallArray]

[H, foreach(obj,largeArray), code: {
   <!-- find match in master -->
   [H: name = json.get(obj,"name")]
   [H: record = getRecord(masterArray,"name",name)]
   [H: newRecord = "{}"]
   [H, if(json.type == "OBJECT"): newRecord = mergeObject(obj,record)]
   [H, if(! json.isEmpty(newRecord)), code: {
      [H: index = record.index(masterArray,"name",name)]
      [H: masterArray = json.set(masterArray,index,newRecord)]
   };{
      [H: masterArray = json.append(masterArray,obj)]
   }]
}]

[H: masterArray = json.sort(masterArray,"a","name")]

Code: Select all

<!-- record.get(records,key,value,...): object/records
   records - A json array of objects where the objects have the same keys in each array element
   key - Main key name that makes record unique. The value of key is unique.
   value - value to find index of   
   object/records - the full object of the record or an array of multiple matching records or none
   
AM 11-25-21
   This function will return the object of a set of records where a key/value combo can be found.

Updates
   AM 10-27-22 - updated to use multiple keys to find match
-->

[H: assert(math.isOdd(argCount()),"<b color=red>record.get(records,key,value,...) must have matching pairs of key and value.</b>",0)]

[H: readCondition = ""]
[H, for(i,1,argCount(),2): readCondition = json.append(readCondition,strformat("\$[?(@.['%s'] == '%s')]",arg(i),arg(i+1)))]

[H: records = json.path.read(arg(0),json.toList(readCondition," && "),"ALWAYS_RETURN_LIST,SUPPRESS_EXCEPTIONS")]

[H, if(json.length(records) == 1): macro.return = json.get(records,0); macro.return = records]

Code: Select all

<!-- record.index(records,key,value): index
   records - A json array of objects where the objects have the same keys in each array element
   key - Main key name that makes record unique. The value of key is unique.
   value - value to find index of   
   index - where in the record this key/value can be found. returns "" if not found
   
AM 11-25-21
   This function will return the index of a set of records where a key/value combo can be found.
   
-->

[H: macro.return = replace(json.path.read(arg(0),strformat("\$[?(@.['%s'] == '%s')]",arg(1),arg(2)),"ALWAYS_RETURN_LIST,AS_PATH_LIST,SUPPRESS_EXCEPTIONS"),"[^0-9]","")]
The all you need to do is write the UDF "mergeObject" to meet your specification.

busterbluth
Cave Troll
Posts: 53
Joined: Sat Aug 17, 2019 3:49 pm

Re: advice needed on combining JSON objects

Post by busterbluth »

aliasmask wrote:
Thu Jul 04, 2024 9:03 pm
Well, if you have 2 arrays of objects you're wanting to merge, then ..
I'm actually looking for like objects to combine within the SAME array. That's why I was asking earlier if I could do a nested foreach on the same array.

User avatar
aliasmask
RPTools Team
Posts: 9061
Joined: Tue Nov 10, 2009 6:11 pm
Location: California

Re: advice needed on combining JSON objects

Post by aliasmask »

Ah, like if you have arrows in 2 different locations. Using json.path.get where you get more than one result will tell you if you have duplicates, then you can modify one and remove the other. But you should build a new array while doing it, then overwrite original when done. Although, until you're satisfied with testing it, I wouldn't overwrite it, just output the json using a dialog and Wiki: json.indent().

busterbluth
Cave Troll
Posts: 53
Joined: Sat Aug 17, 2019 3:49 pm

Re: advice needed on combining JSON objects

Post by busterbluth »

I think I've got a workable solution, and I'll try to fully code it and test it tonight. This seems much simpler than what I had originally written.

Code: Select all


arrNew = "" (will store the new Items array)
arrCheck = "" (JSON array that will simply store the IDs for items already checked)
arrItemsB = arrItemsA (i'm assuming you need 2 different array variables for a nested for/each)

OUTER LOOP
foreach topItem in arrItemsA

 - get the Item ID from topItem (each item in the items array has a unique ID #)
 - if topItem.item ID is not in arrCheck (will need a custom function to do this check) then do the inner loop
 
 INNER LOOP
 foreach subItem in arrItemsB
 
 -Qty = 0
 - check that subItem ID is not in arrCheck
 - check that subItem is a match for topItem (already have a custom function to compare Item fields for 2 items and return 0 or 1)
 - if we pass both checks, Qty = Qty + subItem.Qty and add subItem.ID to arrCheck
 
 BACK TO OUTER LOOP (also conditional on topItem.item ID is not in arrCheck )
 update topItem with Qty value
 arrNew = append updated topItem

 
 

busterbluth
Cave Troll
Posts: 53
Joined: Sat Aug 17, 2019 3:49 pm

Re: advice needed on combining JSON objects

Post by busterbluth »

My code is written but for some reason, I'm generating an infinite loop even though I'm only using nested ForEach roll options. I get an error/prompt for the value of arrItems, a variable in my code below:

macro code, Combine_Like:

Code: Select all

[h: arrItems = arg(0)]
<!-- arrNew will hold the updated Items array -->
[h: arrNew = ""]
<!-- copy of Items array -->
[h: arrSubItems = arrItems]
<!-- using a stringlist to track matched items; can use a built-in find function -->
[h: lstIDs = ""]

<!-- loop / nested loop to look at each item to possibly combine -->
[h, foreach(item,arrItems), CODE: {
	[h: bMatch = 0]
	[h: itemID = json.get(item,"ID")]
	<!-- reset item quantity sum -->
	[h: ind = 0]
	[h: iQty = 0]
	[h, foreach(subitem, arrSubItems), CODE: {
		[h: subitemID = json.get(subitem,"ID")]
		<!-- if the relevant key values match AND object hasn't already been compared, then flag for combining -->
		[h: bMatch = subCombine_Like(item, subitem)]
		<!-- if this item has already been matched then set bMatch to zero -->
		[h: bMatch = if(listFind(lstIDs, subitemID) >= -1,0,bMatch)]
		<!-- list any matched items in the Items array -->
		[h, if(bMatch > 0): lstIDs = listAppend(lstIDs, subitemID)]
		<!-- update Quantity -->		
		[h: iQty = iQty + if(bMatch > 0, json.get(subitemID, "Quantity"),0)]
	}]
	<!-- update with new Quantity value -->
	[h: item = json.set(item, "Quantity", iQty)]
	<!-- throw modified item object into updated item array, if the Quantity > 0 -->
	[h: arrNew = if(iQty > 0, json.append(arrNew, item), arrNew)]	
}]

[h: macro.return = arrNew]

my called function, subCombine_Like is defined below:

Code: Select all

[h: itemA = arg(0)]
[h: itemB = arg(1)]
[h: bMatch = 0]
[h: bMatch = if(json.get(itemA,"ItemName") == json.get(itemB,"ItemName"),1,0)]
[h: bMatch = if(json.get(itemA,"Desc") == json.get(itemB,"Desc"),1,0) ]
[h: bMatch = if(json.get(itemA,"SecretInfo") == json.get(itemB,"SecretInfo"),1,0)]
[h: bMatch = if(json.get(itemA,"Value") == json.get(itemB,"Value"),1,0)]
[h: bMatch = if(json.get(itemA,"Appraised") == json.get(itemB,"Appraised"),1,0)]
[h: bMatch = if(json.get(itemA,"Weight") == json.get(itemB,"Weight"),1,0)]
[h: bMatch = if(json.get(itemA,"Quality") == json.get(itemB,"Quality"),1,0)]
[h: bMatch = if(json.get(itemA,"MagicItem") == json.get(itemB,"MagicItem"),1,0)]
[h: bMatch = if(json.get(itemA,"Container") == json.get(itemB,"Container"),1,0)]

[h: macro.return = bMatch]
and my driver code:

Code: Select all


[h: arrItems = Combine_Like(Items)]
[h: Items = arrItems]
Items is a token property, an array of JSON objects; here's an example:

Code: Select all

[{"ItemName":"Arrow - Longbow","Desc":" ","SecretInfo":" ","Category":"Ammunition","Value":0.2,"Appraised":0.2,"Weight":0.1,"Quantity":8,"Quality":3,"Capacity":0,"WeightOverride":0,"MagicItem":-1,"HitBonus":0,"DmgBonus":0,"WeaponType":"None","ChargeDie":"0d6","MaxCharges":0,"Charges":0,"Container":3,"CurrentHand":-1,"ID":15},{"ItemName":"ep","Desc":" ","SecretInfo":" ","Category":"Coins","Value":0.5,"Appraised":0.5,"Weight":0.02,"Quantity":1,"Quality":3,"Capacity":0,"WeightOverride":0,"MagicItem":-1,"HitBonus":0,"DmgBonus":0,"WeaponType":"none","ChargeDie":"0d6","MaxCharges":0,"Charges":0,"Container":2,"CurrentHand":-1,"ID":14},{"ItemName":"Oil - flask","Desc":" ","SecretInfo":" ","Category":"Gear","Value":0.1,"Appraised":0.1,"Weight":1,"Quantity":8,"Quality":3,"Capacity":0,"WeightOverride":0,"MagicItem":-1,"HitBonus":0,"DmgBonus":0,"WeaponType":"Oil","ChargeDie":"0d6","MaxCharges":0,"Charges":0,"Container":1,"CurrentHand":-1,"ID":13},{"ItemName":"Iron Spike","Desc":" ","SecretInfo":" ","Category":"Gear","Value":0.01,"Appraised":0.01,"Weight":0.08,"Quantity":6,"Quality":3,"Capacity":0,"WeightOverride":0,"MagicItem":-1,"HitBonus":0,"DmgBonus":0,"WeaponType":"none","ChargeDie":"0d6","MaxCharges":0,"Charges":0,"Container":1,"CurrentHand":-1,"ID":12},{"ItemName":"Horseshoe","Desc":" ","SecretInfo":" ","Category":"Gear","Value":0.03,"Appraised":0.03,"Weight":2.5,"Quantity":1,"Quality":3,"Capacity":0,"WeightOverride":0,"MagicItem":-1,"HitBonus":0,"DmgBonus":0,"WeaponType":"none","ChargeDie":"0d6","MaxCharges":0,"Charges":0,"Container":1,"CurrentHand":-1,"ID":11},{"ItemName":"Torch","Desc":0,"SecretInfo":" ","Category":"Gear","Value":0.17,"Appraised":0.17,"Weight":0.17,"Quantity":10,"Quality":3,"Capacity":0,"WeightOverride":0,"MagicItem":-1,"ACbonus":0,"ACshield":0,"HitBonus":0,"DmgBonus":0,"WeaponType":"Torch","ChargeDie":"0d6","Charges":0,"Container":1,"ID":10,"KnownValue":0,"Secret":0,"CurrentHand":-1},{"ItemName":"Torch","Desc":0,"SecretInfo":" ","Category":"Gear","Value":0.17,"Appraised":0.17,"Weight":0.17,"Quantity":4,"Quality":3,"Capacity":0,"WeightOverride":0,"MagicItem":-1,"ACbonus":0,"ACshield":0,"HitBonus":0,"DmgBonus":0,"WeaponType":"Torch","ChargeDie":"0d6","Charges":0,"Container":1,"ID":9,"KnownValue":0,"Secret":0,"CurrentHand":-1},{"ItemName":"Oil - flask","Desc":" ","SecretInfo":" ","Category":"Gear","Value":0.1,"Appraised":0.1,"Weight":1,"Quantity":1,"Quality":3,"Capacity":0,"WeightOverride":0,"MagicItem":-1,"HitBonus":0,"DmgBonus":0,"WeaponType":"Oil","ChargeDie":"0d6","MaxCharges":0,"Charges":0,"Container":1,"CurrentHand":-1,"ID":8},{"ItemName":"Leather Armor","Desc":" ","SecretInfo":" ","Category":"Armor","Value":10,"Appraised":10,"Weight":15,"Quantity":1,"Quality":3,"Capacity":0,"WeightOverride":0,"MagicItem":-1,"HitBonus":0,"DmgBonus":0,"WeaponType":"none","ChargeDie":"0d6","MaxCharges":0,"Charges":0,"Container":-1,"CurrentHand":-1,"ID":7},{"ItemName":"Shield","Desc":" ","SecretInfo":" ","Category":"Armor","Value":7,"Appraised":7,"Weight":5,"Quantity":1,"Quality":3,"Capacity":0,"WeightOverride":0,"MagicItem":-1,"HitBonus":0,"DmgBonus":0,"WeaponType":"none","ChargeDie":"0d6","MaxCharges":0,"Charges":0,"Container":-1,"CurrentHand":1,"ID":6},{"ItemName":"Longbow","Desc":" ","SecretInfo":" ","Category":"Weapons","Value":6,"Appraised":6,"Weight":3,"Quantity":1,"Quality":3,"Capacity":0,"WeightOverride":0,"MagicItem":-1,"HitBonus":0,"DmgBonus":0,"WeaponType":"Longbow","ChargeDie":"0d6","MaxCharges":0,"Charges":0,"Container":-1,"CurrentHand":-1,"ID":5},{"ItemName":"Longsword","Desc":" ","SecretInfo":" ","Category":"Weapons","Value":10,"Appraised":10,"Weight":4,"Quantity":1,"Quality":3,"Capacity":0,"WeightOverride":0,"MagicItem":-1,"HitBonus":0,"DmgBonus":0,"WeaponType":"Longsword","ChargeDie":"0d6","MaxCharges":0,"Charges":0,"Container":-1,"CurrentHand":2,"ID":4},{"ItemName":"Quiver","Desc":" ","SecretInfo":" ","Category":"Gear","Value":0.1,"Appraised":0.1,"Weight":1,"Quantity":1,"Quality":3,"Capacity":1,"WeightOverride":0,"MagicItem":-1,"HitBonus":0,"DmgBonus":0,"WeaponType":"none","ChargeDie":"0d6","MaxCharges":0,"Charges":0,"Container":-1,"CurrentHand":-1,"ID":3},{"ItemName":"Beltpouch","Desc":" ","SecretInfo":" ","Category":"Gear","Value":0.1,"Appraised":0.1,"Weight":0.1,"Quantity":1,"Quality":3,"Capacity":2,"WeightOverride":0,"MagicItem":-1,"HitBonus":0,"DmgBonus":0,"WeaponType":"none","ChargeDie":"0d6","MaxCharges":0,"Charges":0,"Container":-1,"CurrentHand":-1,"ID":2},{"ItemName":"Backpack","Desc":" ","SecretInfo":" ","Category":"Gear","Value":0.4,"Appraised":0.4,"Weight":0.1,"Quantity":1,"Quality":3,"Capacity":40,"WeightOverride":0,"MagicItem":-1,"HitBonus":0,"DmgBonus":0,"WeaponType":"none","ChargeDie":"0d6","MaxCharges":0,"Charges":0,"Container":-1,"CurrentHand":-1,"ID":1}]
Any idea why I'd be getting an infinite loop? If it's easier, I can definitely share my CMPGN file.

User avatar
aliasmask
RPTools Team
Posts: 9061
Joined: Tue Nov 10, 2009 6:11 pm
Location: California

Re: advice needed on combining JSON objects

Post by aliasmask »

I haven't looked over your code very closely, but one thing that jumped out at me was the bMatch. You overwrite the value for each if() so the only one that matters is the last check. If you want to consider all the checks I would make it a running total. So, if they all match, then the value has to equal to all the checks. So,

Code: Select all

[h: bMatch = bMatch + if(json.get(itemA,"ItemName") == json.get(itemB,"ItemName"),1,0)]
...
[H, if(bMatch == 9): macro.return = 1; macro.return = 0]
As for the infinite loop, they only thing I would watch out for is changing the values of the object in the foreach, but I don't see that you do that, so that's not it. You also don't want to call yourself.

busterbluth
Cave Troll
Posts: 53
Joined: Sat Aug 17, 2019 3:49 pm

Re: advice needed on combining JSON objects

Post by busterbluth »

yeah, i'll change the sub function so that bMatch has to get a sum of all the comparisons being made.

But I'm really stumped by this endless looping, and prompting for the value of arrItems ... it makes no sense to me at all.

busterbluth
Cave Troll
Posts: 53
Joined: Sat Aug 17, 2019 3:49 pm

Re: advice needed on combining JSON objects

Post by busterbluth »

This is the pop-up error I"m getting; it keeps popping up, and re-appearing, like it's stuck in some kind of stack overflow

https://1drv.ms/i/s!Aj9v68YqT0Zfg542O29 ... A?e=tHnM0Z

User avatar
aliasmask
RPTools Team
Posts: 9061
Joined: Tue Nov 10, 2009 6:11 pm
Location: California

Re: advice needed on combining JSON objects

Post by aliasmask »

I recommend using some debug tools within your loop. It could be as simple as a broadcast() of a variable value. I do have some functions in this lib token you can use: https://drive.google.com/file/d/1VTAjKS ... drive_link

The syntax would be p.watch("variableName",...) or p.pause("variableName",...) and it'll give you a popup showing the values of those variable. Make sure to put the name in quotes.

Post Reply

Return to “Macros”