RPTools.net

Discussion and Support

Skip to content

It is currently Mon Dec 11, 2017 11:57 am 






Reply to topic  [ 99 posts ]  Go to page Previous  1, 2, 3, 4, 5, 6, 7  Next

Previous topic | Next topic 

  Print view

Author Message
User avatar  Offline
Dragon
 
Joined: Wed May 21, 2008 11:15 pm
Posts: 460
 Post subject: Re: Macro Best Practices
PostPosted: Mon Sep 28, 2009 1:31 pm 
Craig wrote:
Well you did say something special is going on to create the reference that is different from primitives so its was only "technically" pass by value :) Which is the bit that was incorrect.

and I still say something "special" is going on, but I'll admit, whether the behavior is "special" in which language is debatable...

Here's my lousy attempt at an explanation


Top
 Profile  
 
User avatar  Offline
Dragon
 
Joined: Wed May 21, 2008 11:15 pm
Posts: 460
 Post subject: Re: Macro Best Practices
PostPosted: Mon Sep 28, 2009 1:47 pm 
Rumble wrote:
The upshot of all of this being: macros can go real slow if you use excessive JSONs. :P

Yeah, so back to my original question...
It appears that passing a single property is indeed more efficient than passing a large JSON, Why? Because the a new JSON gets copied each time we pass it as a parameter? Can we pass a JSON array or a JSON object into a macro and have the original be modified? or are users just passing around data JSONs to be used and discarded? (and things like loops are killing performance when you really only needed a couple of properties?)


Top
 Profile  
 
User avatar  Offline
Deity
 
Joined: Tue Jul 01, 2008 6:48 pm
Posts: 6237
 Post subject: Re: Macro Best Practices
PostPosted: Mon Sep 28, 2009 2:43 pm 
wyrmwood wrote:
Rumble wrote:
The upshot of all of this being: macros can go real slow if you use excessive JSONs. :P

Yeah, so back to my original question...
It appears that passing a single property is indeed more efficient than passing a large JSON, Why? Because the a new JSON gets copied each time we pass it as a parameter? Can we pass a JSON array or a JSON object into a macro and have the original be modified? or are users just passing around data JSONs to be used and discarded? (and things like loops are killing performance when you really only needed a couple of properties?)



Can't answer the former, but loops appear to hit performance regardless. I did a simple loop over token states, and then modified it to directly manipulate the special "state.Whatever" variables, and the second was much faster, even though the loops involved no JSONs or any passed variables.

_________________

What I'm Working On

MapTool Tutorials:
Introduction to Tokens
Introduction to Properties
Introduction to Macro Writing
Introduction to Light and Sight


Top
 Profile  
 
 Offline
Great Wyrm
 
Joined: Sun Jun 22, 2008 6:53 pm
Posts: 2102
Location: Melbourne, Australia
 Post subject: Re: Macro Best Practices
PostPosted: Mon Sep 28, 2009 3:57 pm 
wyrmwood wrote:
Craig wrote:
Well you did say something special is going on to create the reference that is different from primitives so its was only "technically" pass by value :) Which is the bit that was incorrect.

and I still say something "special" is going on, but I'll admit, whether the behavior is "special" in which language is debatable...

Here's my lousy attempt at an explanation


Please reread my posts, your understanding is flawed. If you want equivalent code then you should change your C++ code to.
Code:
{   
      Person *Bob = new Person();
      Bob->m_nHP = 20;
      cout << "Bob has " << Bob->m_nHP << " HP" << endl;
 
      HurtMeByValue(Bob, 5);
...

void HurtMeByValue(Person *a, int nDamage)
{
      a->m_nHP -= nDamage;
}


Because in Java
Person p;
is the same as the C++
Person *p;
Do that and then tell me they are doing something different.

edit for further explanation

In Java
Person p = new Person() is an object allocated in heap space

In C++
Person *p = new Person() is an object allocated in heap space
Person p is a temporary object allocated on the stack and has no equivalent in Java.

You compare apples from one store with oranges from a different store (instead of the seconds stores apples) and since you see a difference you are attributing that to something special that the second store is doing to its oranges so that they no longer look like apples.


Last edited by Craig on Mon Sep 28, 2009 5:02 pm, edited 2 times in total.

Top
 Profile  
 
 Offline
Great Wyrm
 
Joined: Sun Jun 22, 2008 6:53 pm
Posts: 2102
Location: Melbourne, Australia
 Post subject: Re: Macro Best Practices
PostPosted: Mon Sep 28, 2009 4:04 pm 
wyrmwood wrote:
Rumble wrote:
The upshot of all of this being: macros can go real slow if you use excessive JSONs. :P

Yeah, so back to my original question...
It appears that passing a single property is indeed more efficient than passing a large JSON, Why? Because the a new JSON gets copied each time we pass it as a parameter?

I already explained this is not the case. Please reread earlier posts.

wyrmwood wrote:
Can we pass a JSON array or a JSON object into a macro and have the original be modified? or are users just passing around data JSONs to be used and discarded? (and things like loops are killing performance when you really only needed a couple of properties?)

Are you wanting mutable JSON objects or pass by reference? Pass by reference will make things even slower, mutable JSON objects is possible but then breaks how everything currently works causing people to rewrite all their macros.


Top
 Profile  
 
 Offline
Great Wyrm
 
Joined: Sun Jun 22, 2008 6:53 pm
Posts: 2102
Location: Melbourne, Australia
 Post subject: Re: Macro Best Practices
PostPosted: Mon Sep 28, 2009 5:24 pm 
I just want to add to this thread token properties are always going to be faster than using JSON objects -- for 1.3 at least. This is because of the following reasons.

  • JSON objects under certain conditions (reading/writing from/to token properties) requires conversion from one internal representation to another.
  • MapTool supports storing JSON objects as string representations -- this is one of the main points behind JSON -- and strings are immutable in MapTool script, therefore JSON objects need to be immutable.

It has nothing to do with pass by value/references.When using [macro(): ] calls passing a JSON object (or property, or number) is essentially the following code in the MapTool.
Code:
    newScope.put("macro.args", args);

Which creates a new variable in the new scope.

If you have your JSON object inside of a token property (say for example WeaponsList) and do the following
Code:
      [c(100), code: {
           [macro("blah@lib:token"): WeponList]
      }]


You will take a internal conversion hit from token property representation to internal variable representation on each and every time through the loop. You could easily speed this up by doing the following for large JSON object.
Code:
      [wList = WeaponList]
      [c(100), code: {
           [macro("blah@lib:token"): wList]
      }]



User defined functions occur differently. User defined functions in reality are not function calls. What they do is create a new string that is passed to the parser, so when you have a large argument this parsing takes extra time -- this is not to be confused with copying on pass by value because this is not what it is. So there are two solutions to this problem, keep your arguments small or use [macro(): ], but again be careful about "invisible" conversions when placing the token property directly in the call.


While allowing JSON objects to be mutable would solve several of the speed issues with modifying JSON objects it is not possible to make them mutable in all cases which means you end up with a large list of exceptions where JSON objects would still be immutable. Also supporting mutable JSON objects withing user defined functions would require a lot more processing than is currently done and significantly slow down calls to user defined functions.


In 1.4 there is a large opportunity for scripting performance and the two things that will provide the greatest performance are

JavaScript
This is not because JavaScript is an inherently faster language, but because of the way that parsing currently occurs in MapTool. The JavaScript engine will parse all input once and generate an intermediate code that it can process a lot faster.

The way MapTool currently works there is a Parser module (not part of the MapTool code) that parses all text and generates intermediate code that is faster to process, but this only parses/executes the text that MapTool passes to it. The MapTool input scanner breaks up macros and sends then essentially line by line to the Parser module, so when you loop you re parse the same text over and over and over.

JSON improvements
This has nothing to do with the implementation of JSON in JavaScript itself, but in 1.4 hopefully the parts of MapTool that expect strings and only strings to be stored in token properties can be fixed so that they wont bomb out when something other than a string is stored, this way there will be no conversion hit when moving JSON objects to and from token properties.


Top
 Profile  
 
User avatar  Offline
Deity
 
Joined: Tue Jul 01, 2008 6:48 pm
Posts: 6237
 Post subject: Re: Macro Best Practices
PostPosted: Mon Sep 28, 2009 6:03 pm 
Ahhhh...very useful. I guess my next project will be evaluating my framework's loops and make sure I'm not requiring a representation change on every iteration.

Craig, if nobody else says it, thanks - it's a huge help to have someone who knows the intricacies of the parser and the scripting language lay some knowledge on us.

_________________

What I'm Working On

MapTool Tutorials:
Introduction to Tokens
Introduction to Properties
Introduction to Macro Writing
Introduction to Light and Sight


Top
 Profile  
 
User avatar  Offline
Dragon
 
Joined: Wed May 21, 2008 11:15 pm
Posts: 460
 Post subject: Re: Macro Best Practices
PostPosted: Mon Sep 28, 2009 7:01 pm 
Rumble wrote:
Craig, if nobody else says it, thanks - it's a huge help to have someone who knows the intricacies of the parser and the scripting language lay some knowledge on us.

copy that


Top
 Profile  
 
User avatar  Offline
Dragon
 
Joined: Sat Apr 12, 2008 3:17 pm
Posts: 278
 Post subject: Re: Macro Best Practices
PostPosted: Wed Nov 18, 2009 6:19 pm 
Ah, I remember the days when this thread contained a discussion of macro best practices. ;)


Top
 Profile  
 
User avatar  Offline
Dragon
 
Joined: Mon Jul 21, 2008 6:38 pm
Posts: 742
 Post subject: Re: Macro Best Practices
PostPosted: Mon May 10, 2010 10:56 am 
On the subject of Best Practices and JSON objects, just to be sure I'm currently on the right page, and I may not be...

I'm currently storing some JSON arrays of JSON objects in token properties. I don't pass them as macro arguments, I just use them directly as needed.
I haven't noticed a significant memory hit so far, except once when I just left Maptools alone and open all night with three frames open (by accident) and the memory used (As per the little bar) ended up creeeeeping. No macro activity, so I'm assuming that's related to something in Maptools directly that I may have triggered by creating the FRAMEs.

EDIT: Helps if I finish the question...

When working with the token property containing something JSON, is the best practice that:
I should access the token property as few times as possible, and instead work with a local variable where possible and "commit" my changes back to the token property at the end?


Top
 Profile  
 
User avatar  Offline
Demigod
 
Joined: Mon Jun 29, 2009 9:37 am
Posts: 3108
Location: Germany
 Post subject: Re: Macro Best Practices
PostPosted: Mon May 10, 2010 11:20 am 
Mrugnak wrote:
When working with the token property containing something JSON, is the best practice that:
I should access the token property as few times as possible, and instead work with a local variable where possible and "commit" my changes back to the token property at the end?


Yes. Each time you read/store a json object from/to a property it is transformed into/created by a simple string. This conversion is performance-critical. As long as you use a variable containing a json object there are no conversions.

_________________
HELP: GETTING STARTED WITH MAPTOOLS - TUTORIALS, DOCS, VIDEOS, TOOLS, ETC.

most complete list of my maptool stuff


Top
 Profile  
 
User avatar  Offline
Dragon
 
Joined: Mon Jul 21, 2008 6:38 pm
Posts: 742
 Post subject: Re: Macro Best Practices
PostPosted: Mon May 10, 2010 11:44 am 
CoveredInFish wrote:
Yes. Each time you read/store a json object from/to a property it is transformed into/created by a simple string. This conversion is performance-critical. As long as you use a variable containing a json object there are no conversions.


OK! That's thankfully pretty easy to adjust the way I've got my macros set up so far (glad I caught this before getting too involved though!)


Top
 Profile  
 
User avatar  Offline
Deity
 
Joined: Tue Nov 10, 2009 6:11 pm
Posts: 7964
Location: Bay Area
 Post subject: Re: Macro Best Practices
PostPosted: Mon May 10, 2010 4:13 pm 
I found setting the property to a json object takes longer than getting it from a property. Maybe it's my imagination.

_________________
Downloads:


Top
 Profile  
 
 Offline
Kobold
 
Joined: Thu Aug 18, 2011 9:51 am
Posts: 16
 Post subject: Re: Macro Best Practices
PostPosted: Fri Aug 26, 2011 4:30 am 
After reading over some of the info in this thread about user defined functions, macros, and JSON objects, I had a couple questions I'm hoping someone can answer.

1.) JSON objects/arrays are slower to load from and save to string properties, but are there any speed advantages in working with JSON objects/arrays once they are loaded or as temporary variables? Is json.get() faster or slower than listGet(), for example?

2.) It was mentioned that passing a JSON object/array to a user defined function in a loop would be particularly slow, because each loop would require translating the object to and then from a string; am I correct in thinking that, in a situation like that, it would be considerably faster to use a function defined with newScope set to false to access the JSON object/array, instead of passing it as an argument?

i.e. instead of this:
Code:
[h: defineFunction(exampleFunc, exampleMacro)]

...
 later when the function is called ...

[
h: someJsonObject = json.set("{}", "Key1", Value1, "Key2", Value2, ... , "KeyN", ValueN)]
[
h, count(someLoopCounter), CODE:
{
  [h: exampleFunc(someJsonObject)]
}]
 


something like this:
Code:
[h: defineFunction(exampleFunc, exampleMacro, 0, 0)]

...
 later when the function is called ...

[
h: someJsonObject = json.set("{}", "Key1", Value1, "Key2", Value2, ... , "KeyN", ValueN)]
[
h: targetObject = "someJsonObject"]
[
h, count(someLoopCounter), CODE:
{
  [h: exampleFunc(targetObject)]
}]
 


Top
 Profile  
 
 Offline
Kobold
 
Joined: Thu Aug 18, 2011 9:51 am
Posts: 16
 Post subject: Re: Macro Best Practices
PostPosted: Mon Aug 29, 2011 8:34 am 
Barrodin wrote:
2.) It was mentioned that passing a JSON object/array to a user defined function in a loop would be particularly slow, because each loop would require translating the object to and then from a string; am I correct in thinking that, in a situation like that, it would be considerably faster to use a function defined with newScope set to false to access the JSON object/array, instead of passing it as an argument?


Did a bit of testing, and it turns out that it is much, much faster to use a local scope function to pass and return a JSON Object by name, rather than passing the actual object itself. Since this involves using eval() to retrieve the value, it is absolutely not suitable for anything where you're dealing with uncertain input, but I think it could be a very useful trick for speeding up common function calls where you can be sure of the values passed.

Passing an object like this:
Code:
[h: testObject = json.set("{}", "Key 1", value1, "Key 2", value2, ... , "Key N", valueN)]
[
h: test_function("testObject")]

...
 Inside test_function ...
[
h: local_testObject = eval(arg(0))] 


...instead of this:
Code:
[h: testObject = json.set("{}", "Key 1", value1, "Key 2", value2, ... , "Key N", valueN)]
[
h: test_function(testObject)]

...
 Inside test_function ...
[
h: local_testObject = arg(0)] 


...was around 4x as fast, and returning JSON objects from a local function using a similar method was even faster (~16x as fast, but it was hard to be sure, because at that point each test was only about 1 second from start to finish).

Keep in mind that any variables in a function will overwrite functions in the macro from which it is called, so be conservative creating new variables and always use a distinctive name (I tend to add the prefix local_ to the variable names inside such functions, as a reminder). If you do decide to use this trick it is probably best to create as few variables inside the fucntion as possible, to avoid the possibility of collisions.

EDIT: On a side note, some sort of built in timestamp function would be enourmously helpful in tests like this.


Top
 Profile  
 
Display posts from previous:  Sort by  
Reply to topic  [ 99 posts ]  Go to page Previous  1, 2, 3, 4, 5, 6, 7  Next

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:

Who is online

In total there is 1 user online :: 0 registered, 0 hidden and 1 guest (based on users active over the past 5 minutes)
Most users ever online was 243 on Sun Nov 04, 2012 6:14 am

Users browsing this forum: No registered users and 1 guest





Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group

Style based on Andreas08 by Andreas Viklund

Style by Elizabeth Shulman