[1.3.b66] StackOverflowError using long macro

[Azhrei]

So far nothing new, its a very complex macro with lots and lots of brackets so a stack overflow is bound to happen.

I'll say it again: "Regular expressions require stack space based on the amount of text entered and how that text might match the regex." (That's what I posted towards the top of this page.)

The errors are more likely when the text contains a lot of braces, brackets, and/or single/double quotes. But none of those are REQUIREMENTS for having a stack overflow.

Please don't try to analyze this problem until you have read the link I posted above and understand what the issues are. I mean, you can analyze it if you want but you're not going to accomplish anything.

[Craig]

Craig come on! You're accusing me of exactly what you're doing right now. If told numerous times that I was NOT talking about macros brackets HTML or anything else that involves code. But just clear and plain alphanumeric text. In the numerous answers you gave including this response you have ignored that fact every time.

Thats not true, let me repeat what I have said many times before in full.
Ways to avoid running out of stack.
1. Break up your macros into smaller parts (especcially where you have brackets or quotes, but not just in these cases)
2. If you have long static strings store them in a token property instead of the macro body. Lib tokens work well for this.
3. Use strformat() to create your html, prefereably with the format string stored in a token property.
4. If you get stack issues its better to refactor yuor code than bump up the stack.

If the macros that you posted are any indication its clear you have followed none of the advice so I am not surprised if you are often running into stack issues.


So instead of following the steps outlined many times before you have decided to hijack this thread and go on a rant about how now one is listening to what you have said,and how we probably don't have a clue whats causing the issues or how ridiculas it is, and given out bad advice on tackling the problem. Given the circumstances and the tone of your post I think the criticism was fair.

[wolph42]

Ok Craig, let me first thank you for the great advice, especially no 2. as its the first time I read this. No 3. I've seen before but in a completely different context and didn't connect the dots yet.

As for your criticism on me, here are the three post I either started of 'Hijacked' as you call it. Now I might have overlooked something but the ONLY advice/explanation given is that
1. brackets (in all there forms) raise stack issues
2. raising stack will prevent an stack error.

http://forums.rptools.net/viewtopic.php ... k+overflow
http://forums.rptools.net/viewtopic.php ... k+overflow
http://forums.rptools.net/viewtopic.php ... k+overflow

And that's the world I'm living in. Now I'm not saying that you have given the advice numerous times before, but until 5 minutes ago I've never seen it. Same goes for the 'backtracking' explanation Ahz has given, thats a first too.

Basically there are two things I would like to see answered when I have a problem:
1. why (does it happen) (thanks Ahz)
2. how (can I solve it) (thanks Craig)

And up to now I've never had those answers. I always got the same response: too many brackets. Which, by the way, is why I posted my earlier post: if you completely remove all the brackets and just leave a string, then MT overflows even sooner. I know you can't make that comparison so easily, which is why I left the conclusion open.

Anyway thank you for the advice however I'm still of the opinion that your criticism towards me is unduly, Craig.

[Azhrei]

Now I'm not saying that you have given the advice numerous times before, but until 5 minutes ago I've never seen it. Same goes for the 'backtracking' explanation Ahz has given, thats a first too.

The first time I mentioned backtracking was in Dec 2008 but I certainly don't expect you to be reading back that far!!

if you completely remove all the brackets and just leave a string, then MT overflows even sooner.

Because of the way the regex is structured, you might be able to get more HTML text into a macro by using this construct:

Code: Select all

[r: "html text here"] 

You'll notice it's inside brackets. That means the first RE used by the parser will match and the code will take a path that's different than the one it takes when the text is outside of brackets. That other path uses different REs to then break down what's inside the brackets and you might not get a stack overflow as quickly.

But anything that relies on a specific stack size should be avoided anyway. It requires too much effort on the part of the user to run the program with the right settings. So properties are probably best, or tables if the text won't be changing. Because tokens are modifiable and making any change requires them to be retransmitted to all connected clients, it's best to avoid modifying large tokens.

Of course, this all goes away with JS macros, but that doesn't help you right now.

[wolph42]

thnx for the many tips (both Ahz and Craig), I hope one works. The test code I uploaded earlier in this thread concerns a html form that I really want to extend with a couple of options but one extra line of anything throws the java error and while I am already on a stack of 5 (where I really want to draw the line) I'm basically stuck. I have considered raising it yet one more but my user group has grown exponentially the last couple of weeks (the campaign file is downloaded 90 times a week on average since january) so I want to be really sure that I don't release a version that will either bug out on a lot of users or getting too heavy on memory...

So upto now I just bolted the option door to the users, but with these new insights who knows...

As for 1.4... that really will be a knife that cuts on two sides, I agree with you that the current MT script is used way beyond what it was ever intended but on the other hand, reprogramming 13,000 lines of code is not something Im looking forward to (and possibly won't do at all)...

[Azhrei]

As for 1.4... that really will be a knife that cuts on two sides,

Heh-heh, yep.

"... is a three-edge sword. Their side, your side, and the truth."

But with all of the extra support for JS within the industry this will be a huge step in the right direction.

Who knows? Maybe some enterprising individual will hack together a patch for MT that traces the execution of MTscript and dumps out the corresponding JS code? That seems more likely than some kind of translator. But it won't happen anyway until the JS has been out for awhile since there would be no way to know what to translate the MTscript into...

[wolph42]

2. If you have long static strings store them in a token property instead of the macro body. Lib tokens work well for this.

This works for forms!! Thank you! You don't even need to store it into a tokenproperty, just into a tmpCache value is enough. Which in my case is good because its a dynamic value with a onSelectionChange event build in the form. But just this:

Code: Select all

[h:tmpCache = buildFormMeleeAttack()]
[dialog('Attack'):{[r:tmpCache]}] 

Instead of putting the form directly inside the macro just works!!
tmpCache is a localVar, not tokenProp.

its slightly slower, but beats the stack overflow issue Im having for months now.

[CoveredInFish]

I see a lot of stuff around here that would be great in the wiki. Anyone up to it?

[CoveredInFish]

I put together a page in the wiki: Avoiding_Stack_Overflow

Could the gurus please check if it can stand as it is or if I should change parts? I'd be happy about this especially as I quoted mostly your text here...

[wolph42]

I put together a page in the wiki: Avoiding_Stack_Overflow

Could the gurus please check if it can stand as it is or if I should change parts? I'd be happy about this especially as I quoted mostly your text here...

You hero you. I thought of adding it to my 'speed up your code' wiki page, but then realized it wasn't the right place... then a new page... and well got distracted and... now you've it already.
Read the article, looks good to me (save a couple of typos). But then again, I am (to paraphrase Craig) a 'ranting thread hijacker' when it comes to this subject and thus not a Guru... (or are Guru's just that? )

Edit: added my form-stack-overflow-issue as an example (and fixed a couple of typos)

[Azhrei]

I've corrected a couple of typos and added some "look nice" features, along with a description of how to set the memory size values on the various operating systems. I also added a link to the "memory setting" page.

Thanks, CiF!

[CoveredInFish]

Thanks for making this look nice (Man, I like this community!)

[Zandel]

I have a question on this before I start tearing macro's apart. I have 2 players getting these out of 5 players. Once they get a stack error on the complex macro, it seems that they get them on every macro, even the simplest macros that worked fine before and have not been touched generate stack errors Is that normal for this issue? Wondering if it could just be something like a Java version issue.

[wolph42]

I have a question on this before I start tearing macro's apart. I have 2 players getting these out of 5 players. Once they get a stack error on the complex macro, it seems that they get them on every macro, even the simplest macros that worked fine before and have not been touched generate stack errors Is that normal for this issue? Wondering if it could just be something like a Java version issue.

no thats not normal behaviour. normal is only stack errors for macros that push too much on the stack. A simple solution though is to block all output from those macros and only use broadcast. In a lot of cases this already solves the problem.

[Zandel]

no thats not normal behaviour. normal is only stack errors for macros that push too much on the stack. A simple solution though is to block all output from those macros and only use broadcast. In a lot of cases this already solves the problem.

Good advice thanks. I will do that if I have to, but with 228 macro's in my campaign obviously very time consuming. Not sure I could even get it done by my next game in 2 weeks. I may start slowly doing that for the macros, so I clean it up over time, but the macro that causes the initial issues is a Character sheet macro with lots of HTML, it has frames and does not output to the text box at all. Once that one throws a Stack overflow, then the players get them all the time on the rest of the macros. Going to check their java versions first.

Thanks for the advice.