MapTool 1.3 Development Build 47

[Craig]

In an attempt to replicate that lockup of MT I had. Which I mentioned in Maptools section telling everyone to save often . I tried to think of a way to quickly fill up memory. Since I was doing a LOT of debuging on macros I noticed that the memory in MT seems to get larger over time. Other then a memory leak the only thing that I figure which could account for this is chat box. So I tested with my "CRASH MT MACRO". Or at least that's what I named it hehe

Code: Select all

[C(200),CODE: {
[C(200),CODE: {
THIS IS MEGA SPAM!! BRING ON THE SPAM!! OH YEA CRAZY SPAM!! <BR>
}]
}]

You only need to run it like 3 times in the MT 256MB and it brings MT to it's breaking point. It does take several seconds to run but it also does a nice job of filling up the memory. Double-clicking on the green memory allocation bar does not free up the memory either.

Also after 3 tries I found it locks up everytime if you run the macro 3 times then type /clear in order to clear the chat box which is the common method for freeing up the chat box's memory usage which I tested after running it only 2 times.

Next I noticed SEVERAL other odd things that happen when I do this. First off is even though in the task manager it says "Not Responding" next to MT in the process section you can see the javaw.exe file using up over 50% processing and 300MB memory with it constantly changing so it seems to be doing something.

Well I can explain this one. One sentence "Fundamental flaw in the way the windows was designed". Seriously the whole "Not Responding" thing in windows is a pile of well you get the idea. They made some bad decisions when creating the OS and so you get totally meaningless status reported back to the OS. If anything that is a MS bug report (but er I wouldn't bother .

But then a SECOND tab for MT appears on the task bar even though no new MT have started running or show up in task manager. Then a little bit after that the MT with java icon next to it appears on the tab bar which is the same one that runs on startup of a new copy of MT running. It then stays on their for a while then disappears for a second followed by it reappearing. Also like it was trying to start another copy of MT.

This one is strange.

I know some people want to keep all their chat info but perhaps we could have an option in settings that would out the oldest messages after a curtain point say maybe it uses up 1/2 of the memory that way if you run the 1GB setup you can have a larger stored chat boxed before it starts clearing out the old stuff. Also if it's an option in settings people can choose if they want it to auto manage their chat logs from getting to large.

I wonder if this is really a big issue. After all if you were to try and type in all of the above that your spam macro created it would take someone who can type 70wpm about 4 days to type (and that's straight no resting).

With debugging when the build with macro dialog boxes comes out you could always create a temporary dialog box and send all output to that (it allows all the same formatting as the chat window). If it is a temporary dialog box when you click on the close button the memory is no longer held so could be garbage collected (unlike text in chat box).

[Phergus]

His test is flawed. He crashed some part of the parsing process just a few iterations into the 40000.

Of course this is why attempting to debug macros and such in MT when you start it with "javaw.exe" is such a bad idea.

The chat log is already constrained. I don't recall what size Trevor set it at.

[PyroMancer2k]

His test is flawed. He crashed some part of the parsing process just a few iterations into the 40000.

Of course this is why attempting to debug macros and such in MT when you start it with "javaw.exe" is such a bad idea.

The chat log is already constrained. I don't recall what size Trevor set it at.

The macro it self doesn't crash MT. It's trying to /clear the chat that causes it to crash.

[RPTroll]

RP, I've foudn this to be the case when someone is using a copy of a token that was created in an older version and then not saved to the newer version.

I had that show up with some old D&D tokens that UntoldGlory had made ages ago that I stuck into a new campaign.

Another case was when a guy joined a game and his token always showed up as a "?" until we had him save it in an updated version and rejoin the game later.

Jay

It was a new token, just an image dropped as a token really. Then I tried dropping it as a object and background with the same result.

It is normal for a question mark to show during the image transfer process. Chances are their actions have nothing to do with the image appearing. It just finally transferred to them and a screen update took place.

Well, there's always the coincident factor but we were talking on the phone while it was happening and it wasn't until he put something in chat and hit return that it showed up in both chat and on the screen.

/me shrugs

I was just trying to give you our last experience with it. My friend and I may be running through a Nazis in space adventure development next Friday morning at 8 a.m. Central time if any of the devs want to log in and see if we can break it.

[Phergus]

The macro it self doesn't crash MT. It's trying to /clear the chat that causes it to crash.

Didn't say it crashed MT. I said it crashes the parser. In specific the Java regex parser when it creates an input string that is too long for it to handle. Just 15 or so iterations of the inner loop causes this:

Code: Select all

Exception in thread "AWT-EventQueue-0" java.lang.StackOverflowError
	at java.util.regex.Pattern$Loop.match(Unknown Source)
	at java.util.regex.Pattern$GroupTail.match(Unknown Source)
	at java.util.regex.Pattern$BranchConn.match(Unknown Source)
	at java.util.regex.Pattern$CharProperty.match(Unknown Source)
	at java.util.regex.Pattern$Branch.match(Unknown Source)
	at java.util.regex.Pattern$GroupHead.match(Unknown Source)

Are you running 64-bit Java?

Did you save the console log out of Eclipse while you were testing?

[Phergus]

I was just trying to give you our last experience with it. My friend and I may be running through a Nazis in space adventure development next Friday morning at 8 a.m. Central time if any of the devs want to log in and see if we can break it.

And the input is appreciated. If the images do finally show up in your sessions however that isn't the same problem. The problem being discussed is where the image never gets displayed. The question mark remains no matter what the clients do. If I recall correctly it will persist even across client restarts which would suggest that the local asset caching is involved somehow.

Unfortunately I won't be around Friday morning. Maybe someone else will be able to connect up.

[PyroMancer2k]

The macro it self doesn't crash MT. It's trying to /clear the chat that causes it to crash.

Didn't say it crashed MT. I said it crashes the parser. In specific the Java regex parser when it creates an input string that is too long for it to handle. Just 15 or so iterations of the inner loop causes this:

Code: Select all

Exception in thread "AWT-EventQueue-0" java.lang.StackOverflowError
	at java.util.regex.Pattern$Loop.match(Unknown Source)
	at java.util.regex.Pattern$GroupTail.match(Unknown Source)
	at java.util.regex.Pattern$BranchConn.match(Unknown Source)
	at java.util.regex.Pattern$CharProperty.match(Unknown Source)
	at java.util.regex.Pattern$Branch.match(Unknown Source)
	at java.util.regex.Pattern$GroupHead.match(Unknown Source)

Are you running 64-bit Java?

Did you save the console log out of Eclipse while you were testing?

I don't have Eclipse. And I'm not running 64-bit Java either.

[Azhrei]

If I recall correctly it will persist even across client restarts which would suggest that the local asset caching is involved somehow.

Yep, this is correct. That's why I plan to add an entry to a logfile for EVERY attempt to set the GUID, MD5Key, or PortraitImage, as well as a log entry every time the "?" image has to be used (which is apparently only in ServerMethodHandler.getAsset()).

I'm looking at two .rptok files, one with a broken image as the CharacterImage and another that has all three images as broken. They refer to the same MD5Key for the broken image (which makes sense, as the MD5Key is based on the path to the image), but with different GUID's. I don't yet know what the GUID is for, but it must be some kind of global ID number. Can anyone familiar with the code shed some light on this?

[Phergus]

The GUIDs are, as you surmised, globally unique IDs so that every asset being tracked by MT will have a unique ID independent of the name or source.

It is based off of the host IP, the timestamp at the time the GUID is created and an auto-incrementing counter.

net.rptools.maptool.model.GUID

Pretty much every time you add a new object within MT it gets its own GUID.

[Azhrei]

And it looks like the MD5Key is an MD5 hash of the asset path. Not sure yet if that's the "asset://" URL or the one that represents the image in the repository. But I'm guessing the former.

Okay, I have enough logging to be able to see that with an existing token, the currentImage is set to an MD5Key that doesn't exist in the assetcache directory nor in the AssetManager. So the program chooses the "broken.png" image as a replacement. My only thinking at this point is that the asset was not sent over the network reliably?

Tomorrow I'm going to turn on the logging and save it in a text file. I'm hoping that if I track every attempt to call setId() on Assets and all image calls on Tokens, I might be able to trace backwards and locate the problem. We'll see...

[Phergus]

My only thinking at this point is that the asset was not sent over the network reliably?

This is my theory on it.

* Client B drops down a new token/image. Server is notified and transfer begins.
* Server knows there is a new image but doesn't have it yet. Server tells all other clients that there is a new image on the map.
* Client A gets notice of new image and lacking the image puts up broken.png. Client A requests image from server.
* Server gets request it doesn't have it yet because the transfer is still in progress. Server gets confused and flags the image as not available. Transfer is lost somehow.

[DrVesuvius]

Has anybody managed to get this functionality to work?

* When impersonating a token in the impersonate panel, you can use macros on other tokens (via the selection panel or Apply To Each Selected options) without breaking your impersonation. It runs the macro on the currently selected tokens, then automatically re-impersonates whichever token you had been impersonating originally.

I've got a "Give player a Benny" macro which basically increments the token's Bennies property by one. At the moment I'm giving each player token that macro with instructions that only the GM is allowed to use it.

It would be nice if I could just have the macro on my GM token, and use the "Run for each Selected" with a player token selected. But whenever I do, the macro still acts on the GM token, not the player. The same happens with a simple "hello world" macro, which gets displayed as coming from the GM token rather than the player's, so it's not a matter of rights to token properties.

Am I doing something wrong, or have I misunderstood the original paragraph?

[Azhrei]

My only thinking at this point is that the asset was not sent over the network reliably?

This is my theory on it.

* Client B drops down a new token/image. Server is notified and transfer begins.
* Server knows there is a new image but doesn't have it yet. Server tells all other clients that there is a new image on the map.
* Client A gets notice of new image and lacking the image puts up broken.png. Client A requests image from server.
* Server gets request it doesn't have it yet because the transfer is still in progress. Server gets confused and flags the image as not available. Transfer is lost somehow.

And from my reading of the code in the ImageLoaderCache class, once the image resolves to the "broken.png", it will never be tested again to see if the image has suddenly become available. For this to happen, there would need to be an ImageObserver set up that does notice when an image is available and then goes through and modifies all of the imageMap's for tokens using that image. That is VERY non-practical!

When the broken image is going to be used, it is referenced as the constant BROKEN_IMAGE (a BufferedImage) in the util.ImageManager class. If there were some way to cause any use of that object in a token to result in adding a listener (or observer, or whatever you want to call it ) to a list somewhere, then whenever a new image becomes available, that list could be checked to see if the newly available image is one that was previously being displayed as broken.

But I don't know of any easy way to do that. The current code stores the MD5Key of BROKEN_IMAGE in the Token's imageAssetMap. What we need to have stored in the imageAssetMap is the correct MD5Key and when the image needs to be used and can't be found, replace it BROKEN_IMAGE. But that's a lot of overhead to do that at every lookup. It might work to store the image reference in the MD5Key, though. So the MD5Key for my portrait would contain a null, and whenever the object is asked for the image, it returns BROKEN_IMAGE if the reference is null?

I think this would require some significant restructuring, but I haven't looked at it in detail yet. I thought of it just now while making this reply.

Trevor, what do you think? Is my analysis of the potential problem (once the image is broken, it can never be fixed) correct?

[jfrazierjr]

My only thinking at this point is that the asset was not sent over the network reliably?

This is my theory on it.

* Client B drops down a new token/image. Server is notified and transfer begins.
* Server knows there is a new image but doesn't have it yet. Server tells all other clients that there is a new image on the map.
* Client A gets notice of new image and lacking the image puts up broken.png. Client A requests image from server.
* Server gets request it doesn't have it yet because the transfer is still in progress. Server gets confused and flags the image as not available. Transfer is lost somehow.

And from my reading of the code in the ImageLoaderCache class, once the image resolves to the "broken.png", it will never be tested again to see if the image has suddenly become available. For this to happen, there would need to be an ImageObserver set up that does notice when an image is available and then goes through and modifies all of the imageMap's for tokens using that image. That is VERY non-practical!

When the broken image is going to be used, it is referenced as the constant BROKEN_IMAGE (a BufferedImage) in the util.ImageManager class. If there were some way to cause any use of that object in a token to result in adding a listener (or observer, or whatever you want to call it ) to a list somewhere, then whenever a new image becomes available, that list could be checked to see if the newly available image is one that was previously being displayed as broken.

But I don't know of any easy way to do that. The current code stores the MD5Key of BROKEN_IMAGE in the Token's imageAssetMap. What we need to have stored in the imageAssetMap is the correct MD5Key and when the image needs to be used and can't be found, replace it BROKEN_IMAGE. But that's a lot of overhead to do that at every lookup. It might work to store the image reference in the MD5Key, though. So the MD5Key for my portrait would contain a null, and whenever the object is asked for the image, it returns BROKEN_IMAGE if the reference is null?

I think this would require some significant restructuring, but I haven't looked at it in detail yet. I thought of it just now while making this reply.

Trevor, what do you think? Is my analysis of the potential problem (once the image is broken, it can never be fixed) correct?

Is there not a common entry point into getting an image asset? something like token.getImageAssetMap().getImage()? Unfortunately, I don't have the code in front of me on this computer so I can't see what you are looking at(and I am not familiar with that code anyway). I would hope that all calls to get one of the token images should be pulled from a single method and chained upward. If that was the case, then your solution would work as long as the MD5Key is the same on all machines. In this case, it should be simple to get the assigned image's MD5Key(after changing to store the real one instead of the broken image), and then on each call check to see if the image exists in the asset cache and if not, send the BROKEN_IMAGE to the caller. If this can get fixed, then this also make it easier to add a ServerCommand to update the tokens image, and again, when the image gets to the client's asset cache, the update would happen the next time the image is retrieved from the token object.

[syntruth]

I haven't looked at the code, but just reading Azhrei's post got an idea in my head:

If it's true that Client A is requesting an image from the server that has not fully been transferred from Client B, could the server not just cache that request until the transfer is finished?

Psuedo-code:

Code: Select all

  method requestImageAsset(host, key) {
    if imageAssetLocked(key) {
      cacheRequest(host, key)
    } 
    else {
      return imageAsset(key)
    }
  }

...then have the server poll that cache, and once an asset is complete on the server side, it initiates the send to the original requesting host. It would only require one look-up per host requesting the asset and the server periodically checking that cache, or probably better, once an asset is fully recieved, it checks the cache and initiates sending.

Again, I am so not familiar with the source (yet! After the holidays...) but figured I'd type this out while it was fresh in my head.