Difficulties running maps with objects larger than 1 MB.

Thoughts, Help, Feature Requests, Bug Reports, Developing code for...

Moderators: dorpond, trevor, Azhrei

Forum rules
PLEASE don't post images of your entire desktop, attach entire campaign files when only a single file is needed, or generally act in some other anti-social behavior. :)
Post Reply
Tyshalle
Giant
Posts: 173
Joined: Sat Jul 10, 2010 5:45 pm

Difficulties running maps with objects larger than 1 MB.

Post by Tyshalle »

I don't really get this, so I'm just trying to figure out if it's just me, and if there's anything I can do to fix this.

Basically my issue is simple: Using objects larger than 1 MB tend to cause me issues in large numbers. If I put a 2 MB image on the map, half the time I have at least one player unable to see it (it shows up as a big red X for them). If the image is 4 MB or higher, forget about it, half my players can't see it without them restarting MapTool and maybe dumping their cache folder. If the image file is more than say, 10 MB, half the time I can't even see it.

I don't understand anything about the tech behind MapTool or Java or whathaveyou, but it seems like in 2016 we wouldn't have such serious restrictions on this sort of stuff. I have some pretty beautiful maps I've seen and made that run in the 80 MB range for file sizes, and there's no way I can get maptool to run it.

Anything to be done, or is this just the way it is?

User avatar
wolph42
Winter Wolph
Posts: 9999
Joined: Fri Mar 20, 2009 5:40 am
Location: Netherlands
Contact:

Re: Difficulties running maps with objects larger than 1 MB.

Post by wolph42 »

it sounds like you memory settings are incorrect.

- first off: do you use the *launcher* to launch maptool, the .bat or do you run it directly?
- next, which java version do you have installed (and what is you pc architecture) x32 or x64? Note that per default java installs x32 which limits your memory extensively! To fix that you need to uninstall java (ALL YOUR PLAYERS!!!) and download the 64 bits version of java and install that
- assuming you use the launcher most optimal settings:
min: dont touch (default is 64 IRC)
max: 2048
stack: 4

you "max mem" decides the size of the picture. However using bigger stack (e.g. 8) will diminish the overall available memory!
If one of your players have 32 bit system then you need to limit you memory to around 1500. I still would advice to install j64 on the other systems as its about 1.6x faster than j32.

note: I have a maptool campaign with multiple map images around 100 mb!! (they work!)

User avatar
aliasmask
RPTools Team
Posts: 9024
Joined: Tue Nov 10, 2009 6:11 pm
Location: Bay Area

Re: Difficulties running maps with objects larger than 1 MB.

Post by aliasmask »

Looking at some old maps, I have 1 that's 6 MB and we didn't have a problem with it. Me and all the players run at least 1400 max mem, but I recommend with 64bit OS and 64bit java to run at 2048. I'm not sure if any of my players has a lower setting than 2048.

You should verify your player's memory settings and make sure they're full utilizing their system's memory. The most common thing is players install the default 32bit java when they really want the manual download of the 64bit version. I also recommend using the latest MT version.

You may want to create a repository file and upload it to a 3rd party web client and then go to your campaign properties and put the link to source there. It's basically just a zip file of all the images in the campaign. This will load the images in to the player's cache automatically.

Another possible solution is to split the image and put it down as tiles. They can be big tiles 2000px x 2000px for example and still be less than 1 MB each. There are several free online web based sources that will do this for you. You can also, reduce the size of the image by 50% and then stretch it to double size in MT. You can go to 25% but beyond that and you'll experience some pixelation. Make sure to save as a jpg file and not a png.

As for the technology involved, it's probably based on code written 10 years ago so I can't say what should and shouldn't work. The people at MOTE rewrote much of the base code to try and speed things up. You can do a comparison there and see if it's any different.

User avatar
Jagged
Great Wyrm
Posts: 1306
Joined: Mon Sep 15, 2008 9:27 am
Location: Bristol, UK

Re: Difficulties running maps with objects larger than 1 MB.

Post by Jagged »

Another thing you might want to consider is compressing the map.

I bet you can open your 6M map in GIMP or some similar tool and export it with a lower compression rating and not be able to see the difference. I tend not to use large images for my maps, but on the few occasions I have, I have been able to reduce the size significantly.

The next step would be to then create a repository and upload that somewhere. That won't necessarily help with memory usage but will help with speed.

Tyshalle
Giant
Posts: 173
Joined: Sat Jul 10, 2010 5:45 pm

Re: Difficulties running maps with objects larger than 1 MB.

Post by Tyshalle »

wolph42 wrote:first off: do you use the *launcher* to launch maptool, the .bat or do you run it directly?
I use launcher-130829.01.jar.

wolph42 wrote:next, which java version do you have installed (and what is you pc architecture) x32 or x64? Note that per default java installs x32 which limits your memory extensively! To fix that you need to uninstall java (ALL YOUR PLAYERS!!!) and download the 64 bits version of java and install that
I am using Java 7, Update 55, the 64 bit version, which is right for my computer.

My Min is the default 64, my max is 1024, my stack is 5. I'll try switching it to 2048 and stack 4 and see if that helps.
wolph42 wrote:note: I have a maptool campaign with multiple map images around 100 mb!! (they work!)
As in, you drop a single .png or .jpeg image of 100 MB on the map and that works fine? I have made maps out of like, a hundred or more 0.8 MB images that has given the map a pretty hefty file size and it's worked OK, but I have never in my life been able to drop a single image on maptool of that size and had it work okay.
aliasmask wrote:You may want to create a repository file and upload it to a 3rd party web client and then go to your campaign properties and put the link to source there. It's basically just a zip file of all the images in the campaign. This will load the images in to the player's cache automatically.

Another possible solution is to split the image and put it down as tiles. They can be big tiles 2000px x 2000px for example and still be less than 1 MB each. There are several free online web based sources that will do this for you. You can also, reduce the size of the image by 50% and then stretch it to double size in MT. You can go to 25% but beyond that and you'll experience some pixelation. Make sure to save as a jpg file and not a png.
Yeah, I use repositories fairly often. I also have in the past made tiles of many files, I just find that to be a pain in the donkey, especially if one of my players winds up getting a big red X on one of the tiles and then we have to troubleshoot for several minutes. And yeah, I know all the photoshop tricks, but you've played in my campaigns before, you know the kind of OCD perfectionist I can be, always wanting my maps to look pristine and as high quality as possible. I'm able to make things work with these tricks, I was just wondering if there was something I could do to fix MapTool or Java or my computer or something so this wouldn't be such a problem.

User avatar
aliasmask
RPTools Team
Posts: 9024
Joined: Tue Nov 10, 2009 6:11 pm
Location: Bay Area

Re: Difficulties running maps with objects larger than 1 MB.

Post by aliasmask »

I think wolph42 is talking about 100 MB campaign file. If you're using MT 1.4 then it's safe to use the latest version of java 8. 1024 max mem is good for most campaigns, but I too have moved over to 2048 and my games seem to run smoother now. But you can't have that size unless you have a 64bit OS and 64bit java.

User avatar
Full Bleed
Demigod
Posts: 4736
Joined: Sun Feb 25, 2007 11:53 am
Location: FL

Re: Difficulties running maps with objects larger than 1 MB.

Post by Full Bleed »

Jagged wrote:Another thing you might want to consider is compressing the map.

I bet you can open your 6M map in GIMP or some similar tool and export it with a lower compression rating and not be able to see the difference. I tend not to use large images for my maps, but on the few occasions I have, I have been able to reduce the size significantly.
Compressing an image won't change its memory footprint significantly, it will just change the file size and transfer more quickly.
The next step would be to then create a repository and upload that somewhere. That won't necessarily help with memory usage but will help with speed.
You are correct, it won't fix this memory issue. I get that you and Alias are trying to help with this suggestion, but it's really kind of off topic and, potentially, an unnecessary complication. Tyshalle could spend a bunch of time trying to figure out how to set up a repository, only to (inevitably) find that it will have no impact on this image display problem.

The memory issue is very likely 1 of 2 problems (the first being far more likely than the second and covered by Wolph):

1) Bad memory settings. Get the Help>Gather Debug Information memory settings and java versions from affected players and report back. Try to get all your players on the same settings, preferably using 64 bit Java and a 2048 Max Mem. I usually suggest a 4 stack, Alias typically suggests a 3 stack.

2) File type variation. Believe it or not, some standard file types seem to have some variations that MT doesn't like. For instance, recently a friend on a Mac created a PNG state. It displayed fine on his computer. None of the other players (all on PC's) could see the state properly. He sent me the state, I opened it in photoshop, resaved it as a PNG file and sent it back to him. He reapplied the state to his campaign file and it then worked properly for everyone. I'm not sure what image program he used to make the original PNG, but there must have been some variation to the format that caused the display problem. This has happened a few time over the years. This isn't related to file size though.


Probably the largest image file I've ever used in a campaign is 9446 x 6996 (a nearly 9 MB JPG file of the World of Greyhawk). None of my players have a hard time displaying the image (including one 32 bit java user, though I do believe it's probably nearing their limit on load-in due to the way MT spikes memory usage on load-in... i.e. loading into the map spikes to around 1200 mb... but flush the memory cache after loading in and memory usage drops significantly and stays in the 400-800mb range.)
Maptool is the Millennium Falcon of VTT's -- "She may not look like much, but she's got it where it counts."

Tyshalle
Giant
Posts: 173
Joined: Sat Jul 10, 2010 5:45 pm

Re: Difficulties running maps with objects larger than 1 MB.

Post by Tyshalle »

I will try your suggestions at my next session this Sunday. I already attempted this with a 60 MB image and brought a friend in for a test run and it actually seemed to work a lot better than it has in the past. I think I've just been off the forums too long, I had no idea people were going up to 2048 or that was even possible. Last time I bumped it up to 1024 from 512 I remember conventional thinking around here was that might be pushing it, but that was also, I don't even know, maybe 5+ years ago!

Anyway, thanks for your help guys, I'll let you know how it works.

User avatar
Full Bleed
Demigod
Posts: 4736
Joined: Sun Feb 25, 2007 11:53 am
Location: FL

Re: Difficulties running maps with objects larger than 1 MB.

Post by Full Bleed »

Tyshalle wrote:I already attempted this with a 60 MB image
60 MB campaign file, or a single 60 MB image asset? I suspect the former.

Keep in mind that the size of the campaign file isn't too relevant... players only load into memory the assets of the map they are currently on. It's the size and number of assets on a single map that determine the memory allocation needed.
Maptool is the Millennium Falcon of VTT's -- "She may not look like much, but she's got it where it counts."

Tyshalle
Giant
Posts: 173
Joined: Sat Jul 10, 2010 5:45 pm

Re: Difficulties running maps with objects larger than 1 MB.

Post by Tyshalle »

No, 60 MB image asset. As in, a single object dropped onto the background layer that is 60 MB. Once you factor in all the other objects it comes to around 70 MB. I've been using MapTool since like b45 as a DM, so I know a lot of tricks to working around some of its issues, and so I mostly keep my individual maps under 15 MB, with all the assets under 0.5 MB if I can help it. Sometimes if I find a cool map on RPGMapShare or Dundjinni or elsewhere I'll lay it down on the background, but some of these maps are like, 10, 20, 50, 100 MB images, and I almost always have to shrink them down and heavily compress them, which loses a lot of the quality, which the perfectionist in me hates doing, but I've always done it.

I can continue to do it if it's necessary, but I started this thread because I'm not actually sure if everyone has this issue, or if there's something I could be doing differently so that I could drop a 60 MB .jpeg onto a map and have it be used without causing issues. Doubling the Max memory to 2048 seems like a good place to start, so we'll see.

User avatar
aliasmask
RPTools Team
Posts: 9024
Joined: Tue Nov 10, 2009 6:11 pm
Location: Bay Area

Re: Difficulties running maps with objects larger than 1 MB.

Post by aliasmask »

Using the 2048 max mem, I could load a 17 MB image (5100 x 3300 which seem like really bad compression because I have another one at 8 MB but at 4000x6000 and looks great), but not a 28 MB image. I don't have anything in jpg format in between that.

User avatar
Full Bleed
Demigod
Posts: 4736
Joined: Sun Feb 25, 2007 11:53 am
Location: FL

Re: Difficulties running maps with objects larger than 1 MB.

Post by Full Bleed »

Tyshalle wrote:No, 60 MB image asset.
What's the pixel dimensions and file format of this asset?

Again, keep in mind that once MT loads the asset into memory it's not (primarily) the physical size of the image file that matters, it's the dimensions of the asset. Compression will reduce the color range a bit and, consequently, reduce the memory footprint a bit, but it should not be significant for most images. That is to say that you could take a 60 MB file, and compress it to 6 MB, open both in MT and they will probably be within 10% of total memory usage despite one file being 10 times "larger" with regard to the space it takes up on disk (so long as the pixel dimensions are the same).
aliasmask wrote:Using the 2048 max mem, I could load a 17 MB image (5100 x 3300 which seem like really bad compression because I have another one at 8 MB but at 4000x6000 and looks great), but not a 28 MB image. I don't have anything in jpg format in between that.
It may be bad compression, or it may just be a much more dynamic image where compression is less effective/efficient.
Maptool is the Millennium Falcon of VTT's -- "She may not look like much, but she's got it where it counts."

Tyshalle
Giant
Posts: 173
Joined: Sat Jul 10, 2010 5:45 pm

Re: Difficulties running maps with objects larger than 1 MB.

Post by Tyshalle »

One of my images that wouldn't work without shrinking and compressing it significantly was 6000x6000. Another one is 77.8 MB and 7000 x 9800. One I really want to use is 11,000 x 8000, but is only 40 MB. And you do seem to be right, that it's more about the dimensions than the actual file size.

On one of my maps, I took a simple transparent shadow object, stretched it to be about 30 times bigger than it was and copied/pasted itself on top of itself several times over to give the impression of a huge staircase, and even though the original object was only like, 100 KB in size, I get some of my worst slowdown when I'm looking at that.

User avatar
Full Bleed
Demigod
Posts: 4736
Joined: Sun Feb 25, 2007 11:53 am
Location: FL

Re: Difficulties running maps with objects larger than 1 MB.

Post by Full Bleed »

Tyshalle wrote:On one of my maps, I took a simple transparent shadow object, stretched it to be about 30 times bigger than it was and copied/pasted itself on top of itself several times over to give the impression of a huge staircase, and even though the original object was only like, 100 KB in size, I get some of my worst slowdown when I'm looking at that.
That might actually be an issue with the way java renders layered transparency and might not be memory related at all. I bet if you did the layering for the image in an outside image program to get the same visual effect (i.e. Photoshop or Gimp) and then imported a flat image you wouldn't have the same performance issue.
Maptool is the Millennium Falcon of VTT's -- "She may not look like much, but she's got it where it counts."

Post Reply

Return to “MapTool”