Memory size in Launcher to larger than available physical memory?

Developer discussion regarding MapTool 1.4

Moderators: dorpond, trevor, Azhrei

Forum rules
Posting now open to all registered forum users.
Post Reply
adventuremagic123
Cave Troll
Posts: 39
Joined: Sun Jan 06, 2019 12:29 pm

Memory size in Launcher to larger than available physical memory?

Post by adventuremagic123 »

When we set the memory size using the Launcher, is it okay to use a value larger than available physical memory (and depend on virtual memory)?

So, for example, I have players with only 4GB RAM machines. I'd like for us to allow MapTool 1.4.1.8 to use up to 4GB or maybe 5GB.

A standard rule for virtual memory was that you could have up to twice as much virtual memory as physical.

Do JVMs have to be locked into physical memory?

taustinoc
Dragon
Posts: 516
Joined: Mon Aug 03, 2015 6:30 pm

Re: Memory size in Launcher to larger than available physical memory?

Post by taustinoc »

I believe the answer is "no, you can't do that." Java requires contiguous memory, and gets darned cranky about it. And if you set it wrong in the launcher, MapTool won't launch at all, and with the command box suppressed by the launcher, you won't have any idea why.

Phergus
Deity
Posts: 7132
Joined: Fri May 12, 2006 8:56 pm
Location: Middle of Nowhere, NM
Contact:

Re: Memory size in Launcher to larger than available physical memory?

Post by Phergus »

What Taustinoc said.

But what are you doing that you need more than 4GB of RAM for MapTool? You can load a 16k x 16k map image and still have more than half available. What problem are you trying to solve with more memory?

adventuremagic123
Cave Troll
Posts: 39
Joined: Sun Jan 06, 2019 12:29 pm

Re: Memory size in Launcher to larger than available physical memory?

Post by adventuremagic123 »

We tried my largest map, the lighting and VBL did not appear to be working, and as I explored that eventually a null pointer exception was thrown on the players' clients. I'm not sure where to look to determine the size of the map. I'm guessing it would be where we edit the map.

We currently have our memory limit set to 3GB.

Seems clear the players' clients ran out of memory. I can't remember if I saw it on my server. I also use 3GB there.

After I switched to a smaller map, we had no more trouble.

BTW, as a test, I set the launcher to 5GB on a 4GB machine, and MapTool came up -- I guess using virtual memory.
Attachments
From a player's machine.
From a player's machine.
unknown.png (54.58 KiB) Viewed 8300 times
Last edited by adventuremagic123 on Sat Aug 10, 2019 11:55 pm, edited 1 time in total.

adventuremagic123
Cave Troll
Posts: 39
Joined: Sun Jan 06, 2019 12:29 pm

Re: Memory size in Launcher to larger than available physical memory?

Post by adventuremagic123 »

taustinoc wrote:
Sat Aug 10, 2019 1:11 am
I believe the answer is "no, you can't do that." Java requires contiguous memory, and gets darned cranky about it. And if you set it wrong in the launcher, MapTool won't launch at all, and with the command box suppressed by the launcher, you won't have any idea why.
Does "contiguous memory" rule out virtual memory?

taustinoc
Dragon
Posts: 516
Joined: Mon Aug 03, 2015 6:30 pm

Re: Memory size in Launcher to larger than available physical memory?

Post by taustinoc »

adventuremagic123 wrote:
Sat Aug 10, 2019 11:34 pm
Does "contiguous memory" rule out virtual memory?
Apparently not. Especially if you have told the launcher to use more memory than the physical RAM on your computer without a problem. And I just confirmed that works with 1.4.0.5, with 24 GB listed in the launcher (and confirmed as available in Gather Debut Information) on a machine with only 8 GB of physical RAM.

Contiguous means "all in one block of memory," rather than "a bit here and a bit there, as available." Most programs don't really care, but design decisions were made for Java the prioritize speed of execution over pretty much everything else (and rightly so), so it requires contiguous memory.

However, after doing a little research, apparently one the points of virtual memory is to provide larger contiguous blocks of memory than is available in physical memory. And if the OS handles it right (and they all do), the program (like Java) can't tell the difference.

However, there are a couple of caveats:

1. You (and all of your players) will have to have 64 bit Java (and a 64 bit operating system). There are hard limits to addressable memory for 32 bit programs, and it's never more than about 3.5 GB (and usually less).

2. When you exceed the available physical contiguous memory (and the limit will likely be smaller when you're using virtual memory), the operating system will start paging. That means it is saving the current block (or "page") of memory to disk, and reading a different "page" from disk. This is much slower than reading physical RAM. You may notice the difference. You may (or may not) notice is a lot.

3. If one of your players has 32 bit Java, and they put in too large a number for max memory, when they hit Launch, the launcher will disappear and nothing else will happen (because the launcher suppresses the black command box that tells you what the error is). The easy fix is to back down that number until it launches. The real fix is to install 64 bit Java - but only if they have a 64 bit operating system.

Since you're still using the launcher, it's pretty painless to experiment, even for your players. As long as you know what to tell them if it breaks.

User avatar
Full Bleed
Demigod
Posts: 4736
Joined: Sun Feb 25, 2007 11:53 am
Location: FL

Re: Memory size in Launcher to larger than available physical memory?

Post by Full Bleed »

adventuremagic123 wrote:
Sat Aug 10, 2019 11:25 pm
Seems clear the players' clients ran out of memory. I can't remember if I saw it on my server. I also use 3GB there.

After I switched to a smaller map, we had no more trouble.
They didn't run out of memory. They ran into a bug.

The issue you're experiencing is this one: viewtopic.php?f=86&t=27954

I've opened the test campaign in that thread in 1.5.3 and it doesn't have the issue... so I'm guessing it has been fixed.


I've used some pretty massive maps with hundreds of tokens on them and none of my players have set their max mem above 2G. You can see how much memory you're using in the bottom right hand corner... and your players are, likely, using less.
Maptool is the Millennium Falcon of VTT's -- "She may not look like much, but she's got it where it counts."

Phergus
Deity
Posts: 7132
Joined: Fri May 12, 2006 8:56 pm
Location: Middle of Nowhere, NM
Contact:

Re: Memory size in Launcher to larger than available physical memory?

Post by Phergus »

adventuremagic123 wrote:
Sat Aug 10, 2019 11:25 pm
I'm not sure where to look to determine the size of the map. I'm guessing it would be where we edit the map.
If you have drawn the map using the drawing tools then the memory it takes up is dependent upon the dimensions of the textures (if any) you used (not file size) and what assets you used for set dressing like chests or trees or rocks. Drawn maps typically require less resources.

For actual map images that you load from disk, and were created in a mapping program or paint program, you can find the image dimensions by just hovering your mouse over them in the Windows file explorer for a second or so. You'll get a pop-up that tells you the dimensions. Multiplying (width x height x 4) tells you how many bytes of memory are needed to hold it in memory. A similar pop-up in the MapTool Resource Library will also give you Dimensions of an image. Note the size shown is the file size and not memory size.
Seems clear the players' clients ran out of memory.
Nothing you have provided so far suggests that. Null pointer exceptions are rarely caused by low memory conditions. Instead you'll get:

Code: Select all

java.lang.OutOfMemoryError: Java heap space
    at sun.awt.image.ImageRepresentation.setPixels(ImageRepresentation.java:562)
    ...
As your screenshot cut off a lot of the relevant information there isn't a lot to be gleaned there.
After I switched to a smaller map, we had no more trouble.
Correlation does not imply causation. MapTool provides information in the status bar at the bottom about how much memory it is using and how much is available. Without information about the map type (drawn vs. image) and dimensions there is not much to go on.

adventuremagic123
Cave Troll
Posts: 39
Joined: Sun Jan 06, 2019 12:29 pm

Re: Memory size in Launcher to larger than available physical memory?

Post by adventuremagic123 »

I got the stack trace from a player. Since we were playing a game, I was focused on recovery, rather than collecting information.

The map was a JPG I clipped out of the Iron Gods AP Part 1 (book 1) -- Fires of Creation. It was the image of the map of Torch. I used an HP 34" monitor using 3K resolution (not 4K) to get as high a resolution as possible. I added VBL to all the buildings and the Black Hill as X's. I had FoW and set it to "Dark". When were playing, I had only about 4 tokens on the map when the exception occurred. None of the players could see the map, even though they had Shared FoW. I was trying to research that problem. The token was owned by a player and it had Darkvision. The other tokens I had just put on the map to provide light sources.

When I get a chance, I'll try to reproduce this to get the complete stack trace -- but that won't be for a while. I've have a lot of commitments right now.

I don't remember if I also got this on the server.

adventuremagic123
Cave Troll
Posts: 39
Joined: Sun Jan 06, 2019 12:29 pm

Re: Memory size in Launcher to larger than available physical memory?

Post by adventuremagic123 »

Here's an update:

I was able to work around this problem by deleting the VBL on this map and turning off FOW. The map I was using was a map of the town and had VBL for each building.

After I did that, we were able to play on the map.

Post Reply

Return to “MapTool 1.4”