When switching Vision to Night, MT slows to a crawl.

Developer discussion regarding MapTool 1.4

Moderators: dorpond, trevor, Azhrei

Forum rules
Posting now open to all registered forum users.
User avatar
Full Bleed
Demigod
Posts: 4736
Joined: Sun Feb 25, 2007 11:53 am
Location: FL

Re: When switching Vision to Night, MT slows to a crawl.

Post by Full Bleed »

jfrazierjr wrote:At the time I first was playing with IFoW code, the dev team went into several days worth of discussion about the topic.
It wouldn't surprise me if there were some other legitimate or technical reasons that the community just wasn't privy to... but, on the surface, it's not optimal to go through the song and dance with new users that run into problems with it. It's a feature that requires a full grasp and understanding of vbl, light, vision, and FoW... and all the options available in the server settings. It's a lot to ask of new users trying to get the "full MT experience" before they understand what all the pitfalls/caveats are or each feature.
Maptool is the Millennium Falcon of VTT's -- "She may not look like much, but she's got it where it counts."

User avatar
jfrazierjr
Deity
Posts: 5176
Joined: Tue Sep 11, 2007 7:31 pm

Re: When switching Vision to Night, MT slows to a crawl.

Post by jfrazierjr »

Full Bleed wrote:
jfrazierjr wrote:At the time I first was playing with IFoW code, the dev team went into several days worth of discussion about the topic.
It wouldn't surprise me if there were some other legitimate or technical reasons that the community just wasn't privy to... but, on the surface, it's not optimal to go through the song and dance with new users that run into problems with it. It's a feature that requires a full grasp and understanding of vbl, light, vision, and FoW... and all the options available in the server settings. It's a lot to ask of new users trying to get the "full MT experience" before they understand what all the pitfalls/caveats are or each feature.
As I said, if the dev team agrees, I don't see a huge downside to it outside of retraining people who are used to that functionality. On the flip side, I know there has always been a want to re-write that code anyway from the ground up, but no one with deep design understanding of how geometry works in games has stepped up because it is an immense amount of time/effort to do so.

As for the "beginning" the dev team also did not really have a good idea how just how CRAZY(stupid?) people might use FoW(ie, hundreds of tokens, hundreds of maps, stupidly large outdoor scales, etc) and how IFoW would have an impact on performance with very large groups of exceptional map implementations. I know for me, the idea is to always have a new campaign file for each few areas/sessions (for the most part) AND I am local only, so the thought of people using MapTool as a complete campaign repository is alien to me. It was never designed to be that and users should not try to use Maptool to keep track of hundreds of maps in a single campaign file even WITHOUT FoW being utilized.
I save all my Campaign Files to DropBox. Not only can I access a campaign file from pretty much any OS that will run Maptool(Win,OSX, linux), but each file is versioned, so if something goes crazy wild, I can always roll back to a previous version of the same file.

Get your Dropbox 2GB via my referral link, and as a bonus, I get an extra 250 MB of space. Even if you don't don't use my link, I still enthusiastically recommend Dropbox..

Dracwrym
Cave Troll
Posts: 28
Joined: Mon Apr 10, 2017 5:50 am

Re: When switching Vision to Night, MT slows to a crawl.

Post by Dracwrym »

I have a couple of questions about the code of the FoW mainly because of my lack of understanding of Java. I can guess that there is a lot of math involved as there may be overlapping light sources (brightness), all the layers must be taken into account on what is visible, the VBL blocking light sources, and so on. I mainly code C++, so I looked at Java benchmarks of math. It seemed that older versions of Java were really slow at math, I mean very slow.

1) Is the code written for older versions of Java features, like 1.5 or 1.6?
2) Would updating the code using newest features of v1.8 help (it'd kill previous version support, though)?
3)Also, I found there are third party Java libraries, one called Jafama which does fastMath, not 100% accurate math, but that's not needed anyways, and one called JBLAS: Would using these help speed up the calculations?
( Jafama benchmarks are here: http://blog.element84.com/improving-jav ... afama.html and it seems all you need to do is change Math.function to fastMath.function so conversion shouldn't be too bad... I might even be able to do that and compile it to run bench marks. )

I'd help look at the code, but my understanding of Java is very basic, as in I started a course in it, so I can do 'Hello World'. From what I'm learning, Java code can be a bit slower than native binaries sometimes, but it shouldn't be as slow as what I, and others, experience, even if there are tons of sight tokens, light sources, and VBL. I have a really powerful Core i7 system, as well.

Cheers

Dracwrym
Cave Troll
Posts: 28
Joined: Mon Apr 10, 2017 5:50 am

Re: When switching Vision to Night, MT slows to a crawl.

Post by Dracwrym »

I found someone on GitHub that updated the Jafama code, which was last updated in 2015. I downloaded the source files and added them to RPLib from Jamz's repo (https://github.com/JamzTheMan/maptool/tree/1.4.3.21) since I use the Nerps version of MT. I carefully vetted all finds of Math.function and changed them to FastMath.function. I was able to make a fat jar and copy that over to the existing jar. I couldn't build the whole thing as I get this error "** Reason: javax.imageio.IIOException: Can't read input file!", so it can't build the all in one wrapper thing, but at least I got the MapTools Fat Jar so I could run it.

I was able to add a lot of light sources and have 7 PCs with vision, some dark vision and some normal. It slowed down some, but it wasn't as bad as before. I found a different FastMath library that might be better, so I am going to try that out as well. Java has precise math, so it takes a long time to calculate, but the math for this doesn't need to be that accurate, because there will be no visual difference. This may be something to help improve the speed. Maybe the rewrite that someone mentioned earlier will help along with the FastMath.

I will make FastMath changes against the official master tomorrow instead of Jamz's and post the diff here for others to try. I don't know enough about Java to figure out all that is going on with Jamz's and the error.

User avatar
JamzTheMan
Great Wyrm
Posts: 1872
Joined: Mon May 10, 2010 12:59 pm
Location: Chicagoland
Contact:

Re: When switching Vision to Night, MT slows to a crawl.

Post by JamzTheMan »

@Dracwrym, All the FoW, VBL, Lighting isn't really driving by Math functions. It's all driven using Java's Area class: https://docs.oracle.com/javase/7/docs/a ... /Area.html

The Area's are then "intersected" and combined (using Area's build in methods) to get your various views. For IFoW, each token has it's own Area(s) stored for it (basically, there are some classes that extend area and such). So you would need to find a library that improved upon Java's native geometry Area class. (IIRC I did a quick look a while back but didn't find one)

Given that, the thought really is to rewrite using Ray Casting which should be faster than the Area class calculations (and give other benifits). For instance, implementing Box2DLights not only gives you faster results, you also get soft lighting and OpenGL support (so yes, video card caclulations vs straight CPU calculations). The problem is, it's a big undertaking to translate everything over to libdgx bodies, lights, world, etc) Particularly translating VBL "Area" to non-concave polygons via earclipping (I got it mostly working but some translation issus still pop up).

If you want to see this in effect, go to Help -> Box2D menu on my build to see a demo.
-Jamz
____________________
Custom MapTool 1.4.x.x Fork: maptool.nerps.net
Custom TokenTool 2.0 Fork: tokentool.nerps.net
More information here: MapTool Nerps! Fork

Dracwrym
Cave Troll
Posts: 28
Joined: Mon Apr 10, 2017 5:50 am

Re: When switching Vision to Night, MT slows to a crawl.

Post by Dracwrym »

@JamzTheMan: Thanks for pointing me in the right direction. I looked through the source code for Area and all the things it depends one, and I was right before, there is a ton of math in those functions. Also, as mentioned before, the Math lib in Java does precise math, which is very slow for sin, cos, tan, and all the other functions like that. That level of accuracy is not needed for something like MT, and because of that accuracy, Area calculations are painfully slow. However, after searching for newer, updated FastMath libs, I came across the one from Apache (http://commons.apache.org/proper/common ... d_math.cgi). Just using that and going through the MT only code to replace Math.function with FastMath.function (and adding in the proper import) actually does speed things up a bit as there are some calculations done in MT before handing it off to Area and like classes. I tried my hand at converting Area and dependencies to FastMath, but that was a failure as I kept running into import issues and private functions in classes only accessible to module classes, and then other classes not being compatible for some odd reason. I don't know enough about Java to over come this. This is also one of the reasons to I dislike the Java VM as it's difficult to change the default classes.

1) Would it be beneficial for you to have the internal MT code I modified to use FastMath to help with some speed ups and also if you need math in your new code?

2) Would you like me to investigate making a new area and co classes using FastMath to make things faster until your rewrite is finished? I know rewrites, so I feel your pain on that. I'd like to help, but not sure how.

3) IntelliJ gives me lots of warnings about functions not in use anywhere in the code and functions that can be changed to be simpler. To make your rewrite easier, would you like help in cleaning up these warnings and modernizing the code a bit? These warnings are based on Java 8 specs though.

(Also, you have Nerps_Logo.png in the .gitignore, so that's not on GitHub, and thus caused the compile fail. I created a fake logo to make it work, and disable build for the two OSes I don't use with JWrapper.)

Cheers.

User avatar
JamzTheMan
Great Wyrm
Posts: 1872
Joined: Mon May 10, 2010 12:59 pm
Location: Chicagoland
Contact:

Re: When switching Vision to Night, MT slows to a crawl.

Post by JamzTheMan »

That's interesting news!

A test class or at least some log timer may be useful in comparing speed improvements? One "adhoc" test I will do is zoom out to ~10% and then on a brand new launch/map, drag the hero token with dark vision, "expose only on waypoints off" and zig zag move the token back and forth (with waypoint) a set distance (like 10,000 feet). This causes MT to caclulate vision for every cell along that path and can take several seconds and is usually somewhat consistent.

1. Sure, if we can improve the code by x% that is ALWAYS welcomed!

2. Sure, again, if you can tackle that and improve upon it, go for it! A rewrite is probably still a long way off. Note, look the AreaTree class. You don't have to change every Area class reference to gain improvements right off. Also, keep in mind, I need to change this anyway to convex polygons eventually to work with LIBDGX. So really, it doesn't HAVE to be Area's. It can be a collection of convex polygons. I have a earclipping routine in my gdx code. If fact, IF everything was converted from Area to convex Polygons now, it would save me time going forward and be compatible! (if you are interested in this front, let me know. The libgdx code is a little more game orientated and interesting to work with)

3. Yea, I get a lot of the warning in Eclipse as well (until you turn some of them off lol). There is a lot of legacy code here but luckily NOW we don't HAVE to support it, so Java 8 spec is fine. HOWEVER, I would refrain from cleaning that up at this time. It should be a separate effort (or done as part as moving to JavaFX). A lot of the Swing code will go away anyway and if you perform evergreening on the code now, we'll just end up with a lot of class/code changes which will make debugging the math changes more of an issue.

Regarding Nerps_Logo, ya, I try to keep what is unique to my build separate from the master so it's easier on the pull requests. Did you Fork from my build? What branch? (One option is to Fork from 1.4.1-Dev from Rptools instead. Your work can be submitted to the master branch for consideration then as well (and I can merge it upstream). However, if you want to work on the libgdx and my fork, that is cool with me as well)

Do note, please run the Spotless gradle tasks before submitting any PR's. Since you are using IntelliJ, I'm sure your "Formatting" is not going to be exactly the same as what we have defined for Eclipse and comparing merges suck when every line is flagged because of tabs vs spaces and such. :)
-Jamz
____________________
Custom MapTool 1.4.x.x Fork: maptool.nerps.net
Custom TokenTool 2.0 Fork: tokentool.nerps.net
More information here: MapTool Nerps! Fork

Dracwrym
Cave Troll
Posts: 28
Joined: Mon Apr 10, 2017 5:50 am

Re: When switching Vision to Night, MT slows to a crawl.

Post by Dracwrym »

@Jamz: This is the diff between your 1.4.3.20 branch, not .21 as you haven't released that yet, so you can easily see the differences so far. I think I found all the Math functions to convert to FastMath. I didn't do .js files as I wasn't sure how to handle those. Somethings are faster, but not all as the Area functions use a ton of math. I'm looking at changing those. However, this will help see if I did things right or should change how I do things before going all out. There are still things to be optimized I noticed.

https://github.com/dracwyrm/maptool/com ... b4a2e9957f

Cheers

User avatar
JamzTheMan
Great Wyrm
Posts: 1872
Joined: Mon May 10, 2010 12:59 pm
Location: Chicagoland
Contact:

Re: When switching Vision to Night, MT slows to a crawl.

Post by JamzTheMan »

So far so good. A pretty simple change on the face of things and I don't anticipate any issues.

Do you have any numbers on performance increases?

Also, I would create a new branch just for this (based on 1.4.3.20 is just fine). I'm currently working on 1.4.3.21 but should be just a few minor changes and should be able to merge things just fine. When you are done with the first pass, submit a PR and I'll see if I can get things merged.
-Jamz
____________________
Custom MapTool 1.4.x.x Fork: maptool.nerps.net
Custom TokenTool 2.0 Fork: tokentool.nerps.net
More information here: MapTool Nerps! Fork

User avatar
JamzTheMan
Great Wyrm
Posts: 1872
Joined: Mon May 10, 2010 12:59 pm
Location: Chicagoland
Contact:

Re: When switching Vision to Night, MT slows to a crawl.

Post by JamzTheMan »

Just FYI: https://github.com/google/guava/wiki/MathExplained

(I was looking at Guava for some other functions and saw this...)
-Jamz
____________________
Custom MapTool 1.4.x.x Fork: maptool.nerps.net
Custom TokenTool 2.0 Fork: tokentool.nerps.net
More information here: MapTool Nerps! Fork

Dracwrym
Cave Troll
Posts: 28
Joined: Mon Apr 10, 2017 5:50 am

Re: When switching Vision to Night, MT slows to a crawl.

Post by Dracwrym »

JamzTheMan wrote:Just FYI: https://github.com/google/guava/wiki/MathExplained

(I was looking at Guava for some other functions and saw this...)
Thanks for the link. I'll switch out Apache Commons for this and Benchmark it, but I will do that when I get back from my week trip. I leave tomorrow. :)

Post Reply

Return to “MapTool 1.4”