New Dev Questions, and ideas on source / build organization

[whited]

Hi guys,

Firstly I'd like to say I've just recently found MapTool, and I'm already a fan - Great job!

I'd like to get involved with contributing, but I'm finding the instructions and details around getting started is rather spread out. I'm finding it, but some of it looks dated, and it's hard to tell if something is current.

I figured I'd take my perspective as a newcomer to the project and mention the stuff I'm looking for - some of this I've already found, but I've added them here for reference. I'll write a follow-up post later with stuff I've found so far.

*Note:* Edited inline with answers

1. Are there any eclipse plugins I should have in advance of getting started?
   • Subclipse to get source using Eclipse
   • ANTLR IDE if you want to mess with the macro parser
2. What are all the projects in source that I should be paying attention to? (For MT in particular) the project appears to be split into a few projects now, including a library and the MT executable - I haven't dug too deeply yet, there looks to be potentially more...
   • common.build (not required for eclipse builds)
   • maptool
   • maptool.resources
   • parser (if you want to mess with the macro parser)
   • rplib
   • rplib.resources
3. How do I get these projects into my eclipse environment?
   • See this post from mid-2006
4. Not that I'd be doing this, but what is the build process? How is a new build generated and where is it stored for consumption? (It appears to be built with a lot of manual steps and uploaded to the rptools.net website lib.php/<jar_name>
5. I found a post regarding code style and auto-formatting here. Is there anything similar for testing? Unit tests, integration tests, regression tests? (I'm guessing there isn't much)
   • Nope, not much testing
6. I've seen a couple of different places where devs should post to indicate what they're working on - which one is the current one? Sourceforge has an issue list, and there is a wiki page for release notes. All great stuff, but there's a lot of places to look for this.
   • No specific place at the moment

I realize I'm new here, so I don't know what the prevailing opinions are, but here's some ideas that might help:

1. Apache Maven. It's not perfect, and it has a learning curve, but if a project is using it, there is a predictable place for source code, resources, sub-projects, project settings, and so forth. Also, if you pull the whole source tree and run "mvn package" to build it all. To deploy, (assuming you had the right permissions), it should be a matter of running "mvn deploy" to build, package, verify, test, and upload.
2. Jar Dependency Management with Apache Maven or Apache Ivy. You shouldn't have to force your contributors to download jars from your source code repository. From what I've seen the Maven Repositories are rather comprehensive when it comes to open source jars.
3. Source Repository: Have you guys considered Git? It makes it much easier branch/merge operations for contributors, and patches should be easier to manage. SourceForge appears to support git, as does github.com, and assembla.com. Granted, Git is not as well integrated into Eclipse, but there are great advantages to be had.
4. Bug/Feature lists should be tracked in a tool. There appears to be a Tracker on SourceForge, but it's pretty clunky compared to, for example Assembla Tickets or something similar. Also, with that kind of setup, there are Eclipse Integrations for MyLyn (see next point )
5. Eclipse MyLyn! This is included in basic Eclipse installs these days - in a nutshell, you can integrate your bug/feature list in eclipse, edit the tickets directly in eclipse, and work with "contexts" which automatically narrows your workspace to what you've recently been looking at - it's hard to describe properly and do it justice. You can do things like fix a bug, package up the context and attach it to the ticket, so that if someone else needs to revisit the bug, they can import the context from the bug to see just the relevant parts of the project for that bug.

I'm getting more long-winded as the evening wears on (sorry!) - I'll stop here and wait for some reactions

[Phergus]

Items b & c are covered in the sticky thread in this forum on how to get the source.

[Azhrei]

Hi guys,

Welcome to RPTools, whited.

I'd like to get involved with contributing, but I'm finding the instructions and details around getting started is rather spread out. I'm finding it, but some of it looks dated, and it's hard to tell if something is current.

Yeah, the euphemism around here is "organically grown".

Are there any eclipse plugins I should have in advance of getting started?

(It's not easy to quote list elements so I'm not going to try.)

There are only two and they're not required so much as convenient: ANTLR and Subversion. And ANTLR isn't needed unless you want to change the MTscript grammar and that isn't likely. (But if you have your parser project set to autobuild and you go in editing files, when Eclipse tries to build them and you don't have ANTLR installed the old build files are removed and not recreated. That will thwart any attempt to then build that project. I posted about this somewhere here...)

What are all the projects in source that I should be paying attention to? (For MT in particular) the project appears to be split into a few projects now, including a library and the MT executable - I haven't dug too deeply yet, there looks to be potentially more...

Yeah, it's a bit confusing right now. The contributors are considering options. The easiest thing to do is get the entire repo. Be aware that there are dependencies between projects.

How do I get these projects into my eclipse environment?

SVN (Subversion). Instructions are in a thread here somewhere, started by Trevor mostly likely.

Not that I'd be doing this, but what is the build process? How is a new build generated and where is it stored for consumption? (It appears to be built with a lot of manual steps and uploaded to the rptools.net website lib.php/<jar_name>

I posted a thread somewhere about things I learned while doing a build. Search my posts.

Some parts of the process (uploading to rptools.net) is manual. My short- and mid-term goal is to get everything automated. But given some of the other changes happening right now, this is a lower priority.

I found a post regarding code style and auto-formatting (I read somewhere you checked in an eclipse formatting preferences file) - is there anything similar for testing? Unit tests, integration tests, regression tests? (I'm guessing there isn't much)

Trevor created tests for some things, but I haven't been keeping up with that (nor have I been requiring contributors to keep up with it either). It's becoming a mess as the code base for MT has "grown organically" into spaghetti code of the worst kind.

Our next major release is likely to be OSGi-based and I'm hoping that the modularity it enforces will help with unit and regression testing as well. Some unit tests will need to be inside the bundles, but then I also expect to have regression and integration tests outside the bundles. Unfortunately, it can be difficult to apply many of these kinds of tests against GUI code.

I've seen a couple of different places where devs should post to indicate what they're working on - which one is the current one? Sourceforge has an issue list, and there is a wiki page for release notes. All great stuff, but there's a lot of places to look for this.

We're pretty much in a feature freeze right now. Of the contributors you read about here only Craig and I have SVN commit access. While people are welcome to post what they're working on (I don't know why there would be multiple threads; I only know of the one linked to in my .sig) as a way of coordinating with others, I also recommend contacting Craig or me to be sure that we'd commit your patches to SVN anyway. This saves you effort as well since your patches may be for a feature that we're not ready to integrate/maintain/document.

I realize I'm new here, so I don't know what the prevailing opinions are, but here's some ideas that might help:

Those are all good ideas. We have considered most if not all of them. Going forward we will probably be implementing some/most of them, although we may use a simpler approach than some of the projects you mention have taken.

As this is strictly a "free time hobby project" for all of us, we add these pieces in as time and need dictates. One of my primary goals is better organization and automation -- many of the ideas you listed fit right in. The influencing factors include (1) time available in Real Life, (2) motivation, and (3) cost/benefit ratio (bug fixes for major problems are better for users than a behind-the-scenes change that they won't notice).

[whited]

Thanks for the comprehensive answer, quite helpful! I've edited my initial post with some answers and links/posts that I've dug up.

I recall reading your post on build process at one point, but I can't find it now Searching your 500-page post history for "build", "dist", "distribution", etc. just makes phpBB laugh at you... (well - OK, it complains about keywords that are too common) It's not too big of a deal, it will show up eventually.

I'm thinking since I'm new to the code base, I'll follow this general path:
(1) Get MT running out of eclipse
(2) Do a bit of unit tests and code cleanup (remove unused private members and unused imports, correct or squelch warnings, that sort of thing) Are such changes likely to be committed?
(3) I'll look around for a bug or two to squash
(4) If there is interest, I can take a stab at converting to Maven. I could make a fork in github to work from, which could be kept or tossed depending on the results

[Azhrei]

I'm thinking since I'm new to the code base, I'll follow this general path:
(1) Get MT running out of eclipse
(2) Do a bit of unit tests and code cleanup (remove unused private members and unused imports, correct or squelch warnings, that sort of thing) Are such changes likely to be committed?

Depends. If the change affects the save file format for campaigns, then a corresponding Transform would need to be added (see ModelVersionManager).

(3) I'll look around for a bug or two to squash

Heh, I have plenty of those.

A few that I'd like to see: rotated images don't seem to resize properly, change macro tab to use editable dropdowns for colors and font size, add another tab to texture/background image selection that provides existing textures/images as options, and any performance enhancements regarding macros or the UI (selecting a dragging a token with a lot of macros is slow, for example).

If you pick one of those (or a different one) let us know as we might have some guidance on how we'd like to see them fixed.

(4) If there is interest, I can take a stab at converting to Maven. I could make a fork in github to work from, which could be kept or tossed depending on the results

You're welcome to do that of course. Learning as you go through the process can only be a Good Thing(tm). That's experience that perhaps the rest of the team can take advantage of later.

You can either use git-svn from the SF SVN repo, or there's a Git repo at SF now too. I can't guarantee that the Git repo will be kept up to date though as we're still experimenting with it.

[whited]

So I'm looking at the list of bugs you mentioned

(1) The Rotation bug - I've found that a stamp in the background layer when rotated totally dances a jig if you try to resize after it's rotated. It looks like the drag operation's dx/dy is not being translated properly for the object's current rotation. I've done some preliminary poking around, and I think the code I need to comb through is in net.rptools.maptool.client.tool.StampTool, and possibly net.rptools.maptool.client.ui.zone.ZoneRenderer (please correct me if I'm wrong) I'm going to examine this one a bit more (No promises on a fix though!)

(2) editable combos and extra image tabs I haven't looked into. I'm going to look at other stuff first

(3) Macro Performance, and Token/Macro/UI interaction Performance - haven't looked into this yet, but it's interesting stuff for me - I bought a book on ANTLR (v3 though) - so I might bite off a big chunk here at some point and see what it would take to convert to ANTLR v3. This might help with macro performance, but the greater portion is likely to be in the semantics and implementation of the macro's processing engine. Can you (or anyone) point me to some campaigns or token downloads that exhibit this problem?

[Craig]

So I'm looking at the list of bugs you mentioned

(1) The Rotation bug - I've found that a stamp in the background layer when rotated totally dances a jig if you try to resize after it's rotated. It looks like the drag operation's dx/dy is not being translated properly for the object's current rotation. I've done some preliminary poking around, and I think the code I need to comb through is in net.rptools.maptool.client.tool.StampTool, and possibly net.rptools.maptool.client.ui.zone.ZoneRenderer (please correct me if I'm wrong) I'm going to examine this one a bit more (No promises on a fix though!)

I took a quick look, it is StampTool that you want. If you look round about line 709 thats where the adjusting of the rotation anchor is done. I haven't looked at the code before but if I had to guess I would say that the problems occur because the X,Y location of the Token/Stamp is its top left corner not the centre which makes it easier to draw but causes rotation oddness as your centre of rotation will wobble. I am hoping to change the X,Y location of the Token/Stamp to be the centre of the Token in 1.4 as this will solve other issues as well.

(2) editable combos and extra image tabs I haven't looked into. I'm going to look at other stuff first

(3) Macro Performance, and Token/Macro/UI interaction Performance - haven't looked into this yet, but it's interesting stuff for me - I bought a book on ANTLR (v3 though) - so I might bite off a big chunk here at some point and see what it would take to convert to ANTLR v3. This might help with macro performance, but the greater portion is likely to be in the semantics and implementation of the macro's processing engine. Can you (or anyone) point me to some campaigns or token downloads that exhibit this problem?

The current performance issues are not with the ANTLR portion in the Parser project but with net.rptools.maptool.client.MapToolLineParser which is really a large mess. In 1.4 a lot of that mess is going to disappear as MapToolLineParser either won't exist at all of will be a shadow of its former self. Moving from ANTLR2 to ANTLR3 requires a rewrite of much of the grammar file as they are different between the two. There is a need to move to ANTLR3 for 1.4 so that we can properly support unicode characters in macros, I don't think anyone else has put their hand up for that yet.

The other big performance issue that is related to macros is the net.rptools.maptool.client.ui.macrobuttons.panels.SelectionPanel class which draws the macro buttons on the selection panel based on which Tokens you have selected, at the moment this deletes all the buttons and recreates them when token selection changes which can be a huge performance issue when you have either a lot of tokens or tokens with lots of macros.

[jfrazierjr]

So I'm looking at the list of bugs you mentioned

(1) The Rotation bug - I've found that a stamp in the background layer when rotated totally dances a jig if you try to resize after it's rotated. It looks like the drag operation's dx/dy is not being translated properly for the object's current rotation. I've done some preliminary poking around, and I think the code I need to comb through is in net.rptools.maptool.client.tool.StampTool, and possibly net.rptools.maptool.client.ui.zone.ZoneRenderer (please correct me if I'm wrong) I'm going to examine this one a bit more (No promises on a fix though!)

I took a quick look, it is StampTool that you want. If you look round about line 709 thats where the adjusting of the rotation anchor is done. I haven't looked at the code before but if I had to guess I would say that the problems occur because the X,Y location of the Token/Stamp is its top left corner not the centre which makes it easier to draw but causes rotation oddness as your centre of rotation will wobble. I am hoping to change the X,Y location of the Token/Stamp to be the centre of the Token in 1.4 as this will solve other issues as well.

In my testing with the onTokenMove macro stuff, I found out that Tokens at least depends on the grid type. Square using the top left, while gridless and Hex use the grid center. Just something to think about during testing in case it's the same for stamps.

[Azhrei]

In my testing with the onTokenMove macro stuff, I found out that Tokens at least depends on the grid type. Square using the top left, while gridless and Hex use the grid center. Just something to think about during testing in case it's the same for stamps.

That sounds like Square should use the center as well (for consistency if nothing else!) but I bet that change will touch A LOT of code!

[whited]

In my testing with the onTokenMove macro stuff, I found out that Tokens at least depends on the grid type. Square using the top left, while gridless and Hex use the grid center. Just something to think about during testing in case it's the same for stamps.

Actually, that helps explain a bit. Values added where you think they would be subtracted, that sort of thing. Definitely good to know for testing too.

From what I can tell, Stamps are essentially also Tokens. The rotation and sizing logic appears to be doing more work than you would think it should need to. I've already cut out a couple of variables that are set up but then not used anywhere.

The stamp's footprint bounds appear to be getting used improperly - I'm assuming the footprint bounds are the rectangle that the rotated image is contained in (?) I'll examine that next.

I spent a bunch of time last night studying and poking - I was able to stabilize the rotated resize a bit, but the stamp's [x,y] is still wandering around pretty significantly. I'm obviously still not taking something into account. I'll keep at it - I should have some more time this evening to take another crack at it.

That sounds like Square should use the center as well (for consistency if nothing else!) but I bet that change will touch A LOT of code!

It would touch a lot of places, but I bet it would reduce a lot of complexity too. ZoneRenderer.java has a lot of comments like this...

Code: Select all

// TODO: FIX THIS
// TODO: I really don't like this, it should be optimized
// TODO: I don't like this hardwiring
// TODO: Why not just call token.getLayer() ???
// TODO: Optimize this by doing it once per token per facing
// TODO: Man, this is horrible, there's gotta be a better way to do this

That's definitely a deep dark rabbit hole I'm hoping to avoid for now

[jfrazierjr]

In my testing with the onTokenMove macro stuff, I found out that Tokens at least depends on the grid type. Square using the top left, while gridless and Hex use the grid center. Just something to think about during testing in case it's the same for stamps.

Actually, that helps explain a bit. Values added where you think they would be subtracted, that sort of thing. Definitely good to know for testing too.

From what I can tell, Stamps are essentially also Tokens. The rotation and sizing logic appears to be doing more work than you would think it should need to. I've already cut out a couple of variables that are set up but then not used anywhere.

Yes, that is correct. Any image dragged into the map view port is a token.

The stamp's footprint bounds appear to be getting used improperly - I'm assuming the footprint bounds are the rectangle that the rotated image is contained in (?) I'll examine that next.

Note sure about the improperly or not, but the footprint is the smallest containing SQUARE that will hold the entire image. Something to keep in mind since all squares are rectangles, but not all rectangles are squares.

That's definitely a deep dark rabbit hole I'm hoping to avoid for now

Lol.. yeah me to for the most part. Probably a good idea... I expect Craig will hack this to large bits in 1.4 since he will be doing a lot of the changes to sight/light/vision blocking.

[Phergus]

Note sure about the improperly or not, but the footprint is the smallest containing SQUARE that will hold the entire image.

That's something that needs to change badly. The selection box should be the footprint and not just a square that would contain it. This is especially problematic for images that overlap their containing grid cell. The idea for footprints, IIRC, was that they would return the appropriate area (in whole cells) to be highlighted for the various sizes and shapes on both square and hex grids. Gridless tokens are a horse of a different color.

[Azhrei]

Yeah, but from a cursory look that I gave the code awhile ago, that would require some significant tweaking (although I agree that it should be done).

IMO the shape drawn on the Config tab of the Token Editor dialog is the size and shape that the "bounding box" should be, so if the user resizes the token image up or down or moves it off-center they get the correct result on the screen.

The problem is that the bounding box rectangle is used to locate tokens for stacking and movement purposes. Or at least, that's what I seem to recall. It was back in the early days that I first looked at this so things could've changed by now (although they haven't on my watch).

Hm. Thinking about this for 1.4 the Token should probably have two methods, one that returns the outer bounding box (for clipping and collision detection) and a second one that returns the bounding box of the cell(s) that the Token occupies (including the funky shapes that might be occupied when a Token cover multiple hexes; boy, that's going to be some ugly code for calculating the Area object, although the Grid could perhaps either pre-calculate them or at least cache them once queried).

[jfrazierjr]

Hm. Thinking about this for 1.4 the Token should probably have two methods, one that returns the outer bounding box (for clipping and collision detection) and a second one that returns the bounding box of the cell(s) that the Token occupies (including the funky shapes that might be occupied when a Token cover multiple hexes; boy, that's going to be some ugly code for calculating the Area object, although the Grid could perhaps either pre-calculate them or at least cache them once queried).

yea, I think Trevor went for the easier Shape.getBounds() instead of writing a bunch of custom code. And I agree it would be nice to change, but will take a big effort.

[Phergus]

Thinking about this for 1.4 the Token should probably have two methods, one that returns the outer bounding box (for clipping and collision detection) and a second one that returns the bounding box of the cell(s) that the Token occupies (including the funky shapes that might be occupied when a Token cover multiple hexes; boy, that's going to be some ugly code for calculating the Area object, although the Grid could perhaps either pre-calculate them or at least cache them once queried).

I would expect the TokenFootprint class to handle both methods. The current footprints highlight all occupied cells so you don't need to try and figure out the Area object that would enclose the cells.