RPTools.net

I think that's fair. An API generally comes with documentation defining specific terms and how they're used. At the earliest stages though, can we agree that programming terminology should take a back seat to functional terms. I know what a Map interface looks like, and so would any Java developer wanting to use your API. I don't think it would be confusing though, to say "Image image = MapServer.getCurrentMapImage()" is fairly explicit as to what it we're really requesting (in context).



From the perspective of a Map-Client, I would say that "getting a map" is getting enough data for me to render the image of the map (including FOW, VBL and other 'map specific' data). Depending on the nature of the Client, I might be authorized to get these elements "as is"- that is to say, the list of layers, and the data they contain, or I might only get the current image bytes in it's final rendered form (i.e. with LOS already calculated for me). (These would probably be two different methods in the API, one which requires authentication credentials)

In either case, the decision to support references vs deep copies is probably dependent on the kinds of plugins you're wanting to support. I expect that people wanting to extend MapTools architecture at a deep level (needing references and direct object manipulation) are probably going to prefer contributing to the main code-base, rather than coding a client or plugin.



Well that's a knotty problem. I imagine spending some time thinking about limited feature exposure. A small interface that exposes these limited features can insulate your code from external dependencies. Third Party Code should not have to depend on your implementation directly.


Well, graphics subsystems might well be something that a client could end up being responsible for.
In a different architecture, MapTools could be coded as a thick-client against it's own API


Exactly.

And that's fine if the return value is a rendered image. What if the return is the data structure instead? See the next point.


Okay, so a method that retrieves the Zone.

Here's the fun part: does it provide a reference to the actual Zone? If so, internal changes in the Zone could cause the extension to fail later on without causing any change in the interface. An alternative is to return an opaque type (a.k.a. "a black box"). Ask for the map and you get back an object that implements the MapInterface. The details of the object itself don't really matter. This is probably the way to go. It might mean creating some new classes in some cases because we don't want to expose the internal structure of the code. In fact, probably creating a lot of new classes now that I think about it.


So everything returned should be a deep copy? Because people wanting anything else are working inside the code base and not as an extension? That's fine by me but it may have performance implications down the road.


What I meant was that some of the details about how things work are buried in the code and might be difficult to ferret out. So any API we come up with is likely going to change over time. Not because we've added things, but because we don't understand things as they are now.

Well that was my point - we know by context what it is.
If you're getting the data structure MapData myMap = MapServer.getCurrentMapData() would be explicit. As would Zone myZone = MapServer.getCurrentZone(), if you've decided to keep that terminology in the API.

I can tell you that an Interface is always the way to build an API
It might not require a *lot* of new classes, if you limit the methods you expose. This comes down to what you want the API for. If all you want to do is let other clients communicate with a MapTools server, then it can be abstracted well enough. IF you want them to be able to control the servers every aspect, then it may well become very messy.

Actually, it might not be required to even send a deep copy. Say we create a message-based protocol that lets the clients be very "dumb". Don't give a deep-copy of the whole object. Just give them an ID and some methods to call by making a request to the server. The server decides whether that client is authorized to perform that operation on that object, does the work, and sends the response. This response might be the new rendered Image of the map, or it might be the new stats for the token, or whatever.
The key is that the client doesn't need to do much of the work. For example : why make the client recalculate line of sight if the server already does that function?

I don't know. Perhaps that's not the direction you want to go. Perhaps you want to be able to have a client that can do fancy 3D rendering and sits over the top of the server, with fingers deep into the object model.. Such a client would need to know the state of the entire Zone as quickly as possible to be performant. It's possible though, that you can insulate yourself from changes to the Zone object by making it as generic an interface as possible .. e.g. IZone { layers[] } .. Layer { blocks (vision/movement), pixels, rendering_hints }.. RenderingHint { additive|multiplicative|opaque }.. or whatever. Then you can pass it by reference and avoid deep-copy performance issues.

I think you're talking two different (overlapping) APIs though : the "authenticating client" and the "trusted client". The first communicates via a request/response kind of protocol, and the second can operate like a server-extension : right on the data in the same JVM. Not sure if this is a meaningful distinction, but I do feel one emerging.



Well APIs do change over time - e.g. ArrayList vs Vector. The best you can hope for is that you expose as little as necessary to accomplish the minimal extensions you want to support, and enhance functionality over time rather than have to deprecate things when you find out they don't work right

Consider that, as open source, you already have an "API" : your entire codebase. I can come along and get full referential access to your entire object model and muck around with anything, potentially breaking it as I go.

All an official API does is make a commitment to a set of features that will remain constant from version to version. I'm sure that, regardless of implementation, there is _already_ a set of existing features that the developers expect to remain in future versions.

The current public API is the intersection of those features with the set of features a Client extension might need - we just need to identify and formalize it.

Ok, so I'm going to take a quick stab at a set of *requirements* for a Client API
Some Caveats :
1/ I'm not a "thick client" GUI developer by trade. I spend most of my time in Web Applications of one sort or another.
2/ I'm approaching this from the perspective of what someone wanting to use MapTools as a Server might desire. I'm not sure what other plug-in developers might want to do. Does it make sense for them to be able to extend MapTools capabilities in other directions? I don't yet have a clue what that might entail.

To Start :
Lets talk about functionality.
I envisage MapTools being able to provide services to clients. How this happens can be decided after we decide the kinds of services it could provide.

1/ Maps
(MapTools code calls them Zones, but I am calling them Maps because that's the domain specific term as understood by the users)

A) The Client should be able to get a map.
Depending on it's authentication level (GM or Player right now, but this should be extensible) it may be restricted to only an Image of the Map (with line of sight etc already calculated for the client before the request). Or it may be allowed full access to the data structure of the Map.
-How that structure is exposed is of some concern. That is something that can be left to a future extension of the API. Perhaps the initial client api only provides rendered images.

B) Get a specific View of the Map.
getMapViewAsToken(tokenId, [dimensions])
returns the rendered image of the map as seen by that token (calculated by the LOS of the token) If the optional dimensions parameter is sent, then center the image on the token and crop the image to those dimensions.

Questions: Should Dimensions be in grid-space or image-pixels? Should we have separate methods for both?


2/ Tokens
The Token API is probably quite tricky. How much do we need to expose to a client? Initially a client may only need the token id, it's location, and it's tokenImage. It needs to be able to reposition tokens, so perhaps consider a request/response paradigm : e.g. Client calls Server.requestSetTokenPosition(gridX, gridY) returns TokenResponse("Ok") or TokenResponse("failed:no movement remaining") or whatever.

I guess the key thing is that the Server is exposing a limited feature set for tokens. Just enough information to display and interact with them. If the Token has attached Macros (I have no clue how they're implemented in MapTools!), then it should publish the list to the Client. Then the client just shows them to the user and lets the user pick which ever one to activate. This is then sent back to the Server in a requestToActivateMacro(token.macros[index]) or something similar.
The Server takes care of activating the macro. See 3/ Events for more ideas.

getTokensOnMap()
getToken(id)

For a more advanced client :
addTokenToMap(Token, position) <- this is where it gets knotty since it the Client needs to know more about Token's internal structure. IF this is an exposed API call, then 3rd party developers would need to be made aware that future development might invalidate their Token creation code.


3/ Events :

More generally we can consider the Client to be a listener to Events and to generate some events of it's own.

So I imagine something like
registerAsListener(EventType, authenticator)
registerAsEventSource(EventType, authenticator)

The Server publishes a list of EventTypes ( token_drag_start, token_drag_stop, token_updated, chat, map_view_update, etc)
Clients can subscribe to events of whatever types they find. (Some of them may require authentication to send/receive, or otherwise be correctly targeted. e.g. A Whisper Chat event needs to know which player it's going to. A particular Campaign may restrict token movement. etc

Some token events:
setTokenPosition(token, x,y) returns success/fail
possibly
startTokenDrag(token, x,y)
updateTokenPosition(token, x,y)
endTokenDrag(token, x,y)

activateMacro(tokenId, macroId)


All these are just a bit of stream of consciousness to get things off the ground.

I have no predetermined focus for the discussion. I'm simply responding to the queries and/or ideas you've put out. But I agree that there are probably two different targets placed in our sights.

How about we pick one and we'll develop the API needed for that extension to be written? In order to avoid deep copies for now, we'll assume a built-in extension -- not something written as a service.

How about a game-specific character sheet? An extension that has all the functionality of what is currently provided by one of the FWs, but which is implemented inside MT to make it faster and provide some options that are not accessible to MTscript...?


Base assumption: All data retrieved from the server has a sequence number (aka generation number, aka version number) attached to it. When an extension wants to make a change, it sends the version number along with the new data. If the server sees an old version number (i.e. the object has been updated in the mean time), the request by the extension is rejected. Somehow the extension author will need to program for this. No idea on how that should be done, but it should be decided earlier instead of later as exceptions need to be considered while the interfaces are developed.

1/ Maps
1A/ Get a map.
For this target we'll retrieve the map object and its attendant data structure. We will assume that all of the information we need is available (which won't be much since we're just building a charsheet).

1B/ Get a specific view of the map.
You propose getMapViewAsToken() and I'm fine with that. Out to what distance should the vision be taken? Should the vision be with or without VBL? (Such as a Player view vs. a GM view.) What about fog -- layer it on top or leave it out? Should a grid be overlaid on top of the resulting image? Perhaps each of those options is passed as a parameter; an array of VisionOption objects or some such?

(Implementation note: if the return value is an instance of JLayeredPane, then each layer could contain the various elements. For example, the fog layer could be the 60% opaque image and the caller could decide what to do with it. For the VBL layer, a GM sees the equivalent of soft fog while a Player seeks a blacked out area. The caller chooses the layers and then has the image it wants.)

I think dimensions should be in "grid space". For gridless maps that means 1 grid cell == 1 pixel.

2/ Tokens
2A/ Information to expose.
All properties of the token are exposed: size, shape, top_down vs. portrait, images, properties, macros, speech text, and so on. Changing the location of a token means changing the Location property. However, retrieving a token only retrieves a token reference. Specific properties have to be individually requested. This limits the amount of network traffic in situations where the extension is remote, but also reduces the likelihood of conflicting changes -- something MT suffers from right now. (I change a token at the same time someone else changes the token. Whose changes stick? See base assumption, above.)

2B/ Adding new tokens.
Tokens are never added to the map by the extension. Instead, it asks the server to create an empty token. That token is returned to the extension to be filled in by them. A newly created token is "locked" and is not accessible elsewhere in the program until the extension defines a Location for the token.

2C/ Activating token macros.
Macros attached to tokens can be activated by an extension. The extension provides a "variable context," including the "impersonated token" and/or "selected token(s)," and passes this context as a parameter. The context includes predefined attributes for the default output destination ("GM only", "self only", "GM+self", and so on), default font characteristics, and ...?

3/ Events
3A/ Event categories.
Events might be physical actions such as simple real-time alarms, user interface actions, or network actions. They may also be abstract such as in-game alarms, tokens being moved into illegal locations, or a chat message either inbound or outbound.

The Java model of using Listeners is well-established and will be familiar to Java programmers. Based on the well-defined Observer pattern, they also scale reasonably well since the implementation can execute all listeners in sequence or in parallel. All listeners have a name and registering is done by specifying the name and an optional group of components that the listener applies to (pass null for all objects). Listeners can be ADDED, REMOVED, IGNORED, BLOCKED, and UNBLOCKED. Ignoring a listener means that events destined for that listener are dropped as they are received. (This is like removing the listener, but is likely to have performance advantages.) Blocking a listener causes the event to be queued for later delivery, and unblocking it allows them to actually be delivered.

Question: do extensions run on their own thread? If yes, should events being delivered run within the thread of the extension or on the Swing event queue? There are pros and cons both ways.

So.............................................................................................

What methods does our hypothetical charsheet extension need? Here's a rough draft of how I see the code being laid out. Comments welcome!



Depending on how the thread model is going to work, the callback object's method may be inside of some kind of thread synchronization block; a read-mostly lock seems most likely, but how to decide in an automated manner which objects are being locked? Is each separate event going to lock different kinds of objects? (Just making this up, but...) So a selectionChange event locks the newly selected token, unless it's on the Object layer when it locks all tokens within 5 grid cells? (I can't envision that being necessary, but I do know enough to know that I don't know enough to know what isn't possible. I think. )

Before I jump deeper into this - I think we should talk about authentication.
In much the same way as users can join a server with GM privileges if they know the password, client extensions should identify themselves to the server. The Server can then decide whether it will allow that client to register as a source/listener to that kind of event.

Re: Base Assumption - this sounds like the way Hibernate checks for stale data in it's cache. No problem.

Anyways- I'm running my 2nd session with MapTools tomorrow, so I need to get some sleep Brain fuzzy.

I'll continue with this later.

I've done a simple extension implementation trying to get MT & CT integrated. I used a listener method similar to what Azhrei posted. This works pretty well and is fairly simple to understand. I didn't extend any of Trevor's APIs, I just used what was available. I did have to change the access mode on some of them, but for the most part they work fine. This made my extension reliant on MT's code. Any changes to MT could require changes to my code as well. A pain, but I felt a necessary one. Changing all of the MT code really wasn't an option for me. It didn't cause too much pain in practice however. I don't think any of the methods I was using had changed signatures for awhile.

The other thing I had to extend were the client/server commands. I found I needed to create my own events that were propagated to all of the clients. This was fairly easy to do as well. I created a single client/server command that would check a registry of named command handlers. Then all I had to do was implement an interface, give it a unique name and then register an instance of the handler with the base server code.

The big problem I had with my implementation was with the networking stuff. First off, it doesn't just send the modifications to an object, it sends the entire object. While MT has this working fairly well, it was almost impossible for my plugin to work without screwing this up. My app caused there to be more changes to tokens and that caused them to get out of sync easier. While debugging this I found out that the Tokens tend to go back and forth a LOT; Up to three times for a single change. I was not really sure why it did this, and it made things very difficult to debug.

The other problem I had was with Hession, which is the lib used to connect server to client. It appears to be designed to be used across coding languages (i.e. Java to C) and it would change my object types for certain data. In my case I had a specialized Map object that was converted back to a regular HashMap. Not good for data integrity.

I'm not sure why there needs to be any authentication, unless you expect remote apps to log in to MT and start mucking with things. This was the original proposal of the dev team years ago, but it required a big update to the networking scheme to handle it. That was part of the original 1.3 code update, but it was never implemented.

Have you checked in your changes, jay? Or created a branch to work on them?


Yep -- surprise!


Yeah, I've noticed this as well. It seems to be some kind of feedback resonance field generated by the quadric bit signature of the SHA sums of all properties and macros.*


Yeah, the supported data types are documented on the Hessian project pages. I noticed that as well and for the most part figured it didn't matter. One thing you could do would be to package up your data as XML and send that instead. The Hessian lib supports the same readObject() and writeObject() as used by the Serializable lib, IIRC. (Hmm, could be that the XStream lib supports that, not Hessian. I'd have to look.)


I don't understand either, but I was just going to ignore it until things moved along a little further. Anyone who installs a plugin into the MapTool code gets what they deserve (if you don't trust the author, don't install the plugin). Remote clients won't have any problem because they'll need the same GM/Player password system that normal clients use.


*Note: In case you hadn't figured it out, this is just nonsense.

While I got all of the extension support to work I wasn't having a lot of luck getting CT data to transfer across the network so I never checked it in. I can check in the extension event support w/o too much trouble to anybody. I placed it in the RPLib so that CT & IT can use it too. The other stuff is in MT and should probably wait until after the final drop for inclusion.

Exactly!!!!

I did switch to XML and I got that to work pretty easy because I use XStream to save all of my data in CT. But the switching back and forth kept increasing the feedback resonance.... *

Let me take a look at what I did tonight and at least post some of it here for discussion.

*Note: Still just nonsense.

I think this applies here so i'll go ahead and throw it in. I think an add on/extension/script/etc should be able do do anything a player can do. Can a player move a token? yes, with a few exceptions the add on should also adhere to. Can a player reveal FOW? no, but if he/she moves his token to a location, that action can.

I think the best approach is to set out a list of actions that a player can perform, assign function names to them, and then tie the underlying MT functions to those "alias" functions. For example:

Player action: move a token
Alias function: APIMoveToken()
psudeo code: function APIMoveToken(tokenID,newLoc,addonName){ <<insert the same code the MT runs when a player drags a token>>}

that way as long as you keep the names of the alias functions the same, it wont matter as much what changes in MT bc the functionality can always be tied to the alias function. New versions of the code are just a fact of life, if a new release breaks an old addon, someone will update it if it's a popular plugin.

All that being said, I think MT should declare certain builds as the current "stable" release that is only updated when a new major feature is added and a majority of the bugs are worked out, and the rest as betas. That way addon dev's can focus on compatibility with the current stable release and not try to keep up so much with the blistering pace of development right now.

This seems to be the approach taken by the projects with the most successful mod communities that i know of, ie neverwinter nights

Sounds cool. You can start such a list on the wiki, where others will be able to add to it or use the Discussion page to comment on it.

Please read the two threads titled, RFC: Wargamers, what do you need from MapTool? and Exploring UI changes: permissions first. They contain background information you'll need before you start your list.


That's what we do. Have you read the Version section of the Tools page on the main site? Visit the main site and click Tools in the toolbar. Then clicking Version will expose a longer discussion. Or checking the Launch page will provide similar insight (although the first technique has more detail).

There's also a sticky thread here on the forum somewhere that discusses the concept behind "builds" and the "b##" number.


Heh. While I appreciate having MapTool compared with NWN -- at least as far as the modding community! -- I don't really want to compete at that level.

not really sure what you mean by "compete" at that level. could you clarify?

I was mostly thinking of the fancy NWN editors when I wrote that and not the idea of 3pp building on a "stable" base. Does that help?

ok, yeah I see. I definitely wouldn't spend too much time building editing tools and such. Basically what i was getting at was that if the files are user editable, that's more likely to attract modders than requiring source code edits or learning a new language (MTscript) to implement something. It doesn't really matter what form the files take (xml, sql, csv) as long as they are in a fairly common format and can be edited and reloaded without having to recompile. If the format is common and the integration process (save file in X directory, restart MT) is simple, it's much more likely that someone who is good with file editors and gui's will come along and make a simple editor to make it more user friendly (ala the mapscale tool).

While i'm at it, let me throw this one on the fire. It would be cool to see MT implement GET/POST requests. A lot of the framwork and drop in tools that i've seen here could greatly benefit from this. For example aliasmask's SRD reference. If he'd been able to just build a generic parser, seems like his dev time would have been slashed as well as making it more generic, ie referencing magic items, weapon props and etc. Being able to pull down character stats from a web site? update that site when the char is updated? great possibilities =)

While this would be "cool" there are a number of issues related to security and denial-of-service attacks. For these reasons you're not likely to ever see MTscript being able to access any type of external storage (files, URLs, databases, whatever).

With that said, there has been talk of building a simple web server into MT. Such a server would allow MT clients to make requests to the server using HTTP. By itself there is no huge advantage to this, but if/when MT has a plugin architecture then it might open some doors.

It's also possible that it could be used by a web browser to see a current image of the map, for example. This would allow an iPhone or other mobile device (iPad?) to see MT maps. Beyond some quick discussion of the possibilities there hasn't been any movement on this front -- there are just too many other things to work on.