Ultimate Game Table

[dlan1000]

Hey,

I've seen several posts about gametables using wiimote and IR pens. I am interesting in constructing a gametable that employs this method but is supplemented with additional camera-based object tracking. I am writing to see if anyone else is interested in this idea and whether some of the people on this board that have some experience with making interactive gametables and an intimate knowledge with maptool might be able to answer some questions I have.

This is what I would like to do, in summary:
I would like to mount a camera underneath a glass gametable and use object tracking software to recognize when physical tokens are placed on the board. I have found that the trackmate project (http://trackmate.media.mit.edu/) could be used to track objects such as tokens by printing out circular barcode images (1"x1") and affixing them to the bottom of the token. In order to use this software, I would need some way to be able to add tokens to a maptool session from some external program. i.e., when someone puts a physical token on the board, maptool adds the corresponding token onto the map. I am not sure if there is a method to do this through a networked session. In the gametable projects that I have seen so far, many people run a player (non-DM) maptool session for the gametable projection and then use a separate laptop running a DM maptool session so that the DM can see private information (e.g., unseen npcs). One possibility to accomplish adding tokens automatically in response to physical token placement on the board would be for a separate piece of software that is handling object tracking to send network packets to the client gametable maptool session by "masquerading" as the DM client. Does anyone that is familiar with the maptool networking code know how such a thing could be accomplished? I am a fairly competent programmer and I would like to contribute a token-tracking tool to the community.

[dochogan]

My thoughts, although they may not specifically answer your question(s):

For object coding, I would recommend looking into QR Code based bar-coding. It is a more universal format, and similar to (if not the same as) what is being used for MS Surface-based computing.

If you are going to building such a table, I would highly recommend generalizing a little more than just the intent to use it for VTTing. The technology is there, overall, for multi-touch surface computing. If you base off of that/those technolgy, general support for VTTs, including MT, should be inherent.

For specific VTT use (as enhancement to the above), I would think that you would first be working externally in a database environment, providing the host system with a means of recognizingg, archiving, and manipulating objects and devices identified via QR codes (or whichever recognition code system you choose). Optimally, this would allow for MS Surface-type device interaction (accessing multimedia from PDAs, etc), and then specifically recognizing game-piece codes as a form of input device, perhaps.

One concern I would have, and have not yet had time to research, is how to have an object's barcode even *seen* by a visible-light camera. In order for a projected screen image to be seen on the table, a semi-opaque medium (drafting celluloid, commonly) is used to catch the projected image. I've yet to figure out how a barcode would be seen through this required layer. It can be done, I'm sure, as the MS Surface tables are capable of it, just haven't come up with a way to do it in a homebrew.

These are the *general* thoughts I have, as I too am mulling over beginning a multi-touch table project. The first resource I would point you to would be:
http://www.maximumpc.com/article/featur ... e_computer
Which will provide an overview of a simple-ish implementation, and then:
http://www.nuigroup.com
which is the Natural User Interface Group site, specifically focussed on multi-touch tables.

[hrs]

Hi, I'm also interested in such thing, and there are other people around here that posted similar topics

http://forums.rptools.net/viewtopic.php?f=6&t=13411

http://forums.rptools.net/viewtopic.php?f=6&t=8827

http://forums.rptools.net/viewtopic.php ... 28&start=0

http://forums.rptools.net/viewtopic.php ... game+table

At the moment I'm building a table too, my intent is more general because I'd like to use it also as a everyday pc (just surfing and reading no real work), but using it as game table is another intended use.

I surfed around reading about interest technology to use (I'm not a pro programmer so I hope somebody develops the right tools for me to assemble ) and I found this

http://hackaday.com/2009/01/18/multitou ... op-gaming/

that seems to be able to recognize tokens. (and yes there is also MS Surface)

By the way I'm not sure barcode tracking is absolutely necessary, if the miniature is put over the maptool token displayed, and the base of the min is seen by the IR camera as a cluster, the the only thing to do is to drag the miniature like a finger in a drag style motion (I mean without lifting it from the table).
In this case is mandatory that MapTool is able to manage multiple inputs

[dochogan]

By the way I'm not sure barcode tracking is absolutely necessary, if the miniature is put over the maptool token displayed, and the base of the min is seen by the IR camera as a cluster, the the only thing to do is to drag the miniature like a finger in a drag style motion (I mean without lifting it from the table).

I've been thinking they same thing for the most part; as long as the input system recognizes the IR displacement, it should be fine as a generic cluster. I do think that object ID is a core feature I'd want for a MTT though, and would be a major enhancement for minitaures play. For one, it would allow for a miniature to be lifted and relocated, and be recognized at the new position, for example.

In this case is mandatory that MapTool is able to manage multiple inputs

I believe that there are a few things in MT that would have/should be modified to make MTT technology more feasable. as well as the multi-input, one of the more troublesome things I've had to deal with as a TabletPC nut, and which would be an identical issue with MTT, is interface design. UI widgets are almost purely designed for mouse-type interfacing. Scroll bars, min/max/restore/close are a PITA with TabletPC, and I can only imagine them being MUCH more annoying to use with the gross input area of a fingertip. Just something to consider. Thankfully, it shouldn't be too incredibly difficult to hack in an alternate, optional widget style at some point. I think

[dlan1000]

hrs and dochogan,

You both bring up interesting points. I agree that in general a VTT should have more functionality (such as generalized multitouch input capabilities or generalized object recognition and tracking). There is hardware/software out there to both of these ends (for example, I've seen homebrew multitouch VTTs using edgelighting and frustrated internal reflection). That being said, I am interested in using existing technologies and software together... For example, rather than putting the onus on maptool devs to work in multitouch support or the ability to accept token positions from an input stream, I would like to see what can be done with what exists already.

About barcode tracking specifically: I hadn't thought about whether barcodes may not be visibile through a semi-opaque surface. However, IR light certainly gets through rear-projected VTTs with semi-opaque layers, so it may be a matter of ensuring that the contrast is high enough to make the barcode visible to the camera. I do think this problem can be solved with either the right material choices, lighting conditions, or via another alternative (for example, suppose barcodes are backlit by a visible spectrum LED embedded in each 'token').

So why barcodes? Because we would like a way to differentiate one token placed on the board from another and using LEDs cannot accomplish this task well (ok, differentation based on color or blinking frequency could work but is more complicated to implement over image-recognition methods such as barcodes). There are certainly other methods of doing so, but there is existing and quite usable software that does a great job at this and requires only a webcam and a printer to start using (http://www.instructables.com/id/Trackma ... iffhanger/ ). The LucidOSC approach allows you to use the trackmate software to stream object positions through TCP/IP so that you can plug into it in a way that is language-independent.

I guess there are two ways to approach making a gametable, you could try to develop new Human-Computer interaction technologies, or you could try to compile existing solutions into a package. There are already a great many talented people working on the former problem; I am interested in the latter problem. In particular, I would like to find the cheapest method (and thereby accessible to a large population of users) that will allow gamers to use maptool with a rich interaction environment. The wiimote/IR pen tracking combination has made simple interaction relatively affordable and technically accessible and it can be used with maptool without any software modification (since it's just mouse emulation). I think token tracking is nearly as affordable and accessible with the barcode approach.

In fact, what I am proposing is a very simple piece of software middleware (the translator, below) that sits as a layer in between the object tracker (trackmate) and maptool:

object tracker [stream of object positions]-> Translator [reads object positions from stream and sends out a network packet to maptool clients by masquerading as a maptool DM client]-> maptool client


I was hoping to get some answer to my post from someone that knows how maptool handles networking to see if this is possible. If the translator software were built out of the skeleton of maptools' networking module, this should be achievable (though I wonder: can maptool handle multiple DM clients? Can it pass DM control between connected users?)

The translator middleware would probably need to know the state of all tokens 'in play' in maptool because adding a 'new' token to the board is not the same as moving an existing token. Using this reasoning, it sort of makes sense to build the middleware from a heavily pared down version of the maptool software itself (since a client can know the states of existing tokens and a DM client can control new token addition, etc.)

Anyone see where I'm coming from here? Sorry for the long-winded post

Dylan

[Azhrei]

I was hoping to get some answer to my post from someone that knows how maptool handles networking to see if this is possible. If the translator software were built out of the skeleton of maptools networking module, this should be achievable (though I wonder: can maptool handle multiple DM clients? Can in pass DM control between connected users?)

I didn't write the networking layer and I'm only familiar with it in passing, but I'll tell you what I know. I don't think the issue will be the networking, though.

MapTool uses a library called the Hessian Protocol. This is essentially a way of sending data and remote procedure calls across a data stream in a language-neutral manner. For example, when you start a MapTool server and fill in the RPTools Alias field, a Hessian packet arrives at the rptools.net web site in a particular PHP script. That PHP scripts adds the server listing to a MySQL table and the contents of that table are sent to any client that uses the Connect to Server dialog (so it includes the IP address and port to connect to).

When a token is dragged across the map, the local client calculates the A* route and renders the movement. My understanding is that the destination grid location (or map-relative x/y coordinate in gridless mode) is sent to all clients and they calculate their own A* routing for display locally. If this is indeed the way it works, your Translator idea should be fairly easy to implement as you can ignore everything except the network portion -- you don't need to do any rendering, for example.

It should be possible for your Translator to act as a GM instance of MapTool without much trouble. As I describe below, I would start with the Translator being inserted into the MT code base in the beginning and move to a separate tool later.

The translator middleware would probably need to know the state of all tokens 'in play' in maptool because adding a 'new' token to the board is not the same as moving an existing token. Using this reasoning, it sort of makes sense to build the middleware from a heavily pared down version of the maptool software itself (since a client can know the states of existing tokens and a DM client can control new token addition, etc.)

You would certainly want to do some testing wherein you run wireshark or similar traffic capture program and watch what happens on the wire as you drag a token or drop a new token on the map. And for testing purposes, you might use the full MapTool base and simply add your Translator and see if you can get the network portion completed. If so, it might be best to bundle the Translator as a new MT JAR in the initial phases and work on a standalone implementation later. The MT code base is huge with a lot of complex interactions between pieces, so I would think that a new tool with just the Hessian library and your own code would be the way to go.

When a client drops a token onto the map, the information must be presented to the server and from there to the other clients. Ditto for the GM dropping a token. As you surmised, your Translator would need to keep track of the map state in regards to token existence and placement, but this might get tricky for MT 1.3. As the code stands right now, when a token is moved the entire contents of the Token POJO are sent to each machine (POJO=plain old Java object) and this means properties, asset IDs, etc. Your Translator could work much better if all it needed to send were the x/y coordinates. That is the planned situation for MT 1.4. I say this because without that ability, your Translator would need to listen to all other token manipulations and track all that data. Not a big deal except that the MT code base is not constrained to a fixed API which means the data structures can change from build to build as long as the external campaign format is upward compatible. It seems to me that could make your Translator rather fragile in regards to which versions of MT it will work with.

Ideally, MT would have some kind of plug-in architecture so that your Translator could be picked up and used by an existing MT instance. Currently there is no such architecture. I would sorely love to see one in 1.4 so I do have plans to come up with something. I suppose I should work on it soon so I can present it to Trevor and he can be working with it in mind...

[dlan1000]

Thanks Azhrei,

That information is exactly what I was looking for. I had a peek at the some of the network code for the repo and I see how Hessian is being used. I also hear your point about transferring the entire token object; it could be complicated to track all of the changes to token objects (the code base is indeed very complicated, though I am impressed by the level of organization and how simple it is to read [definitely not the first time I've been impressed with rptools so props to all the devs]) and the only pertinent information is the x/y coords of each token--also it would be a shame to get something working only to have it break with a new release. I would also love to see a plugin architecture for maptools as I think the hurdle of understanding the wealth of modules and how they all interact makes stepping into it as an unfamiliar developer very time-consuming. I am curious to know, do you and the other devs have working docs that detail the logic of how the modules in the MT codebase interact and if you have a board where you discuss existing and planned future work?

thanks,

Dylan

[Azhrei]

The Java Developer forum under Developer Notes (I think?) is the closest, but no, most of the discussion about code happens here on the boards with one contributor giving clues to the others.

I really want to see 1.4 done with more structure. It's confining, and programmers don't like to work that way (generally), but the more contributors there are the more it's needed IMO.

I've been adding javadoc as I write/modify code and I know Craig has been doing the same, but much of the code isn't documented in terms of how the pieces fit together. I'd love to get the UML2 plugin for Eclipse to generate some static diagrams for me (a class diagram would be a big start) but whenever I take a look at it I get lost in trying to get it to reverse engineer a drawing from existing code. One of these days...

[eRaz0r]

I'd be interested in seeing MapTools developed with an API in mind.
I know that the main thrust of 3rd party development has been in creating macro-suites. This, to me, can only go so far. There's a lot of things you can do with macros, but changing major UI elements is not really one of them.

Now, I've only just started looking at the code, but conceptually, MapTools is basically just a Map Server. It shows a map, and allows clients to interact with that map - with map editing tools, and/or with tokens.

All the other stuff is icing on the cake.

Ideally an API doesn't need to be very complex. What does a client of a MapServer need it to provide?

get the Map
get the current ViewPort of that Map
get the tokens on the map

register as a listener for events
(token movement, other map events such as FOW reveal, Chat)

register as a source for actions (either as an Event, or as a direct method invocation whatever..)
- chat
- token movement, token addition etc.
- map editing


Am I way off base here? Like I said - I'm still dabbling in the source code, but how hard would this be?

[Azhrei]

Ideally an API doesn't need to be very complex. What does a client of a MapServer need it to provide?

get the Map
get the current ViewPort of that Map
get the tokens on the map

register as a listener for events
(token movement, other map events such as FOW reveal, Chat)

register as a source for actions (either as an Event, or as a direct method invocation whatever..)
- chat
- token movement, token addition etc.
- map editing

Am I way off base here? Like I said - I'm still dabbling in the source code, but how hard would this be?

Difficulty isn't the issue. It would take only an hour or so to build a set of interfaces to the major system components.

The problem is defining an API that won't change tomorrow. So many APIs (think "Microsoft Windows") are created as a thin veneer on top of existing functions. This is great for getting something out the door, but it's terrible for long-term maintenance or even short-term development.

Let's design an API for that first chunk of functionality you mentioned: get map/viewport/token. First, MapTool calls them Zones, ZoneRenderers, and Tokens. Do we use that same terminology? Good OOA&D says we should use real-world terms and not geek jargon. Yet the term "Map" is a type of data structure in Java (more accurately, it's a type of interface to a data structure). How do you want to resolve that? I'll stick with Zone for discussion purposes.

What does it mean to "get a map"? Are you getting a reference to the original? A copy of it? If you get a copy of the Zone, is it a deep copy? Or do you give the programmer the option of either of the above?

What is the structure of a Zone? How is it likely to change in the future? When it does change, how do we protect the API so that existing extensions don't have to be modified to compensate? If that third-party code does need to be modified (let's say it's inescapable), how can we minimize the exposure?

Ask and answer those same questions for the ZoneRenderer and Tokens and you begin to get an idea of what's involved. Anyone who's worked on a significant size application knows that interface design is likely THE most important aspect of programming. (Note I said "programming" and not "software development". The most important aspect of software development is probably drawing up specifications so that the program solves the correct problem.)

Last, add in that many of the details of the graphics subsystem are stored in Trevor's head and you begin to get an idea of just how involved this could be.

All that said, if you want to develop a set of interfaces, I would love to have someone start that work. Whatever you come up with, do not look at the existing MapTool code! The interface should be independent of the application. That's how we'll be able to ensure that internal changes don't affect third-party extensions.

[Azhrei]

I've split eRaz0r's reply into a separate thread in the Developer Notes forum. We have started getting into the details of how an extension API to MapTool might look and this thread is more about the core concept of the Ultimate Gaming Table. I have let this thread go for a little while because it seemed to be related to how one might use a gaming table with MapTool, but the last couple of posts are really a programmer topic. Hence the split.

You may now continue your discussion of the UGT.

[Stitched]

Don't know if people saw this or how they feel about it but it's maybe a cheaper alternative to building a flat-screen/multi-touch surface.

You still need a way to project an image (I suppose) or a perhaps a flat screen mounted inside a table / large screen TV in the living-room.

http://www.instructables.com/id/Low-Cos ... -Wii-Remo/

They have a multi-touch project posted at the low-cost of $2500 (a bit steep for me) but putting an LED emitter in a pen and buying a Wii remote seems a lot more affordable.

[arronlee]

Hi,
As for me, I am testing the related java barcode generator these days. Do you have any ideas about it? Or any good suggestion? I am totally a green hand on barcode generating field. Any suggestion will be appreciated. Thanks in advance.


Best regards,
Arron

[JamzTheMan]

Hi,
As for me, I am testing the related java barcode generator these days. Do you have any ideas about it? Or any good suggestion? I am totally a green hand on barcode generating field. Any suggestion will be appreciated. Thanks in advance.


Best regards,
Arron

Are you trying to use it real-time to scan individual mini's like fiducials?

On a "separate" note, I may have found a new toy/macro for my build, thanks! For our Pathfinder game (we have a unique multi-gm game) I started using QR codes for magic items/treasure. I'd find a cool pic for said item, then copy it's stats (full magic stats, lvl, spellcraft check, gp worth, etc) and encode it into a QR code and print it out on photo paper (usually fit 3-4 items on a 4x6. (I would then copy the same info into MT if needed).

Then I could hand the items out to the players. If 4 weeks of real time go by before they can identify the item, I'm (or next GM) doesn't have to look back into the module/notes and say "wait, which Longsword, where did you find it?" flip, flip, flip. Instead, GM just whips out smartphone, click, "Ok, make a spellcraft...success, it's blah blah blah". Also, the player can do the same later for a reference. (I suppose you could encrypt the QR code as well but my players don't scan the cards until told to).

ANYWAY, maybe I'll throw this into my build of MT that can generate QR codes based on GM notes for tokens (items). Maybe an option then to print token pic + QR code or save on players token, email...mmmm....wheel's are turning....

Anyone have any other thoughts/uses for QR codes in MT?

[RPTroll]

The QR image could be included in the token handout field. Don't know if a phone could pick that up or not.