Mote/MapTool: Thoughts on a DB choice

If you have an idea for a new feature, please discuss it in the main MapTool forum first, then post a summary of the discussion here. Use the first Sticky as a template.

Moderators: dorpond, trevor, Azhrei, giliath, jay, Mr.Ice

User avatar
RPTroll
TheBard
Posts: 3148
Joined: Tue Mar 21, 2006 7:26 pm
Location: Austin, Tx
Contact:

Re: Mote/MapTool: Thoughts on a DB choice

Post by RPTroll »

I speak from complete ignorance as to how they plan to integrate the DB. :mrgreen:

The problem JML refers to is that if you have multiple database types supported then you also have to support porting between them. Data won't just magically go from one to the other without a tool, some work, and a bit of cussing. From a dev point of view, integrating with one will be easier. My hope is that the binding with the database is loose so that others can be supported if need be but during development they should pick one and stick with it.

In your case, you're speaking of dev, test, and prod and its not uncommon to have different environments there. Now think of those same statements only in prod, which would be the case for a deployed MapTool. That's a harder case to make for swappable DBs.

Not that you can't just plop in a driver. The problem comes in the data and leveraging special features of each database. If you use something other than H2 then there is a reason, something you want out of the other database that might be different in another. Accounting for that in code will send the devs running into the hills screaming.

My 2 cents.

Cheers,
Keith
ImageImage ImageImageImageImage
Support RPTools by shopping
Image
Image

User avatar
Jagged
Great Wyrm
Posts: 1306
Joined: Mon Sep 15, 2008 9:27 am
Location: Bristol, UK

Re: Mote/MapTool: Thoughts on a DB choice

Post by Jagged »

RPTroll wrote:The problem JML refers to is that if you have multiple database types supported then you also have to support porting between them.

Would you see that as a job for the application developers?

As it happens, with the example I am working on above, the developers of the application in question do have a migration tool (or rather are working on one). We haven't made use of it though, and have just used the tools Microsoft provide to migrate from Oracle.

I am assuming that the reason MOTE wants a database, is to enable third-party hosting. That probably means MySql, since that seems to be the most popular database offering from small independent Web Hosting companies in the UK.

User avatar
JML
Dragon
Posts: 515
Joined: Mon May 31, 2010 7:03 am
Location: Blagnac, France

Re: Mote/MapTool: Thoughts on a DB choice

Post by JML »

Jagged wrote:What exactly do you mean by not SQL? Did you have any databases in mind?

I mean another DB model as in NoSQL, with no specific product in mind as my knowledge is really shallow. I'm just wondering what DB model would be more suitable for framework coders. I'm not thinking of Mote/MapTool developpers here.

RPTroll wrote:The problem JML refers to is that if you have multiple database types supported then you also have to support porting between them. Data won't just magically go from one to the other without a tool, some work, and a bit of cussing. From a dev point of view, integrating with one will be easier. My hope is that the binding with the database is loose so that others can be supported if need be but during development they should pick one and stick with it.
[…]
Not that you can't just plop in a driver. The problem comes in the data and leveraging special features of each database. If you use something other than H2 then there is a reason, something you want out of the other database that might be different in another. Accounting for that in code will send the devs running into the hills screaming.

Not exactly what I had in mind, but not very far. You guys have a professional view point. I'm thinking of how would complete newbies deal with all this. It's going to be hard enough to get the hang of dealing with a DB system, whatever model it is. Giving them choice will only bring confusion IMO.

Example: framework A was developed with DB xx and framework B with DB yy. How do we make both work together ?
Professionals are used to deal with this. I don't think we expect MapTool users to be this expert.

Jagged wrote:I am assuming that the reason MOTE wants a database, is to enable third-party hosting. That probably means MySql, since that seems to be the most popular database offering from small independent Web Hosting companies in the UK.

Good point I didn't even think of… which is why I'm asking you guys.

User avatar
JML
Dragon
Posts: 515
Joined: Mon May 31, 2010 7:03 am
Location: Blagnac, France

Re: Mote/MapTool: Thoughts on a DB choice

Post by JML »

JML wrote:
Jagged wrote:I am assuming that the reason MOTE wants a database, is to enable third-party hosting. That probably means MySql, since that seems to be the most popular database offering from small independent Web Hosting companies in the UK.

Good point I didn't even think of… which is why I'm asking you guys.

Wait, no. Didn't they chose H2 because it's embedded and Mote needs to be host independent ???

Which would mean any Java embeddable DB would be ok ? (technically speaking)

Lee
Dragon
Posts: 958
Joined: Wed Oct 19, 2011 2:07 am

Re: Mote/MapTool: Thoughts on a DB choice

Post by Lee »

I have to keep this short for now, sorry.

JML wrote:Wait, no. Didn't they chose H2 because it's embedded and Mote needs to be host independent ???


This. We have a branch where we're tooling with iciql, and JaQu, which are friendly with H2, to evaluate performance vs. constructing queries from parameters passed through MapToolScript. Also the choice for H2 is also to leverage MVStore, something we're experimenting with, along with several other similar projects, to further reduce the application's memory footprint. But that's for another time.

It's simple enough to allow for arbitrary db engines, and it's currently written that way, really, where all one needs is to drop the db engine, pass the connection string, and have at it. But we chose to do it this way in these initial stages, for the sake of users who either easily confused or distract on what to do. If we provided an arbitrary process, we either have to maintain an embedded instance and allow for arbitrary deployment of some other db, or ditch an embedded instance, and let the user decide on what they want to deploy.

A lot of people will find the latter easy, but for people who problems just getting Java to work, we're dubious things will be silky smooth for them in this case. Most want something that works, and a lot of this number will be interested in a performant solution, to save them all the trouble down the road. So, who gets to give them this info? Where will they find help? Will these sources perform the necessary evaluations periodically to find what's best, so that the info they give will always be pertinent? We all know the dangers of constantly worrying about optimization, and when it comes to writing a framework, we feel that this is an unnecessary distraction, for most potential users. Lastly, we want to keep the deployment of Mote (files et. al), and its installation, as simple as possible. If we can keep it to one file where stuff gets expanded upon first launch, then it's best for us, and the average end-user.

All scenarios point out to providing more support rather than lessening the need for it, if we allow for arbitrariness. We're after a solution that offers stability, and reasonable performance while getting the job done. After a lot of evaluation done for the sake of the end-user, we ended up with H2.

JML wrote:Which would mean any Java embeddable DB would be ok ? (technically speaking)


Technically, yes. But not all are created equal. Again, we made the choice based on the breadth of their feature list, as well as technologies that might help improve Mote down the line. The bottom line is, we want to abstract much of the technical stuff from the average end user as possible, while providing handy tools for those who know how. To answer the original intent of this post, we're all about providing multiple avenues in getting the job done. SQL won't find its way into every framework, and new users also do not need to learn SQL to work with Mote. It's something that's there, like a screwdriver in your toolbox. Ultimately, SQL is there to inspire projects suited to its use, like well-built libraries/compendiums of the different game systems out there. If we have such things in place, people will not have to roll their own version of game data every time someone makes a new framework for game X. It is a move toward a standard approach, and a standard set of "libraries", something that was always hard to do with the current system.

On the reference to NoSQL solutions, we think it's overkill to expose one for the needs of a framework designer. We are, however, evaluating several to help with managing Mote's internals, particularly the models that make up the token, maps, and vision system.

GTG, sorry to be brief. We'll talk more as things go along!

User avatar
Jagged
Great Wyrm
Posts: 1306
Joined: Mon Sep 15, 2008 9:27 am
Location: Bristol, UK

Re: Mote/MapTool: Thoughts on a DB choice

Post by Jagged »

Lee wrote:GTG, sorry to be brief.

Quite happy to know you are busy :D

User avatar
JML
Dragon
Posts: 515
Joined: Mon May 31, 2010 7:03 am
Location: Blagnac, France

Re: Mote/MapTool: Thoughts on a DB choice

Post by JML »

Lee wrote:On the reference to NoSQL solutions, we think it's overkill to expose one for the needs of a framework designer.

Which seems to imply you (Mote team) think SQL will be much easier to grasp for beginners than a document store DB for instance (yes, I did a little homework lately, hopping that I'd be able to keep up better with you guys). I'd be interested to see you elaborate on this when you'll have time for it.

Lee wrote:GTG, sorry to be brief. We'll talk more as things go along!

Nothing to be sorry about Lee. You gave a very clear insight on Mote team's philosophy, which makes sense to me and makes me more and more confident in Mote's future.

User avatar
RPTroll
TheBard
Posts: 3148
Joined: Tue Mar 21, 2006 7:26 pm
Location: Austin, Tx
Contact:

Re: Mote/MapTool: Thoughts on a DB choice

Post by RPTroll »

Regarding SQL for the framework designer - I'm sure some convenience functions can be created to ease the pain for those not used to SQLese.
ImageImage ImageImageImageImage
Support RPTools by shopping
Image
Image

Lee
Dragon
Posts: 958
Joined: Wed Oct 19, 2011 2:07 am

Re: Mote/MapTool: Thoughts on a DB choice

Post by Lee »

@JML Still quite busy, so this will have to wait a bit :) Though...

RPTroll wrote:Regarding SQL for the framework designer - I'm sure some convenience functions can be created to ease the pain for those not used to SQLese.


...long story made short, we did make convenience functions in MapTool script for the basic SQL functions: SELECT, UPDATE, DELETE, and INSERT, as well as several to create databases and tables, plus simple methods to manage their internals (e.g. list, rename, or drop tables). This is as much abstraction as we can put for this, since anyone who'd want to do any coding in MT will need to learn the forms and functions for MapTool script anyway.

User avatar
JML
Dragon
Posts: 515
Joined: Mon May 31, 2010 7:03 am
Location: Blagnac, France

Re: Mote/MapTool: Thoughts on a DB choice

Post by JML »

Lee wrote:@JML Still quite busy, so this will have to wait a bit :)

No problem Lee. I think the Mote team being busy is quite a good thing :D

Regarding Mote, the choice has been made, so it's just a matter of knowing what made people who took real time to think it over judge SQL would be easier to grasp, as it seems to me the other aspects have already been discussed. And I can surely wait for the answer. That's not a life or death matter :wink:

Post Reply

Return to “Feature Requests”