[Mote] Tools to make tools

If you have an idea for a new feature, please discuss it in the main MapTool forum first, then post a summary of the discussion here. Use the first Sticky as a template.

Moderators: dorpond, trevor, Azhrei, giliath, jay, Mr.Ice

User avatar
aliasmask
RPTools Team
Posts: 9029
Joined: Tue Nov 10, 2009 6:11 pm
Location: Bay Area

Re: [Mote] Tools to make tools

Post by aliasmask »

If you can write an if statement, then you can write sql queries. When you have a bunch of different tables you want to merge data, then the only tricky part is doing it efficiently and excluding the data you don't want. Anyone who understands logical operands shouldn't have a problem writing a query. I haven't written one in so long I'll have to re-learn it myself. Here's the last one I did and it looks pretty scary at first until you start breaking it down in to smaller sections, which is how I built it. This was what I used to get all the spell data from a 3.5 db I found online. Most of this is setting default values and setting flags based on values in the data (creating my own table columns).

The basic format is:
select tableColumn
from dbName
where somethingIsTrue

Code: Select all

select id, name, casting_time, ctUnit, '' ctDuration, range,
	case when formula = '' and personal = 0 and touch = 0 then 'textRange' else formula end as rangeFormula,
	personal, touch, rangeTouch, '' rDistance, target, tCreature,
	case when (tCreature = 0 
		or target like '%object%' 
		or target like '%undead%' 
		or id in (123,138,160,342,385,387,507,517,662,533)) 
	then 1 else 0 end tObject,
	targetLeveled, area, areaShape, areaLeveled, '' areaUnits, '' areaQuantity, effect, '' damageFormula,
	duration, durationUnit, durationLeveled, durationCondition, durationAdder, durationCancel,
	saving_throw,fortSave,willSave,refSave,noSave,saveCondition,
	spell_resistance, noSR, yesSR, srCondition
from (
	select id, name, 
		casting_time,
		case when casting_time like '%free%' then 'free'
			when casting_time like '%standard%' then 'standard'
			when casting_time like '%round%' then 'round'
			when casting_time like '%minute%' then 'minute'
			when casting_time like '%hour%' then 'hour' 
		else 'special' end ctUnit,
		range,
		case when range like 'Close%' then 'Close'
			when range like 'Medium%' then 'Medium'
			when range like 'Long%' then 'Long'
			when range like '1 mile/level' then 'WhisperingWind'
			when range like '40 ft./level' then 'ControlWinds'
			when range like 'Up to 10 ft./level' then 'Repulsion'
			when range like '0 ft%' then 'noRange'
			else '' end as formula,
		case when range like '%personal%' then 1 else 0 end as personal,
		case when range like '%touch%' then 1 else 0 end as touch,
		case when description like '%ranged touch%' then 1 else 0 end as rangeTouch,
		target,
		case when (target like '%creature%' 
			or target = 'You' 
			or id in (153,445,320,473,474,475,341) 
			or target like '%humanoid%' 
			or target like '%animal%' 
			or target like '%plant%') and not id in (109,587,605)
		then 1 else 0 end tCreature, 
		case when target like '%level%' then 1 else 0 end targetLeveled,	
		area,
		case when area like '%radius%' then 'radius'  
			when area like '%square%' then 'square'  
			when area like '%line%' and id <> 563 then 'line'  
			when area like '%cube%' then 'cube'  
			when area like '%cone%' then 'cone' 
			when id = 186 then 'controlwater'
			when area like '%sq. ft.%' then 'square'
			when area like '%apart%' then 'radius'
			when area like '%within%' then 'radius'
			when area like '%see text%' then 'special' else '' end
		as areaShape,
		case when area like '%/level%' then 1 else 0 end as areaLeveled,
		effect,
		duration,
		case when duration like 'Instantaneous%' then 'instant'
			when duration like '%Concentration%' then 'consentrate'
			when duration like '%round%' then 'round'
			when duration like '%minute%' or duration like '%min.%' then 'minute'
			when duration like '%hour%' then 'hour'
			when duration like '%day%' then 'day'
			when duration like 'Permanent%' then 'permanent'
		else 'special' end durationUnit,
		case when duration like '%level%' then 1 else 0 end as durationLeveled,
		case when duration like '% or %' or duration like '%until%' then 1 else 0 end as durationCondition,
		case when duration like '% and %' or duration like '%+%' then 1 else 0 end as durationAdder,
		'' as durationCancel,
		saving_throw,
		case when saving_throw like '%Fortitude half%' then 'half'
			when saving_throw like '%Fortitude negates (harmless)%' then 'negates,harmless'
			when saving_throw like '%Fortitude negates (object)%' then 'negates,object'
			when saving_throw like '%Fortitude negates%' then 'negates'
			when saving_throw like '%Fortitude partial (object)%' then 'partial,object'
			when saving_throw like '%Fortitude partial%' then 'partial' 
		else '' end as fortSave,
		case when saving_throw like '%Will half (harmless)%' then 'half,harmless'
			when saving_throw like '%Will half%' then 'half'
			when saving_throw like '%Will negates (harmless)%' then 'negates,harmless'
			when saving_throw like '%Will negates (harmless, object)%' then 'negates,harmless,object'
			when saving_throw like '%Will negates (object)%' then 'negates,object'
			when saving_throw like '%Will negates%' then 'negates'
			when saving_throw like '%Will disbelief%' then 'disbelief'
			when saving_throw like '%Will partial%' then 'partial' 
		else '' end as willSave,
		case when saving_throw like '%Reflex half%' then 'half'
			when saving_throw like '%Reflex negates (object)%' then 'negates,object'
			when saving_throw like '%Reflex negates%' then 'negates'
			when saving_throw like '%Reflex partial%' then 'partial' 
		else '' end as refSave,
		case when saving_throw like '%none (object)%' then 'object'
			when saving_throw like '%no%' then 'no' 
		else '' end as noSave,
		case when saving_throw like '%see text%' or saving_throw like '% or %' then 1 else 0 end as saveCondition,
		spell_resistance,
		case when spell_resistance like '%no (object)%' then 'object'
			when spell_resistance like '%no%' then 'no' else '' end as noSR,
		case when spell_resistance like '%Yes (harmless)%' then 'harmless'
			when spell_resistance like '%Yes (harmless, object)%' then 'harmless,object'
			when spell_resistance like '%Yes (object)%' then 'object'
			when spell_resistance like '%Yes%' then 'yes' else '' end as yesSR,
		case when spell_resistance like '%see text%' then 1 else 0 end as srCondition
	from spell 
	where id > 93
	) a
order by id

User avatar
JML
Dragon
Posts: 515
Joined: Mon May 31, 2010 7:03 am
Location: Blagnac, France

Re: [Mote] Tools to make tools

Post by JML »

It's been a LONNNNGGGG time since I didn't write one and I don't have much insight on other options, that is to say NoSQL.

You're right in saying basic SQL syntax is easy. There's two points that are making me wonder:
  1. Hidden traps: IRC there's quite a few things happening behind the SQL scene, that can lead to significant queries performance issues.
  2. Database design and flexibility: IRC SQL DB design has got to be learned, first in order to know how to map your data into tables, secondly to avoid hidden performance traps. And then, changing your design once your software has evolved isn't supposed to be this easy.
Does someone know how other DB compare regarding those three points ?
  1. Query language simple enough for beginners.
  2. Few hidden query traps.
  3. Design and evolution simplicity.
I'm especially dubitative regarding the last point as I know most people build their frameworks in an iterative manner, with a fair share of trial and errors.


User avatar
Bone White
Great Wyrm
Posts: 1124
Joined: Tue Aug 23, 2011 11:41 am
Location: Cornwall, UK

Re: [Mote] Tools to make tools

Post by Bone White »

JML wrote:Is there an educated someone around here who could enlighten me on this mater (beginner="non coder" friendliness) ?
It depends more on the beginner's type of learning. Intra-personal learners like myself are very suited to figuring things out themselves, regardless of the steepness of the learning curve. Other learning types are not so fortunate, especially for programming, visual and aural types (if there are no tutorial videos). Some people with the physical learning style will also struggle, but that depends mainly upon their ability to feel a part of their computer.

If you took two people and tried to teach them different programming languages from scratch, their performance would be more controlled by their learning styles and the learning delivery, than the format of the programming language itself. You could take this further and apply it to programming in general, as well as computer gaming and computer usage.

My point being, if people want to learn, they'll try to learn regardless of the method, and the results depend upon the person.

Dracones
Cave Troll
Posts: 68
Joined: Mon Nov 14, 2011 8:04 pm
Location: Fort Pierce, FL

Re: [Mote] Tools to make tools

Post by Dracones »

How about allowing the map to be specified for getTokens()? Right now I'm having to do map switching to grab all the token IDs off another map. Unless there's some other way do this I'm missing out on.

Would it be possible to eliminate the 2 block depth restriction for CODE? That one annoys the piss out of me:

Code: Select all

FOREACH this json, CODE:
  FOREACH a json from that json, CODE:
    IF on that, oh wait, I can't CODE here.... :/
And better debug support on the above would be nice as well. Right now if the IF block fails then it's "Dude, something frakked up past your parent FOREACH statement. Good luck finding it sucker!"

I spend more time breaking apart and chasing down bugs in loops than anything else with Maptool.

User avatar
JML
Dragon
Posts: 515
Joined: Mon May 31, 2010 7:03 am
Location: Blagnac, France

Re: [Mote] Tools to make tools

Post by JML »

[meta thinking on]
Bone White wrote:My point being, if people want to learn, they'll try to learn regardless of the method, and the results depend upon the person.
True if you get no other choice than learning, otherwise some will give up. How many will, heavily depends on the learning curve steepness and means to ease things.

There's no way to get everyone on board, but envisioning to which extent you want to come to to get a significant number of new framework coders doesn't seem such an inadequate move :wink:

Therefore my wondering about a DB choice.
[meta thinking off]

User avatar
aliasmask
RPTools Team
Posts: 9029
Joined: Tue Nov 10, 2009 6:11 pm
Location: Bay Area

Re: [Mote] Tools to make tools

Post by aliasmask »

My thought is any db is better than no db. I'd even use MySQL. Having a central db solves a lot of problems with sharing and moving data around that MT's current config experiences. One thing is speed and another is loss of data if 2 players are writing data to same token.

User avatar
JML
Dragon
Posts: 515
Joined: Mon May 31, 2010 7:03 am
Location: Blagnac, France

Re: [Mote] Tools to make tools

Post by JML »

aliasmask wrote:My thought is any db is better than no db. I'd even use MySQL.
Given no other choice I'd have to fully agree. I'm only wondering if there's really no other choice.
aliasmask wrote:Having a central db solves a lot of problems with sharing and moving data around that MT's current config experiences. One thing is speed and another is loss of data if 2 players are writing data to same token.
You're perfectly right in pointing out that data security trumps execution speed. I'd only be surprised SQL would be the only secure DB system by now.

What's bothering me is I have old memories of SQL data bases being quite heavy to maintain and to evolve, and I'm afraid this could get a bit too much for non IT professionals. I've got the feeling that people usually build their frameworks in an iterative manner. To which extent could this become a problem ?

Again, I'm lacking too much knowledge here and waiting for informations, which you started to give to me. Thanks AM.

Dracones
Cave Troll
Posts: 68
Joined: Mon Nov 14, 2011 8:04 pm
Location: Fort Pierce, FL

Re: [Mote] Tools to make tools

Post by Dracones »

JML wrote: What's bothering me is I have old memories of SQL data bases being quite heavy to maintain and to evolve, and I'm afraid this could get a bit too much for non IT professionals. I've got the feeling that people usually build their frameworks in an iterative manner. To which extent could this become a problem ?
This isn't going to be a problem for something like Mote because you're not going to see databases large enough for tuning. You start to see problems in databases after you get millions and millions of rows. It's a problem in the real world because newer programmers often treat databases as "dump in data, don't worry about it" because it takes so much data for performance to become a problem.

So you have an app that records user logins, never deletes them and in 2 years your app starts to slow down because there's a billion entries in that table.

User avatar
aliasmask
RPTools Team
Posts: 9029
Joined: Tue Nov 10, 2009 6:11 pm
Location: Bay Area

Re: [Mote] Tools to make tools

Post by aliasmask »

I've used mainly oracle dbs and I've experienced very little in problems. Rarely, there would be a collision, but most of the time collisions would just table lock and buffer the data or fail if it waits too long. MySQL is unstable IMO. Rebuilding tables was needed often to keep things working. I worked for a big internet retail company that built their db off some old hacky code and a bazillion tables, but the DB environment was pretty solid. I haven't looked at the proposed H2 until now. Looks pretty good for features, but I'm unsure of the affect of not having Multi-Threaded Statement Processing and if it has another option/solution in place of using that. Seems to be the norm with other dbs.

User avatar
JML
Dragon
Posts: 515
Joined: Mon May 31, 2010 7:03 am
Location: Blagnac, France

Re: [Mote] Tools to make tools

Post by JML »

Thanks for the replies guys. As I think it's becoming a thread hack I created a new one here.

User avatar
wolph42
Winter Wolph
Posts: 9999
Joined: Fri Mar 20, 2009 5:40 am
Location: Netherlands
Contact:

Re: [Mote] Tools to make tools

Post by wolph42 »

JML wrote:Thanks for the replies guys. As I think it's becoming a thread hack I created a new one here.
Thnx
Dracones wrote:How about allowing the map to be specified for getTokens()? Right now I'm having to do map switching to grab all the token IDs off another map. Unless there's some other way do this I'm missing out on.

Would it be possible to eliminate the 2 block depth restriction for CODE? That one annoys the piss out of me:

Code: Select all

FOREACH this json, CODE:
  FOREACH a json from that json, CODE:
    IF on that, oh wait, I can't CODE here.... :/
And better debug support on the above would be nice as well. Right now if the IF block fails then it's "Dude, something frakked up past your parent FOREACH statement. Good luck finding it sucker!"

I spend more time breaking apart and chasing down bugs in loops than anything else with Maptool.
Nice one on the getTokens. More then two code layers IS possible (see sig). The trick with loops is either use bot_getDebugInfo() or remove the loop and replace it for one value of the loop.

Lee
Dragon
Posts: 958
Joined: Wed Oct 19, 2011 2:07 am

Re: [Mote] Tools to make tools

Post by Lee »

Sorry for the late response. It's been busy.

This is a good list. Some are easy to do, while others will have to wait. Without going into detail, implementation will depend on what gets affected in the code. If it's one of the base models, then it gets sent to the backburner, until such time a healthy amount of these are collated so it can be done in one go. The reason being, for each change, especially a reversion, a migration process from the previous version to the next, needs to be written. It can get tricky, and sometimes, near impossible, depending on whether the XStream library MapTool uses for serializing information to save, supports the object addition. Most of the time, it does, but we're at a stage where we're trying things out beyond the standard library (the safest to serialize), and want to avoid making reversions as people may find that they can't use their campaigns anymore.

I'll write a specific response to each post when time becomes more available.

User avatar
wolph42
Winter Wolph
Posts: 9999
Joined: Fri Mar 20, 2009 5:40 am
Location: Netherlands
Contact:

Re: [Mote] Tools to make tools

Post by wolph42 »

Lee wrote:Sorry for the late response. It's been busy.

This is a good list. Some are easy to do, while others will have to wait. Without going into detail, implementation will depend on what gets affected in the code. If it's one of the base models, then it gets sent to the backburner, until such time a healthy amount of these are collated so it can be done in one go. The reason being, for each change, especially a reversion, a migration process from the previous version to the next, needs to be written. It can get tricky, and sometimes, near impossible, depending on whether the XStream library MapTool uses for serializing information to save, supports the object addition. Most of the time, it does, but we're at a stage where we're trying things out beyond the standard library (the safest to serialize), and want to avoid making reversions as people may find that they can't use their campaigns anymore.

I'll write a specific response to each post when time becomes more available.
thanks for the respone.
I'll write a specific response to each post when time becomes more available
- I would suggest you limit yourself to the OP as all info is there (and then some)
- the only thing NOT in the OP is the database discussion, but that has been moved here: http://forums.rptools.net/viewtopic.php?f=26&t=24993

Lee
Dragon
Posts: 958
Joined: Wed Oct 19, 2011 2:07 am

Re: [Mote] Tools to make tools

Post by Lee »

Will reply to the OP then, when we get around to it :)

Post Reply

Return to “Feature Requests”