Spell Reference D&D 3.5/Pathfinder SRD

These are tools and utilities that make it easier to run games. This includes Lib: macro tokens dropped into MapTool to manage the game, a conversion file for CharacterTool to allow use in MapTool, or just about anything else you can think of -- except graphics with macros and anything specific to a particular campaign framework. Those are already covered by the Tilesets subforum and the Links and External Resources forum.

Moderators: dorpond, trevor, Azhrei, giliath, Gamerdude, jay, Mr.Ice

User avatar
aliasmask
RPTools Team
Posts: 9024
Joined: Tue Nov 10, 2009 6:11 pm
Location: Bay Area

Re: Spell Reference D&D 3.5/Pathfinder SRD

Post by aliasmask »

Okay, I'm back to working on version 2.0. Here's some code to create a Spell Search macro:

Code: Select all

[H: hasInput = input("Enter Full Spell Name|Spell=||PROPS|span=true SETVARS=UNSUFFIXED")]
[H: abort(hasInput)]
[H: rSpell = searchSpellLink(Spell)]
[H, if(rSpell == Spell), code: {
   [H: output = strformat('Spell <b>"<font color=red>%s</font>"</b> not found.',rSpell)]
};{
   [H: output = strformat("Researching <b><font color=blue>%s</font></b> Spell.",rSpell)]
   [H: openSpell(Spell)]
}]
[R: output] 
You don't need to put in special characters or capitalization to get the same results, for example: "Bull's Strength, Mass" is the same as "bullsstrengthmass".

User avatar
aliasmask
RPTools Team
Posts: 9024
Joined: Tue Nov 10, 2009 6:11 pm
Location: Bay Area

Re: Spell Reference D&D 3.5/Pathfinder SRD

Post by aliasmask »

Okay, building the new indexing system. Hopefully it'll take less than 2 hours. My guess is around 15 min at most.

Whoa... that was only 3 minutes. That's a vast improvement. I also found spell lists I didn't even know I had in my db. Splitting the properties for spell data and index data in to two lib tokens really helped. So far I have 345 props for indexes and 606+ for spell data.

Time to build the add/edit/delete spell macro now.

neofax
Great Wyrm
Posts: 1694
Joined: Tue May 26, 2009 8:51 pm
Location: Philadelphia, PA
Contact:

Re: Spell Reference D&D 3.5/Pathfinder SRD

Post by neofax »

Will you have a way to mass import spells? The reason I ask is because Pathfinder came out with the Advanced Player's Guide and I would like to add these to your Spell Library, but do not want to do it one by one.

User avatar
aliasmask
RPTools Team
Posts: 9024
Joined: Tue Nov 10, 2009 6:11 pm
Location: Bay Area

Re: Spell Reference D&D 3.5/Pathfinder SRD

Post by aliasmask »

neofax wrote:Will you have a way to mass import spells? The reason I ask is because Pathfinder came out with the Advanced Player's Guide and I would like to add these to your Spell Library, but do not want to do it one by one.
Yep. You can either put the spell data in the format of the existing tables, add each field from scratch or cut/paste json object to a form. It will be able to export spells as json objects as well. I'm sure if you have the raw data, I can convert it in to the necessary json objects for import.

Here's an example format. The blue fields are used to create the indexes and the <p> is needed in the description for formatting.
Acid Arrow
{"spellName":"Acid Arrow",
"school":"Conjuration",
"subschool":"Creation",
"descriptor":"acid",
"spellLevel":"Sorcerer/Wizard 2",
"spellComponents":"V, S, M, F",
"castTime":"1 standard action",
"range":"long (400 ft. + 40 ft./level)",
"target":"",
"area":"",
"effect":"one arrow of acid",
"duration":"1 round + 1 round per three levels",
"savingThrow":"none",
"spellResistance":"no",
"shortDescription":"Ranged touch attack; 2d4 damage for 1 round +1 round/three levels.",
"materialComponents":"rhubarb leaf and an adder's stomach",
"focus":"a dart",
"description":"<p>An arrow of acid springs from your hand and speeds to its target. You must succeed on a ranged touch attack to hit your target. The arrow deals 2d4 points of acid damage with no splash damage. For every three caster levels you possess, the acid, unless neutralized, lasts for another round (to a maximum of 6 additional rounds at 18th level), dealing another 2d4 points of damage in each round.</p>",
"reference":"PFRPG Core"}
These are all the default fields, but more can be added and configured to be displayed. For example, indexProps will be added as a field during the build which will have the links to related indexes. The above may have the following indexProps:

Code: Select all

indexProps = "school.conjuration,subschool.creation,descriptor.acid,class.sorcerer/wizard,class.sorcerer/wizard.2,alpha.a"
An example property on lib:spellindexes would be:

Code: Select all

descriptor.acid = ["acidarrow","acidfog","acidsplash"]
Each spell will be stored on lib:spelldata in the format:

Code: Select all

acidarrow = {...object above...}
The lists of names will be stored separately along with the index pointers to be displayed, but all this is generated from the json object of the spell on import.

User avatar
aliasmask
RPTools Team
Posts: 9024
Joined: Tue Nov 10, 2009 6:11 pm
Location: Bay Area

Re: Spell Reference D&D 3.5/Pathfinder SRD

Post by aliasmask »

See, I am working on it. This is the bare bones, no CSS, no Tab Bar for tokens (spell lists). All the data is now on tokens, super fast and editable. No need for full page caching and I added tooltips to the menu links. This will start off as a spell manager, but will be able to add feats, abilities, skills. All these things will be searchable and will allow a wildcard (*) to give multiple results.

I have a bunch more functions to write, but I think the hard part is behind me, organizing all the data so it runs fast.
Attachments
example.jpg
example.jpg (49.01 KiB) Viewed 6892 times

wrathgon
Dragon
Posts: 735
Joined: Wed Feb 06, 2008 10:28 am
Location: Guildford, UK

Re: Spell Reference D&D 3.5/Pathfinder SRD

Post by wrathgon »

hey can help/suggest how to speed up my manager?

User avatar
aliasmask
RPTools Team
Posts: 9024
Joined: Tue Nov 10, 2009 6:11 pm
Location: Bay Area

Re: Spell Reference D&D 3.5/Pathfinder SRD

Post by aliasmask »

wrathgon wrote:hey can help/suggest how to speed up my manager?
Do you have any one thing specific you would want to speed up first?

I know a 20th level Cleric spell list takes about 3 seconds to load via your code and about another 3 loading my links. You'll be able to knock that down to less than 1 sec for my links with this new lib. I've been avoiding loops where I can and doing things like this:

Code: Select all

[H: spellData = am.sd.glp(arg(0))]
[H: spellData = replace(spellData,";",strformat("&#%s",59))]
[H: varsFromStrProp(json.toStrProp(spellData))]
[H: menuLinks = am.sm.glp("link_menu_index")]
[H: varsFromStrProp(json.toStrProp(menuLinks))]
The above code just assigned locally about 20 variables for spellData and about 300 links in about 100ms or less, whereas using 2 loops would take significantly longer.

If you can call a function once, then that is the way to go. So, if you want spell links you can get them all by:

Code: Select all

[H: spellLinks = am.sm.glp("link_spell_index")]
[H: spellLinks = replace(spellLinks,";",strformat("&#%s",59))]
[H: varsFromStrProp(json.toStrProp(spellLinks))]
There is now 621 spell links defined locally in the format "link.spell.[clean spell name]" in about 40ms.

Code: Select all

@@cleanVar
[H: cleanVarName = lower(arg(0))]
[H: macro.return = replace(cleanVarName,"[^a-z0-9_.]","")]

wrathgon
Dragon
Posts: 735
Joined: Wed Feb 06, 2008 10:28 am
Location: Guildford, UK

Re: Spell Reference D&D 3.5/Pathfinder SRD

Post by wrathgon »

what about reading in large json objects? i been toying with getting lists of the objects and storing them as txt lists so i dont have to get a json to get list of say classes or schools

User avatar
aliasmask
RPTools Team
Posts: 9024
Joined: Tue Nov 10, 2009 6:11 pm
Location: Bay Area

Re: Spell Reference D&D 3.5/Pathfinder SRD

Post by aliasmask »

It's less of a problem reading a large json object than getting values from the large json object and setting values inside a large json object. That's where the slowdown is. The large json object should only be 1 dimensional though, so no nesting of objects. Use a property list for a value or your own custom list if you need more structure inside the object.

I've also been using dot names for object field names and so far I don't have any properties defined in the campaign to worry about when defining variables locally. No, oops, I overwrote my token property basically. It makes dumping all the variables from an object to local call easier using the method above.

Here's an example of how I will do some structures:

Code: Select all

asm.casterClasses = {"asm.wizard":"Wizard","asm.cleric":"Cleric"}
asm.wizard = {"level":20,"abilProp":"Intelligence","abilModProp":"IntB","SPD":"4,4,4,4,4,4,4,4,4,4"}
I use a dot property name as the field defining what it represents for it's value. So, what classes do I have: asm.wizard and asm.cleric defined as "Wizard" and "Cleric". Then use that name as an actual property to define variables I'll need to use for that item. Their vars are contained in asm.wizard property where I would dump locally. I would likely have a loop for asm.casterClasses where the foreach will give me the propName. I would use 1 json.get for the name, although I could skip that part too and just put "className" inside the asm.wizard prop which is what I would do if asm.casterClasses was really large... just make an array or list. Using the "dump all" method is only profitable to speed if you have to get json.get around 4 or more times.

My first go around with speed I did something like this:

Code: Select all

{"name1":"value1","name2":"{eval('propName')}"}
where propName pointed to a complex object. That made getting a large complicated structure very fast with json.evaluate, but I found I really didn't need that structure because I could just get the propName directly anyway if I needed a nested value. Also, it made the number of properties very large on a token. If I need to get a lot of similar values, like all my cached links, then I make one big json.object and dump them all locally. I don't know if json.toStrProp will do complex objects. Also, because I use that, I can't use a property string list inside without having to do some conversions, then it gets sketchy as to which can be an html entity and a value separator. So, I just use a custom separator like "|" or "," depending on the type of data it holds.

User avatar
aliasmask
RPTools Team
Posts: 9024
Joined: Tue Nov 10, 2009 6:11 pm
Location: Bay Area

Re: Spell Reference D&D 3.5/Pathfinder SRD

Post by aliasmask »

Here's an example of how I do the body of my spell description page. I added the comment for the demo, but I delete them in final.
spellPage

Code: Select all

@@spellPage
<!-- get needed spell data -->
[H: spellData = am.sd.glp(arg(0))]
[H: spellData = replace(spellData,";",strformat("&#%s",59))]
[H: varsFromStrProp(json.toStrProp(spellData))]
<!-- get all cached links for menu indexes -->
[H: menuLinks = am.sm.glp("link_menu_index")]
[H: menuLinks = replace(menuLinks,";",strformat("&#%s",59))]
[H: varsFromStrProp(json.toStrProp(menuLinks))]
<!-- update school to link -->
[H: school = eval(strformat("link.menu.school.%s",am.sl.cleanVar(school)))]
<!-- update subschool to list of links with some formatting -->
[H: subschoolList = ""]
[H, foreach(item,subschool): subschoolList = listAppend(subschoolList,eval(strformat("link.menu.subschool.%s",am.sl.cleanVar(item))))]
[H, if(subschool != ""): subschool = strformat("(%s)",subschoolList)]
<!-- update descriptor to list of descriptors with formatting -->
[H: descriptorList = ""]
[H, foreach(item,descriptor): descriptorList = listAppend(descriptorList,eval(strformat("link.menu.descriptor.%s",am.sl.cleanVar(item))))]
[H, if(descriptor != ""): descriptor = strformat("[%s]",descriptorList)]
<!-- create header -->
[H: header2 = strformat("%s%s%s",school,subschool,descriptor)]
<!-- update spellLevel to link list -->
[H: spellLevelList = ""]
[H: casterClassList = lower(am.sl.glp("list.classes"))]
[H: domainClassLevelList = spellLevel]
[H, foreach(domainClass,domainClassLevelList), code: {
   [H: level = listGet(domainClass,1," ")]
   [H: domainClassName = listGet(domainClass,0," ")]
   [H: domainClassNameProp = am.sl.cleanVar(domainClassName)]
   [H: domainClassLevelProp = am.sl.cleanVar(domainClass)]
   [H, if(json.contains(casterClassList,domainClassNameProp)): 
      spellLevelList = listAppend(spellLevelList,eval(strformat("link.menu.class.%s.%s",domainClassNameProp,domainClassLevelProp)));
      spellLevelList = listAppend(spellLevelList,eval(strformat("link.menu.domain.%s.%s",domainClassNameProp,domainClassLevelProp)))
   ]
}]
[H: spellLevel = spellLevelList]
<!-- output template.  Put in to array because not every line is outputted and row class alternates -->
[H: spellTemplate = json.append("",
   "<tr><td class='header' colspan=2>%{spellName}</td></tr>",
   "<tr><td class='header2' colspan='2'>%{header2}</td></tr>",
   "<tr valign=top><td class='%{tRow}'>Level:</td><td class='%{tRow}'>%{spellLevel}</td></tr>",
   "<tr valign=top><td class='%{tRow}'>Components:</td><td class='%{tRow}'>%{spellComponents}</td></tr>",
   "<tr valign=top><td class='%{tRow}'>Casting Time:</td><td class='%{tRow}'>%{castTime}</td></tr>",
   "<tr valign=top><td class='%{tRow}'>Range:</td><td class='%{tRow}'>%{range}</td></tr>",
   "<tr valign=top><td class='%{tRow}'>Target:</td><td class='%{tRow}'>%{target}</td></tr>",
   "<tr valign=top><td class='%{tRow}'>Area:</td><td class='%{tRow}'>%{area}</td></tr>",
   "<tr valign=top><td class='%{tRow}'>Effect:</td><td class='%{tRow}'>%{effect}</td></tr>",
   "<tr valign=top><td class='%{tRow}'>Duration:</td><td class='%{tRow}'>%{duration}</td></tr>",
   "<tr valign=top><td class='%{tRow}'>Saving Throw:</td><td class='%{tRow}'>%{savingThrow}</td></tr>",
   "<tr valign=top><td class='%{tRow}'>Spell Resistance:</td><td class='%{tRow}'>%{spellResistance}</td></tr>",
   "<tr valign=top><td class='%{tRow}'>Material Components:</td><td class='%{tRow}'>%{materialComponents}</td></tr>",
   "<tr valign=top><td class='%{tRow}'>Focus:</td><td class='%{tRow}'>%{focus}</td></tr>",
   "<tr valign=top><td class='%{tRow}'>XP Cost:</td><td class='%{tRow}'>%{xpCost}</td></tr>",
   "<tr valign=top><td class='%{tRow}'>Reference:</td><td class='%{tRow}'>%{reference}</td></tr>",
   "<tr><td colspan=2>%{description}</td></tr>"
)]
<!-- list of fields to be printed -->
[H: varList = "spellName,header2,spellLevel,spellComponents,castTime,range,target,area,effect,duration,savingThrow,spellResistance,materialComponents,focus,xpCost,reference,description"]
<!-- build output to variable -->
[H: tRow = "odd"]
[H: output = "[]"]
[H, foreach(line,varList), code: {
   [H, if(eval(line) != ""), code: {
      [H: item = listFind(varList,line)]
      [H: varStr = json.get(spellTemplate,item)]
      [H: textLine = strformat(varStr)]
      [H: output = json.append(output,textLine)]
      [H, if(tRow == "odd"): tRow = "even"; tRow = "odd"]
   };{}]
}]
[H: macro.return = json.toList(output,"")]
 

User avatar
aliasmask
RPTools Team
Posts: 9024
Joined: Tue Nov 10, 2009 6:11 pm
Location: Bay Area

Re: Spell Reference D&D 3.5/Pathfinder SRD

Post by aliasmask »

Yikes! I was thinking of adding the spell compendium spells. I did not realize there were 1001 spells in that book. No wonder I'm always finding new spells I haven't heard of. Add the 606 from PHB and that's a lot of spells. Imarv has many of the fields (in his own abbr code) for about 692(?) of them, so that will make it a little easier.

Anyone want to volunteer for some db work? If we break this up in to chunks of 100, it shouldn't be so bad. PM me if your interested and I'll give you the CSV file parts to fill in. You can see an example of the fields needed in a post above with "acid arrow" button.

I may just post the broken up CSV files here for ppl to download and call dibbs.

User avatar
Azhrei
Site Admin
Posts: 12086
Joined: Mon Jun 12, 2006 1:20 pm
Location: Tampa, FL

Re: Spell Reference D&D 3.5/Pathfinder SRD

Post by Azhrei »

If someone has them in electronic format, I can probably generate the JSON. I won't have much time to work on this until the end of the week at the soonest though.

Or I can post the Perl scripts I have that do the work and someone else can take a shot at it.

My scripts are rather complex because I broke them up into multiple steps. First step is to download the HTML of the spell descriptions into separate files (such as the d20srd.org pages). Step two is to extract out interesting information and write that to new files. Third step is to convert those files into macros that use [frame:] to display the content.

Obviously, that last step would be changed to generate JSON instead. Or perhaps even the second step could do that but whoever does the work would need to learn a bit about XSL since that's what I used for step two.

I have a separate script that takes a single large document full of spell descriptions and creates JSON from it. This was designed to process a data dump from DM Genie. But it's very dependent on the format of the data coming in (I believe I have versions for both RTF and text as input formats).

Maybe that'll help, maybe not... :)

I also have a web-based conversion tool. (Click the link, then click on Spell2JSON, then on byname-prd-main.html.) This web page will load an HTML page that is a dump from d20pfsrd.org. Basically, you can visit d20pfsrd and dump the spell database in HTML format. The page above will read that HTML and produce JSON objects, one per spell. It's not very configurable as-is; you'd need to edit prd_spells_data.js and tell it which fields you want in the JSON object. I can offer guidance on making any changes you need...

User avatar
aliasmask
RPTools Team
Posts: 9024
Joined: Tue Nov 10, 2009 6:11 pm
Location: Bay Area

Re: Spell Reference D&D 3.5/Pathfinder SRD

Post by aliasmask »

I spent several hours, and still not done yet, getting all the short descriptions for all the Spell Compendium Spells, so that is nearly done. Just a little more formatting and clean up and that will be done.

I'll merge that data with the data I have from IMarv's site. I'll have to clean that data up too to be more like the spell book (he uses a bunch of abbreviated codes for some of the fields). I'll post what I have probably later tonight.

As long as I have a json object with all the fields a couple of posts up, then my library builds everything from there automatically. I don't have to worry about making domain, school, subschool or descriptor lists because it parses it for me and creates the spell links to be cached.

Just to keep things speedy, I may have to make a minor tweak, but I'll actually have the spell compendium on two tokens and the indexes to the compendium on 2 tokens. I'll have 6 tokens total just for spells and spell lists, then one for the manager. It's possible to put it all on one token, but that would be about 3500 properties and I'm pretty sure MT will blow up if I did that. I plan to use the same manager to look up skills and feats with little effort. I spent most of my time on design and it's pretty portable for whatever kind of db you want. You just have to create an import, display page (if one doesn't exist) and give the new lib a prefix and the rest is automatic (more or less). So, the SRD spells start with spell.bane and the compendium would be spell2.armorofdarkness or spell3.tremorsense just to give you an idea.

User avatar
aliasmask
RPTools Team
Posts: 9024
Joined: Tue Nov 10, 2009 6:11 pm
Location: Bay Area

Re: Spell Reference D&D 3.5/Pathfinder SRD

Post by aliasmask »

VOLUNTEERS WANTED!

Okay, I have most of the data cleaned up and in one big XLS file. I'm going to start working on the first 500 doing the long descriptions and filling in the few spells missing stats.

The last 500 have much more data missing and will take you longer, but this is how I have it split up:

Code: Select all

Absorb Weapon                 to    Blinding Spittle         -aliasmask (WIP)
Blindsight                    to    Crumble                  -aliasmask (WIP)
Curse of Ill Fortune          to    Energized Shield         -aliasmask (WIP)
Energized Shield, Lesser      to    Glass Strike             -aliasmask (WIP)
Glowing Orb                   to    Kelpstrand               -aliasmask (WIP)
Kiss of the Vampire           to    One With the Land
Ooze Puppet                   to    Repair Serious Damage
Replicate Casting             to    Sonic Snap
Sonic Weapon                  to    Tidal Surge              -neofax(done)
Tigers Tooth                  to    Zone of Revelation
To avoid duplication of work, please post that you're working on a section and I'll update this post to show the current status and have the latest updated file.
tips for formating long description
Here's the process I'm going through to get the the long descriptions. First I have a PDF of the Spell Compendium with the OCR applied to it. That way I can copy text. Using Notepad++ I have TextFX installed as a plug-in through the plug-in manager.
  • Copy text to blank notepad++
  • I paste the text to notepad++ (several spells descriptions)
  • I then put an @ at the beginning of the first paragraph. This paragraph is the flavor description and I will have a a different format for this paragraph.
  • I do the same for the other paragraphs, but I put a # at the beginning.
  • I use TextFX | Edit - unwrap text
  • I hit ctrl-H to copy and replace. First regex is "\n@([^#]*)" and replacement text is "<p><i>\1</i></p>".
  • The second regex is "#([^#@]*)" and replacement text is "<p>\1</p>"
  • Each spell is now formatted and on it's own line.
Adding the html adds to the readability of the description and allows me to add CSS directed at the formats (like paragraph indenting). Once you get going on the data, it's not so bad. The regex is saved for each one after doing it the first time.
Attachments
Master Worksheet for Spell Compendium.zip
(123.53 KiB) Downloaded 115 times
Last edited by aliasmask on Tue Nov 23, 2010 11:41 am, edited 1 time in total.

neofax
Great Wyrm
Posts: 1694
Joined: Tue May 26, 2009 8:51 pm
Location: Philadelphia, PA
Contact:

Re: Spell Reference D&D 3.5/Pathfinder SRD

Post by neofax »

I can take the last two Sonic to Zone bit. I need the material and the Notepad++ userDefinedLang.xml if you have it(I am currently using a minor change to the one you have in your signature).

BTW, are you doing the APG stuff, or do you want me to do it using this format?

Post Reply

Return to “Drop-In Macro Resources”