A few XML Questions with .rptok

Thoughts, Help, Feature Requests, Bug Reports, Developing code for...

Moderators: dorpond, trevor, Azhrei

Forum rules
PLEASE don't post images of your entire desktop, attach entire campaign files when only a single file is needed, or generally act in some other anti-social behavior. :)
Imper1um
Cave Troll
Posts: 92
Joined: Mon Sep 28, 2009 7:46 am

A few XML Questions with .rptok

Post by Imper1um »

I'm looking at modifying the XML of an .rptok because I'm working on a program that will create the XML and zip up the files just like an .rptok file for use in Maptools.

Unfortunately, I do not know a few things, so I'm wondering a few things:

1. Is the order of the XML required to be in the order? I understand that all keys need to be there, but does the XML need to be in order, such as:
  1. x
  2. y
  3. z
  4. anchorX
  5. anchorY
or, can I reorder it how I want, so it could be:
  1. z
  2. x
  3. anchorX
  4. anchorY
  5. y
and it will not cause any issues?

2. What is the difference between id.baGUID and imageAssetMap.entry.net.rptools.lib.MD5Key.id? They look like two different structured keys, one being an alpha key (id.baGUID) and the other looks like an alphanumeric key (MD5Key). I did tie them down to two different assets in the assets folder, so I know that, but can I make up my own baGUID (Using C#'s Guid.NewGuid()), or does it have to be something specific? If it is something specific, what library do you use to generate the key?

3. Same question for portraitImage.id?

4. Can I not include states in the state fields and it will be okay?

5. In the propertymap.store, is it okay to not include properties?

Thanks for your help. I should be showing off what I'm working on soon. :)

User avatar
Azhrei
Site Admin
Posts: 12086
Joined: Mon Jun 12, 2006 1:20 pm
Location: Tampa, FL

Re: A few XML Questions with .rptok

Post by Azhrei »

Imper1um wrote:I'm looking at modifying the XML of an .rptok because I'm working on a program that will create the XML and zip up the files just like an .rptok file for use in Maptools.

Unfortunately, I do not know a few things, so I'm wondering a few things:

1. Is the order of the XML required to be in the order? I understand that all keys need to be there, but does the XML need to be in order, such as:
Order is not important in general. (I can't think of any instances where order is important off-hand, but it is potentially possible as some complex objects may need information from one field in order to properly interpret the next field. Nothing like that currently, though.)

The XStream library we use is documented here and while much of it is Java-related you should be able to skip around that in order to glean what you need about the XML format.
2. What is the difference between id.baGUID and imageAssetMap.entry.net.rptools.lib.MD5Key.id? They look like two different structured keys, one being an alpha key (id.baGUID) and the other looks like an alphanumeric key (MD5Key). I did tie them down to two different assets in the assets folder, so I know that, but can I make up my own baGUID (Using C#'s Guid.NewGuid()), or does it have to be something specific? If it is something specific, what library do you use to generate the key?
Currently you can make up your own as long as they are unique. The current GUID objects are constructed from a hash based on the IP address and some random data; the actual value isn't important -- they are merely used as "Globally Unique IDentifiers" within the program.
4. Can I not include states in the state fields and it will be okay?
Um, don't know off-hand. You should be able to leave them out and the XStream library will treat them as null when reading them in (if you leave out the definition) or the Java code will treat them as null when it tries to retrieve a particular key, i.e. state name. However, I don't know off-hand if the Java code will convert null into false automatically.
5. In the propertymap.store, is it okay to not include properties?
I would assume so, for the same reason as #4. I would suggest trying it. And as new builds of MapTool come out, be sure to keep an eye on the ChangeLog to ensure your code doesn't have any problem with it.

You will likely want to look at the version field in the properties.xml file as well, since that tells you how old the token file itself is. Right now MapTool doesn't actually use the version number for anything when reading in the token, but in the future it may be needed to apply automatic conversions. (This is actually a bug in MT; it does do conversions when reading in campaigns, but doesn't do any when reading in tokens. Theoretically it's possible that a token in a campaign might be read in just fine but the same token when stored as an RPTOK might fail while being read. :( There's no simple way to fix this in the current version however. I expect to take care of it in 1.4.)
Thanks for your help. I should be showing off what I'm working on soon. :)
Cool. Hopefully it's cross-platform but even if it's not I'm sure there'll be people interested in additional automation tools... :)

Akodo Makama
Giant
Posts: 249
Joined: Mon Apr 20, 2009 9:31 pm

Re: A few XML Questions with .rptok

Post by Akodo Makama »

I made a script a while back that striped the:

propertyMap
propertyType
state
hasSight
sightType
macroPropertiesMap
speechMap

XML entries from the .rptok file, and b70 was able to load them just fine. I had some tokens that were acting corrupted, and didn't want to bother tracking down which of those entries were at fault, so I just ripped them all out.

User avatar
Azhrei
Site Admin
Posts: 12086
Joined: Mon Jun 12, 2006 1:20 pm
Location: Tampa, FL

Re: A few XML Questions with .rptok

Post by Azhrei »

Yep, that's certainly possible.

The tough one is the propertyMap in b63+ (I think that's the version) where variable and property names were no longer case-sensitive. That required saving the lowercase version, plus the real name (without case changed, for output display purposes), plus the value. But that's a nested structure so the XML is pretty ugly and would be a pain to create new entries with anything other than a real* programming language.

.

*Meaning a language with string functions. :)

Imper1um
Cave Troll
Posts: 92
Joined: Mon Sep 28, 2009 7:46 am

Re: A few XML Questions with .rptok

Post by Imper1um »

Heh, that's why C# has XmlWriter. It makes it easy to make this, however, I'm at an...issue now. I believe I've gotten the xmls correct, but I need a little help to find out why this will not import correctly. Since I don't have a full callstack, I can't tell where I'm building the xmls (or whatever) incorrectly.

I've compared the xmls, and now I'm wondering if the order of the xml matters (that would suck, really).
Attachments
Balor.rptok
(12.52 KiB) Downloaded 30 times

User avatar
Azhrei
Site Admin
Posts: 12086
Joined: Mon Jun 12, 2006 1:20 pm
Location: Tampa, FL

Re: A few XML Questions with .rptok

Post by Azhrei »

Imper1um wrote:Heh, that's why C# has XmlWriter. It makes it easy to make this, however, I'm at an...issue now. I believe I've gotten the xmls correct, but I need a little help to find out why this will not import correctly. Since I don't have a full callstack, I can't tell where I'm building the xmls (or whatever) incorrectly.

I've compared the xmls, and now I'm wondering if the order of the xml matters (that would suck, really).
The order of the XML in MapTool should never matter. There was a time way back when the images stored in the assets/ directory inside the rptok (or cmpgn) were XML and that XML required one of the fields to be specified first, but that's been cleaned up and is no longer the case.

Regarding your first paragraph, I'm not sure if this is the issue but my attempt to use unzip on the rptok indicates that the PKZIP format is newer than that supported by my open source tool. I'm thinking that means it's also likely newer than the built-in java.util.Zip library, but I don't know that for a fact. It simply shows up as a "corrupted" ZIP file.*

If that's true, then you may need to do something similar to what I do for the Mac and Linux builds of MapTool: create a ZIP using the tools provided by the system and then add to it. For example, I build the ZIP file on Unix and insert some files with execute permission turned on (something the Windows version of ZIP can't do). Then when I need to build the ZIP file for a new build of MapTool I copy that ZIP file and add to it. That forces the existing entries to have execute permission and requires that added files are compatible with the archive envelope. Not sure if that would be enough in your case, though.


*Here's what the output of unzip looks like:

Code: Select all

mymac2:Desktop frank$ unzip -l Balor.rptok 
Archive:  Balor.rptok
  Length      Date    Time    Name
---------  ---------- -----   ----
    17016  11-19-2010 12:26   content.xml
       90  11-19-2010 12:26   properties.xml
     1392  11-19-2010 12:26   thumbnail
        0  11-19-2010 12:26   assets/
     1392  11-19-2010 12:26   assets/2ef23d3e2a524c318f3759405a00d4ca
     1392  11-19-2010 12:26   assets/2ef23d3e2a524c318f3759405a00d4ca.png
     4669  11-19-2010 12:26   assets/f7431aeb7f4740869d58e89bf07e7fa3
     4669  11-19-2010 12:26   assets/f7431aeb7f4740869d58e89bf07e7fa3.png
---------                     -------
    30620                     8 files
mymac2:Desktop frank$ unzip ../Balor.rptok 
Archive:  ../Balor.rptok
   skipping: content.xml             need PK compat. v4.5 (can do v2.1)
   skipping: properties.xml          need PK compat. v4.5 (can do v2.1)
   skipping: thumbnail               need PK compat. v4.5 (can do v2.1)
   skipping: assets/                 need PK compat. v4.5 (can do v2.1)
   skipping: assets/2ef23d3e2a524c318f3759405a00d4ca  need PK compat. v4.5 (can do v2.1)
   skipping: assets/2ef23d3e2a524c318f3759405a00d4ca.png  need PK compat. v4.5 (can do v2.1)
   skipping: assets/f7431aeb7f4740869d58e89bf07e7fa3  need PK compat. v4.5 (can do v2.1)
   skipping: assets/f7431aeb7f4740869d58e89bf07e7fa3.png  need PK compat. v4.5 (can do v2.1)
mymac2:Desktop frank$ _
By the way, you don't need two copies of the image assets and you certainly shouldn't have both of them be images! Review what the MapTool-generated rptok file contents look like...

Imper1um
Cave Troll
Posts: 92
Joined: Mon Sep 28, 2009 7:46 am

Re: A few XML Questions with .rptok

Post by Imper1um »

So you guys have an outdated package? You might want to look into that (newer engines have superior compression).

Certainly if you're behind by 2 whole major revisions, you're behind by a huge margin. You might want to look into 7zip. :)

In the mean time, I'll have to look into trying to downgrade my version, or at least provide a better compatibility with older versions of zipping utilities (note that if you use the latest zip of Windows 7, it will produce the same issue).

Also, the reason why there are two image files, one with and one without a extension is because that's how the .rptok that I extracted showed up.

User avatar
Azhrei
Site Admin
Posts: 12086
Joined: Mon Jun 12, 2006 1:20 pm
Location: Tampa, FL

Re: A few XML Questions with .rptok

Post by Azhrei »

Imper1um wrote:So you guys have an outdated package? You might want to look into that (newer engines have superior compression).
Perhaps you didn't notice, but that was a command line utility. I'm not saying that the Java library is that old, only that the ZIP you created couldn't be read with an open source command line utility.
Certainly if you're behind by 2 whole major revisions, you're behind by a huge margin. You might want to look into 7zip. :)
Hm, I'll check to see if there's a open source library written in Java...
(note that if you use the latest zip of Windows 7, it will produce the same issue).
Heh. "Doctor, it hurts when I do *this*!?" "Then don't do that."
Also, the reason why there are two image files, one with and one without a extension is because that's how the .rptok that I extracted showed up.
Nope. Look again. 8) And look inside the file or at the file size...

Imper1um
Cave Troll
Posts: 92
Joined: Mon Sep 28, 2009 7:46 am

Re: A few XML Questions with .rptok

Post by Imper1um »

Azhrei wrote:
Imper1um wrote:So you guys have an outdated package? You might want to look into that (newer engines have superior compression).
Perhaps you didn't notice, but that was a command line utility. I'm not saying that the Java library is that old, only that the ZIP you created couldn't be read with an open source command line utility.
Well, I asked what Java was reading incorrectly. I got an error when importing into MapTool, something about the GroupTool.
Azhrei wrote:
Also, the reason why there are two image files, one with and one without a extension is because that's how the .rptok that I extracted showed up.
Nope. Look again. 8) And look inside the file or at the file size...
Weird...what is that extra file for?

Phergus
Deity
Posts: 7132
Joined: Fri May 12, 2006 8:56 pm
Location: Middle of Nowhere, NM
Contact:

Re: A few XML Questions with .rptok

Post by Phergus »

Imper1um wrote:Well, I asked what Java was reading incorrectly. I got an error when importing into MapTool, something about the GroupTool.
Java isn't doing anything incorrectly. You are creating the token files incorrectly.

Java uses ZLIB for jar file and files created using the ZipFile class. These files use normal Zip deflate/inflate. RPTOK files are created this way by MT and are expected to be in that form if you create them on your own. 7Zip has no problem creating compatible files if you select accept the default options for Zip file.

User avatar
Azhrei
Site Admin
Posts: 12086
Joined: Mon Jun 12, 2006 1:20 pm
Location: Tampa, FL

Re: A few XML Questions with .rptok

Post by Azhrei »

Imper1um wrote:Well, I asked what Java was reading incorrectly. I got an error when importing into MapTool, something about the GroupTool.
Yeah, the drag/drop support makes a few assumptions. If none of them pans out, it errors out in the last block of code -- which is the block that tries to interpret the dropped data as a Group object coming from InitiativeTool. So the part of the error that refers to "Group" is bogus. Unfortunately, it's going to be a pain to redo that section of code because doing it right actually means redoing the parent above it and the parent above that. That will happen in 1.4.
Weird...what is that extra file for?
One is XML metadata, the other is the image.

Encoding the binary image as XML when saving RPTOK or CMPGN files was slowing down MT quite a bit (pre 1.3b66) and requiring it to consume more RAM (to hold the string prior to it being output). Having two separate files allows me to dump the binary image directly without going through the XML stream library and speeds things up quite a bit. In addition, the original XML files couldn't be opened and the image viewed using any of the standard tools (GIMP, Paint.NET, etc) but now the image is stored separately -- and with the proper filename extension for those broken tools and operating systems that require it. I also have plans to turn off the Deflate compression for images but for some reason I couldn't get that to work properly the last time I was in there mucking with the code so I gave up. I'll look at it again when the file I/O stuff is reimplemented for 1.4.

Imper1um
Cave Troll
Posts: 92
Joined: Mon Sep 28, 2009 7:46 am

Re: A few XML Questions with .rptok

Post by Imper1um »

So... you couldn't tell me what I'm doing wrong except for the .xml (which I fixed and I'm still having the same error)?

Meh, I'll try a few things, including making the XML the same order as the original XMLs.

Imper1um
Cave Troll
Posts: 92
Joined: Mon Sep 28, 2009 7:46 am

Re: A few XML Questions with .rptok

Post by Imper1um »

Okay, I figured it out. My zip system isn't creating the archive correctly. I have to find a new utility to zip it up. It isn't rptools fault.

Imper1um
Cave Troll
Posts: 92
Joined: Mon Sep 28, 2009 7:46 am

Re: A few XML Questions with .rptok

Post by Imper1um »

...and then I switch Zip Utilities, and add the zips and...

Viola...

Note that this token was NOT made using RPTools. I created it using a program I'm calling 'Character Bridge'. I'll have more details about it soon.

Zac
Attachments
BalorFinal.rptok
(8.83 KiB) Downloaded 33 times

User avatar
Azhrei
Site Admin
Posts: 12086
Joined: Mon Jun 12, 2006 1:20 pm
Location: Tampa, FL

Re: A few XML Questions with .rptok

Post by Azhrei »

Cool. I'm sure Windows users will approve. :)

I'm going to stick to IMarvinTPA's web site I guess. :|

Post Reply

Return to “MapTool”