Using non alphanumeric characters in token names/file names.

[Oryan77]

When I first started using MT years ago, I was using commas in file names (Animated Object, Table.rptok) and token names (Animated Object, Table) to better identify creature types to my eyeballs.

Then using features like the Bag of Tricks started causing problems and I was told to rename ALL of my tokens. At the time, it was suggested that I could rename them using a ; instead (Animated Object; Table) and it would probably be fine. Years later, I am getting errors again using tokens with a ; in the token name and I was told that I should not use any non alphanumeric characters.

A user here made a nice program to rename token names within the rptok files. It seems to work but I have not yet updated the thousands of tokens that I have using this program. I believe I will also need to rename the tokens that already exist on my maps, but I'm sure someone can provide a macro for that (I hope).

I am not here to complain because I understand what MT is and how it is developed. So it being provided for free along with the things I like about it is good enough for me. But I am just curious why file names and token names are so finicky in MT and why this is even an issue this day and age? I believe tokens generate a token ID (cause I see errors about token ID), so I don't understand why MT cares about a token name or file name if there is a token ID that can be referenced?

I am by no means a programmer and I don't fully understand that part of MT. As a user though, it is quite odd that we can't name our fantasy character tokens using non alphanumeric symbols. I don't believe I have been restricted from that sort of thing in any software before. And I have been writing in English for so long that it really gives me a headache trying to view lists of identifying names and not being able to distinguish "groups" by using , or ;.

Are we even limited from using an underscore _ within a token name?

I am not complaining here. I'm just genuinely curious about this and I thought I'd ask before I take the leap and edit all of my tokens so they will work properly in MT.

[aliasmask]

Your problem probably arises from the , and ; has special meaning to certain functions and circumstances. IE ; is used to separate string property list while , separates a normal list among other things. You can use _ because it has no other meaning in strings. You can probably also use ":" and () as separators. But even those could cause issues with output if you or the other players have smilies turned on. Actually, that may have been disabled in the newer versions, so that may not be a problem.

[Oryan77]

I'm glad I asked then. Is there any documentation for us non-programmer types to reference that would show what specific characters we could use and which character we absolutely should not use?

This seems like something that could be very helpful for users like myself. It could give us an idea for what is available to use in place of apostrophes in names and similar naming conventions common in fantasy world names.

I did not know a colon or parentheses is acceptable. I would actually prefer to use a colon rather than an underscore. So that's good to know.

[aliasmask]

I'm glad I asked then. Is there any documentation for us non-programmer types to reference that would show what specific characters we could use and which character we absolutely should not use?

This seems like something that could be very helpful for users like myself. It could give us an idea for what is available to use in place of apostrophes in names and similar naming conventions common in fantasy world names.

I did not know a colon or parentheses is acceptable. I would actually prefer to use a colon rather than an underscore. So that's good to know.

Well, it depends on context as to what is safe to use, like how the names are retrieved and used. You CAN use , and ; in the names because MT allows that for token names, but if you want to use certain functions you need to do a little filtering. The main idea is to definitively say that , is not a , but the character &#44 for html output purposes. But to avoid conversions, picking a safe character like _ or : would be easier. Something like ; works fine for most things unless you're doing some string property conversions. This is one reason why I like using jsons because most characters are safe to use, but there are some really useful functions that don't use jsons, so we end up needing to make restrictions and apply filters.

[wolph42]

the core of the issue lies with several issues regarding token ID's. All macros *should* be using id and not names. However e.g. cut and paste or even starting a server resets these. That and there is a severe ID bug where numerical ID's get truncated.
Hence for now we are forced to use names as that's more reliable...at least as long as one does use 'strange' characters.

I've updated the bot recently with an improved diagnose where you can scan all the maps for 'unallowed' characters AND I've added a macro that allows you to search and replace these characters.

[Oryan77]

I've updated the bot recently with an improved diagnose where you can scan all the maps for 'unallowed' characters.

I did use that (Diagnose Map macro, right?) but it identifies tokens using any non alphanumeric characters such as : or - which aliasmask says should be ok to use. Are we able to filter that macro to ignore certain characters?

I've added a macro that allows you to search and replace these characters.

Oh, this I did not notice. I can definitely make use of this macro. What is it called?

On a side note, the handy tool that a user named badsequel created for me back in 2014 http://forums.rptools.net/viewtopic.php ... 57#p255257 does not seem to work on my new PC. I don't know if it is because I am now using Windows 8.1, but I get an error saying, "FATAL ERROR: Could not load python library". I have no idea what it means, so it looks like I may have to drag and drop 2000 tokens into MT and edit the names using wolph42's macro.

[wolph42]

the new diagnose has 3 settings light, medium, heavy. These check for the different characters. You can run the light version to at least filter out the most 'destructive' characters.
And yes for my tool to use, you have to load them into mt. something i did a long time ago. I have one map with 1000+ tokens that i save as a map and load in a new campaign to use. As i have a search function anyway i find what i need that way.

[Oryan77]

the new diagnose has 3 settings light, medium, heavy. These check for the different characters. You can run the light version to at least filter out the most 'destructive' characters.
And yes for my tool to use, you have to load them into mt. something i did a long time ago. I have one map with 1000+ tokens that i save as a map and load in a new campaign to use. As i have a search function anyway i find what i need that way.

I did try each setting on the diagnose macro but even the light setting displayed tokens with : and - as problem tokens. I have not really seen a difference with the light, medium, heavy settings.

Forgive me if I have just overlooked this, but is there an explanation in your manual that explains what the difference is between those 3 settings in the diagnose macro? As a user, I am not really clear "why" I should select among the 3 different options. Especially when I get the same reports when running light or heavy. I'm just not sure what the light selection is supposed to be ignoring.