Re: Redesign for autosaving campaigns (WIP)
Posted: Sun Sep 10, 2017 6:07 pm
Alright, so here are some issues...
Full Bleed: I would rename the whole process to "Auto Backup" and have Crash Recovery simply draft off of the Auto Backup feature as I suggested above.
This isn't 100% clear to me, but how about this:
The first/most recent autosave file will be called recovery.cmpgn, then older versions of that file will be named autosave-number.cmpgn.
If the interval specified in Preferences is non-zero, the former file will be created at the given interval. A second field in Preferences will control how many should be created and if the user selects 1, they only get that file; if they select >1, the previous file is renamed as described above.
what do you think? How would you change that?
Being able to manually set the directory would be nice. Being able to set it to a faster, safer, drive would be a big plus.
What's the best way to do this? If we store the directory name in the Preferences, it would apply to all campaigns. If we store it in the campaign, it would be specific to each campaign. I see pros and cons each way, but I don't think it's a good idea to provide options for both! (For example, specify the default directory in Preferences and then let a specific campaign override that default.) I'm with Craig on something like this -- too many options is just confusing and besides, how many people would actually use this feature? I know I wouldn't.
Elkad: Number of autosaves should be Per Campaign File, default to something low-ish (4?), but be fully user configurable. If I want 99 autosaves, I want 99 autosaves. Maybe I want to use it for version control.
Well, you might want to use it for version control, but then you're using it the wrong way.
There are plenty of other ways to implement version control and that's not what we're attempting to do with this. Besides, MT automatically taking snapshots periodically doesn't give you any clue where you were at when the autosave occurred.
I think the Auto Backup names should be based on the campaign names so this wouldn't be a problem.
Then you won't get any autosaves until you've manually saved the campaign at least once, because until then there is no campaign name. You would get your recovery file, but that would be it (and it wouldn't work if you have two instances of MT open). Is that what you want?
Give me a checkbox for (maptool/autosave/) or (campaignfilelocation/autosave/). And an option to put in yet another path if I want.
This is what I'm thinking, too. If the autosaves were under .maptool/autosaves/ and each campaign had it's own subdirectory under that, then you'd be able to tell at a glance which campaigns had autosaves and which didn't. It would be easy to clean up as well ("We finished TombOfHorrors, so I can delete everything under that directory.") and make it easy to store them somewhere else (using a symbolic link for the entire .maptool/autosaves/ directory, for example).
The problem is that we don't have a campaign name until the user does their first save. I'm thinking we should use some generic string in that case (like "unknown" + task ID), so that multiple instances of MT don't step on each other's toes. (I doubt anyone is going to have two unsaved campaigns working simultaneously, but if we don't account for it in the code then users will simply report problems that occur as bugs. Much better to consider these things and include the necessary if-statements or whatever to prevent them in the first place. )
And, yes, this will inevitably lead to a request for an "Auto Clean-Up" feature. With something like an option to "delete backups more than X months old" and maybe a size warning if the backup directory was exceeding a certain size.
Those types of options will likely never be implemented as part of MT! I have no issue adding a count limit (no more than 10, for example), but adding time or size expirations means people complaining if MT doesn't delete something even though they haven't run MT for days or weeks. The counter limit is easy to implement and is reliable and predictable.
It'll be the user's responsibility to manage their permissions and performance with regard to using an optional network share location.
Yeah, that would be my inclination, but Windows is pretty brain-dead when it comes to opening files.
Warn (of the collision), Skip (the new save), Inform (that the save didn't happen and that changes should be made to address the issue.)
That sounds like a plan.
I'd say the interval should start after the save finishes.
Okay.
As for saves taking so long that they collide with other saves... there should probably be a timeout that is shorter than the interval (and, as above: warn, skip, inform.)
Hm, good plan. There's no reason that an overlapping autosave couldn't produce the same warn/skip/inform sequence. (And I'm sure Jamz will want to create an "overlapping save" event that will invoke a macro! )
wolph42: I *did* notice the autosaves. Basically entire MT grinds to a halt, quite expectantly and remains unresponsive for 30s or so and then everything works again.
Wow. I can tell you're going to be one of the prime testers, then! I don't see why it would be so bad, but there are a few things that I can do to help ameliorate it -- dropping thread priority, intermittently adding yield(), and maybe other stuff. (See this page if you're interested in reading about some of the complexities involved.)
Jagged: In my opinion the best auto backup (or crash recovery if you prefer) services work on a count down. The countdown starts after a modification is made and is reset by a manual save.
I agree!! But unfortunately, MT doesn't have a single point at which to determine that "something has changed". It will need something like that in the future, though, since we want token data to eventually be split off from the image that appears on the screen. If we implement that properly, we can send only the information that changes from client<->server and save a lot of bandwidth/effort. It can also eliminate some of our current "one token updated on two systems" concurrency issues.
RPTroll: Some sort of notification that an autosave is in progress as well as how long it is taking. (bottom of screen?)
If Autosaves are taking longer than X seconds, somehow notify the GM to let them decide what autosave features to enable. (could be a tooltip on the indicator above)
While new autosave functionality is being worked out - a timer to the next autosave would be handy (could be combined with the first line above).
Wow, I like all of these ideas!
I can easily arrange a notification to appear in the chat window, but I think a progress bar in the status bar at the bottom of the window would be much better. I don't think I can accurately estimate a percentage while the save is occurring, though. At the time the save happens, I don't know what the file size will be, nor do I know how many components will need to be saved, since they are saved on the fly as they are referenced by the maps. I suppose a bar that swipes left and right will be enough, since you'll know an autosave is running. (Maybe the actual file writing can be changed in the future so that a true percentage could be calculated.)
The progress bar area could include some status information when the autosave is not running, like "how long to the next save, how long did the last save take, how much space did the last save consume."
Wow, good stuff everyone. Thanks for the input.
(I'm hunkered down for Hurricane Irma. Don't worry, though; I have my off-site Git repositories of MapTool code fully synced. )
Full Bleed: I would rename the whole process to "Auto Backup" and have Crash Recovery simply draft off of the Auto Backup feature as I suggested above.
This isn't 100% clear to me, but how about this:
The first/most recent autosave file will be called recovery.cmpgn, then older versions of that file will be named autosave-number.cmpgn.
If the interval specified in Preferences is non-zero, the former file will be created at the given interval. A second field in Preferences will control how many should be created and if the user selects 1, they only get that file; if they select >1, the previous file is renamed as described above.
what do you think? How would you change that?
Being able to manually set the directory would be nice. Being able to set it to a faster, safer, drive would be a big plus.
What's the best way to do this? If we store the directory name in the Preferences, it would apply to all campaigns. If we store it in the campaign, it would be specific to each campaign. I see pros and cons each way, but I don't think it's a good idea to provide options for both! (For example, specify the default directory in Preferences and then let a specific campaign override that default.) I'm with Craig on something like this -- too many options is just confusing and besides, how many people would actually use this feature? I know I wouldn't.
Elkad: Number of autosaves should be Per Campaign File, default to something low-ish (4?), but be fully user configurable. If I want 99 autosaves, I want 99 autosaves. Maybe I want to use it for version control.
Well, you might want to use it for version control, but then you're using it the wrong way.
There are plenty of other ways to implement version control and that's not what we're attempting to do with this. Besides, MT automatically taking snapshots periodically doesn't give you any clue where you were at when the autosave occurred.
I think the Auto Backup names should be based on the campaign names so this wouldn't be a problem.
Then you won't get any autosaves until you've manually saved the campaign at least once, because until then there is no campaign name. You would get your recovery file, but that would be it (and it wouldn't work if you have two instances of MT open). Is that what you want?
Give me a checkbox for (maptool/autosave/) or (campaignfilelocation/autosave/). And an option to put in yet another path if I want.
This is what I'm thinking, too. If the autosaves were under .maptool/autosaves/ and each campaign had it's own subdirectory under that, then you'd be able to tell at a glance which campaigns had autosaves and which didn't. It would be easy to clean up as well ("We finished TombOfHorrors, so I can delete everything under that directory.") and make it easy to store them somewhere else (using a symbolic link for the entire .maptool/autosaves/ directory, for example).
The problem is that we don't have a campaign name until the user does their first save. I'm thinking we should use some generic string in that case (like "unknown" + task ID), so that multiple instances of MT don't step on each other's toes. (I doubt anyone is going to have two unsaved campaigns working simultaneously, but if we don't account for it in the code then users will simply report problems that occur as bugs. Much better to consider these things and include the necessary if-statements or whatever to prevent them in the first place. )
And, yes, this will inevitably lead to a request for an "Auto Clean-Up" feature. With something like an option to "delete backups more than X months old" and maybe a size warning if the backup directory was exceeding a certain size.
Those types of options will likely never be implemented as part of MT! I have no issue adding a count limit (no more than 10, for example), but adding time or size expirations means people complaining if MT doesn't delete something even though they haven't run MT for days or weeks. The counter limit is easy to implement and is reliable and predictable.
It'll be the user's responsibility to manage their permissions and performance with regard to using an optional network share location.
Yeah, that would be my inclination, but Windows is pretty brain-dead when it comes to opening files.
Warn (of the collision), Skip (the new save), Inform (that the save didn't happen and that changes should be made to address the issue.)
That sounds like a plan.
I'd say the interval should start after the save finishes.
Okay.
As for saves taking so long that they collide with other saves... there should probably be a timeout that is shorter than the interval (and, as above: warn, skip, inform.)
Hm, good plan. There's no reason that an overlapping autosave couldn't produce the same warn/skip/inform sequence. (And I'm sure Jamz will want to create an "overlapping save" event that will invoke a macro! )
wolph42: I *did* notice the autosaves. Basically entire MT grinds to a halt, quite expectantly and remains unresponsive for 30s or so and then everything works again.
Wow. I can tell you're going to be one of the prime testers, then! I don't see why it would be so bad, but there are a few things that I can do to help ameliorate it -- dropping thread priority, intermittently adding yield(), and maybe other stuff. (See this page if you're interested in reading about some of the complexities involved.)
Jagged: In my opinion the best auto backup (or crash recovery if you prefer) services work on a count down. The countdown starts after a modification is made and is reset by a manual save.
I agree!! But unfortunately, MT doesn't have a single point at which to determine that "something has changed". It will need something like that in the future, though, since we want token data to eventually be split off from the image that appears on the screen. If we implement that properly, we can send only the information that changes from client<->server and save a lot of bandwidth/effort. It can also eliminate some of our current "one token updated on two systems" concurrency issues.
RPTroll: Some sort of notification that an autosave is in progress as well as how long it is taking. (bottom of screen?)
If Autosaves are taking longer than X seconds, somehow notify the GM to let them decide what autosave features to enable. (could be a tooltip on the indicator above)
While new autosave functionality is being worked out - a timer to the next autosave would be handy (could be combined with the first line above).
Wow, I like all of these ideas!
I can easily arrange a notification to appear in the chat window, but I think a progress bar in the status bar at the bottom of the window would be much better. I don't think I can accurately estimate a percentage while the save is occurring, though. At the time the save happens, I don't know what the file size will be, nor do I know how many components will need to be saved, since they are saved on the fly as they are referenced by the maps. I suppose a bar that swipes left and right will be enough, since you'll know an autosave is running. (Maybe the actual file writing can be changed in the future so that a true percentage could be calculated.)
The progress bar area could include some status information when the autosave is not running, like "how long to the next save, how long did the last save take, how much space did the last save consume."
Wow, good stuff everyone. Thanks for the input.
(I'm hunkered down for Hurricane Irma. Don't worry, though; I have my off-site Git repositories of MapTool code fully synced. )