Page 3 of 4

Re: Redesign for autosaving campaigns (WIP)

Posted: Tue Sep 12, 2017 12:11 am
by taustinoc
Full Bleed wrote:
taustinoc wrote:
Full Bleed wrote:How is an imperfect expiration system somehow *worse* than no backup redundancy at all?
That depends on how much hard drive space you have (granted, most people have a lot, but not everybody), and whether or not you have the expertise to figure out why you've run out (and most people do not have that).
Kind of a soft concern. There are so many ways to address this... and most wouldn't even be that hard.
Absolutely. But it does have to be considered from as many angles as possible.
Full Bleed wrote:But no reason to spend any more time on it... it looks like the "redesign" is going to just be a fix. Which, as mentioned, is fine by me. I think it's an important fix and am glad someone (Az!) has taken the time to do it.
I really do think an auto-save system should be a separate system from crash recovery, and of the two, crash recovery is the more important (and it's already mostly done).

Re: Redesign for autosaving campaigns (WIP)

Posted: Tue Sep 12, 2017 10:57 pm
by Elkad
I can happily get by with a batch file to watch a directory and clone the files in it when they change.
But then I'm used to stuff like that. Your average user isn't.

Doing it in a few stages makes sense.
Make it actually work.
Make it respect campaign names.
Those are the two big ones.
After that, fluffier stuff like multiple copies, deleting old copies when a storage threshold is reached, and similar options can be lower priority.

Re: Redesign for autosaving campaigns (WIP)

Posted: Tue Sep 12, 2017 11:46 pm
by taustinoc
Elkad wrote: After that, fluffier stuff like multiple copies, deleting old copies when a storage threshold is reached, and similar options can be lower priority.
I disagree. Those are fundamental requirements for it to be functional, because those are the things that will cause serious problems that the average user has no chance of figuring out on their own.

For an auto-backup system, all the features have to be working at rollout.

Re: Redesign for autosaving campaigns (WIP)

Posted: Wed Sep 13, 2017 1:26 am
by Azhrei
Elkad wrote:Doing it in a few stages makes sense.
Make it actually work.
Make it respect campaign names.
Those are the two big ones.
Yes, the crash recovery process will do those things in the next release.
After that, fluffier stuff like multiple copies, deleting old copies when a storage threshold is reached, and similar options can be lower priority.
Yeah, we'll come back to autosave/autobackup/autowhatever after the next release. In the meantime, we can be thinking some more about how it should work in all of the different situations. :)

Re: Redesign for autosaving campaigns (WIP)

Posted: Wed Sep 13, 2017 2:06 am
by wolph42
concerning the two instances. Good that only 'server' saves. There is however still a potential issue: during development its not unheard of two have two instances running of mt on one client (NOT connected) so they will both autosave over the other instance. Granted its more obscure, but I for one, when working on my FW I rather often also make changes to the BoT at the same time.

Re: Redesign for autosaving campaigns (WIP)

Posted: Wed Sep 13, 2017 3:02 am
by Full Bleed
wolph42 wrote:There is however still a potential issue: during development its not unheard of two have two instances running of mt on one client (NOT connected) so they will both autosave over the other instance. Granted its more obscure, but I for one, when working on my FW I rather often also make changes to the BoT at the same time.
I mentioned this earlier:
Full Bleed wrote:As for concurrency issues between two MT instances, with the same file name open... I guess you could create a "file open flag" in the autosave directory... and if the second instance sees it, have it disable autosave (and make note of it on the status bar.)

Re: Redesign for autosaving campaigns (WIP)

Posted: Wed Sep 13, 2017 3:18 am
by wolph42
Full Bleed wrote:As for concurrency issues between two MT instances, with the same file name open... I guess you could create a "file open flag" in the autosave directory... and if the second instance sees it, have it disable autosave (and make note of it on the status bar.)
ah sry, was a bit behind on reading so must have overlooked that.

Note that your suggestion won't work, cause that particular 'flag' file will remain there forever after MT crashes...defeating the entire purpose.

As far as I can see, the only way to circumvent this issue is when one instance of MT can directly (so no 'flag files') detect the presence of another instance running and then act accordingly, e.g. [the usual autosave name]_2.cmpgn

Re: Redesign for autosaving campaigns (WIP)

Posted: Wed Sep 13, 2017 4:13 am
by aliasmask
My thought is only 1 instance will be a server, so no need to save non server crashes.

Re: Redesign for autosaving campaigns (WIP)

Posted: Wed Sep 13, 2017 4:15 am
by wolph42
aliasmask wrote:My thought is only 1 instance will be a server, so no need to save non server crashes.
...and if you have two instances running (not connected) how do you decide which one will be 'running the server' ?? Cause the point I'm trying to make is that I often have both BoT and FW campaign open and developing both the same time. If one of the two crashes, what will be the autosave when I start MT again?

Re: Redesign for autosaving campaigns (WIP)

Posted: Wed Sep 13, 2017 5:26 am
by Full Bleed
wolph42 wrote:Note that your suggestion won't work, cause that particular 'flag' file will remain there forever after MT crashes...defeating the entire purpose.
I think there is some logic that can work around that... but it was originally part of some other logic that won't be in there.


However, I think that there is a simpler method when you've only got ONE autosave file called "AutoSave.cmpgn" (which is what I think we're getting.)

When an AutoSave is written have it record the Date Modified. When it attempts to create another AutoSave have it looks at the Date Modified. If it matches the date it recorded last time it can AutoSave. Otherwise it skips the AutoSave knowing that something else created the save. This can be noted int he Status bar.

The first AutoSave won't have a Date Modified field to look at. But if there is already an AutoSave it knows it didn't create it. So it skips the AutoSave.

Re: Redesign for autosaving campaigns (WIP)

Posted: Wed Sep 13, 2017 11:35 am
by wolph42
Full Bleed wrote:When an AutoSave is written have it record the Date Modified. When it attempts to create another AutoSave have it looks at the Date Modified. If it matches the date it recorded last time it can AutoSave. Otherwise it skips the AutoSave knowing that something else created the save. This can be noted int he Status bar.

The first AutoSave won't have a Date Modified field to look at. But if there is already an AutoSave it knows it didn't create it. So it skips the AutoSave.
I think that there are some pitfall in that method, but apart from that: it would mean only one instance of MT will be 'autosaved' the other not, again not the most preferable outcome.

Re: Redesign for autosaving campaigns (WIP)

Posted: Wed Sep 13, 2017 12:06 pm
by Azhrei
The crash recovery process will handle having two unrelated MT server instances running. It won't use timestamps.

Each program that is running, whether on Windows or Unix, will have a unique ID number assigned to it. On Windows it's called the task ID and on Unix it's called the process ID.

This unique ID number can be part of the filename, thus ensuring that both instances can create recovery files without any conflicts.

When MT closes down cleanly, it'll delete it's recovery file.

When MT crashes, the recovery file will still be. So at startup, if there are any recovery files, the user gets a popup dialog that shows what the names are and will be asked if any of them should be loaded. (There could be more than one, because multiple instances of MT could all crash simultaneously.) Any recovery file that is loaded is renamed to use the ID of the new MT instance that just loaded the file.

Re: Redesign for autosaving campaigns (WIP)

Posted: Wed Sep 13, 2017 1:27 pm
by wolph42
sounds good! What happens when you want to save both autosaves? Or better put: what does a user need to do to save both? (assuming that after choosing one, the directory will be cleaned up. ).
This also thus means that if you start a second instance (after a while) you will get a 'previous crash report' (same as it is now) as an active MT autosave is detected (of the first running instance). I guess that autoave will be deleted when you start the 2nd instance, but any saves after that will remain. Am I correct in that?

Re: Redesign for autosaving campaigns (WIP)

Posted: Wed Sep 13, 2017 2:24 pm
by taustinoc
What happens if there's an instance of MT running, with a recovery autosave, and a second instance sees that, asks if you want to recover it, and you say yes? Does the second instance rename it to match it's (the second instance) unique ID number? What, then, does the first instance do?

And what happens if that second instance tries to recover the file while the first instance is writing to it?

Is there a way for MT to detect if other instances are running and recover their unique ID number so that it knows an existing file isn't from a crash?

Re: Redesign for autosaving campaigns (WIP)

Posted: Wed Sep 13, 2017 4:07 pm
by Full Bleed
wolph42 wrote:
Full Bleed wrote:When an AutoSave is written have it record the Date Modified. When it attempts to create another AutoSave have it looks at the Date Modified. If it matches the date it recorded last time it can AutoSave. Otherwise it skips the AutoSave knowing that something else created the save. This can be noted int he Status bar.

The first AutoSave won't have a Date Modified field to look at. But if there is already an AutoSave it knows it didn't create it. So it skips the AutoSave.
I think that there are some pitfall in that method
Not sure what they are.
but apart from that: it would mean only one instance of MT will be 'autosaved' the other not, again not the most preferable outcome.
Yes, it means that there would be only 1 autosave (like we have now). But that's what it seemed like we were getting because all methods to create multiple autosaves was taken off the table.

I think the date modified method should work to create 1 autosave tagged to the first instance of MT running.

*If* we used campaign file names in the autosave file, the main issues would be the corner cases of:

A) Changing the file name during play or development.
Solution: A static backup/recovery/autosave name per campaign file that doesn't change with a changing file name.

B) Running two instances of the same file name.
Solution: I'd suggest a user create a separate data directory to manage their separate instances of the SAME file. Using the Date-Modified method, the user would be notified (via the Status Bar) that their second instance wasn't autosaving and could take steps to create a second data directory to rectify the issue.