Last night I had a rather long batch running. Not surprising, I ran out of disk space just before two o'clock.
When today I prepared to restart with the remaining tasks, I noticed that 15 project files had been reset to 0 bytes, so lost their content. In 5 cases there was a .zsj.backup file with (aproximately) the same time stamp as the project file. I found one where the .zsj.bkp file had zero bytes but the .zsj file survived.
The output was written to another disk. The same disk as where the Windows' temp filder resides. The full temp-folder in itself is a source of problems.
I think that the batch processor would better wait for user intervention in case of space problems. Now it seems to continue with the next batch in the list.
Version 1.04 Build T2021-05-28-1930
Zerene: batch problem
Moderators: rjlittlefield, ChrisR, Chris S., Pau
- iconoclastica
- Posts: 487
- Joined: Sat Jun 25, 2016 12:34 pm
- Location: Wageningen, Gelderland
Zerene: batch problem
--- felix filicis ---
- rjlittlefield
- Site Admin
- Posts: 23625
- Joined: Tue Aug 01, 2006 8:34 am
- Location: Richland, Washington State, USA
- Contact:
Re: Zerene: batch problem
Speaking as the author of Zerene Stacker, I agree completely. The handling of errors in batch mode is not good.
In general, the handling for errors in ZS is basically "call human and keep making progress if possible".
Unfortunately, for some classes of errors ZS makes the mistake of acting like progress is possible when really it is not.
I'm pretty sure that level of elegance is more than I can manage. There are at least dozens of places inside ZS where I/O might fail due to space issues, and no existing code structure that would facilitate retry and continue.
But probably I can do something like adding an option to immediately pause operations after "call human". I've added that to the infamous list of requested features.
--Rik
In general, the handling for errors in ZS is basically "call human and keep making progress if possible".
Unfortunately, for some classes of errors ZS makes the mistake of acting like progress is possible when really it is not.
That sounds like the sort of precise diagnosis and recovery that we might hope for in a disk backup program: wait for more space to be made available, then re-do whatever operation failed due to the shortage, and continue seamlessly from there.iconoclastica wrote: ↑Sat Nov 26, 2022 3:57 am... would better wait for user intervention in case of space problems.
I'm pretty sure that level of elegance is more than I can manage. There are at least dozens of places inside ZS where I/O might fail due to space issues, and no existing code structure that would facilitate retry and continue.
But probably I can do something like adding an option to immediately pause operations after "call human". I've added that to the infamous list of requested features.
--Rik
- iconoclastica
- Posts: 487
- Joined: Sat Jun 25, 2016 12:34 pm
- Location: Wageningen, Gelderland
Re: Zerene: batch problem
It's not a complaint, just an observation
Every user is a tester. On the whole, the batch executor did an excellent job, doing overnight what would have taken me over a week of boring work to do manually.
Every user is a tester. On the whole, the batch executor did an excellent job, doing overnight what would have taken me over a week of boring work to do manually.
--- felix filicis ---
-
- Posts: 1527
- Joined: Mon Jan 15, 2018 9:23 pm
- Contact:
Re: Zerene: batch problem
I have run into this too. I plan to fix it by having C raid with another disk, in the future. I love the batch function, it is so useful for diatom stacks.
- iconoclastica
- Posts: 487
- Joined: Sat Jun 25, 2016 12:34 pm
- Location: Wageningen, Gelderland
Re: Zerene: batch problem
It can be easily fixed by running a script against the (compound) batch xml that makes an extra copy of every project file involved.
--- felix filicis ---