Resume a cryosparc job

Features and Functionality
#1

Hi,
I’m very new to CS.
While I was using 2D classification I found out that if a job crashes due to any reason like out of memory or something else, there is no option to continue the job from the GUI.
My query is if there is any option or workaround to resume a failed/crashed job in CS.

Thanks

#2

You can Clear Job, and then re-start it (from the beginning). No way to resume a job (that I know of).

Best,
RJ

#3

Yes that’s an option we have. But sometimes some job crashes due to out of memory error in last iterations.
In such cases an option which might allow to resume the job assigning more memory on clusters would be very helpful. Sad that we don’t have any options like that.

#4

Hi @diffracteD,
Unfortunately we do not have a way to resume 2D Classification or Refinement jobs that failed in cryoSPARC, at the moment. However, this is possible in cryoSPARC Live (https://cryosparc.com/docs/live). We hope to add similar functionality into cryoSPARC soon.

If you are okay to start from raw movies, you can import these into cryoSPARC Live, have the cryoSPARC Live GPU workers take care of preprocessing (motion correction, ctf estimation, picking, extraction), and then use the Streaming 2D Classification job in Live, which if it fails, will be able to continue from a last saved state.

#5

Hi @spunjani ,

Any word on getting this type of functionality in cryosparc proper? I’ve had some long jobs (ex. non-uniform refinement with very large particles) crash 20+hrs in due to cluster instability a few times and have just restarted from scratch; it’d be useful to be able to restart from the most recent iteration’s output at the very least.

Thank you!