Did anyone ever encountered a problem with local motion correction when the job seems to output all extracted particle files, but never gets to output the metadata file and the job is never given as finished?
I have this local motion correction job that is running for over >70 hours and in the last 12 hours it has not output any new file (and I am quite sure it has been through all micrographs). If I kill the job, I know I won’t get the metadata file needed for the follow up jobs… how can I rescue this situation?
The cryoSPARC version is v. 4.1.2. If I had the oportunity, I would upgrade to the latest version, but it isn’t an option at the moment.
I contacted the admin to get privileges to run the command you suggested @wtempel, but before he could attend my request, meanwhile the job failed and I think that was because we ran out of storage in our shared network file storage unit… I guess the extracted particles cannot be used in follow-up jobs without the metadata file?..
There’s also some substitution in the script you have to make specific to your CryoSPARC installation/project.
Once you have everything set up, you can run it from the command-line like this:
Note that I’ve only done some rudimentary testing for this; there may be something else wrong with the job that I have not accounted for which could prevent the script from working. You may modify it as required.
I have beeen struggling a bit to put this into practice, but I overcame parts of it.
There a few questions, though:
First I assumed that the parent_picker_job was the imediate parent job of the local motion correction job, but is it actually the upstream particle picking job? Just so you are aware, I am not local motion correcting particles coming straight from a picker job (but I think you would guess that, as normally people clean well the particle stack before going to local motion correction). In fact, between the particle picking job and the local motion correction job I also curated exposures and rejected a number of them.
I do not understand “zero_shift_frame = frame_start + (frame_end - frame_start) / 2”. What is it suppose to be and should I change anything there?
After running the script, what shall I expect/do? Will the motion correction job be marked as completed? Do I have to restart cryoSPARC to see the differences? I did run already the script successfuly, by suplying the particle picking job as “parent_picker_job” and leaving “zero_shift_frame” as is, but I did not see any changes (in the CryoSPARC GUI nor the directory of the localmo job) after the script completed its task. Currently the workstation is running another long job, so I don’t want to try restarting CryoSPARC now.
I may or may not get my answers once I manage to restart CryoSPARC, however I think other users that may have this problem would like to have answer to these questions as well.
Hi André, glad to hear you’re getting some results with this. To answer you questions:
Your first assumption was correct, parent_picker_job is the job that provides the input particle picks to local motion correction. This may be a picking job, or it may be a filtered subset of the original picks from a classification job.
zero_shift_frame is the frame for local motion correction to consider as the “central” frame, i.e., the frame from which particle offsets across frames are calculated by local motion. For a movie with n frames, this value defaults to n/2
After running the script, you should see a completed “External” job in the selected workspace with the motion-corrected particles as outputs. There is no need to restart CryoSPARC. If you do not see this, please post any errors you see in the output of the Python script.