Hi All. I’m trying to go forward with results from a patch motion correction job that had failed (due to expiration of storage space, which is now resolved), but had completed several thousand images. I’d like to use the completed images in a patch ctf estimation job, but the patch ctf estimation job will not run, it remains queued waiting for completion of the patch motion correction whose completed output files I have set as input to the patch ctf estimation job. I forced completion of the motion correction job by marking it completed within the GUI, but this seems not to help. Has anyone suggestions for how to go forward? FWIW, I am using cyroSPARC version 2.15.0.
Hi @earturo,
Can you please try ‘Clear Job’ on the queued Patch CTF estimation job, and then try Queuing it again?
If that does not work, you can create a new Patch CTF estimation job, and drag over the ‘successful’ micrographs from the Patch Motion job that you manually marked as completed.
I’ve done both things you’ve suggested, and the new patch CTF estimation job is also apparently stuck at ‘Queued’. Is there a different way to ‘stop’ the failed patch motion correction job besides marking it as complete, which has not apparently worked? I hesitate to clear job…
I should clarify a thing: though I pressed ‘Mark Job as Complete’ on the patch motion correction Job, it does not complete; the forced complete seems to fail itself. The output is shown below:
I don’t know what this output means. Can you translate, @spunjani?
Hi @earturo,
This is an odd error- it indicates the file on disk may have a different size than what the function expected (maybe it’s corrupted?).
Either way, is it possible if you can run the following command, and try re-queueing the Patch CTF Estimation job after that?
cryosparcm cli "set_job_status('<project_uid>', '<job_uid>', 'completed')"
@stephan, that command worked to complete the job. However, the queued patch ctf estimation job failed immediately. I deleted the failed patch ctf estimation job and started a new one with the same input (the exposures from patch motion correction, those that are not ‘incomplete’), and this also failed. See screenshot. It’s something about the array size once again. Any ideas?
Hi @earturo,
I think based on this error message, it looks like the metadata output file of your Patch Motion job may have been corrupted. The only way to get around this is to re-import the micrographs using the Import Micrographs job. The micrographs will located in the job folder of the Patch Motion job- you can find the path to by going to the projects page, clicking on the project, and copying the absolute path to the project directory from the project details panel:
If the job uid of the patch motion job was J13, you will find it in a folder named J13/motioncorrected.
You can choose to import the doseweighted micrographs (these files will not have a suffix) to use for subsequent processing (although Patch CTF Estimation will perform better on the non-doseweighted micrographs).
@stephan, it looks like several files had become corrupted when the patch motion correction job failed. When the Import Micrographs job also failed we looked over all the files and noticed that one of the motion corrected files was empty and the sum of images in the incomplete group and the completed group of files did not equal the total number of movies input to motion correction. So I opted to re-run the patch motion correction job and delete the old one as I wasn’t sure there wouldn’t be other issues down the line.
Thanks for your responses, All.