I can see that it’s been some time since anything has been posted on this thread - conclusive or otherwise - so I just wanted to share my recent experience related to the above. Running v3.3.1 it seems that the preprocessing step becomes idle instead of returning an error, if it is not assigned enough RAM. At least I tried a run with 5649 training exposures (which I now know is absurd overkill), which was seemingly running for 55 h until I killed it without it being done preprocessing, whereas another job with 500 micrographs was done preprocessing after 175 s. Every time I run a job, I get the error message above that the micrographs to which the particles are assigned, are not found in the folder of preprocessed micrographs. This was due to the fact that the preprocessed micrographs were patch-aligned and doseweighted, whereas the particle-assigned micrographs it was looking for were rigid-aligned without weighting (suffix of …fractions_rigid_aligned.mrc vs …fractions_patch_aligned_doseweighted.mrc). This error came no matter if the particles and micrographs were taken directly from a re-extraction job, manual picker job or curate exposures job. Since it does, however, preprocess the doseweighted micrographs, which should be the most fitting (and because it usually downsamples the micrographs quite heavily anyway), the fix I used was simply to rename all the .mrc-files in the preprocessed-folder of the failed job and adding the path of that folder to “Absolute path of directory containing preprocessed directory” of a new job, which then ran smoothly. For anyone who isn’t too happy about writing bash, I used the following two loops (run directly in UNIX terminal in the preprocessed folder):
for f in *fractions_>>suffix to remove<<.mrc; do mv “$f” "${f/>>suffix to remove<<}"; done
for f in *.mrc; do mv “$f” “${f/.mrc}”>>suffix to be inserted<<.mrc; done
Hopefully, this will help others experiencing issues.
And if you know that this hack of treating patch-aligned, doseweighted as rigid-aligned, non-doseweighted is inappropriate, please let me know!
2 Likes