Hi,
In case it hasn’t been mentioned before, it would be nice to be able to parallelise Blob Picker over multiple GPUs, similar to the preceding preprocessing jobtypes. We often resort to splitting the input into n groups to feed into n number of picking jobs–something we also do with the MotionCor2 wrapper–before reconsolidating, but this is rather cumbersome.
Cheers,
Yang
+1 have been wishing for this since the beginning. (And for the template picker as well).
Note for MC2 it’s usually significantly faster to run one movie on several GPUs, than to simultaneously run multiple copies of MC2 on single GPUs. Running on a set of movies with -Serial 1 mode is also much faster, in that case MC2 will interleave compute and IO intelligently.
It’s been so long since I’ve run stand-alone MC2, I wasn’t aware of the -Serial 1 option. That’s a helpful hint.
Out of curiosity, I assume these comments relate to MC2 from the command-line? Because RELION’s wrapper seems to spawn a single MC2 process per GPU, parallelised across several GPUs; and CS’s wrapper processes movies serially, each recruiting n GPUs, but with no apparent speed gains (n>1), at least with our in-house GPU node setup.
Cheers,
Yang
Yes, stand-alone. The Relion wrapper is quite out of date at this point, it doesn’t even pass most of the arguments correctly (all the ones that can be two or more numbers, and X and Y are swapped). I’ve never used the cryoSPARC wrapper but I believe it’s at least missing some of the options. It’s too bad because in addition to the highly efficient multi-GPU support and interleaved IO there are some nice features like -InFmMotion 1. The Relion wrapper isn’t needed for polishing anymore, either because of -OutStar 1.
All that said, in my tests it appears that cryoSPARC Patch Motion is giving the best standalone results (without polishing). Perhaps this is largely be because of a greater number of patches being used in most cases.
I’ve started looking more closely into stand-alone MC2, since anticipating losing access to a large CPU cluster on which to run RelionCor efficiently and quirks with how detector defects are handled in ≥v3.1. Thanks for highlighting some of MC2’s more modern features.
This is consistent with what we’ve been seeing as well.
Cheers,
Yang
We took not of your suggestion. Thanks!
Preprocessing from motion correction through particle extraction using multiple workers is already available in CryoSPARC Live and simulates a parallel particle extraction capability.
@wtempel Good point, but if a different motion correction is used this does result in making a hard copy of every input micrograph.
Yes, this.
I imagine an expansion of CryoSPARC Live to accept Imported Micrographs from a workspace as source data (as opposed to parsing as single-framed moviestacks to be motion-corrected?) would be useful.
Cheers,
Yang