Only do unfinished? [Feature request]



Hi, would it be possible, for specific “per-micrograph” job types, such as CTF estimation, picking, and extraction, to add a “only do unfinished” option when re-running a job?

I use cryosparc a lot for on the fly processing of data as it is collected, by importing aligned micrographs from the scope. The only thing is that as data collection proceeds, the number of micrographs in each set increases, and everything becomes slower, largely because I have to redo CTF estimation, picking and extraction for micrographs that had already been completed earlier. An option to rerun these jobs, only adding micrographs that were not processed in the previous round, would be very helpful for on the fly processing.

Even better would be to also separate out these new micrographs (e.g. as a new_micrographs/new_particles blob), so I can monitor the current quality of data being collected.

Ultimately, having a streaming mode where cryosparc continuously looks for new micrographs and processes them in batches would be optimal, but this would I think help in the interim.



Just adding a note for those of you who visit this post through search - we designed cryoSPARC Live specifically for on the fly processing, it covers this use case.


Somewhat - would still be good to incorporate some of the cryoSPARC live goodies into the “regular” cryosparc interface IMO