Hi, would it be possible, for specific “per-micrograph” job types, such as CTF estimation, picking, and extraction, to add a “only do unfinished” option when re-running a job?
I use cryosparc a lot for on the fly processing of data as it is collected, by importing aligned micrographs from the scope. The only thing is that as data collection proceeds, the number of micrographs in each set increases, and everything becomes slower, largely because I have to redo CTF estimation, picking and extraction for micrographs that had already been completed earlier. An option to rerun these jobs, only adding micrographs that were not processed in the previous round, would be very helpful for on the fly processing.
Even better would be to also separate out these new micrographs (e.g. as a new_micrographs/new_particles blob), so I can monitor the current quality of data being collected.
Ultimately, having a streaming mode where cryosparc continuously looks for new micrographs and processes them in batches would be optimal, but this would I think help in the interim.
Cheers
Oli