Dynamic reassembly of Card View

Editing automated workflows after applying the workflow (i.e. generating the 40-50 job cards) exacerbates a desire to “replace” a card with a different job, much like how replacing with a “blueprint” which changes the parameters works, or otherwise change the order of jobs after they’re created without the need to change the J###. Right-click, change GPU version of micrograph extraction to CPU version. Darn, I deleted micrograph denoiser but now I want to add it back but it goes to the end of the list and that is much less satisfying than each job having inputs come from prior jobs, at least in card view position.

We have also observed low-level inputs are empty when generated, but replacing the card after the input job is complete then fills out the low-level inputs.. So queuing 40 jobs has lead to several cases where the parent job needs to be relaoded as input and breaks the chain of automation. This is in addition to issues of our own making when replacing micrograph preprocessing in the workflow with preprocessed exposures from LIVE.

Did I miss some way to incorporate micrograph junk detector and/or denoiser into LIVE?

Any way to have the “final” NU-refine after RBMC have matched parameters with the random prior job that generated the best resolution (which is then select volumes). would be nice to automatically use whichever parameters yielded highest res prior to RBMC for the final.

Halo sandbox mode and worldwide collaboration/utilization of custom works gives me the idea that having some repository where users could load their automated workflows would induce some friendly competition (fastest/best) and provide starting points for the many other complicated scenarios that users will automate, for general use. 5-stars rating system would be important to separate those which perform well from those which don’t. Fastest way to have the community generate and benefit from automated workflows for common SBDD repeat targets (like GPCRs that got it started), universal microscope benchmarking with Apoferritin or other, big protein vs small, 50k particles vs 10mil, discrete heterogeneity vs continuous, etc.

2 Likes

No way of doing that yet, as far as I know. Denoiser might be tricky due to training (can’t do until minimum number of mics processed?)

Hi @CryoEM2,

Thanks for your musings and feedback! Please see my responses below

Re: Editing in card view

We have found the easiest way to edit workflows is to have a workspace dedicated to such and using the tree view. This will maintain the correct ordering of the jobs. But, I have made a note on our end to look into whether we can make editing workflows easier or simpler.

LLIs

I have noted that maintaining/re-creating low level inputs are of interest in workflows.

Did I miss some way to incorporate micrograph junk detector and/or denoiser into LIVE?

No, you did not. Although with the GPCR automation preprint we did release a workflow file that takes as input, micrographs preprocessed (PMC, CTF est) in CS-live (export exposures is a parent job for the workflow) and these mics can then be used with the full workflow including denoising and junk detection.

“final” NU-refine after RBMC

This is a good suggestion and something we could look into for future versions of automation. It should be noted that those particles will carry though the refined CTF values, although we understand that redoing Global CTF refinement post RBMC can sometimes boost resolution.

Repository of workflows

We have noted this idea!

Cheers,
Kye