Non uniform (New) sometimes proceeds to CTF refinement before convergence (bug)

Troubleshooting
1

Hi,

The new Non uniform job with on the fly CTF refinement works great! However I have noticed that with certain particle subsets, it starts the on the fly CTF refinement after iteration 1, before refinement had fully converged. As a result, the CTF refinement then only uses low spatial frequencies and completely messes up the rest of the refinement.

In the most recent case, this did not happen with the full particle set of 200k particles, but it did happen with a 40k subset of particles. It wasn’t an issue with the particles being bad - when I switch off defocus and higher order CTF refinement and manually alternate non uniform and CTF refinement, I get good results. Perhaps the convergence criteria need tweaking?

Cheers
Oli

2

I think perhaps CTF refinement is being switched on as soon as cryoSPARC is using the full dataset, whereas in the case of small particle sets at least I would want to wait until refinement has fully converged without CTF refinement before iterating CTF and defocus refinement (so one can initialize it with the highest resolution reconstruction).

1 Like
3

Hi @olibclarke I think you’re right - this is a bug. It looks like CTF refinement starts once we’ve seen the whole dataset once rather than when converged. We’ll change this.

1 Like
4

Thanks @apunjani!! Also, would it be possible to add at least optional on the fly beamtilt/trefoil refinement (and maybe per particle defocus) to Live?

This would be very helpful to diagnose alignment issues during data collection - maybe we could have a separate config section for refinement options?

Also it would still be fantastic to see an option to have at least two rounds of 2D in Live - this would make a huge difference for small particles, where we often need two rounds of 2D to get a usable particle set, particularly when starting from blob picker

Cheers
Oli

1 Like
5

Also streaming heterogeneous refinement would in some cases be more useful than streaming homogeneous - e.g. in cases where we really want to get an idea whether a sub-class is present on the fly…

1 Like
6

Also would be helpful to have the option to display plots in the overview tab in Live as simple histograms, rather than mapped to exposure ID. Both are useful, but a simple histogram gives a much better idea of the distribution of e.g. relative ice thickness, and makes it easier to see the mean and to compare with previous datasets

Cheers
Oli