I’m testing different approaches to get to reach high resolution on EMPIAR 10200.
I’ve managed to do so following CTF refinement workflow getting these results in local ctf estimation:
Now, when I prune some mics (based on 2100<defocusU<20000 or estimated resolution <4A) it seems that I can not get the same nice histogram:
Particles are not exactly the same but there is a huge overlap.
Hi @pconesa1,
Could you provide more details about the specific workflow used after the initial CTF refinement job? A screenshot of the job tree might help us see if something else (the volume reference, mask, etc.) has changed that would cause this, as it seems unexpected that just filtering micrographs & particles by defocus and CTF fit would cause such a huge discrepancy.
Best,
Michael
Thanks mmclean. Let me clarify one thing, I’m doing this through Scipion, so there is a chance that the problem reasides somewhere in metadata conversion. But so far, several attempts have gone great.
Once I have “pruned particles” (though 3 iteration of ab initio (A) or just removing “bad” ctfs (B) I got an initial volume. In A case is already produced since the pruning consist of several ab initio rounds keeping particles of the good volume. In B case have an “ab initio” with 2 classes keeping the “good ones”.
After initial volume, steps are quite straightforward: homogeneous refinement, global ctf refinement, local ctf refinement, final homogeneous refinement.
Since I want to keep both workflow independent, each of the approaches provides their own reference and mask. Mask is a relaxed mask done in both cases with the same parameters. Their reference volume is the best, but I guess is kindoff random.
I’ve made a presentation summarizing the 2 approaches: here.
All changes in global ctf refinement for the second approach, and the ctf local refinement goes nuts?
Hi! did anyone have the chance to have a look at this?
Hi @pconesa1,
Apologies for the delay, and thank you for making this presentation with a very clear depiction of the workflow. In approach B, it definitely looks like something went wrong with the CTF data between slides 9 and 11. The homogeneous refinement job in slide 9 looks like it worked great, but the global CTF refinement job in slide 11 doesn’t look as expected, and this is confirmed by the erroneous downstream refinement job in slide 13.
Where are the particles coming from in the CTF refinement jobs in slides 11 and 12? Are they from the same particle stack that was input into the homogeneous refinement in slide 9? Is the input volume from the homogeneous refinement in slide 9?
If all yes – would it be possible to try to recreate this workflow entirely in cryoSPARC directly, perhaps using Manually Curate Exposures to prune micrographs instead of Scipion? This would help us rule out whether the issue arose in cryoSPARC directly, or whether it was a side-effect of interfacing.
Best,
Michael
Sorry I replied by email a few days ago, but seems it does not reach here.
Where are the particles coming from in the CTF refinement jobs in slides 11 and 12? Are they from the same particle stack that was input into the homogeneous refinement in slide 9?
No, I’m trying independent workflows, so they are not literally the same particles, but they have been produced with same method and parameters. So the stack is very similar, only there are fewer particles in approach B.
I’ll try to do as much as possible inside CS, avoiding Scipion side-effect possibility.
I’ve managed to run a ctf global refinement inside CS from a previous homogenous refinement (this one triggered by scipion).
It has worked well. On the other hand some of the global ctf refinement job send from Scipion have worked fine.
I’ve reviewed logs, metadata, masks, … the only thing I’m looking now is that we are importing particles, without optics group. Could this be affecting the results?
Got some more info.
When running global ctf refinement from Scipion3, we are previously running the required import jobs:
import particles
import volume
import mask
There is a difference in the import volume. Basically, it is just the volume, without the halves!
Again, in one case this hasn’t been a problem at all, but I’m wondering if this is reducing the success chances.
In any case, we would like to import properly, not just the refinement volume but also the halves.
Can this be done programmatically using “cryosparcm cli”?
Hi @pconesa1,
This might be the source of the discrepancy. Local CTF refinement requires access to the half-maps, but global CTF refinement just uses the full map.
In any case, you shouldn’t have to worry about these details so long as you never connect an import volume job directly to a CTF refinement job, and instead only use the imported volume as an initialization for a homogeneous refinement. CTF refinement was built assuming the inputs always come right out of a homogeneous refinement in cryoSPARC.
I’d be concerned if you saw the same issues in CTF refinement when using volume+particles+mask inputs that all came right out of the same homogeneous refinement job.
Best,
Michael
