I have seen this on 3 of around >150 data sets or so. These examples images above are both for sub 3A unbinned data sets.
For one case the raw movies were redone and particles reimported and the noise variance returned to ascending as a function of resolution similar to what is seen right above.
I was curious as to 1) why this is, 2) how might one overcome these issues when they arise and 3) any further or deeper insights anyone might have.
Could I ask when you did the initial motion correction for the datasets that show this issue (i.e. that have the flat noise model)? In v3.0 and v3.1, there was a transient issue in Patch Motion Correction that caused dose weighting to be incorrectly applied in some cases (this was caused by the introduction of variable-dose support in v3.0). This was fixed as of March 29, 2021 in v3.2 and we noticed the issue through a similar observation to your own – so it would explain your observations if the raw data was motion corrected in v3.0 or 3.1.
For best results, I’d recommend re-doing the motion correction on these datasets. Empirically it’s generally seen that the noise variance increases at higher frequencies with most datasets.
Just found this and it retrospectively explains a number of things we have observed - mainly that we had a few datasets for which we were seeing unusually large improvements from Bayesian polishing, and often from multiple rounds of Bayesian polishing (e.g. one improved from 2.7 to 2.2). Looking back at them, these datasets were ~late 2020 → ~early 2021, and they exhibited flat noise models like this, which changed to ascending after polishing. So I guess any data collected between Dec 2020 and March 2021 is worth reprocessing.
It very much depends - for CTF refinement depends on particle size (for local CTF refinement) and data collection (for global CTF refinement). Gains from global CTF refinement can be minimal or huge, depending on beam tilt (and I would always recommend at least running one global CTF as a diagnostic). Gains from polishing usually on the order of 0.1Å plus or minus.
What does a flat noise model variance implies, reading the discussion in this thread it was indicated that it could be due to an earlier version issue with motion correction, however, I still see the flat noise curve like the one on the left of the original post? I am using v3.3.1+220215 and had done a path motion correction. Interestingly in another unrelated dataset where I have smaller particles ~200 KDa (the flat one has 1.2 MDa complex), I have noise model variance as on the right graph.
Please help me understand this and do I need to change something in my motion correction as my current map is stuck at 3.5A. Many Thanks
We have a similar plot behaviour when we import sums into CS and only use those (noise model variance being flat) vs imported movies with motioncorrection done in CS (noise model variance ascending) using the same dataset. motion correction in CS with noise model variance ascending leads to resolution improvement around 0.3 -0.5 Å and also better B-factor estimation. As far as I understand the model looks at differences before and after ctf to get an idea of the noise. Any input on this behaviour) would be highly appreciated.
When using CryoSPARC’s motion correction algorithms, the ascending noise variance is generally expected if the noise in each movie frame starts out as white noise. This is because of how the dose weighting works: when summing aligned frames to produce a micrograph from a movie, low frequency data have nearly equal weights over the different frames, but high frequency data have strongly unequal weights (most of the weight comes from the first few frames, later frames hardly contribute due to radiation damage). See the following dose weight plot, where the vertical spacing between adjacent curves indicates the amount of weight received by that frame at a given frequency.
The result of this dose weighting is that, if we assume noise to be white in each frame, then low frequencies have a variance that goes as ~1/N where N is the number of frames (this comes from the variance of a sample mean). Since high frequencies have strongly unequal weights, only the first few frames contribute significant mass to the sum, and so the variance of the sum goes as something greater than ~1/N.
Out of curiosty, when you import already-summed micrographs into CryoSPARC that exhibit a flat noise variance, which motion correction software are you using?
thank you so much for your explanation. That makes a lot of sense. So we are usually motioncorr2. we haven’t tried the CS implementation though. I would, however, anticipate that this should be also true for other motion correction software. Is there anything missing when importing sums? I have realised that for the patch CTF the non DW blob from the motion correction job is also used.
FWIW, we’ve always seen a flat-ish noise variance curve when working on particle images extracted in CS from dose-weighted RelionCor sums (Bfactor 150). Similar to Oli’s observation, the polished particles then assume a rising noise variance.
In most instances, where a comparison had been made, CS Patch Motion-derived particle images gave slightly better results at the outset (vs RelionCor/MC2), with the shiny particles making up the difference and then some later.
I had always rationalised this with differences in the fidelity of patch motion tracking, e.g. finer interpolation with the knots-approach, with the noise variance behaviour being coincident. Is there more to it than that?
To clarify, CryoSPARC’s patch motion correction does use the same model-based dose-weights as MotionCor2; this was initially described by Grant et al in their eLife publication. I am not sure if RelionCor uses the same dose-weighting method or has changed since its initial development
