Hi everyone,
I am revisiting the processing of a single-particle analysis dataset and encountered an issue that I would like to understand better.
Due to storage limitations, I no longer have the original Motion Correction and Patch CTF output images. To continue processing the dataset, I re-ran Import and Motion Correction. This caused problems when I tried to re-run some existing Extraction and Select 2D Classes jobs.
I suspected that the problem was related to the unique UIDs assigned to each image during the new Motion Correction job. I therefore remapped all .cs files generated by the new Motion Correction job, as well as the image filenames in the motioncorrected directory, so that they matched the original image filenames used by the existing jobs. I then ran Patch CTF again using the remapped Motion Correction output.
After this, I was able to successfully re-run the existing particle Extraction jobs. More importantly, I obtained the same number of correctly extracted particles as in the original processing.
However, I then noticed an unexpected difference downstream (also see figures below, left is from old processing and right is from latest processing):
-
Homo Refinement using exactly the same volume map from Ab Initio and particles from the same particle Extraction job produced a different resolution compared with the original processing.
-
The subsequent Local CTF Refinement also produced a substantially different defocus histogram. This is not just limited to my case, but also happened to my lab mate’s data processing as well.
-
The difference is not simply a small variation in resolution; the CTF refinement results look noticeably different from the original processing.
This makes me wonder whether there is some information from the original Motion Correction/Patch CTF processing that is not fully captured by the remapping of the .cs files and image filenames, or is there anything strange about Motion correction, Patch CTF estimation, local/global CTF ref.
I acknowledge that manually remapping UIDs may sound questionable. However, if the remapping were incorrect in terms of the particle-to-image correspondence, I would expect the downstream Extraction job to fail, or at least to produce a different/incorrect particle set. Instead, the existing Extraction job runs successfully and produces the same number of particles as before, suggesting that the particle-image correspondence has been restored correctly, at least at the level required for extraction.
So I am wondering:
-
Are there additional pieces of metadata associated with Motion Correction or Patch CTF that are used during Homo Refinement or Local CTF Refinement, but are not preserved by simply remapping the image UIDs and filenames?
-
Could differences in UIDs or other internal metadata affect refinement even if particle extraction successfully reproduces the same particle count?
-
Is there a way to verify that the newly generated particles are truly equivalent to the particles from the original processing, beyond comparing the particle count?
-
Could there be something different in Motion correction, Patch CTF estimation, local/global CTF ref in the latest version of CryoSparc that causes the problem?
I would appreciate any insight into what information CryoSPARC uses downstream from Motion Correction/Patch CTF that might explain this behavior.
I can provide the relevant .cs files, job configurations, or comparisons of the original and new refinement/CTF-refinement results if that would help diagnose the issue.
Thank you everyone in advance!
Haochen