I imported data collected using a 3x3 image shift pattern in SerialEM based on the unique identifier in the file name for each optics group using manually override exposure group at import. for examples files with name _0000 got assigned as exposure group 0, _0001 got assigned as exposure group 1, etc.
The various imported movies were pooled at the motion correction step and processed together to a refined particle stack that I would like to run global ctf refinement. At this step it complains of only 1 exposure group (0). I generated a star file from the parent NU refinement job and indeed while the data is from an approx even distribution of movies across all exposure groups (based on naming), however, the exposure groups id is 0 irrespective of the previous assignment.
Is this a known bug? Data was imported with cryosparc v2.15.0 and processing was carried out between versions 2.15.0 and 3.1
Hi @Joe, I am not able to reproduce the issue you’re describing in the latest cryoSPARC v3.1. Here’s what I did:
Import two disjoint subset of movies from the dataset, overriding the exposure group to 1 and 2
Combine the two datasets in a Patch Motion Correction step
Pick particles from the two movies during refinement (the exposure group ID is carried over via the ctf/exp_group_id field in the ctf output result for the picked particles
Refine a volume using the picked particles with both Homogeneous and Non-Uniform refinment
Both jobs produced particles with ctf/exp_group_id values 1 and 2 from both groups.
I converted the particles to star files, and both exposure groups were represented in the _rlnOpticsGroup field as values 2 and 3.
There have been no major changes between v2.15 and v3.1 that would wipe out the exposure group references.
Does the above differ from what you did? If yes, can you describe how?
Please also send the .cs files from the following outputs and jobs:
imported_movies > mscope_params from both Import jobs where you manually specified the exposure group ID
micrographs > mscope_params from the completed motion correction job where you combined the two exposure groups
particles > ctf from the job you used to pick the particles
particles > ctf from the computed NU refinement job
Imported disjointed subsets of movies into 9 exposure groups using the manual override command
Combined all dataset in a patch motion correction step
Continued to ctf determination using gctf (at the time patch ctf multi was not working on our systems)
Curated exposures using manual curate tool
picked particles using templates and extracted 4xbinned particles
2D classification
2 x heterogenous refinement
used particle sets to take the intersection of the binned refined particles stack from an unbinned particle stack (extracted the same as above)
NU-refinement
From there I tried running local and global CTF refinement where I noticed the exposure groups were set to 0.
I can try and upload links to the various CS files later once I get on the system. Any others you would like?
Update: I traced the error to the ctf determination step using gctf. The mscope_params for that job have the exposure group array from [0-8] however the ctf/exp_group_id only has [0].
Hi @Joe, thanks for confirming, the gctf wrapper is indeed not carrying over the group ID in the current version of cryoSPARC. We’ll fix this bug in the next release. Sorry about that!
