Exposure groups set at import lost during processing - possible bug?

I imported data collected using a 3x3 image shift pattern in SerialEM based on the unique identifier in the file name for each optics group using manually override exposure group at import. for examples files with name _0000 got assigned as exposure group 0, _0001 got assigned as exposure group 1, etc.

The various imported movies were pooled at the motion correction step and processed together to a refined particle stack that I would like to run global ctf refinement. At this step it complains of only 1 exposure group (0). I generated a star file from the parent NU refinement job and indeed while the data is from an approx even distribution of movies across all exposure groups (based on naming), however, the exposure groups id is 0 irrespective of the previous assignment.

Is this a known bug? Data was imported with cryosparc v2.15.0 and processing was carried out between versions 2.15.0 and 3.1

Hi @Joe, I am not able to reproduce the issue you’re describing in the latest cryoSPARC v3.1. Here’s what I did:

  • Import two disjoint subset of movies from the dataset, overriding the exposure group to 1 and 2
  • Combine the two datasets in a Patch Motion Correction step
  • Pick particles from the two movies during refinement (the exposure group ID is carried over via the ctf/exp_group_id field in the ctf output result for the picked particles
  • Refine a volume using the picked particles with both Homogeneous and Non-Uniform refinment

Both jobs produced particles with ctf/exp_group_id values 1 and 2 from both groups.

I converted the particles to star files, and both exposure groups were represented in the _rlnOpticsGroup field as values 2 and 3.

There have been no major changes between v2.15 and v3.1 that would wipe out the exposure group references.

Does the above differ from what you did? If yes, can you describe how?

Please also send the .cs files from the following outputs and jobs:

  • imported_movies > mscope_params from both Import jobs where you manually specified the exposure group ID
  • micrographs > mscope_params from the completed motion correction job where you combined the two exposure groups
  • particles > ctf from the job you used to pick the particles
  • particles > ctf from the computed NU refinement job

You can download a .cs file as indicated

The files may be several hundred MB in size, so feel free to use a file sharing service such as Dropbox, Google Drive, OneDrive, etc.


Hi @nfrasser,
My workflow was as follows:

  • Imported disjointed subsets of movies into 9 exposure groups using the manual override command
  • Combined all dataset in a patch motion correction step
  • Continued to ctf determination using gctf (at the time patch ctf multi was not working on our systems)
  • Curated exposures using manual curate tool
  • picked particles using templates and extracted 4xbinned particles
  • 2D classification
  • 2 x heterogenous refinement
  • used particle sets to take the intersection of the binned refined particles stack from an unbinned particle stack (extracted the same as above)
  • NU-refinement

From there I tried running local and global CTF refinement where I noticed the exposure groups were set to 0.

I can try and upload links to the various CS files later once I get on the system. Any others you would like?

Update: I traced the error to the ctf determination step using gctf. The mscope_params for that job have the exposure group array from [0-8] however the ctf/exp_group_id only has [0].


1 Like

Hi @Joe, thanks for confirming, the gctf wrapper is indeed not carrying over the group ID in the current version of cryoSPARC. We’ll fix this bug in the next release. Sorry about that!

No problem, and many thanks for taking a look. I probably should have dug deeper from day 0.

A fix for this is included in v3.2, released March 29, 2021. Release notes available here: https://cryosparc.com/updates