Ctf/anisomag key error on 2nd full iteration during Class3D

3D Classification
1

Hi,

One of my 3D classification jobs failed with the attached error, during the second full iteration after O-EM iterations ended. Previous Class3D runs on the same set of particles worked fine, and no refinement of magnification anisotropy has been performed for this dataset.

Cheers
Oli

UPDATE: This now happened for a second Class3D job on the same system, with a different set of particles. Also on the second full iteration.

[CPU:   1.10 GB  Avail: 327.44 GB]
-------------- Iteration 81 --------------

[CPU:   1.10 GB  Avail: 327.41 GB]
    [Full EM] Spooled entire dataset, 317466 particles

[CPU:   1.10 GB  Avail: 327.38 GB]
    Computing expectation step (batch A), processed 0 / 158733 particles

[CPU:   1.02 GB  Avail: 327.50 GB]
Traceback (most recent call last):
  File "cryosparc_master/cryosparc_compute/run.py", line 96, in cryosparc_compute.run.main
  File "cryosparc_master/cryosparc_compute/jobs/class3D/run.py", line 619, in cryosparc_compute.jobs.class3D.run.run_class_3D
  File "cryosparc_master/cryosparc_compute/jobs/class3D/run.py", line 1131, in cryosparc_compute.jobs.class3D.run.class3D_engine_run
  File "cryosparc_master/cryosparc_compute/jobs/class3D/run.py", line 1142, in cryosparc_compute.jobs.class3D.run.class3D_engine_run
  File "cryosparc_master/cryosparc_compute/engine/newengine.py", line 551, in cryosparc_compute.engine.newengine.EngineThread.scan_all_images
  File "/home/user/software/cryosparc/cryosparc2_worker/cryosparc_tools/cryosparc/row.py", line 36, in __getitem__
    return self.cols[key][self.idx]
KeyError: 'ctf/anisomag'
2

Hi @olibclarke how old are the other jobs in this project? Any chance these particles were extracted using a much older version of CryoSPARC? The ctf/anisomag field was not always present, it could be that the particles do not have this field if they were extracted long ago. Can you try running the “Patch CTF Extraction” job to just re-output the CTF values for these particles?

3

Hi @apunjani these are new particles, fairly new project, all post v4.1… This did happen just after we upgraded to 4.2.1-privatebeta1 this morning, so I am suspicious that that upgrade may be the root of the issue. We ran Class3D jobs with the previous version with no problems yesterday (same set of particles)

4

All of our 3D classification jobs on this system are dying, including jobs from other projects, and all are dying at the second full iteration - jobs on other systems still on 4.1x are unaffected. Other job types (e.g. NU refine) run and complete fine with the same particle set.

5

Hi @olibclarke thanks for the further details - we are investigating!

1 Like
6

Thanks Ali - let me know if you need any more details, logs etc!

7

@olibclarke we’ve reproduced and found a likely fix. As a temporary workaround until this is released can you try to run 3D class with Output results after every F-EM iteration turned off? that should stop this from happening

1 Like
8

Ok will do, thanks for the quick fix!

1 Like
9

Can confirm that this fix works, thanks Ali!