Topaz Training Assertion errors 1 and -9 after CryoSPARC live Motion Correction and CTF estimation

Particle Picking
1

I’ve been running into AssertionError: Subprocess exited with status 1 and AssertionError: Subprocess exited with status -9 while trying to train a topaz model.

For context I did the following:

  • Patch Motion correction and Patch CTF estimation in cryoSPARC live (Session 1, S1).
  • Exported Exposures from cryoSPARC live to regular cryoSPARC (J8).
  • Did some manual picking then template picking and iterated a few times, I used 2D classification to get an idea on particle quality.
  • Selected suitable 2D classes.
  • Attempted a Topaz Train job inputting all Exposures from J8 and the particles from my latest Select 2D job.

If I don’t specify the Absolute path of directory containing preprocessed directory then it doesn’t find the micrographs, skipping them all at the point of splitting the dataset, and exits with the AssertionError status 1 (it gives the ValueError: arrays must all be same length immediately before the AssertionError) I tried specifying the motion corrected cryoSPARC live directory S1/motioncorrected, it appears to find the micrographs and exits with AssertionError status -9 right after starting training. If I point it to the J8 directory with the Live Exposure Export it also exits with AssertionError status 1.

I’m not sure if I’m missing something obvious. Any suggestions?

Other users in my department have successfully ran topaz training using cryoSPARC, but it doesn’t appear anyone has done this with cryoSPARC live before.

2

@JamesLeCornu Please keep in mind that

  • the Absolute path of directory containing preprocessed directory parameter pertains to the output of the
    topaz preprocess step, not to the outputs of CryoSPARC (Live) motion correction
  • the currently supported version 0.25 of Topaz can be overwhelmed by long lists of exposures paths (a large number of exposures and/or long absolute paths to exposures)

Very likely, given your use case, you may want to

  1. leave blank the Absolute path of directory containing preprocessed directory parameter of Topaz Train
  2. limit training input to a few hundred exposures. You can create a subset of exposures with the Exposure Sets tool.

If these suggestions do not help, please post the outputs of these commands

csprojectid="P99" # substitute actual project ID
csjobid="J999" # substitute actual job ID
cryosparcm cli "get_job('$csprojectid', '$csjobid', 'job_type', 'version', 'instance_information', 'status',  'params_spec', 'errors_run', 'input_slot_groups')"
cryosparcm joblog $csprojectid $csjobid | tail -n 20
cryosparcm eventlog $csprojectid $csjobid | tail -n 40

for a Topaz Train job for which you left blank the Absolute path of directory containing preprocessed directory parameter and to which you connected a subset of the exposures exported from CryoSPARC Live.

3

Just to confirm, splitting the exposures into subsets (3,000 exposures per split) and leaving the absolute file path blank fixed this.

Thanks!

1 Like