Motion correction failure in v2.13

[Juno]

Motion correction on empiar subset sample data, fails at the end, missing reference to micrographs.micrograph_thumbnail_blob_1x
The same job succeeds after downgrading back to v2.12.4

Traceback (most recent call last):
File “cryosparc2_worker/cryosparc2_compute/run.py”, line 78, in cryosparc2_compute.run.main
File “cryosparc2_worker/cryosparc2_compute/jobs/motioncorrection/run.py”, line 349, in cryosparc2_compute.jobs.motioncorrection.run.run_rigid_motion_correction_multi
File “cryosparc2_worker/cryosparc2_compute/jobs/motioncorrection/run.py”, line 285, in cryosparc2_compute.jobs.motioncorrection.run.run_rigid_motion_correction_multi.make_outputs
File “cryosparc2_compute/jobs/runcommon.py”, line 442, in output
result = com.query(job[‘output_results’], lambda r: r[‘group_name’] == group_name and r[‘name’] == name, error=“No output result named %s.%s in job.” % (group_name, name))
File “cryosparc2_compute/jobs/common.py”, line 349, in query
assert res != default, error
AssertionError: No output result named micrographs.micrograph_thumbnail_blob_1x in job.

[stephan]

Hi @Juno,

Did you start a new job? Clearing an existing job and re-running it would cause this error. If that doesn’t work, run cryosparcm cli "refresh_job_types()"

[stavros]

I have the same issue, and I even created a new session for the processing of the data with the same error.

[Juno]

I started a new test run from scratch. Movies were imported with v2.13, then a new motion correction job created. – I assume this would be easy to reproduce, I’m using the distributed empiar test data.

[stephan]

Hi @Juno, @stavros,

Can you follow this post:

Import Micrographs broken in 2.12?

Hi @Peiyu-Xu, Can you confirm you’re creating a new job and not cloning the original job that caused the issue? If that doesn’t work, can you make sure there are no zombie processes: cryosparcm stop ps -ax | grep “supervisord” (kill only the process that is running from your cryosparc2 install) ps -ax | grep “cryosparc2_command” (kill all the matching processes related to your cryosparc2 instance) ps -ax | grep “mongod” (kill only the process running your cryosparc2 database) e.g. kill 82681…

[Juno]

ps -ax | grep “supervisord”

SOLVED… OK, I did “cryosparcm stop” and found that supervisord was still running, with a run time that meant it never stopped at the last upgrade/downgrade. The problem seems to have been supervisord not being shut down correctly, which I think has happened before. Stopped all processes, repeated the upgrade, and now it’s all working. But also the release updated to 2.13.2