Can't delete failed 2.15 jobs in 3.0

Troubleshooting
1

I get a stale job error message when I try to delete a failed job that was started in v2.15 before my upgrade to 3.0. (Extract particles, because of the missing recenter shifts parameter). I ran the refresh jobs command.

2

Hi @DanielAsarnow,

Are there any errors in the logs? cryosparcm log command_core

- Suhail

3

This one seems triggered by the delete attempt. The message log in the webui is blank.

Traceback (most recent call last):
  File "/mnt/tyr-data/cryosparc/cryosparc2/cryosparc2_master/cryosparc_command/command_core/__init__.py", line 115, in wrapper
    res = func(*args, **kwargs)
  File "/mnt/tyr-data/cryosparc/cryosparc2/cryosparc2_master/cryosparc_command/command_core/__init__.py", line 2353, in delete_job
    clear_job(project_uid, job_uid, nofail=nofail, isdelete=delete_job_data)
  File "/mnt/tyr-data/cryosparc/cryosparc2/cryosparc2_master/cryosparc_command/command_core/__init__.py", line 124, in wrapper
    raise e
  File "/mnt/tyr-data/cryosparc/cryosparc2/cryosparc2_master/cryosparc_command/command_core/__init__.py", line 115, in wrapper
    res = func(*args, **kwargs)
  File "/mnt/tyr-data/cryosparc/cryosparc2/cryosparc2_master/cryosparc_command/command_core/__init__.py", line 4857, in clear_job
    builder.validate_inputs()
  File "/mnt/tyr-data/cryosparc/cryosparc2/cryosparc2_master/cryosparc_compute/jobs/extract/build.py", line 86, in validate_inputs
    recenter_picks = params['recenter_using_shifts'] and (alignments2D_connected or alignments3D_connected)
  File "/mnt/tyr-data/cryosparc/cryosparc2/cryosparc2_master/cryosparc_compute/util/__init__.py", line 50, in __getitem__
    """%(key)
AssertionError: "recenter_using_shifts" parameter not available. This job may be stale.
    Please try the following steps in order until the issue is resolved:
    1. Build a new job and try again.
    2. Run the following via command line: cryosparcm cli "refresh_job_types()".
    3. Ensure no orphaned cryoSPARC processes are running in the background.
4

Hi @DanielAsarnow,

This is a bug- we’re going to fix in via a patch that will be released before the end of next week. For the time being, you can do the following to get around jobs that fail with this error when trying to delete them:

  1. Mark the job as deleted: cryosparcm cli "update_job('<project_uid>', '<job_uid>', {'deleted':True})"
  2. Manually delete the job folder (You can get the path by running the command: cryosparcm cli "get_job_dir_abs(<project_uid>, <job_uid>)"
5

I thought this was fixed in 3.0.1 but maybe I’m mistaken? It seems like it’s still an issue when at least when clearing jobs.

Also - is there a function to just clear the job? Folks like to just clear their extractions in order to keep the input parameters.

6

Just to update this - deleting jobs works fine, but they can’t be cleared either through the GUI or on the command line. I told people to clone their jobs and delete the originals to preserve inputs and make space.

7

Hi @DanielAsarnow,

Thanks for reporting this workaround. In our next release (cryoSPARC v3.1.0), this issue should be fixed and Extract from Micrographs jobs from pre-v3.0 (and 2D Classification jobs) should be able to be cleared successfully.

Best,
Michael