Cryosparc 4.1 upgrade leading to failed inspect picks job

Troubleshooting
1

Cross-posting from (Remove duplicates doesn't work) since the original issue from the post seems to be closed.

Hi, after updating to cryosparc 4.1, I tried running an inspect picks job that fails and doesn’t finish.

Error currently looks like this:

Traceback (most recent call last):
  File "cryosparc_master/cryosparc_compute/run.py", line 93, in cryosparc_compute.run.main
  File "/local1/local/storage/software/cryosparc3/cryosparc_master/cryosparc_compute/jobs/interactive/run_inspect_picks_v2.py", line 171, in run
    mic_uid_to_particle_dset = particles_dset.split_by('location/micrograph_uid')
  File "/local1/local/storage/software/cryosparc3/cryosparc_master/cryosparc_tools/cryosparc/dataset.py", line 1310, in split_by
    return {val: self.take(idx) for val, idx in idxs.items()}
  File "/local1/local/storage/software/cryosparc3/cryosparc_master/cryosparc_tools/cryosparc/dataset.py", line 1310, in <dictcomp>
    return {val: self.take(idx) for val, idx in idxs.items()}
  File "/local1/local/storage/software/cryosparc3/cryosparc_master/cryosparc_tools/cryosparc/dataset.py", line 1251, in take
    return type(self)([(f, self[f][indices]) for f in self])
  File "/local1/local/storage/software/cryosparc3/cryosparc_master/cryosparc_tools/cryosparc/dataset.py", line 638, in __init__
    self.add_fields([entry[0] for entry in populate])
  File "/local1/local/storage/software/cryosparc3/cryosparc_master/cryosparc_tools/cryosparc/dataset.py", line 933, in add_fields
    ), f"Could not add {field} with dtype {dt}"
AssertionError: Could not add ('uid', '<u8') with dtype uint64

Any other files I could send over to help figure out what’s wrong?

Job log appears like so:


================= CRYOSPARCW =======  2022-12-18 11:27:34.332588  =========
Project P37 Job J163
Master cbsukellogg.biohpc.cornell.edu Port 8015
===========================================================================
========= monitor process now starting main process
MAINPROCESS PID 3681526
MAIN PID 3681526
interactive.run_inspect_picks_v2 cryosparc_compute.jobs.jobregister
========= monitor process now waiting for main process
***************************************************************
INTERACTIVE JOB STARTED ===  2022-12-18 11:27:56.726596  ==========================
========= sending heartbeat
========= sending heartbeat
========= sending heartbeat
========= sending heartbeat
========= sending heartbeat
========= sending heartbeat
========= sending heartbeat
========= sending heartbeat
========= sending heartbeat
========= sending heartbeat
========= sending heartbeat
========= sending heartbeat
========= sending heartbeat
========= sending heartbeat
========= sending heartbeat
uid: invalid handle 562949953454080, wrong generation counter (given 1, expected 5) (errno 1: Operation not permitted)
add column: invalid handle 562949953454080, wrong generation counter (given 1, expected 5) (errno 1: Operation not permitted)
**** handle exception rc
set status to failed
========= main process now complete.
========= monitor process now complete.```

Can I get some help on why this might be happening?
2

A post was merged into an existing topic: Remove duplicates doesn’t work

3