Topaz extraction -struct.error: unpack requires a buffer of 1024 bytes

Particle Picking
1

Dear colleagues,

Running the Topaz Extract I received the following error - struct.error: unpack requires a buffer of 1024 bytes

Traceback (most recent call last):
File “cryosparc_master/cryosparc_compute/run.py”, line 115, in cryosparc_master.cryosparc_compute.run.main
File “/home/cryosparc_user/cryosparc/cryosparc_worker/cryosparc_compute/jobs/topaz/run_topaz.py”, line 1182, in run_topaz_wrapper_extract
utils.run_process(extract_command)
File “/home/cryosparc_user/cryosparc/cryosparc_worker/cryosparc_compute/jobs/topaz/topaz_utils.py”, line 99, in run_process
assert process.returncode == 0, f"Subprocess exited with status {process.returncode} ({str_command})"
AssertionError: Subprocess exited with status 1 (/home/cryosparc_user/.conda/envs/topaz/bin/topaz extract --radius 24 --threshold -6 --up-scale 9 --assignment-radius -1 --min-radius 5 --max-radius 100 --step-radius 5 --num-workers 16 --device 0 --model /data/dmitry/cryosparc/CS-cs-XXXX/J467/mod…)

Any ideas?

Thank you.

Kind regards,
Dmitry

2

@Dmitry Please can you post a log excerpt that shows

  • the lines leading up and including struct.error
  • the relative position of struct.error with respect to the Traceback you posted already
  • the lines between the struct.error and Traceback
1 Like
3

Hello @wtempel and all,

The issue seemed to be solved.
One has to be careful when using a big dataset with Topaz (about 20 000 images).
Splitting dataset on pieces (about 8000) with later on Topaz extraction fixed the error.

Thank you.

Kind regards,
Dmitry

1 Like
4

Hello,

I am having a similar problem, but I already split my dataset into pieces of ~5700 micrographs. Anyone have any ideas on how to fix this issue?

5

Hello @pyoung,

some possible strategy to move with.

  1. use defaults settings for the Topaz (fill-in what is absolutely needed - radius, expected number of particles per movie image)
  2. Use minimum (default) recourses CPU/GPU
  3. Check the input - good if it is the micrographs after the motion correction. Maybe you may want to repeat the splitting.
  4. Check the old projects - did Topaz performed well in some of them? Did you update it recently? If so - copy and re-run Topaz on the successful protocol again to see if it works at all.
  5. Post the error you have so cryoSPARC team can check it more closely.

Hopefully the issue will be solved.

Kind regards,

Dmitry