Topaz error "struct.error: unpack requires a buffer of 1024 bytes"

Troubleshooting
1

Dear All,

My topaz extract run is halting with the following message.

“struct.error: unpack requires a buffer of 1024 bytes”
Full log is below.

Is somebody able to help me troubleshooting this ?
With many thanks.

GIA

[CPU: 296.1 MB Avail: 380.51 GB]
Starting extraction by running command /usr/local/bin/topaz extract --radius 7 --threshold -6 --up-scale 4 --assignment-radius -1 --min-radius 5 --max-radius 100 --step-radius 5 --num-workers 8 --device 0 --model /data/cryoEM/XXX/C1924_GIA_XXX_041124/cryosparc/CS-gia-XXX-c1924/J177/models/model_epoch34.sav -o /data/cryoEM/XXX/C1924_GIA_XXX_041124/cryosparc/CS-gia-XXX-c1924/J196/topaz_particles_prediction.txt [8998 MICROGRAPH PATHS EXCLUDED FOR LEGIBILITY]

[CPU: 296.4 MB Avail: 376.51 GB]
Traceback (most recent call last):

[CPU: 296.4 MB Avail: 376.51 GB]
File “/usr/local/bin/topaz”, line 11, in

[CPU: 296.4 MB Avail: 376.51 GB]
sys.exit(main())

[CPU: 296.4 MB Avail: 376.51 GB]
File “/usr/local/lib/python3.6/dist-packages/topaz/main.py”, line 148, in main

[CPU: 296.4 MB Avail: 376.51 GB]
args.func(args)

[CPU: 296.4 MB Avail: 376.51 GB]
File “/usr/local/lib/python3.6/dist-packages/topaz/commands/extract.py”, line 288, in main

[CPU: 296.4 MB Avail: 376.51 GB]
for path,score,coords in nms_iterator(stream, radius, threshold, pool=pool):

[CPU: 296.4 MB Avail: 376.51 GB]
File “/usr/local/lib/python3.6/dist-packages/topaz/commands/extract.py”, line 79, in nms_iterator

[CPU: 296.4 MB Avail: 376.51 GB]
for name,score,coords in pool.imap_unordered(process, scores):

[CPU: 296.4 MB Avail: 376.51 GB]
File “/usr/lib/python3.6/multiprocessing/pool.py”, line 735, in next

[CPU: 296.4 MB Avail: 376.51 GB]
raise value

[CPU: 296.4 MB Avail: 376.51 GB]
struct.error: unpack requires a buffer of 1024 bytes

[CPU: 296.4 MB Avail: 380.46 GB]
Traceback (most recent call last):
File “cryosparc_master/cryosparc_compute/run.py”, line 95, in cryosparc_master.cryosparc_compute.run.main
File “/data/software/cryosparc/cryosparc_worker/cryosparc_compute/jobs/topaz/run_topaz.py”, line 1128, in run_topaz_wrapper_extract
utils.run_process(extract_command)
File “/data/software/cryosparc/cryosparc_worker/cryosparc_compute/jobs/topaz/topaz_utils.py”, line 99, in run_process
assert process.returncode == 0, f"Subprocess exited with status {process.returncode} ({str_command})"
AssertionError: Subprocess exited with status 1 (/usr/local/bin/topaz extract --radius 7 --threshold -6 --up-scale 4 --assignment-radius -1 --min-radius 5 --max-radius 100 --step-radius 5 --num-workers 8 --device 0 --model /data/cryoEM/XXX/C1924_GIA_XXX_041124/cryosparc/CS-gia-XXX-c1924/J177/models/mo…)

13m 48s

default

1

NVIDIA RTX A5000
1

2

Hi @giax,

There can be a couple reasons as to why this crops up. If you have over 8,000 or so micrographs in your dataset or if during preprocessing, a micrograph fails.

To fix, ensure your dataset is split into ~8,000 micrograph batches for extraction and make sure all of the micrographs present in the preprocessing directory have the same file size. If you see any that are really small (like a few KB) compared to the others, then delete the small mics and rerun the job.

If neither of these work, please let me know.

Best,
Kye

3

Hi and thanks,

actually I split my dataset in 2 batches, containing each ~4000
the first one is working fine, but the second one, GPU/CPU give the same error… can be a broken file or something like that ?

GIA

4

Yes, it is likely that preprocessing failed on a micrograph present in your second batch. Please navigate to the folder containing the TOPAZ preprocessed micrographs and make sure all of the micrographs present in the preprocessing directory have similar file sizes. If you see any that are really small (like a few KB) compared to the others, then delete the small mics from the TOPAZ preprocessing folder and rerun the job.

Best,
Kye