I met the problem in NU refinement

3D Refinement
1

Using Filter Radius 22.190 (12.806A) | Previous: 9.472 (30.000A)

Non-uniform regularization with compute option: GPU

Running local cross validation for A …

====== Job process terminated abnormally.

I use A100， box size is 384，and detail log is：gpufft: creating new cufft plan (plan id 8 pid 2367825)
gpu_id 0
ndims 3
dims 192 192 192
inembed 192 192 97
istride 1
idist 3575808
onembed 192 192 194
ostride 1
odist 7151616
batch 1
type C2R
wkspc manual
Python traceback:

/data/cryosparc/cryosparc_worker/cli/cryosparcw.py:290: UserWarning: Cannot manually free CUDA array; will be freed when garbage collected
return run(conf)
/data/cryosparc/cryosparc_worker/cli/cryosparcw.py:290: RuntimeWarning: invalid value encountered in divide
return run(conf)
2026-07-13 05:18:13,177 core heartbeat INFO | ========= Updating heartbeat
gpufft: creating new cufft plan (plan id 9 pid 2367825)
gpu_id 0
ndims 3
dims 384 384 384
inembed 384 384 193
istride 1
idist 28459008
onembed 384 384 386
ostride 1
odist 56918016
batch 1
type C2R
wkspc manual
Python traceback:

2026-07-13 05:18:23,205 core monitor INFO | ========= monitor detected main process exit.
2026-07-13 05:18:23,311 core monitor INFO | ========= monitor process now complete.

2

Welcome to the forum @liuwenzhe .
What version of CryoSPARC do you use?
Please can you also confirm whether an actual traceback is or is not displayed following the

line inside the log.

4

I also make a test with different version in a100, but the NU refinement and Local refinement are the same problem.I make a test in rx3090, the version is 5.0.6, this is no problem with two refinement. How can I do next?

5

Thanks @liuwenzhe. Good to know that you didn’t see this problem with the other GPU type.
Please can you post the outputs of these commands

  • run on the CryoSPARC master
    project_uid=P999 # replace with actual project ID
j_rtx3090=J99 # replace with ID of RTX3090 successful job
j_a100=J100 # replace with ID of A100 failed job
cryosparcm cli "dict(filter(lambda x: x[0] in ['uid', 'type', 'status', 'instance_information'], api.jobs.find_one('$project_uid', '$j_rtx3090').model_dump().items()))"
cryosparcm cli "dict(filter(lambda x: x[0] in ['uid', 'type', 'status', 'instance_information'], api.jobs.find_one('$project_uid', '$j_a100').model_dump().items()))"
  • run on both the A100 and RTX3090 hosts:
    uname -a
nvidia-smi