Detected NaN values during Cryosparc v5 negstain ab-ini

every time i do ab-ini, it will reported error, and it always happen in 51 iterations, while other jobs seem to be normal

Thanks @J6H55 for your post. I modified the topic title with a more specific summary of the problem.
Please can you the text output as text (instead of a screenshot) to improve forum search results? In CryoSPARC v5, one can print the event log to the terminal with the command

cryosparcm job events P999 J9

(with project and job IDs modified as appropriate).

sure!

2026-10-09 21:53:37.75\] \[CPU: 659.6 MB

[Avail: 210.75 GB]----------- Iteration 35 (epoch 1.022). radwn 15.66 resolution 20.00A minisize 200 beta 0.10

2026-10-09 21:53:38.35\] \[CPU: 659.6 MB
Avail: 210.73 GB

– Class 0 – lr: 0.40 eps: 4.39 step ratio : inf ESS R: 8925.129 S: 12.998 Class Size: 100.0% (Average: 100.0%)

2026-10-09 21:53:38.36\] \[CPU: 659.6 MB
Avail: 210.73 GB

Done iteration 00035 of 00761 in 0.620s. Total time 64.7s.

2026-10-09 21:53:38.37\] \[CPU: 659.6 MB

[Avail: 210.73 GB]----------- Iteration 36 (epoch 1.050). radwn 15.66 resolution 20.00A minisize 200 beta 0.10

2026-10-09 21:53:39.05\] \[CPU: 659.6 MB
Avail: 210.73 GB

– Class 0 – lr: 0.40 eps: 4.39 step ratio : inf ESS R: 8925.132 S: 12.998 Class Size: 100.0% (Average: 100.0%)

2026-10-09 21:53:39.06\] \[CPU: 659.6 MB
Avail: 210.73 GB

Done iteration 00036 of 00761 in 0.690s. Total time 65.4s.

2026-10-09 21:53:39.08\] \[CPU: 659.6 MB

[Avail: 210.73 GB]----------- Iteration 37 (epoch 1.079). radwn 15.66 resolution 20.00A minisize 200 beta 0.10

2026-10-09 21:53:39.81\] \[CPU: 659.6 MB
Avail: 210.73 GB

– Class 0 – lr: 0.40 eps: 4.39 step ratio : inf ESS R: 8925.129 S: 12.998 Class Size: 100.0% (Average: 100.0%)

2026-10-09 21:53:39.82\] \[CPU: 659.6 MB
Avail: 210.73 GB

Done iteration 00037 of 00761 in 0.750s. Total time 66.1s.

2026-10-09 21:53:39.84\] \[CPU: 659.6 MB

[Avail: 210.73 GB]----------- Iteration 38 (epoch 1.107). radwn 15.66 resolution 20.00A minisize 200 beta 0.10

2026-10-09 21:53:40.57\] \[CPU: 661.3 MB
Avail: 210.68 GB

– Class 0 – lr: 0.40 eps: 4.39 step ratio : inf ESS R: 8925.122 S: 12.998 Class Size: 100.0% (Average: 100.0%)

2026-10-09 21:53:40.58\] \[CPU: 661.3 MB
Avail: 210.68 GB

Done iteration 00038 of 00761 in 0.744s. Total time 66.9s.

2026-10-09 21:53:40.60\] \[CPU: 661.3 MB

[Avail: 210.68 GB]----------- Iteration 39 (epoch 1.135). radwn 15.66 resolution 20.00A minisize 200 beta 0.10

2026-10-09 21:53:41.32\] \[CPU: 661.3 MB
Avail: 210.73 GB

– Class 0 – lr: 0.40 eps: 4.39 step ratio : inf ESS R: 8925.131 S: 12.998 Class Size: 100.0% (Average: 100.0%)

2026-10-09 21:53:41.33\] \[CPU: 661.3 MB
Avail: 210.73 GB

Done iteration 00039 of 00761 in 0.742s. Total time 67.6s.

2026-10-09 21:53:41.35\] \[CPU: 661.3 MB
Avail: 210.73 GB

----------- Iteration 40 (epoch 1.164). radwn 15.66 resolution 20.00A minisize 200 beta 0.10

2026-10-09 21:53:42.07\] \[CPU: 661.3 MB
Avail: 210.68 GB

– Class 0 – lr: 0.40 eps: 4.39 step ratio : inf ESS R: 8925.131 S: 12.998 Class Size: 100.0% (Average: 100.0%)

2026-10-09 21:53:42.08\] \[CPU: 661.3 MB
Avail: 210.68 GB

Done iteration 00040 of 00761 in 0.737s. Total time 68.4s.

2026-10-09 21:53:42.10\] \[CPU: 661.3 MB

[Avail: 210.68 GB]----------- Iteration 41 (epoch 1.192). radwn 15.66 resolution 20.00A minisize 200 beta 0.10

2026-10-09 21:53:42.83\] \[CPU: 661.1 MB
Avail: 210.73 GB

– Class 0 – lr: 0.40 eps: 4.39 step ratio : inf ESS R: 8925.124 S: 12.998 Class Size: 100.0% (Average: 100.0%)

2026-10-09 21:53:42.84\] \[CPU: 661.1 MB
Avail: 210.73 GB

Done iteration 00041 of 00761 in 0.744s. Total time 69.1s.

2026-10-09 21:53:42.86\] \[CPU: 661.1 MB

[Avail: 210.73 GB]----------- Iteration 42 (epoch 1.221). radwn 15.66 resolution 20.00A minisize 200 beta 0.10

2026-10-09 21:53:43.59\] \[CPU: 661.3 MB
Avail: 210.73 GB

– Class 0 – lr: 0.40 eps: 4.39 step ratio : inf ESS R: 8925.127 S: 12.998 Class Size: 100.0% (Average: 100.0%)

2026-10-09 21:53:43.60\] \[CPU: 661.3 MB
Avail: 210.73 GB

Done iteration 00042 of 00761 in 0.750s. Total time 69.9s.

2026-10-09 21:53:43.62\] \[CPU: 661.3 MB

[Avail: 210.73 GB]----------- Iteration 43 (epoch 1.249). radwn 15.66 resolution 20.00A minisize 200 beta 0.10

2026-10-09 21:53:44.35\] \[CPU: 661.3 MB
Avail: 210.73 GB

– Class 0 – lr: 0.40 eps: 4.39 step ratio : inf ESS R: 8925.128 S: 12.998 Class Size: 100.0% (Average: 100.0%)

2026-10-09 21:53:44.36\] \[CPU: 661.3 MB
Avail: 210.73 GB

Done iteration 00043 of 00761 in 0.745s. Total time 70.7s.

2026-10-09 21:53:44.38\] \[CPU: 661.3 MB

[Avail: 210.73 GB]----------- Iteration 44 (epoch 1.277). radwn 15.66 resolution 20.00A minisize 200 beta 0.10

2026-10-09 21:53:45.10\] \[CPU: 661.3 MB
Avail: 210.68 GB

– Class 0 – lr: 0.40 eps: 4.39 step ratio : inf ESS R: 8925.125 S: 12.998 Class Size: 100.0% (Average: 100.0%)

2026-10-09 21:53:45.11\] \[CPU: 661.3 MB
Avail: 210.68 GB

Done iteration 00044 of 00761 in 0.742s. Total time 71.4s.

2026-10-09 21:53:45.13\] \[CPU: 661.3 MB

[Avail: 210.68 GB]----------- Iteration 45 (epoch 1.306). radwn 15.66 resolution 20.00A minisize 200 beta 0.10

2026-10-09 21:53:45.86\] \[CPU: 661.3 MB
Avail: 210.73 GB

– Class 0 – lr: 0.40 eps: 4.39 step ratio : inf ESS R: 8925.124 S: 12.998 Class Size: 100.0% (Average: 100.0%)

2026-10-09 21:53:45.87\] \[CPU: 661.3 MB
Avail: 210.73 GB

Done iteration 00045 of 00761 in 0.746s. Total time 72.2s.

2026-10-09 21:53:45.89\] \[CPU: 661.3 MB

[Avail: 210.73 GB]----------- Iteration 46 (epoch 1.334). radwn 15.66 resolution 20.00A minisize 200 beta 0.10

2026-10-09 21:53:46.62\] \[CPU: 661.3 MB
Avail: 210.73 GB

– Class 0 – lr: 0.40 eps: 4.39 step ratio : inf ESS R: 8925.128 S: 12.998 Class Size: 100.0% (Average: 100.0%)

2026-10-09 21:53:46.63\] \[CPU: 661.3 MB
Avail: 210.73 GB

Done iteration 00046 of 00761 in 0.747s. Total time 72.9s.

2026-10-09 21:53:46.65\] \[CPU: 661.3 MB

[Avail: 210.73 GB]----------- Iteration 47 (epoch 1.362). radwn 15.66 resolution 20.00A minisize 200 beta 0.10

2026-10-09 21:53:47.38\] \[CPU: 661.3 MB
Avail: 210.73 GB

– Class 0 – lr: 0.40 eps: 4.39 step ratio : inf ESS R: 8925.125 S: 12.998 Class Size: 100.0% (Average: 100.0%)

2026-10-09 21:53:47.39\] \[CPU: 661.3 MB
Avail: 210.73 GB

Done iteration 00047 of 00761 in 0.746s. Total time 73.7s.

2026-10-09 21:53:47.41\] \[CPU: 661.3 MB

[Avail: 210.73 GB]----------- Iteration 48 (epoch 1.391). radwn 15.66 resolution 20.00A minisize 200 beta 0.10

2026-10-09 21:53:48.15\] \[CPU: 661.3 MB
Avail: 210.66 GB

– Class 0 – lr: 0.40 eps: 4.39 step ratio : inf ESS R: 8925.128 S: 12.998 Class Size: 100.0% (Average: 100.0%)

2026-10-09 21:53:48.16\] \[CPU: 661.3 MB
Avail: 210.66 GB

Done iteration 00048 of 00761 in 0.753s. Total time 74.5s.

2026-10-09 21:53:48.17\] \[CPU: 661.3 MB

[Avail: 210.66 GB]----------- Iteration 49 (epoch 1.419). radwn 15.66 resolution 20.00A minisize 200 beta 0.10

2026-10-09 21:53:48.90\] \[CPU: 661.3 MB
Avail: 210.71 GB

– Class 0 – lr: 0.40 eps: 4.39 step ratio : inf ESS R: 8925.124 S: 12.998 Class Size: 100.0% (Average: 100.0%)

2026-10-09 21:53:48.91\] \[CPU: 661.3 MB
Avail: 210.71 GB

Done iteration 00049 of 00761 in 0.741s. Total time 75.2s.

2026-10-09 21:53:48.93\] \[CPU: 661.3 MB
Avail: 210.71 GB

----------- Iteration 50 (epoch 1.448). radwn 15.66 resolution 20.00A minisize 200 beta 0.10

2026-10-09 21:53:49.94\] \[CPU: 661.3 MB
Avail: 210.71 GB

– Class 0 – lr: 0.40 eps: 4.39 step ratio : inf ESS R: 8925.128 S: 12.998 Class Size: 100.0% (Average: 100.0%)

2026-10-09 21:53:49.97\] \[CPU: 661.3 MB
Avail: 210.71 GB

Done iteration 00050 of 00761 in 1.052s. Total time 76.3s.

2026-10-09 21:53:49.99\] \[CPU: 661.3 MB

[Avail: 210.71 GB]----------- Iteration 51 (epoch 1.476). radwn 15.66 resolution 20.00A minisize 200 beta 0.10

2026-10-09 21:53:50.41\] \[CPU: 609.0 MB
Avail: 210.76 GB

Traceback (most recent call last):
File “cli/run.py”, line 106, in cli.run.run_job
File “cli/run.py”, line 211, in cli.run.run_job_function
File “compute/jobs/abinit/run.py”, line 436, in compute.jobs.abinit.run.run_homo_abinit
File “compute/engine/engine.py”, line 1204, in compute.engine.engine.process
File “compute/engine/engine.py”, line 1205, in compute.engine.engine.process
File “compute/engine/engine.py”, line 1146, in compute.engine.engine.process.work
File “compute/engine/engine.py”, line 431, in compute.engine.engine.EngineThread.compute_error
ValueError: Detected NaN values in engine.compute_error. 23212800 NaNs in total, 200 particles with NaNs.

Thanks @J6H55 .
Please can you also post

  1. the output of this command (with prject and job IDs adjusted as appropriate):
    cryosparcm cli "api.jobs.find_one('P999', 'J9999').params"
    
  2. whether the same particle stacks were used successfully in other job types, like 2D classification.
  3. whether you have checked the same particle stacks for corruption.

i check the particles and no error occurs.

i’ve tried 16bit or normal particles, both cant work, the mic and particle are all with constant CTF.

the same or downstream particles can be used in Job: Homogeneous Ab-Initio Refinement (BETA) or Ab-initio (High Symmetry), but when i turn to nomarl ab-ini, it always reported error at iteration 51.

this is the output of command:

even when doing homogenous ab, it still reported error???

somehow it can just continue to run :exploding_head:

still, mic and particles(from cryolo) are all with constant ctf

Please can you input the same particles that were input to the failed ab initio 3D reconstruction to a 2D classification job and post a screenshot of the resulting 2D classes.

I collect a new dataset and no error being reported again :smiling_face_with_tear: not a problem now