Hi all,
I’m encountering a recurring issue during ab-initio reconstruction.
> Apr 3, 2026, 2:06:36 PM
Traceback (most recent call last):
File “cli/run.py”, line 105, in cli.run.run_job
File “cli/run.py”, line 210, in cli.run.run_job_function
File “compute/jobs/abinit/run.py”, line 436, in compute.jobs.abinit.run.run_homo_abinit
File “compute/engine/engine.py”, line 1204, in compute.engine.engine.process
File “compute/engine/engine.py”, line 1205, in compute.engine.engine.process
File “compute/engine/engine.py”, line 1146, in compute.engine.engine.process.work
File “compute/engine/engine.py”, line 431, in compute.engine.engine.EngineThread.compute_error
ValueError: Detected NaN values in engine.compute_error. 979776000 NaNs in total, 500 particles with NaNs.
The error does not occur at a fixed iteration:
-
In some runs, it appears around iteration ~43
-
In others, it happens much later (e.g., ~1900 iterations)
-
Before crashing, the reconstruction often becomes abnormal
-
Dear @pey123456 ,
Apologies for the delay. We would appreciate if you could answer the following questions:
- Can you confirm whether this is a single-class ab-initio job, multi-class ab-initio job, or an ab-initio refinement job?
- Have there been any successful runs using this particle stack, or have all of them resulted in these errors at some point in the job’s execution?
- Has this error been observed across multiple particle stacks, or is it specific to this one?
- In the failing runs, do any intermediate iterations produce intact results that appear normal looking, or are they consistently abnormal from the start of the job?
- Could you provide the values of all non-default parameters used in these runs?
Let me know if you are also willing to share full event logs of the jobs with us to help us understand the failure, and I can provide upload instructions.
Michael
Dear Michael,
Thank you for your response.
- This is a multi-class ab-initio job. I have tried using both 6 classes and 12 classes, and both settings resulted in the same error.
- All of the ab-initio jobs I have attempted with this particle stack have eventually failed.
- So far, I have only observed this issue with this particular dataset. Another dataset with a similar biochemical sample and collected under the same microscope data acquisition parameters ran normally.
- Yes, some intermediate iterations did produce results that looked normal and intact.
- No problem. These are the parameters from the two failed jobs:
Thanks for bearing with me @pey123456. Based on your replies, we would be able to debug this most effectively if we could test directly on your data. Are you willing to share data with our team, in confidence? Let me know; if so, I can provide upload instructions.
Kind regards,
Michael
Hi Michael,
Thank you very much for your kind offer and willingness to help.
Unfortunately, we are planning to re-prepare the samples and recollect the data, and the previous raw and processed data have already been deleted.
If we encounter the same issue again in the future, I would be very happy to share the data with you for debugging.
Best,
Hao
1 Like