Hello, is it possible that this “pairwise distance matrix” check could be canceled, or have an option to bypass? We noticed that even for normal single particle cryoEM (e.g., nucleosome sample rather than filament), as long as two particles are too closed to each other it will cause this error, even if they are actually two different particles.
DIE: particle pairwise distance matrix is singular, check for duplicate particles (sgetrf failed)
========= sending heartbeat at 2023-12-29 13:04:35.686924
========= main process now complete at 2023-12-29 13:04:38.713599.
========= monitor process now complete at 2023-12-29 13:04:38.759795.
Hello, it happened both in reference based motion correction jobs, no matter the parameter search, imperial dose weight nor motion correction of particles.
Quite a few people have reported this. I believe the CryoSPARC team are looking into it. In the mean time, a workaround is split your data into subsets (remove duplicates works fine, then take both outputs into different RBMC jobs) and recombine afterwards (use the same training params for all subsets).
@rbs_sci is correct. The issue is that some particles are too close together. We are indeed looking into this, as it is something that several users have reported.
When this job is submitted with duplicates in the stack, the job fails for the reasons reported here.
If the job is submitted to the default queue, the matrix singularity error appears.
However, if the job is sent to SLURM, the only error message is “Job terminated abnormally”, with no mention of the matrix issue.
If duplicate particles remain an issue after the fix, it would be good to fix the error reporting for SLURM jobs too.
Thanks
@Andrea Please can you check whether you can see the error message in the job log (under Metadata|Log) in the case of a SLURM job that failed due to matrix is singular.
[…]
DIE: particle pairwise distance matrix is singular, check for duplicate particles (sgetrf failed)
========= main process now complete at 2024-01-10 15:28:09.940007.
========= monitor process now complete at 2024-01-10 15:28:10.003739.