Local Motion Correction error: 'all the input arrays must have same number of dimensions'

Motion Correction
1

Hi everyone,

I’m running Local Motion Correction with downsampled particles after homogeneous refinement. Micrographs come from “Micrograph curation” (the particles were extracted after curation as well). After some time (plenty of micrographs processed, if not all), I see:

  File "cryosparc_worker/cryosparc_compute/run.py", line 93, in cryosparc_compute.run.main
  File "cryosparc_worker/cryosparc_compute/jobs/motioncorrection/run_local.py", line 398, in cryosparc_compute.jobs.motioncorrection.run_local.run_local_motion_correction_multi
  File "<__array_function__ internals>", line 6, in concatenate
ValueError: all the input arrays must have same number of dimensions, but the array at index 0 has 3 dimension(s) and the array at index 4819 has 1 dimension(s)

And have no idea how to fix that.
I’m running v4.0.2 CryoSPARC.

2

an update to that: the failure was with the downsampled particles (extract with box size 640 but downsample to 128). However, with the re-extracted 640px particles, everything went smoothly and finished successfully.

1 Like
3

Thanks for sharing this fix. I ran into the same error on v4.4.1 using homogeneous refinement with downsampled particles (from local motion correction-singlegpu) as input to local motion multigpu (with less downsampling). What’s strange is the first run of local motion correction input was downsampled particles (380->62px) from a select 2D class job. This first run of local motion correction was performed with downsampling of 400->260px. Now I want to take a subset of those particles and run local motion correction again with less downsampling.

~3,000 out of 5,000 movies seem to have been processed successfully, but the job failed after running for 14hr with the following error:
Traceback (most recent call last):
File “cryosparc_master/cryosparc_compute/run.py”, line 95, in cryosparc_master.cryosparc_compute.run.main
File “cryosparc_master/cryosparc_compute/jobs/motioncorrection/run_local.py”, line 408, in cryosparc_master.cryosparc_compute.jobs.motioncorrection.run_local.run_local_motion_correction_multi
File “<array_function internals>”, line 5, in concatenate
ValueError: all the input arrays must have same number of dimensions, but the array at index 0 has 3 dimension(s) and the array at index 957 has 1 dimension(s)

Here’s the pathway
patch motion, patch ctf, picking, extract with binning 380->62px, multiple rounds of 2D, 2D select particles as input for local motion correction (single-GPU) with binning 400->260px, more rounds of 2D, select all side views and subset of top views, homogeneous refine, local motion correction (multi-GPU) with binning 480->374

Maybe the problem is with multi-gpu local motion, or with using local motion-singlegpu downsampled particles as input to local motion-multi-gpu, or with using homo refine downsampled particles as input, instead of 2D select as input. There are many things to test and they may take 14hrs each, so it would be great to narrow it down to a single factor to change. I may just repeat the original local motion correction job with less binning but I really only need a small subset of those particles.

4

I ran into the same issue again with a cloned job but using the 2D selected particles as input, but the job went farther. This time it got through 4,741 of 5,341 movies. Is there a way to write out the completed particles and only process the incomplete movies? Mark job as complete does not make the particles available.

Exact error is:
Traceback (most recent call last):
File “cryosparc_master/cryosparc_compute/run.py”, line 95, in cryosparc_master.cryosparc_compute.run.main
File “cryosparc_master/cryosparc_compute/jobs/motioncorrection/run_local.py”, line 408, in cryosparc_master.cryosparc_compute.jobs.motioncorrection.run_local.run_local_motion_correction_multi
File “<array_function internals>”, line 5, in concatenate
ValueError: all the input arrays must have same number of dimensions, but the array at index 0 has 3 dimension(s) and the array at index 957 has 1 dimension(s)

5

@cryofun Please can you try

  1. a Manually Curate Exposures job with exposures and particle inputs to reject all exposures where the Number of picked particles is not at least 1.
  2. Retry Local Motion Correction with only the accepted exposures from the previous step.

Does this help?