Patch Motion Correction error

Motion Correction
1

Hi all,
There is an error messenge when I do the Patch motion correction. Does anyone know what happen?

-----------------------------------------------------------------------------------------------------------------
Error occurred while processing J3/imported/011447104231546704801_20170629_00022_frameImage.tiff
Traceback (most recent call last):
  File "/home/cryosparc/software/cryosparc_worker/cryosparc_compute/jobs/pipeline.py", line 60, in exec
    return self.process(item)
  File "cryosparc_master/cryosparc_compute/jobs/motioncorrection/run_patch.py", line 117, in cryosparc_compute.jobs.motioncorrection.run_patch.run_patch_motion_correction_multi.motionworker.process
  File "cryosparc_master/cryosparc_compute/jobs/motioncorrection/run_patch.py", line 132, in cryosparc_compute.jobs.motioncorrection.run_patch.run_patch_motion_correction_multi.motionworker.process
  File "cryosparc_master/cryosparc_compute/blobio/prefetch.py", line 71, in cryosparc_compute.blobio.prefetch.synchronous_native_read
RuntimeError: Error ocurred (Input/output error) at line 666 in sendfile

IO request details:filename:    /activeEM/data/cyfu/fuchiyu/YC_test2/CS-tutorial-2/J3/imported/011447104231546704801_20170629_00022_frameImage.tiff
filetype:    1
header_only: 0
idx_start:   0
idx_limit:   -1
eer_upsampfactor: 1
eer_numfractions: 40
num_threads: 6
buffer:      (nil)
nx, ny, nz:  0 0 0
dtype:       0
total_time:  -1.000000


Marking J3/imported/011447104231546704801_20170629_00022_frameImage.tiff as incomplete and continuing.
-----------------------------------------------------------------------------------------------------------

Any help is appreciated

2

Looks like a corrupt micrograph. Check your micrographs, check your filesystem for corruption (and/or disks for bad sectors) or a failure in the network connection if a network filesystem.

3

Hi @cafbi,

Is your installation running under docker, or is there anything else in any way peculiar about your installation? Is it a cluster? Also, how much memory does your machine have?

Regardless, though, you can probably avoid this by opening the file /path/to/cryosparc_worker/config.sh and adding the line export CRYOSPARC_TIFF_IO_SHM=false. That will turn off shared-memory I/O, wherein we read an entire TIFF file into memory before decompressing it (rather than decompressing it piece-by-piece directly from disk). We added this as an optimization, and turning this off may have a negative performance impact if you’re using a cluster file system like Lustre.

—Harris

5

Hi @hsnyder ,

What if the problem is with .eer?

I’m having the message

Error occurred while processing J1/imported/018020536438261153333_FoilHole_13203630_Data_13184437_24_20240913_193718_EER.eer
Traceback (most recent call last):
File “/home/cryosparc_user/software/cryosparc/cryosparc_worker/cryosparc_compute/jobs/pipeline.py”, line 59, in exec
return self.process(item)
File “cryosparc_master/cryosparc_compute/jobs/motioncorrection/run_patch.py”, line 149, in cryosparc_master.cryosparc_compute.jobs.motioncorrection.run_patch.run_patch_motion_correction_multi.motionworker.process
File “cryosparc_master/cryosparc_compute/jobs/motioncorrection/run_patch.py”, line 164, in cryosparc_master.cryosparc_compute.jobs.motioncorrection.run_patch.run_patch_motion_correction_multi.motionworker.process
File “cryosparc_master/cryosparc_compute/blobio/prefetch.py”, line 82, in cryosparc_master.cryosparc_compute.blobio.prefetch.synchronous_native_read
OSError:

IO request details:
Error ocurred (Input/output error) at line 1096 in sendfile

filename: /data/20240913_LBNC6g3_data/CS-nc-ip6/J1/imported/018020536438261153333_FoilHole_13203630_Data_13184437_24_20240913_193718_EER.eer
filetype: 2
header_only: 0
idx_start: 0
idx_limit: -1
eer_upsampfactor: 1
eer_numfractions: 40
num_threads: 6
buffer: (nil)
buffer_sz: 0
nx, ny, nz: 0 0 0
dtype: 0
total_time: -1.000000
io_time: 0.000746

Marking J1/imported/018020536438261153333_FoilHole_13203630_Data_13184437_24_20240913_193718_EER.eer as incomplete and continuing…

Should I do similar thing that you suggested? Thanks!!

6

Hi @Briganti,

The EER format actually uses TIFF as a container format, so much of the advice relating to TIFF files applies to EER as well. So yes, you could indeed try what I suggested. However, if you upgrade to CryoSPARC v4.6, there should be no need to do so, as the way we read TIFF files has changed, and we don’t make sendfile calls to read TIFFs anymore.

– Harris