EER file import error

Import and Export
1

Hi,
This is a really stupid question but somehow I’m stuck here. For very first time I got some movies in .eer format. One data set gave no hassle but in another dataset while running import movie with .eer files I’m getting this following error:

[CPU: 6.65 GB]   Traceback (most recent call last):
  File "cryosparc_master/cryosparc_compute/run.py", line 84, in cryosparc_compute.run.main
  File "/home/emPackage/CS_3.2/cryosparc_master/cryosparc_compute/jobs/imports/run.py", line 922, in run_import_movies_or_micrographs
    mov_raw = eer.read_eer(abs_path, num_fractions, upsampfactor=1)
  File "/home/emPackage/CS_3.2/cryosparc_master/cryosparc_compute/blobio/eer.py", line 86, in read_eer
    eer = open_eer(fname)
  File "/home/emPackage/CS_3.2/cryosparc_master/cryosparc_compute/blobio/eer.py", line 15, in open_eer
    return eerdecompressor.ElectronCountedFramesDecompressor(str(fname))
RuntimeError: unable to open tiff file

The end of log file of the job is as follows:

========= sending heartbeat

========= sending heartbeat

========= sending heartbeat

========= sending heartbeat

Can not read TIFF directory countFailed to read directory at offset 862900252========= sending heartbeat

Unknown field with tag 65100 (0xfe4c) encounteredUnknown field with tag 65100 (0xfe4c) encounteredUnknown field with tag 65100 (0xfe4c) encounteredUnknown field with tag 65100 (0xfe4c) encounteredUnknown field with tag 65100 (0xfe4c) encounteredUnknown field with tag 65100 (0xfe4c) encounteredUnknown field with tag 65100 (0xfe4c) encounteredUnknown field with tag 65100 (0xfe4c) encounteredUnknown field with tag 65100 (0xfe4c) encounteredUnknown field with tag 65100 (0xfe4c) encounteredUnknown field with tag 65100 (0xfe4c) encounteredUnknown field with tag 65100 (0xfe4c) encounteredUnknown field with tag 65100 (0xfe4c) encounteredUnknown field with tag 65100 (0xfe4c) encounteredCan not read TIFF directory countFailed to read directory at offset 862923212min: 481.894072 max: 646.770601

min: 8534.238495 max: 9524.394318

min: 8534.238495 max: 9524.394318

min: 2056.168755 max: 2456.969917

**** handle exception rc

set status to failed

========= main process now complete.

========= monitor process now complete.

Please help me out regarding how to resolve this issue.

Thank you

2

Hi @diffracteD, please see this post and let us know if you are able to have a look at the job log.

3

Hi @spunjani, I am getting a similar error -
TIFFReadDirectory: Failed to read directory at offset …
I was hoping to get insight from your reply. You mentioned “please see this post…”, but I don’t see any link or referral attached to the response. Am I missing something?

4

@aapeters I had linked to this post: Local motion correction error: RuntimeError: unable to open tiff file but I think discourse’s formatting isn’t the most obvious :slight_smile:

5

Thank you so much for responding.
Finally, after extensive checking, I could figure out that CS is not being able to run motioncorrection on some movies (not understood why from the error). So, I ran same job multiple times to get all the names of the corrupted movies as per CS.
Then I removed those movies manually from the directory. After that when I ran CS patch motion correction again, it finished successfully.
I never used EER files before. For tiff movies, CS normally skips the corrupted movies and keep processing.

6

Hi @diffracteD,

It definitely sounds like that .eer file is corrupt. However, as you’ve said, normally patch motion correction skips over unreadable movies and proceeds processing the rest. I will look into why this is not happening with EER data, this sounds like a bug.

–Harris

7

Hello, I am also having this issue. However, it seems not to be just one eer files, but most of them.

8

Please can you paste the errors you saw, and let us know the job type as well as your CryoSPARC version and patch level?

Were there any EER files in the set that were successfully processed?

9

It seems that they get successfully imported, but the patch motion correction job has some traceback errors.

[CPU: 214.3 MB] Error occurred while processing J2/imported/009148797359298835130_FoilHole_9006262_Data_8993834_8993836_20220909_010506_EER.eer
Traceback (most recent call last):
File “/home/renee.arias/cryosparc/cryosparc_worker/cryosparc_compute/jobs/pipeline.py”, line 60, in exec
return self.process(item)
File “cryosparc_master/cryosparc_compute/jobs/motioncorrection/run_patch.py”, line 118, in cryosparc_compute.jobs.motioncorrection.run_patch.run_patch_motion_correction_multi.motionworker.process
File “cryosparc_master/cryosparc_compute/jobs/motioncorrection/run_patch.py”, line 133, in cryosparc_compute.jobs.motioncorrection.run_patch.run_patch_motion_correction_multi.motionworker.process
File “cryosparc_master/cryosparc_compute/blobio/prefetch.py”, line 71, in cryosparc_compute.blobio.prefetch.synchronous_native_read
RuntimeError: Error ocurred (Input/output error) at line 903 in open
Could not open file. Make sure the path exists.

IO request details:filename: /home/renee.arias/data/em/220908_beta_gal_glacios/proc/P9/J2/imported/009148797359298835130_FoilHole_9006262_Data_8993834_8993836_20220909_010506_EER.eer
filetype: 2
header_only: 0
idx_start: 0
idx_limit: -1
eer_upsampfactor: 2
eer_numfractions: 35
num_threads: 6
buffer: (nil)
nx, ny, nz: 0 0 0
dtype: 0
total_time: -1.000000

Marking J2/imported/009148797359298835130_FoilHole_9006262_Data_8993834_8993836_20220909_010506_EER.eer as incomplete and continuing…

10

Were there any eer files for which this error did not occur during motion correction?
What are the outputs of these (each in a single line) commands:

readlink /home/renee.arias/data/em/220908_beta_gal_glacios/proc/P9/J2/imported/009148797359298835130_FoilHole_9006262_Data_8993834_8993836_20220909_010506_EER.eer
ls -Ll /home/renee.arias/data/em/220908_beta_gal_glacios/proc/P9/J2/imported/009148797359298835130_FoilHole_9006262_Data_8993834_8993836_20220909_010506_EER.eer