emailed files to xxxxx@xxxxxx.xxx
The json file for the upstream import job, which you emailed us, specifies a Gain reference path that may not be in MRC format. I wonder if this caused a problem during import, which in turn may have lead to the KeyError.
If the gain reference is indeed not already in MRC format, you may want to retry the upstream import and this motion correction job after converting the gain reference to MRC format. Your microscope facility may be able to help with the conversion.
Tried that using both tif2mrc and newstack. Still have the same error.
Any update on this issue? I have used multiple file conversion and still achieve the same error
This appears to be related to the update. All works fine in v3.3.1. This error only occurs in v3.3.2+220518
To help us identify the cause, please clone a job that succeeded on v3.3.1 (“old job” ) and execute that job on v3.3.2+220518 (“new job”). After confirming that the clone fails, please collect for this pair of jobs:
job.jsonjob.log
In addition:
-
job.jsonfrom the (common) upstream import job - one raw movie file that’s cited in the error message of the “new” motion correction job
- the error message from the “new” job’s Overview tab
- the gain reference and defect file (if used during import)
I will send you direct message regarding data transfer arrangements.
Hello, I am also having this issue. I am using version 3.3.2. The import movies seems to work okay, but when looking at the imported eer files, they only come out to 139 bytes.
I guess the 139 bytes is not a huge deal. I went back and looked at other import jobs and this is on about the same level as previous.
Because several issues have been discussed under this topic, please post
- the error message you observed
- the patch level of you CryoSPARC instance, if any
- a screen shot of the failed jobs Inputs and Parameters tab
I was running v3.3.2 (no patch) for these errors.
Here is my error message for one of the eers, but this occurred again and again for most of the movies:
[CPU: 228.9 MB] Error occurred while processing J1/imported/004745586021228548817_FoilHole_9006262_Data_8993834_8993836_20220909_010506_EER.eer
Traceback (most recent call last):
File “/home/renee.arias/cryosparc/cryosparc_worker/cryosparc_compute/jobs/pipeline.py”, line 60, in exec
return self.process(item)
File “cryosparc_worker/cryosparc_compute/jobs/motioncorrection/run_patch.py”, line 117, in cryosparc_compute.jobs.motioncorrection.run_patch.run_patch_motion_correction_multi.motionworker.process
File “cryosparc_worker/cryosparc_compute/jobs/motioncorrection/run_patch.py”, line 139, in cryosparc_compute.jobs.motioncorrection.run_patch.run_patch_motion_correction_multi.motionworker.process
File “/home/renee.arias/cryosparc/cryosparc_worker/cryosparc_compute/blobio/eer.py”, line 86, in read_eer
eer = open_eer(fname)
File “/home/renee.arias/cryosparc/cryosparc_worker/cryosparc_compute/blobio/eer.py”, line 15, in open_eer
return eerdecompressor.ElectronCountedFramesDecompressor(str(fname))
RuntimeError: unable to open tiff file
Marking J1/imported/004745586021228548817_FoilHole_9006262_Data_8993834_8993836_20220909_010506_EER.eer as incomplete and continuing…
And here is the Inputs and Parameters screenshot:
I just implemented the latest patch (I think it is 220824) and am currently importing the images again. Perhaps that will be the fix.
After implementation of the patch, I got a new error:
Error occurred while processing J2/imported/015153351232888067961_FoilHole_9006286_Data_8993834_8993836_20220909_010807_EER.eer
Traceback (most recent call last):
File “/home/renee.arias/cryosparc/cryosparc_worker/cryosparc_compute/jobs/pipeline.py”, line 60, in exec
return self.process(item)
File “cryosparc_master/cryosparc_compute/jobs/motioncorrection/run_patch.py”, line 118, in cryosparc_compute.jobs.motioncorrection.run_patch.run_patch_motion_correction_multi.motionworker.process
File “cryosparc_master/cryosparc_compute/jobs/motioncorrection/run_patch.py”, line 133, in cryosparc_compute.jobs.motioncorrection.run_patch.run_patch_motion_correction_multi.motionworker.process
File “cryosparc_master/cryosparc_compute/blobio/prefetch.py”, line 71, in cryosparc_compute.blobio.prefetch.synchronous_native_read
RuntimeError: Error ocurred (Input/output error) at line 903 in open
Could not open file. Make sure the path exists.
IO request details:filename: /home/renee.arias/data/em/220908_beta_gal_glacios/proc/P9/J2/imported/015153351232888067961_FoilHole_9006286_Data_8993834_8993836_20220909_010807_EER.eer
filetype: 2
header_only: 0
idx_start: 0
idx_limit: -1
eer_upsampfactor: 2
eer_numfractions: 35
num_threads: 6
buffer: (nil)
nx, ny, nz: 0 0 0
dtype: 0
total_time: -1.000000
Marking J2/imported/015153351232888067961_FoilHole_9006286_Data_8993834_8993836_20220909_010807_EER.eer as incomplete and continuing…
I did verify that the path is correct and exists.
Inside the directory that you specified as Movies data path, please can you run the command
stat FoilHole_9006286_Data_8993834_8993836_20220909_010807_EER.eer and post the command’s output here?
Here it is (I replaced my colleague’s name with the generic colleague.name):
File: 'FoilHole_9006286_Data_8993834_8993836_20220909_010807_EER.eer'
Size: 901309466 Blocks: 1760993 IO Block: 131072 regular file
Device: 27h/39d Inode: 1935 Links: 1
Access: (0777/-rwxrwxrwx) Uid: ( 1001/colleague.name) Gid: ( 1001/colleague.name)
Access: 2022-09-12 14:41:12.906309647 -0700
Modify: 2022-09-09 01:07:55.818618200 -0700
Change: 2022-09-12 14:41:12.915309677 -0700
Birth: -
Were any of the eer files successfully imported?
The problem seems to come with the patch motion job, where the previously mentioned errors occur.
Is this the information you were expecting from the “stat” command?
I also tried converting the .gain file to a .mrc file with EMAN2, but this did not help, I saw the same errors.
Just to confirm: is this the same issue as
?
You are correct! 
Thank you