Reference based motion correction: empty chart and no results

Motion Correction
1

Reference based motion correction gives no results. Any insight?

image
image948×665 59.7 KB

WechatIMG277
WechatIMG277829×934 19.9 KB

WechatIMG278
WechatIMG278680×823 18.7 KB

2

It happens sometimes. Try re-running (if you haven’t already) and checking for particles containing NaNs (“Check for Corrupt Particles” job). Some more details on the parameters used might provide more insight… are the movies TIFF, MRC, EER? Super res? Gain corrected or separate gain? Box size? Fourier cropping during Patch Motion Correction? Particle stack box size and downsampling? How clean is the stack?

3

All the movies are EER file with separated gain reference file. Particles box size is 400 without downsampling and cropping. I have done the RBMR job before with this dataset but differnet particles groups, and it worked for me. After that, I delete the gain reference by accident and add the file agin to the same path. I re-run the job and the issue still happens with empty charts and no results.

4

The log file show like this:

========= sending heartbeat at 2024-03-20 21:30:36.864136
========= sending heartbeat at 2024-03-20 21:30:46.880701
TIFFReadDirectory: Warning, Unknown field with tag 65001 (0xfde9) encountered.
TIFFReadDirectory: Warning, Unknown field with tag 65100 (0xfe4c) encountered.
TIFFReadDirectory: Warning, Unknown field with tag 65001 (0xfde9) encountered.
TIFFReadDirectory: Warning, Unknown field with tag 65100 (0xfe4c) encountered.
TIFFReadDirectory: TIFFReadDirectory: Warning, Unknown field with tag 65001 (0xfde9) encountered.
TIFFReadDirectory: Warning, Unknown field with tag 65001 (0xfde9) encountered.
TIFFReadDirectory: Warning, Unknown field with tag 65001 (0xfde9) encountered.
TIFFReadDirectory: Warning, Unknown field with tag 65001 (0xfde9) encountered.
Warning, Unknown field with tag 65001 (0xfde9) encountered.
TIFFReadDirectory: Warning, Unknown field with tag 65001 (0xfde9) encountered.
TIFFReadDirectory: TIFFReadDirectory: Warning, Unknown field with tag 65100 (0xfe4c) encountered.
TIFFReadDirectory: Warning, Unknown field with tag 65100 (0xfe4c) encountered.
Warning, Unknown field with tag 65100 (0xfe4c) encountered.
TIFFReadDirectory: Warning, Unknown field with tag 65100 (0xfe4c) encountered.
TIFFReadDirectory: Warning, Unknown field with tag 65100 (0xfe4c) encountered.
TIFFReadDirectory: Warning, Unknown field with tag 65100 (0xfe4c) encountered.
TIFFReadDirectory: Warning, Unknown field with tag 65001 (0xfde9) encountered.
TIFFReadDirectory: Warning, Unknown field with tag 65100 (0xfe4c) encountered.
TIFFReadDirectory: Warning, Unknown field with tag 65001 (0xfde9) encountered.
TIFFReadDirectory: Warning, Unknown field with tag 65100 (0xfe4c) encountered.
TIFFReadDirectory: Warning, Unknown field with tag 65001 (0xfde9) encountered.
TIFFReadDirectory: Warning, Unknown field with tag 65100 (0xfe4c) encountered.
TIFFReadDirectory: Warning, Unknown field with tag 65001 (0xfde9) encountered.
TIFFReadDirectory: Warning, Unknown field with tag 65001 (0xfde9) encountered.
TIFFReadDirectory: Warning, Unknown field with tag 65001 (0xfde9) encountered.
TIFFReadDirectory: Warning, Unknown field with tag 65001 (0xfde9) encountered.
TIFFReadDirectory: Warning, Unknown field with tag 65001 (0xfde9) encountered.
TIFFReadDirectory: Warning, Unknown field with tag 65001 (0xfde9) encountered.
TIFFReadDirectory: TIFFReadDirectory: Warning, Unknown field with tag 65100 (0xfe4c) encountered.
TIFFReadDirectory: Warning, Unknown field with tag 65100 (0xfe4c) encountered.
Warning, Unknown field with tag 65100 (0xfe4c) encounteredTIFFReadDirectory: Warning, Unknown field with tag 65100 (0xfe4c) encountered.
.
TIFFReadDirectory: Warning, Unknown field with tag 65100 (0xfe4c) encountered.
TIFFReadDirectory: Warning, Unknown field with tag 65100 (0xfe4c) encountered.
TIFFReadDirectory: Warning, Unknown field with tag 65001 (0xfde9) encountered.
TIFFReadDirectory: Warning, Unknown field with tag 65100 (0xfe4c) encountered.
========= sending heartbeat at 2024-03-20 21:30:56.896231
gpufft: creating new cufft plan (plan id 0   pid 2210314) 
	gpu_id  0 
	ndims   3 
	dims    400 400 400 
	inembed 400 400 402 
	istride 1 
	idist   64320000 
	onembed 400 400 201 
	ostride 1 
	odist   32160000 
	batch   1 
	type    R2C 
	wkspc   automatic 
	Python traceback:

ElectronCountedFramesDecompressor: reading using TIFF-EER mode.
TIFFOpen: /cephfs2/qchen/Cryosaprc/J2/imported/008423197056849766219_FoilHole_9250221_Data_9247583_9247585_20231109_144752_EER.eer: No such file or directory
C++ Exception: std::exception
python: [refmotion worker 0 (NVIDIA GeForce GTX 1080 Ti)] ERROR: Failed to read file: /cephfs2/qchen/Cryosaprc/J2/imported/008423197056849766219_FoilHole_9250221_Data_9247583_9247585_20231109_144752_EER.eer
Error ocurred (State not recoverable) at line 1101 in eer_codec_create

See job log for further details.
EER I/O error, see job log: No such file or directory
ElectronCountedFramesDecompressor: reading using TIFF-EER mode.
TIFFOpen: /cephfs2/qchen/Cryosaprc/J1/imported/001240511463603413449_FoilHole_25774664_Data_25767879_25767881_20231025_191448_EER.eer: No such file or directory
C++ Exception: std::exception
python: [refmotion worker 7 (NVIDIA GeForce GTX 1080 Ti)] ERROR: Failed to read file: /cephfs2/qchen/Cryosaprc/J1/imported/001240511463603413449_FoilHole_25774664_Data_25767879_25767881_20231025_191448_EER.eer
Error ocurred (State not recoverable) at line 1101 in eer_codec_create
5

The TIFFRead errors are normal but annoying with EER, as it is effectively a tiff file. The other errors means micrographs are missing… Have the symlinks/network shares broken?

6

Solved. The symlinks to the micrographs somehow broken, not sure when and why. Making a new symlinks works. Great thanks for figuring it out.

7

Glad you got is sorted out! :smiley: