When we use Reference Based Motion Correction, we faced two libtiff error (see below). We have checked the tiff file by header, and it looks good. Any suggestion for further test? Thanks.
Blockquote
[CPU: 220.0 MB Avail: 220.96 GB]
[libtiff error] TIFFReadDirectory: Failed to read directory at offset 511957844
[CPU: 220.0 MB Avail: 220.94 GB]
[libtiff error] TIFFFetchDirectory: Can not read TIFF directory count
[CPU: 24.80 GB Avail: 216.85 GB]
Plotting trajectories and particles for movie 16538045552548172683
J1/imported/016538045552548172683_207_00002.tiff
Blockquote
[CPU: 28.31 GB Avail: 214.43 GB]
No further example plots will be made, but the job is still running (see progress bar above).
[CPU: 236.4 MB Avail: 241.90 GB]
====== Job process terminated abnormally.
[CPU: 221.1 MB Avail: 242.14 GB]
[libtiff error] J1/imported/009605098670889419572_207_00286.tiff: Not a TIFF or MDI file, bad magic number 14674 (0x3952)
Hi @samuelhao, this suggests that one of your input movie files is corrupt. Try excluding that movie using a curate exposures job.
– Harris
I have been having similar issues, and have been employing the suggested workaround by excluding movies that trigger a libtiff error. Strangely enough though, the error trigger almost seems to be happening randomly. Movies that were processed without problem in an earlier RBMC attempt, have failed on me during later RBMC jobs after multiple rounds of testing and manual removal of the movies involved.
E.g. J293 RBMC: 20240423_P_abyssi_collect_2_1000_00001.tif was processed with no problems
However, J293 crashed with the following error:
[libtiff error] TIFFOpen: /mnt/beegfs/EMprocessing/remaut/msleutel/Pyrodyctium_abyssi/csparc/CS-pyrodyctium-abyssi/S1/import_movies/20240423_P_abyssi_collect_2_1215_00001.tif: No such file or directory
J294: Curate exposures: manually removed 20240423_P_abyssi_collect_2_1215_00001.tif
J295: RBMC: crashed with on 20240423_P_abyssi_collect_2_1000_00001.tif that had been succesfully processed in J293, here’s the error message:
[libtiff error] TIFFOpen: /mnt/beegfs/EMprocessing/remaut/msleutel/Pyrodyctium_abyssi/csparc/CS-pyrodyctium-abyssi/S1/import_movies/20240423_P_abyssi_collect_2_1000_00001.tif: No such file or directory
Hi @msleutel,
You’re sure that nothing has gone wrong with the beegfs mount? “No such file” errors are generally quite reliable. I’m not experienced with beegfs, but could this be some kind of timeout on the network aspect of the filesystem?
– Harris
Hi @hsnyder,
thank you for the suggestion. We checked the beegfs mount and the links to the tiff files and everything is fine. However, following your comment, we reasoned that perhaps (although there are no clear errors in the system logs) the beegfs might be the root cause of the libtiff errors. To test this, we copied the tiff files to a fast SSD-raid, updated the symlinks in the S1 import job and reran RBMC which now ran to completion with no errors.
Thanks,
Mike
1 Like
