Hi everyone, my “import movies” job has finished with “Job process terminated abnormally” when loading ~3k tif files into the cryosparc.
I had to manually check for the corrupted files using tiffdump utility (this one), and remove the ones that were damages. After that, “Import movies” went normally.
However, I’d expect the corrupted tif files to end up in the “Failed movies” output group, but not result in the job failing and me checking all the files manually.
For reproducibility, the dataset I was working on was EMPIAR-10926, and the corrupted file was eme1-3_159-16_0005_Aug11_08.58.03.tif.
I removed initial all-movies job, but if I run it on failed micrograph only, I get this:
job.log
================= CRYOSPARCW ======= 2022-12-23 22:50:17.284392 =========
Project P58 Job J15
Master cmm-1 Port 39002
===========================================================================
========= monitor process now starting main process
MAINPROCESS PID 27939
========= monitor process now waiting for main process
MAIN PID 27939
imports.run cryosparc_compute.jobs.jobregister
========= sending heartbeat
========= main process now complete.
========= monitor process now complete.
on a v4.1.1 instance (with an unreleased, unrelated patch).
In case this observation motivates an update of your instance, I recommend preparing the update by ensuring prerequisites for the potential installation of 3DFlex dependencies even if you do not plan to install those dependencies immediately.
I see, thanks. Perhaps this was an issue with wget not ensuring download correctly, and not with the file itself.
Although, it doesn’t change the second part of the message: if one of the tif files is damaged, I’d expect it to fail into the “Failed movies”, and not to fail the whole process itself. Not sure if that was the designed behavior though.
