Troubleshooting: Import Movies "Reading File" Error

Troubleshooting
1

Hi all! I am hoping you can provide some help. I am repeatedly having problems completing the import movies job in cryosparc. The job process terminates abnormally after it tries to read the imported files. Interestingly, the movie files (.tif files) and gain file (.mrc file) are successfully imported into cryosparc, and can be seen in the cryosparc working directory. Please see the job error message below. Note that all directories used were given “777” permission. Any advice/suggestions would be really appreciated. Thank you!

FailedJobError_1movie
FailedJobError_1movie1016×660 102 KB

2

Hi @Dnbrawley,

Is it possible if you can post the output of the job log?
E.g. run the command:
cryosparcm joblog Px Jx
Where Px is the project uid and Jx is the job uid

3

Thanks for the reply! Here’s the job log output from the “cryosparcm” command:

================= CRYOSPARCW ======= 2020-04-07 19:19:16.146519 =========
Project P28 Job J1
Master prince0.hpc.nyu.edu Port 39002

========= monitor process now starting main process
MAINPROCESS PID 114513
========= monitor process now waiting for main process
MAIN PID 114513
imports.run cryosparc2_compute.jobs.jobregister
========= sending heartbeat
========= sending heartbeat
========= sending heartbeat
========= main process now complete.
========= monitor process now complete.

4

Hi @Dnbrawley,

Is this job running on a cluster system? and how much RAM does the node that is running the cryoSPARC master application have?
One possible cause is that the job tries to allocated a lot of system RAM to load the entire .tiff file in memory and decompress it in order to do a simple average for plotting - and during this time the system kills the job because it asked for too much memory.

5

Thank you for your reply! We are running the job on GPU cards on a HPC at New York University. As of present, I don’t know much about the cluster specs, but I forwarded your reply to our system engineer & he will have a look. Is there a certain threshold of RAM that the node running the cryosparc master app will need?

6

Update: We resolved the problem. A memory (4 GB limit on the login node) was the issue preventing movie import and that problem has now been resolved.