CTFFind4 jobs fail

Biorak · October 4, 2018, 2:19am

Hi,

I am running T20S tutorial dataset in cryosparc V2.3.2. The CTFFind jobs fail. The following is the output. Do I need to install CTFFind for this, if so how do I specify the path in cryosparc ?

Processed 0 of 20 micrographs in 0.03s

=== Running CTFFIND for micrograph 000000 ( J4/motioncorrected/14sep05c_00024sq_00003hl_00002es.frames_rigid_aligned.mrc )
CTFFIND did not run correctly. See the log at /home/rakesh/WORK/Projects/4-SPOTSComplex/SParticleProcess/cryoSPARC/P2/J6/ctffind_output/14sep05c_00024sq_00003hl_00002es.frames_rigid_aligned_ctffind.log

=== Running CTFFIND for micrograph 000002 ( J4/motioncorrected/14sep05c_00024sq_00004hl_00002es.frames_rigid_aligned.mrc )
CTFFIND did not run correctly. See the log at /home/rakesh/WORK/Projects/4-SPOTSComplex/SParticleProcess/cryoSPARC/P2/J6/ctffind_output/14sep05c_00024sq_00004hl_00002es.frames_rigid_aligned_ctffind.log

=== Running CTFFIND for micrograph 000001 ( J4/motioncorrected/14sep05c_00024sq_00003hl_00005es.frames_rigid_aligned.mrc )
CTFFIND did not run correctly. See the log at /home/rakesh/WORK/Projects/4-SPOTSComplex/SParticleProcess/cryoSPARC/P2/J6/ctffind_output/14sep05c_00024sq_00003hl_00005es.frames_rigid_aligned_ctffind.log

=== Running CTFFIND for micrograph 000003 ( J4/motioncorrected/14sep05c_00024sq_00006hl_00003es.frames_rigid_aligned.mrc )
CTFFIND did not run correctly. See the log at /home/rakesh/WORK/Projects/4-SPOTSComplex/SParticleProcess/cryoSPARC/P2/J6/ctffind_output/14sep05c_00024sq_00006hl_00003es.frames_rigid_aligned_ctffind.log

=== Running CTFFIND for micrograph 000004 ( J4/motioncorrected/14sep05c_c_00003gr_00014sq_00002hl_00005es.frames_rigid_aligned.mrc )
CTFFIND did not run correctly. See the log at /home/rakesh/WORK/Projects/4-SPOTSComplex/SParticleProcess/cryoSPARC/P2/J6/ctffind_output/14sep05c_c_00003gr_00014sq_00002hl_00005es.frames_rigid_aligned_ctffind.log

=== Running CTFFIND for micrograph 000006 ( J4/motioncorrected/14sep05c_c_00003gr_00014sq_00005hl_00002es.frames_rigid_aligned.mrc )
CTFFIND did not run correctly. See the log at /home/rakesh/WORK/Projects/4-SPOTSComplex/SParticleProcess/cryoSPARC/P2/J6/ctffind_output/14sep05c_c_00003gr_00014sq_00005hl_00002es.frames_rigid_aligned_ctffind.log

=== Running CTFFIND for micrograph 000005 ( J4/motioncorrected/14sep05c_c_00003gr_00014sq_00004hl_00004es.frames_rigid_aligned.mrc )
CTFFIND did not run correctly. See the log at /home/rakesh/WORK/Projects/4-SPOTSComplex/SParticleProcess/cryoSPARC/P2/J6/ctffind_output/14sep05c_c_00003gr_00014sq_00004hl_00004es.frames_rigid_aligned_ctffind.log

=== Running CTFFIND for micrograph 000007 ( J4/motioncorrected/14sep05c_c_00003gr_00014sq_00005hl_00003es.frames_rigid_aligned.mrc )
CTFFIND did not run correctly. See the log at /home/rakesh/WORK/Projects/4-SPOTSComplex/SParticleProcess/cryoSPARC/P2/J6/ctffind_output/14sep05c_c_00003gr_00014sq_00005hl_00003es.frames_rigid_aligned_ctffind.log

=== Running CTFFIND for micrograph 000008 ( J4/motioncorrected/14sep05c_c_00003gr_00014sq_00005hl_00005es.frames_rigid_aligned.mrc )
CTFFIND did not run correctly. See the log at /home/rakesh/WORK/Projects/4-SPOTSComplex/SParticleProcess/cryoSPARC/P2/J6/ctffind_output/14sep05c_c_00003gr_00014sq_00005hl_00005es.frames_rigid_aligned_ctffind.log

=== Running CTFFIND for micrograph 000009 ( J4/motioncorrected/14sep05c_c_00003gr_00014sq_00006hl_00002es.frames_rigid_aligned.mrc )
CTFFIND did not run correctly. See the log at /home/rakesh/WORK/Projects/4-SPOTSComplex/SParticleProcess/cryoSPARC/P2/J6/ctffind_output/14sep05c_c_00003gr_00014sq_00006hl_00002es.frames_rigid_aligned_ctffind.log

=== Running CTFFIND for micrograph 000010 ( J4/motioncorrected/14sep05c_c_00003gr_00014sq_00006hl_00003es.frames_rigid_aligned.mrc )
CTFFIND did not run correctly. See the log at /home/rakesh/WORK/Projects/4-SPOTSComplex/SParticleProcess/cryoSPARC/P2/J6/ctffind_output/14sep05c_c_00003gr_00014sq_00006hl_00003es.frames_rigid_aligned_ctffind.log

=== Running CTFFIND for micrograph 000011 ( J4/motioncorrected/14sep05c_c_00003gr_00014sq_00006hl_00005es.frames_rigid_aligned.mrc )
CTFFIND did not run correctly. See the log at /home/rakesh/WORK/Projects/4-SPOTSComplex/SParticleProcess/cryoSPARC/P2/J6/ctffind_output/14sep05c_c_00003gr_00014sq_00006hl_00005es.frames_rigid_aligned_ctffind.log

=== Running CTFFIND for micrograph 000012 ( J4/motioncorrected/14sep05c_c_00003gr_00014sq_00007hl_00004es.frames_rigid_aligned.mrc )
CTFFIND did not run correctly. See the log at /home/rakesh/WORK/Projects/4-SPOTSComplex/SParticleProcess/cryoSPARC/P2/J6/ctffind_output/14sep05c_c_00003gr_00014sq_00007hl_00004es.frames_rigid_aligned_ctffind.log

=== Running CTFFIND for micrograph 000013 ( J4/motioncorrected/14sep05c_c_00003gr_00014sq_00007hl_00005es.frames_rigid_aligned.mrc )
CTFFIND did not run correctly. See the log at /home/rakesh/WORK/Projects/4-SPOTSComplex/SParticleProcess/cryoSPARC/P2/J6/ctffind_output/14sep05c_c_00003gr_00014sq_00007hl_00005es.frames_rigid_aligned_ctffind.log

=== Running CTFFIND for micrograph 000014 ( J4/motioncorrected/14sep05c_c_00003gr_00014sq_00008hl_00005es.frames_rigid_aligned.mrc )
CTFFIND did not run correctly. See the log at /home/rakesh/WORK/Projects/4-SPOTSComplex/SParticleProcess/cryoSPARC/P2/J6/ctffind_output/14sep05c_c_00003gr_00014sq_00008hl_00005es.frames_rigid_aligned_ctffind.log

=== Running CTFFIND for micrograph 000015 ( J4/motioncorrected/14sep05c_c_00003gr_00014sq_00009hl_00004es.frames_rigid_aligned.mrc )
CTFFIND did not run correctly. See the log at /home/rakesh/WORK/Projects/4-SPOTSComplex/SParticleProcess/cryoSPARC/P2/J6/ctffind_output/14sep05c_c_00003gr_00014sq_00009hl_00004es.frames_rigid_aligned_ctffind.log

=== Running CTFFIND for micrograph 000016 ( J4/motioncorrected/14sep05c_c_00003gr_00014sq_00010hl_00002es.frames_rigid_aligned.mrc )
CTFFIND did not run correctly. See the log at /home/rakesh/WORK/Projects/4-SPOTSComplex/SParticleProcess/cryoSPARC/P2/J6/ctffind_output/14sep05c_c_00003gr_00014sq_00010hl_00002es.frames_rigid_aligned_ctffind.log

=== Running CTFFIND for micrograph 000017 ( J4/motioncorrected/14sep05c_c_00003gr_00014sq_00011hl_00002es.frames_rigid_aligned.mrc )
CTFFIND did not run correctly. See the log at /home/rakesh/WORK/Projects/4-SPOTSComplex/SParticleProcess/cryoSPARC/P2/J6/ctffind_output/14sep05c_c_00003gr_00014sq_00011hl_00002es.frames_rigid_aligned_ctffind.log

=== Running CTFFIND for micrograph 000018 ( J4/motioncorrected/14sep05c_c_00003gr_00014sq_00011hl_00003es.frames_rigid_aligned.mrc )
CTFFIND did not run correctly. See the log at /home/rakesh/WORK/Projects/4-SPOTSComplex/SParticleProcess/cryoSPARC/P2/J6/ctffind_output/14sep05c_c_00003gr_00014sq_00011hl_00003es.frames_rigid_aligned_ctffind.log

=== Running CTFFIND for micrograph 000019 ( J4/motioncorrected/14sep05c_c_00003gr_00014sq_00011hl_00004es.frames_rigid_aligned.mrc )
CTFFIND did not run correctly. See the log at /home/rakesh/WORK/Projects/4-SPOTSComplex/SParticleProcess/cryoSPARC/P2/J6/ctffind_output/14sep05c_c_00003gr_00014sq_00011hl_00004es.frames_rigid_aligned_ctffind.log

Completed CTF Estimation using CTFFIND.

0 micrographs successfully completed.

20 micrographs failed.

Total time: 0.32s

Compiling job outputs…

Passing through outputs for output group micrographs_success from input group micrographs

This job outputted results [‘ctf’, ‘ctf_plotdata’]

Loaded output dset with 0 items

Passthrough results [‘micrograph_blob’, ‘mscope_params’, ‘background_blob’, ‘rigid_motion’, ‘movie_blob’, ‘gain_ref_blob’]

Loaded passthrough dset with 20 items

Intersection of output and passthrough has 0 items

Passing through outputs for output group micrographs_fail from input group micrographs

This job outputted results [‘ctf’, ‘ctf_plotdata’]

Loaded output dset with 20 items

Passthrough results [‘micrograph_blob’, ‘mscope_params’, ‘background_blob’, ‘rigid_motion’, ‘movie_blob’, ‘gain_ref_blob’]

Loaded passthrough dset with 20 items

Intersection of output and passthrough has 20 items

Checking outputs for output group micrographs_success

Checking outputs for output group micrographs_fail

Job complete. Total time 0.61s

Outputs

stephan · October 4, 2018, 2:30pm

Hi @Biorak,

As of v2.3.2, CTFFIND 4.1.8 is bundled with cryoSPARC. You don’t have to install it. You can find the binaries at cryosparc2_worker/deps/external/ctffind/bin
Is it possible if you can post the output of one of the logs created by CTFFIND?

Biorak · October 4, 2018, 6:27pm

/home/rakesh/WORK/Softwares/CryoEM_Fitting/cryosparcV2/cryosparc2_worker/deps/external/ctffind/bin/ctffind: error while loading shared libraries: libtiff.so.3: cannot open shared object file: No such file or directory

Is it not possibel to upload text or zip files ?

stephan · October 4, 2018, 8:57pm

nis_it · December 13, 2018, 5:35pm

We seem to be having a similar issue with v2.6.4 the ctffind job fails using the test dataset but complains that some file can’t be found?

Processed 0 of 20 micrographs in 0.06s

=== Running CTFFIND for micrograph 000001 ( J2/motioncorrected/14sep05c_00024sq_00003hl_00005es.frames_rigid_aligned.mrc )
Running thread Thread-1…

=== Running CTFFIND for micrograph 000000 ( J2/motioncorrected/14sep05c_00024sq_00003hl_00002es.frames_rigid_aligned.mrc )
Running thread Thread-2…

=== Running CTFFIND for micrograph 000002 ( J2/motioncorrected/14sep05c_00024sq_00004hl_00002es.frames_rigid_aligned.mrc )
Running thread Thread-3…

=== Running CTFFIND for micrograph 000004 ( J2/motioncorrected/14sep05c_c_00003gr_00014sq_00002hl_00005es.frames_rigid_aligned.mrc )
Running thread Thread-5…

=== Running CTFFIND for micrograph 000003 ( J2/motioncorrected/14sep05c_00024sq_00006hl_00003es.frames_rigid_aligned.mrc )
Running thread Thread-4…

=== Running CTFFIND for micrograph 000006 ( J2/motioncorrected/14sep05c_c_00003gr_00014sq_00005hl_00002es.frames_rigid_aligned.mrc )
Running thread Thread-7…

=== Running CTFFIND for micrograph 000005 ( J2/motioncorrected/14sep05c_c_00003gr_00014sq_00004hl_00004es.frames_rigid_aligned.mrc )
Running thread Thread-6…

=== Running CTFFIND for micrograph 000007 ( J2/motioncorrected/14sep05c_c_00003gr_00014sq_00005hl_00003es.frames_rigid_aligned.mrc )
Running thread Thread-8…

Traceback (most recent call last):
File “cryosparc2_compute/jobs/ctf/run.py”, line 173, in work
resample=params[‘resample’]
File “cryosparc2_compute/jobs/ctf/ctf_wrapper.py”, line 398, in run_ctf_wrapper
**kwargs)
File “cryosparc2_compute/jobs/ctf/ctf_wrapper.py”, line 375, in run_ctffind
proc = rc.start_external_process([ctffind_bin_path_abs], log_path_abs) # starts and registers subprocess
File “cryosparc2_compute/jobs/runcommon.py”, line 671, in start_external_process
process = com.start_external_process(command, log_file_path)
File “cryosparc2_compute/jobs/common.py”, line 200, in start_external_process
close_fds = True ) # NOTES close_fds is critical! otherwise child process will keep the webserver’s sockets open and cause all kinds of problems!
File “/home/cryosparc_user/cryosparc2_worker/deps/anaconda/lib/python2.7/subprocess.py”, line 394, in init
errread, errwrite)
File “/home/cryosparc_user/cryosparc2_worker/deps/anaconda/lib/python2.7/subprocess.py”, line 1047, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory

=== Running CTFFIND for micrograph 000008 ( J2/motioncorrected/14sep05c_c_00003gr_00014sq_00005hl_00005es.frames_rigid_aligned.mrc )
Running thread Thread-3…

Traceback (most recent call last):
File “cryosparc2_worker/cryosparc2_compute/run.py”, line 78, in cryosparc2_compute.run.main
File “cryosparc2_compute/jobs/ctf/run.py”, line 231, in run_ctf_estimation
pool.map(work, selected_exposure_idxs)
File “/home/cryosparc_user/cryosparc2_worker/deps/anaconda/lib/python2.7/multiprocessing/pool.py”, line 253, in map
return self.map_async(func, iterable, chunksize).get()
File “/home/cryosparc_user/cryosparc2_worker/deps/anaconda/lib/python2.7/multiprocessing/pool.py”, line 572, in get
raise self._value
OSError: [Errno 2] No such file or directory

all the logs are 0b files; but the ctffind bins are in the correct path:
[cryosparc_user@gpu1 root]$ ls /home/cryosparc_user/cryosparc2_worker/deps/ext/ctffind/bin -lah
total 28M
drwxr-xr-x 2 root root 52 Dec 13 09:35 .
drwxrwxr-x 4 cryosparc_user cryosparc_user 55 Dec 13 09:35 …
-rwxr-xr-x 1 cryosparc_user cryosparc_user 28M Jun 7 2017 ctffind
-rwxr-xr-x 1 cryosparc_user cryosparc_user 4.9K Jun 7 2017 ctffind_plot_results.sh

stephan · December 13, 2018, 6:09pm

Hi @nis_it,

Actually, the path you mentioned seems to be the path to the CTFFIND4 binary from a previous version of cryoSPARC (before the update to our dependency manager). Is it possible that an update to your cryosparc2_worker instance somehow failed mid way?
The correct path should be:

sarulthasan@cryoem5:/u/cryosparcprod/production_cryosparc/cryosparc2_worker/deps/external/ctffind-4.1.10/bin$ ll
total 31736
drwxrwxr-x 2 cryosparcprod cryosparcprod        4 Oct 12 16:34 ./
drwxrwxr-x 3 cryosparcprod cryosparcprod        3 Oct 12 16:34 ../
-rwxr-xr-x 1 cryosparcprod cryosparcprod 31545286 Oct 12 16:34 ctffind*
-rwxr-xr-x 1 cryosparcprod cryosparcprod     4963 Oct 12 16:34 ctffind_plot_results.sh*

You can copy the binaries into this directory and it should work again.
I’m currently working on a feature that will allow a user to selectively reinstall dependencies, so until that is released, you might have issues with GCTF as well. A dirty method of fixing this issue is to delete your cryosparc2_worker directory and reinstall it using the “Installation: Standalone Worker” documentation here.

nis_it · December 13, 2018, 7:10pm

It must have had some issue with the upgrade, but there was no error messages during the upgrade, as the version of ctffind4 is 4.1.8 not 4.1.10; i created the path and copied the 4.1.10 binary and the job completed correctly.

chana · February 24, 2019, 10:16pm

I had the same issue after updating and when I checked the path, it was correct.
I just fixed this issue by doing a Forced Update for which the instructions are written in the Installation and management guide.

eswar909 · March 1, 2019, 2:39am

Dear all,

I have installed a recent version of cryosparc and while running the T20 tutorial data and I got the same (CTFFFind4 job fail) error with a log as shown in the screenshot. I carried a forced update and also checked the path of CCTF4 (I hope it is correctly assigned ) (see the screenshot) and I have all the necessary libtiff files.

running on OpenSuse 15.0, Cuda 10.0 with the latest version of cryosparc; installed with a stand-alone option.

Any suggestions are welcome,

stephan · March 1, 2019, 3:50pm

Hi @eswar909,

In the streamlog, you can see CTFFIND4 outputted a message with a path to its error log. Can you check that file and see what it says?

eswar909 · March 1, 2019, 4:19pm

Hi@sarulthasan

Please have a look in to the error log

**** handle exception rc
set status to failed
Traceback (most recent call last):
File “cryosparc2_worker/cryosparc2_compute/run.py”, line 78, in cryosparc2_compute.run.main
File “cryosparc2_compute/jobs/ctf/run.py”, line 271, in run_ctf_estimation
rc.output(‘micrographs_fail’, ‘ctf’, outpath_rel, 0, len(exposures_fail))
File “cryosparc2_compute/jobs/runcommon.py”, line 251, in output
result = com.query(job[‘output_results’], lambda r: r[‘group_name’] == group_name and r[‘name’] == name, error=“No output result named %s.%s in job.” % (group_name, name))
File “cryosparc2_compute/jobs/common.py”, line 265, in query
assert res != default, error
AssertionError: No output result named micrographs_fail.ctf in job.
========= main process now complete.
========= monitor process now complete.
Best,
Eswar

stephan · March 1, 2019, 8:56pm

Hey @eswar909,

Is this the output of the file /home/takagilab/cryosparc_data/eswar/P2/J30/ctffind_output/14sep05...._rigid_aligned_ctffind.log? or the output of cryosparcm joblog P2 J30?

eswar909 · March 1, 2019, 9:23pm

Hi @stephan
Unfortunately I deleted Job 30, but every time I ran CTF correction I got the same error, for your convenience I am sending the screenshot of Job42 and its log file

/home/takagilab/cryosparc_data/eswar/P2/J26/motioncorrected/14sep05c_c_00003gr_00014sq_00011hl_00003es.frames_rigid_aligned.mrc
/home/takagilab/cryosparc_data/eswar/P2/J42/ctffind_output/a165ef_14sep05c_c_00003gr_00014sq_00011hl_00003es.frames_rigid_aligned_ctffind.mrc
0.6575
300.0
2.7
0.07
512
30.0
5.0
5000.0
50000.0
500.0
no
yes
yes
200.0
no
no
**** handle exception rc
set status to failed
Traceback (most recent call last):
File “cryosparc2_worker/cryosparc2_compute/run.py”, line 78, in cryosparc2_compute.run.main
File “cryosparc2_compute/jobs/ctf/run.py”, line 271, in run_ctf_estimation
rc.output(‘micrographs_fail’, ‘ctf’, outpath_rel, 0, len(exposures_fail))
File “cryosparc2_compute/jobs/runcommon.py”, line 251, in output
result = com.query(job[‘output_results’], lambda r: r[‘group_name’] == group_name and r[‘name’] == name, error=“No output result named %s.%s in job.” % (group_name, name))
File “cryosparc2_compute/jobs/common.py”, line 265, in query
assert res != default, error
AssertionError: No output result named micrographs_fail.ctf in job.
========= main process now complete.
========= monitor process now complete.

Best,
Eswar

eswar909 · March 8, 2019, 3:13pm

Hi @stephan,

I assume the error that I got has to do with OS (Open Suse) that we installed. Based on Cryosparc manual and ccpem bulletin board people suggestions we Installed CentOS7 and running cryosparc quite efficiently without any errors.

Many thanks for your help,
Eswar.

PJJ · February 2, 2024, 10:08am

Hi, we seem to be having the same issue when I ran the CTF Estimation (CTFFIND4) job.The following is the output.
Processed 15550 of 15553 micrographs in 414.96s

[2024-02-02 16:54:21.41]
[CPU: 247.3 MB]
=== Running CTFFIND for micrograph 015426 ( J3/imported/001078461686346782165_FoilHole_24021134_Data_24022943_22_20240126_022214_EER.mrc )
CTFFIND did not run correctly. See the log at /work/home/faculty/yhzhang/YZ_lab/PJJ/20240124_25_TCC_R1-848m-r2zfd6-T1-mg/CS-20240121-tcc-r1-848m-r2zfd6-t1-mg/J4/ctffind_output/90d0ff_001078461686346782165_FoilHole_24021134_Data_24022943_22_20240126_022214_EER_ctffind.log

[2024-02-02 16:54:21.52]
[CPU: 247.3 MB]
=== Running CTFFIND for micrograph 015427 ( J3/imported/006780604574872826451_FoilHole_24021135_Data_23985022_36_20240126_014323_EER.mrc )
CTFFIND did not run correctly. See the log at /work/home/faculty/yhzhang/YZ_lab/PJJ/20240124_25_TCC_R1-848m-r2zfd6-T1-mg/CS-20240121-tcc-r1-848m-r2zfd6-t1-mg/J4/ctffind_output/beff4d_006780604574872826451_FoilHole_24021135_Data_23985022_36_20240126_014323_EER_ctffind.log

[2024-02-02 16:54:21.71]
[CPU: 247.3 MB]
=== Running CTFFIND for micrograph 015428 ( J3/imported/013745997178797269470_FoilHole_24021135_Data_23985025_36_20240126_014332_EER.mrc )
CTFFIND did not run correctly. See the log at /work/home/faculty/yhzhang/YZ_lab/PJJ/20240124_25_TCC_R1-848m-r2zfd6-T1-mg/CS-20240121-tcc-r1-848m-r2zfd6-t1-mg/J4/ctffind_output/41681f_013745997178797269470_FoilHole_24021135_Data_23985025_36_20240126_014332_EER_ctffind.log

[2024-02-02 16:54:21.88]
[CPU: 248.6 MB]
Completed CTF Estimation using CTFFIND.

[2024-02-02 16:54:21.88]
[CPU: 248.6 MB]

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
Neither the name of the Howard Hughes Medical Institute nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS “AS IS” AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, ANY IMPLIED WARRANTIES OF MERCHANTABILITY, NON-INFRINGEMENT, OR FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; REASONABLE ROYALTIES; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

Rohou, A, Grigorieff N. 2015. CTFFIND4: Fast and accurate defocus estimation from electron micrographs. J Struct Biol. 192:216-221.

[2024-02-02 16:54:21.89]
[CPU: 248.6 MB]
0 micrographs successfully completed.

[2024-02-02 16:54:21.89]
[CPU: 248.6 MB]
15553 micrographs failed.

[2024-02-02 16:54:21.89]
[CPU: 248.6 MB]
Total time: 415.44s

[2024-02-02 16:54:21.94]
[CPU: 244.5 MB]

[2024-02-02 16:54:21.94]
[CPU: 236.2 MB]
Compiling job outputs…

[2024-02-02 16:54:21.95]
[CPU: 236.2 MB]
Passing through outputs for output group exposures_success from input group exposures

[2024-02-02 16:54:21.95]
[CPU: 236.2 MB]
This job outputted results [‘ctf’, ‘ctf_plotdata’]

[2024-02-02 16:54:21.96]
[CPU: 236.2 MB]
Loaded output dset with 0 items

[2024-02-02 16:54:21.96]
[CPU: 236.2 MB]
Passthrough results [‘micrograph_blob_non_dw’, ‘micrograph_blob’, ‘mscope_params’]

[2024-02-02 16:54:22.23]
[CPU: 245.3 MB]
Loaded passthrough dset with 15553 items

[2024-02-02 16:54:22.24]
[CPU: 245.5 MB]
Intersection of output and passthrough has 0 items

[2024-02-02 16:54:22.25]
[CPU: 245.5 MB]
Passing through outputs for output group exposures_fail from input group exposures

[2024-02-02 16:54:22.29]
[CPU: 245.5 MB]
This job outputted results [‘ctf’, ‘ctf_plotdata’]

[2024-02-02 16:54:22.30]
[CPU: 245.5 MB]
Loaded output dset with 15553 items

[2024-02-02 16:54:22.30]
[CPU: 245.5 MB]
Passthrough results [‘micrograph_blob_non_dw’, ‘micrograph_blob’, ‘mscope_params’]

[2024-02-02 16:54:22.34]
[CPU: 247.7 MB]
Loaded passthrough dset with 15553 items

[2024-02-02 16:54:22.35]
[CPU: 249.9 MB]
Intersection of output and passthrough has 15553 items

[2024-02-02 16:54:22.84]
[CPU: 250.1 MB]
Checking outputs for output group exposures_success

[2024-02-02 16:54:22.87]
[CPU: 250.1 MB]
Checking outputs for output group exposures_fail

[2024-02-02 16:54:22.98]
[CPU: 259.0 MB]
Updating job size…

[2024-02-02 16:54:42.00]
[CPU: 259.5 MB]
Exporting job and creating csg files…

[2024-02-02 16:54:42.59]
[CPU: 259.5 MB]

[2024-02-02 16:54:42.59]
[CPU: 259.5 MB]
Job complete. Total time 436.24s

And，one of the error logs created by CTFFIND4 is as follows:

    **   Welcome to Ctffind   **

        Version : 4.1.10
       Compiled : Jan 18 2018
           Mode : Scripted

Input image file name : /work/home/faculty/yhzhang/YZ_lab/PJJ/20240124_25-mg/CS-20240121-mg/J3/imported/009449807666329243632_FoilHole_24018184_Data_23985025_2_20240126_081307_EER.mrc
e[31mError: mode 12 MRC files not currently supported
e[0m
By the way,the ctffind bins are in the correct path:
[yhzhang@mgt bin]$ ls /work/home/faculty/yhzhang/cryosparc/cryosparc_worker/deps/external/ctffind-4.1.10/bin -lah
total 32M
drwxr-xr-x 2 yhzhang faculty 4.0K Feb 2 17:19 .
drwxr-xr-x 3 yhzhang faculty 4.0K Dec 23 2022 …
-rwxr-xr-x 1 yhzhang faculty 31M Dec 23 2022 ctffind
-rwxr-xr-x 1 yhzhang faculty 4.9K Dec 23 2022 ctffind_plot_results.sh

So,I want to know how can ideal with this?Thankyou very much.

rbs_sci · February 2, 2024, 2:38pm

You can’t use CTFFIND 4.1.10 with 16-bit MRC, it’s not supported.

You’ll either need to compile the experimental version of CTFFIND from cisTEM2 and move it to the appropriate place yourself (unsupported but will probably work), or use the Patch CTF estimation from CryoSPARC itself.

edit:

Except for very high magnification data, or a couple of other limited scenarios, Patch CTF does a good job. If it really struggles, set the patches to 1x1 so it behave like CTFFIND and gives a micrograph global defocus/astigmatism estimate.

CTFFind4 jobs fail

[2024-02-02 16:54:21.94] [CPU: 244.5 MB]

[2024-02-02 16:54:21.94]
[CPU: 244.5 MB]