Gctf output error

Troubleshooting
1

Hi,
Since updating to the latest version everything is running smoothly, except gctf does not work anymore… I get a green light after about 30 seconds as if it ran normally but there is no output. I get this error message:

Using files in wildcard path /media/ben/ben003/CryoSparcV2_output/P6/J79/motioncorrected/FoilHole_134*.mrc

Writing /media/ben/ben003/CryoSparcV2_output/P6/J79/input_ctfstar.star

Gctf command: /media/ben/cryoem1/CryoSparc_v2/cryosparc2_worker/deps/external/gctf-1.06/bin/Gctf-v1.06_sm_30_cu8.0_x86_64 --apix 1.06 --kV 300.0 --Cs 2.7 --ac 0.1 --ctfstar /media/ben/ben003/CryoSparcV2_output/P6/J79/micrographs_all_gctf.star --input_ctfstar /media/ben/ben003/CryoSparcV2_output/P6/J79/input_ctfstar.star --gid 1 --boxsize 1024 --boxsuffix _automatch.star --logsuffix _gctf.log

Running process 3695

ERROR Gctf failed to produce output file /media/ben/ben003/CryoSparcV2_output/P6/J79/micrographs_all_gctf.star

any help would be greatly appreciated :slight_smile:

2

Hi @BWise,

Is it possible if you can provide the logs for GCTF and the cryoSPARC job itself?
cryosparcm joblog P6 J79
The GCTF logs will be found in the actual job directory.

3

Hi,
Sorry for the slow reply, but here is the log: it seems i am missing a library file…
/media/ben/cryoem1/CryoSparc_v2/cryosparc2_worker/deps/external/gctf-1.06/bin/Gctf-v1.06_sm_30_cu8.0_x86_64: error while loading shared libraries: libcufft.so.8.0: cannot open shared object file: No such file or directory

Running job J108 of type ctf_estimation_gctf
Running job on hostname %s Arctic
Allocated Resources : {u’lane’: u’default’, u’target’: {u’lane’: u’default’, u’name’: u’Arctic’, u’title’: u’Worker node Arctic’, u’resource_slots’: {u’GPU’: [0, 1], u’RAM’: [0, 1, 2, 3, 4, 5, 6, 7], u’CPU’: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15]}, u’hostname’: u’Arctic’, u’worker_bin_path’: u’/media/ben/cryoem1/CryoSparc_v2/cryosparc2_worker/bin/cryosparcw’, u’cache_path’: u’/home/ben/Cryosparc2_cache/’, u’cache_quota_mb’: None, u’resource_fixed’: {u’SSD’: True}, u’cache_reserve_mb’: 10000, u’type’: u’node’, u’ssh_str’: u’ram@Arctic’, u’desc’: None}, u’license’: True, u’hostname’: u’Arctic’, u’slots’: {u’GPU’: [0], u’RAM’: [0], u’CPU’: [0]}, u’fixed’: {u’SSD’: False}, u’lane_type’: u’node’}

========= main process now complete.
========= monitor process now complete.

thanks for your help!
ben

4

Hi @BWise,

This version of GCTF by Dr. Kai Zhang is only compiled for CUDA 8.0 and lower- is it possible that you are using cryoSPARC with CUDA 9.0+?

5

Hi,
Yes, good guess!
We have CUDA 9.1 installed, everything works fine except gctf. How can we solve this?
thanks!

6

You could check where CryoSPARC puts the Gctf binary, and replace it with the CUDA9 one from Kai Zhang’s website - I did that to upgrade CTFFIND to 4.1.13

7

good idea, thanks. however, unless i am missing something (which could be possible)… i could not find a cuda9 version…

8

Oh you’re right, for 1.06 there isn’t - there is for 1.18 though (https://www.mrc-lmb.cam.ac.uk/kzhang/Gctf/Gctf_v1.18_b2/bin/) - you might try that, though I’m not 100% sure it will be compatible with the cryosparc wrapper. You can always replace with 1.06 if it doesn’t work

Cheers
Oli

9

thanks! i downloaded a cuda9 version. but how do i add the path in cryosparc to the executable?

10

you’d need to replace this file:

/home/user/cryosparc2_worker/deps/external/gctf-1.06/bin/Gctf-v1.06_sm_30_cu8.0_x86_64

(obvs you’ll need to rename the binary so that it has the same name as this one)

so e.g.

cp ~/Downloads/gctf_cuda9_file /home/user/cryosparc2_worker/deps/external/gctf-1.06/bin/Gctf-v1.06_sm_30_cu8.0_x86_64

should do the trick. I’d copy the existing binary to a backup first though, so you can replace it if it doesn’t work

Cheers
Oli

11

thanks! we are moving in the right direction… it ran to completion, but it created empty plots for the Gctf 1D diagnostic plot for exposure graph. and completely black pictures for the Gctf global 2-D diagnostic plot for exposure… the output files are very small… (2MB)

12

Yeah the diagnostic images for 1.18 are slightly different than 1.06 IIRC - but maybe the CTF estimates are still okay? Depends where compatibility between the wrapper and Gctf is breaking… is it giving defocus estimates in the log?

13

both the defocus values don’t change… all the micrographs say this: (only the Angle changes between micrographs)
Defocus_U Defocus_V Angle CCC
4500.00 5500.00 1.63 -nan Final Values

14

Hmmm, looks like 1.18 isn’t compatible with the Gctf wrapper… I think it will require an update to the Gctf wrapper to support.

In the meantime, if you update CTFFIND to 4.1.13 and turn off the “exhaustive search” option it is lightning fast, assuming you have a few CPU cores. In my hands it gives comparable if not slightly better results than Gctf.

Cheers
Oli

15

thanks a lot Oli,
I shall give that a try :slight_smile:

16

Hi Oli,

I have been perusing the forums quite a lot recently and I always find your input very helpful. Do you know if Gctf 1.18 is supported or compatible with the cryosparc wrapper now? Also, what was/is your need to update CTFFIND4 to 4.1.13? It looks like for CryoSPARC v3.4.0, CTFFIND is v4.1.10 and allows the user to toggle off the exhaustive search so I am just curious about the differences between the two versions.

Thanks,
Karl

17

Hi Karl,

Not sure - I mostly use Patch CTF these days, it has been a while since I used Gctf… can’t remember exactly why I needed or wanted that version of CTFFIND at the time, sorry…

Cheers
Oli