Cufft library version 10 is not listed

Install, Configure and Update
1

CryoSPARC 3.3.2
Cudatoolkit 11.1
RHEL 8.5

Patch motion correction (multi) fails with:

  File "/projects/MOLBIO/local/cryosparc-della-test-2/cryosparc_worker/cryosparc_compute/skcuda_internal/cufft.py", line 53, in <module>
    raise OSError('cufft library not found')

That seems to be because libcufft is installed with these names – .so, .so.10, and .so.10.3.0.74:

# ls -l /usr/local/cuda-11.1/lib64/libcufft.so*
lrwxrwxrwx. 1 root root        14 Sep 16  2020 /usr/local/cuda-11.1/lib64/libcufft.so -> libcufft.so.10
lrwxrwxrwx. 1 root root        21 Sep 16  2020 /usr/local/cuda-11.1/lib64/libcufft.so.10 -> libcufft.so.10.3.0.74
-rwxr-xr-x. 1 root root 235963680 Sep 16  2020 /usr/local/cuda-11.1/lib64/libcufft.so.10.3.0.74

The variable _linux_version_list in /projects/MOLBIO/local/cryosparc-della-test-2/cryosparc_worker/cryosparc_compute/skcuda_internal/cufft.py does not list those:

_linux_version_list = [11.0, 10.1, 10.0, 9.2, 9.1, 9.0, 8.0, 7.5, 7.0, 6.5, 6.0, 5.5, 5.0, 4.0]

Adding 10 to the list fixes the problem.

_linux_version_list = [10, 11.0, 10.1, 10.0, 9.2, 9.1, 9.0, 8.0, 7.5, 7.0, 6.5, 6.0, 5.5, 5.0, 4.0]

– Matthew Cahn

2

Hi Matthew,

Did your installation/configuration workflow include
/projects/MOLBIO/local/cryosparc-della-test-2/cryosparc_worker/install.sh [..] --cudapath /usr/local/cuda-11.1
or
/projects/MOLBIO/local/cryosparc-della-test-2/cryosparc_worker/bin/cryosparcw newcuda /usr/local/cuda-11.1
?
Do all worker nodes that use the installation at /projects/MOLBIO/local/cryosparc-della-test-2/cryosparc_worker have access to the CUDA toolkit at the same /usr/local/cuda-11.1 path?

3

Yes, I did the worker install as follows:

cd cryosparc_worker

./install.sh --license $LICENSE_ID --cudapath /usr/local/cuda-11.1

Yes, all the worker nodes have the same cuda path.

Note that it’s quite clear that the versions of libcufft.so that are installed are not listed in cryosparc_worker/cryosparc_compute/skcuda_internal/cufft.py – the listed versions don’t match any of the symbolic links for the installed library. So it’s just a matter of adding a version number that actually exists to the list in that py file. Or, conversely, making the appropriate symbolic links for libcufft – but I think the cuda installation we have is a standard thing, so I believe we have the links that anyone running RHEL 8 would have.

4

Thank you for bringing this issue to our attention.