@wtempel
The Cryosparc version is 4.2.1,no patch.
Traceback like this:
Traceback (most recent call last):
File "cryosparc_master/cryosparc_compute/run.py", line 83, in cryosparc_compute.run.main
File "/opt/cryosparc/cryosparc_worker/cryosparc_compute/jobs/jobregister.py", line 442, in get_run_function
runmod = importlib.import_module(".."+modname, __name__)
File "/opt/cryosparc/cryosparc_worker/deps/anaconda/envs/cryosparc_worker_env/lib/python3.8/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
File "<frozen importlib._bootstrap>", line 991, in _find_and_load
File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 1174, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "cryosparc_master/cryosparc_compute/jobs/flex_refine/run_train.py", line 12, in init cryosparc_compute.jobs.flex_refine.run_train
File "cryosparc_master/cryosparc_compute/jobs/flex_refine/flexmod.py", line 19, in init cryosparc_compute.jobs.flex_refine.flexmod
File "/opt/cryosparc/cryosparc_worker/deps/anaconda/envs/cryosparc_worker_env/lib/python3.8/site-packages/torch/__init__.py", line 229, in <module>
from torch._C import * # noqa: F403
ImportError: libnccl.so.2: cannot open shared object file: No such file or directory
Below are output of 2 commands
[user@local cryosparc_worker]$ ldd /opt/cryosparc/cryosparc_worker/deps/anaconda/envs/cryosparc_worker_env/lib/python3.8/site-packages/torch/_C.cpython-38-x86_64-linux-gnu.so
linux-vdso.so.1 => (0x00007ffd0ab95000)
libtorch_python.so => /opt/cryosparc/cryosparc_worker/deps/anaconda/envs/cryosparc_worker_env/lib/python3.8/site-packages/torch/lib/libtorch_python.so (0x00007f71e50b2000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f71e4e96000)
libc.so.6 => /lib64/libc.so.6 (0x00007f71e4ac8000)
libshm.so => /opt/cryosparc/cryosparc_worker/deps/anaconda/envs/cryosparc_worker_env/lib/python3.8/site-packages/torch/lib/libshm.so (0x00007f71e6343000)
libtorch.so => /opt/cryosparc/cryosparc_worker/deps/anaconda/envs/cryosparc_worker_env/lib/python3.8/site-packages/torch/lib/libtorch.so (0x00007f71e631d000)
libnvToolsExt.so.1 => /usr/local/cuda-10.1/targets/x86_64-linux/lib/libnvToolsExt.so.1 (0x00007f71e48bf000)
libtorch_cpu.so => /opt/cryosparc/cryosparc_worker/deps/anaconda/envs/cryosparc_worker_env/lib/python3.8/site-packages/torch/lib/libtorch_cpu.so (0x00007f71cba0c000)
libtorch_cuda.so => /opt/cryosparc/cryosparc_worker/deps/anaconda/envs/cryosparc_worker_env/lib/python3.8/site-packages/torch/lib/libtorch_cuda.so (0x00007f71a594b000)
libc10_cuda.so => /opt/cryosparc/cryosparc_worker/deps/anaconda/envs/cryosparc_worker_env/lib/python3.8/site-packages/torch/lib/libc10_cuda.so (0x00007f71e62ad000)
libc10.so => /opt/cryosparc/cryosparc_worker/deps/anaconda/envs/cryosparc_worker_env/lib/python3.8/site-packages/torch/lib/libc10.so (0x00007f71e61f2000)
libcudart.so.11.0 => /usr/local/cuda11.1/targets/x86_64-linux/lib/libcudart.so.11.0 (0x00007f71a56c6000)
libcudnn.so.8 => not found
libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00007f71a53be000)
libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f71a51a8000)
/lib64/ld-linux-x86-64.so.2 (0x00007f71e6167000)
librt.so.1 => /lib64/librt.so.1 (0x00007f71a4fa0000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007f71a4d9c000)
libm.so.6 => /lib64/libm.so.6 (0x00007f71a4a9a000)
libgomp-a34b3233.so.1 => /opt/cryosparc/cryosparc_worker/deps/anaconda/envs/cryosparc_worker_env/lib/python3.8/site-packages/torch/lib/libgomp-a34b3233.so.1 (0x00007f71a4870000)
libcupti.so.11.7 => not found
libcusparse.so.11 => /usr/local/cuda11.1/targets/x86_64-linux/lib/libcusparse.so.11 (0x00007f7196637000)
libcurand.so.10 => /usr/local/cuda-10.1/targets/x86_64-linux/lib/libcurand.so.10 (0x00007f71925d6000)
libcudnn.so.8 => not found
libnccl.so.2 => not found
libcufft.so.10 => /usr/local/cuda-10.1/targets/x86_64-linux/lib/libcufft.so.10 (0x00007f7189f9c000)
libcublas.so.11 => /usr/local/cuda11.1/targets/x86_64-linux/lib/libcublas.so.11 (0x00007f7181b80000)
libcublasLt.so.11 => /usr/local/cuda11.1/targets/x86_64-linux/lib/libcublasLt.so.11 (0x00007f7173b8c000)
[user@local cryosparc_worker]$ cat /opt/cryosparc/cryosparc_worker/deps/anaconda/envs/cryosparc_worker_env/lib/python3.8/site-packages/torch/version.py
__version__ = '2.0.0+cu117'
debug = False
cuda = '11.7'
git_version = 'c263bd43e8e8502d4726643bc6fd046f0130ac0e'
hip = None
From the output，like @rbs_sci said， seems like I need install these “not found” libraries into the cuda of OS.