Cuda problem after downgrading back to 4.3.1

I have the following setup:
1x Nvidia GT 710
2x Nvidia GTX 1070
nvidia-driver-470
2 screens connected to the 710
Ubuntu 20.04.6 LTS
Everything was working fine with Cryosparc 4.3.1

Updating to 4.4 required to upgrade to at least nvidida-driver-520. The GT 710 however doesnt support that driver. So I managed that only the 1070 are using the 520 driver while the 710 is using the neauvou. In this configuration I could update CS to 4.4 and all worked fine expect that only one screen was working at the 710 (there was no signal for the second screen neither from the 710 nor from the 1070).
So I downgraded both, Cryosparc and the drivers and somehow managed to mess up a lot…

CS is back to 4.3.1, my driver is back to 470, both screens working, but CS cannot find cuda anymore…

2D classification gives the following error:

Traceback (most recent call last):
File “cryosparc_master/cryosparc_compute/run.py”, line 82, in cryosparc_compute.run.main
File “/home/phz-17br01/Programme/cryosparc_worker/cryosparc_compute/jobs/jobregister.py”, line 448, in get_run_function
runmod = importlib.import_module(“…”+modname, name)
File “/home/phz-17br01/Programme/cryosparc_worker/deps/anaconda/envs/cryosparc_worker_env/lib/python3.8/importlib/init.py”, line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File “”, line 1014, in _gcd_import
File “”, line 991, in _find_and_load
File “”, line 975, in _find_and_load_unlocked
File “”, line 671, in _load_unlocked
File “”, line 1174, in exec_module
File “”, line 219, in _call_with_frames_removed
File “cryosparc_master/cryosparc_compute/jobs/class2D/run.py”, line 13, in init cryosparc_compute.jobs.class2D.run
File “/home/phz-17br01/Programme/cryosparc_worker/cryosparc_compute/engine/init.py”, line 8, in
from .engine import * # noqa
File “cryosparc_master/cryosparc_compute/engine/engine.py”, line 9, in init cryosparc_compute.engine.engine
File “cryosparc_master/cryosparc_compute/engine/cuda_core.py”, line 4, in init cryosparc_compute.engine.cuda_core
ModuleNotFoundError: No module named ‘pycuda’

Anyone has an idea what I could do to fix that?

Fixed it by

  1. purging nvidia:
    sudo apt-get purge '^nvidia-.'*
  2. reinstalling working driver that is supported by both GPUs
    sudo apt-get install nvidia-driver-470
  3. installing cuda
    sudo apt install nvidia-cuda-toolkit
  4. telling Cryosparc where cuda is
    navigate to the cryosparc_worker directory
    ./bin/cryosparcw newcuda /usr/local/cuda-11.5
1 Like