ModuleNotFoundError: no module named 'pycuda' when trying to install cryosparc_worker

Hello,

I am trying to install the cryosparc worker node as I was getting an error before when running jobs where “License is valid” would continually print to console.

I run the following command, and the master installs fine. I can access the web interface. But when it tries to install the worker, I get "ModuleNotFoundError: No module named 'pycuda’

./install.sh --standalone --license $LICENSE_ID --worker_path /home/CryoSPARC/cryosparc_worker --cudapath /usr/local/cuda --nossd --initial_email "zzz" --initial_password "zzz" --initial_username "zzz" --initial_firstname "zzz" --initial_lastname "zzz" --port 8000

So, I was able to get pycuda installed, but I still cannot install the cryosparc_worker.

My path was missing /usr/local/cuda/bin, which then allowed pycuda to install via pip3.

I guess I have two installations of python on my machine, 2.7 and 3.8. Does CryoSPARC use python 2.7? pycuda installs with pip3 without a hitch, but with pip2 it doesn’t seem to be working.

Here is the bottom of my .bashrc file:

 Added by cryoSPARC:
export PATH="/home/CryoSPARC/cryosparc_master/bin":$PATH
# CUDA
export PATH="/usr/local/cuda/bin:${PATH}"
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH/usr/local/cuda/lib64

I am still stuck on installation though (same error as in the image above), and don’t know what to do :frowning:

Hi @NKHO, try removing the cryosparc_worker/deps folder, then run the installation again.

Hi @nfrasser,

Thank you for your response. Unfortunately, I still get the same error. Here is the (almost) full output of installation:

******* CRYOSPARC SYSTEM: WORKER INSTALLER ***********************

 Installation Settings:
   License ID              : ---
   Root Directory          : /home/nick/CryoSPARC/installation/cryosparc_worker
   Standalone Installation : true
   Version                 : v3.1.0

******************************************************************

 CUDA check..
 Found nvidia-smi at /usr/bin/nvidia-smi

 CUDA Path was provided as /usr/local/cuda
 Checking CUDA installation...
 Found nvcc at /usr/local/cuda/bin/nvcc
 The above cuda installation will be used but can be changed later.

******************************************************************

 Setting up hard-coded config.sh environment variables

******************************************************************

 Installing all dependencies.

Warning: conda environment not found; this indicates that a cryoSPARC installation is either incomplete or in progress
Checking dependencies... 
Dependencies for python have not changed.
Currently checking hash for ctffind
Dependencies for ctffind have not changed.
Currently checking hash for gctf
Dependencies for gctf have not changed.
Completed dependency check. 

******* CRYOSPARC WORKER INSTALLATION COMPLETE *******************

 In order to run processing jobs, you will need to connect this
 worker to a cryoSPARC master.

******************************************************************

******************************************************************

 Connecting the cryoSPARC worker to the master...

Warning: conda environment not found; this indicates that a cryoSPARC installation is either incomplete or in progress
 ---------------------------------------------------------------
  CRYOSPARC CONNECT --------------------------------------------
 ---------------------------------------------------------------
  Attempting to register worker Desktop to command Desktop:8002
  Connecting as unix user nick
  Will register using ssh string: nick@Desktop
  If this is incorrect, you should re-run this command with the flag --sshstr <ssh string> 
 ---------------------------------------------------------------
  Connected to master.
 ---------------------------------------------------------------
  Current connected workers:
 ---------------------------------------------------------------
  Autodetecting available GPUs...
Traceback (most recent call last):
  File "bin/connect.py", line 233, in <module>
    gpu_devidxs = check_gpus()
  File "bin/connect.py", line 107, in check_gpus
    num_devs = print_gpu_list()
  File "bin/connect.py", line 24, in print_gpu_list
    import pycuda.driver as cudrv
ModuleNotFoundError: No module named 'pycuda'

During the installation, when it is trying to install pycuda:

nstalling collected packages: scikit-cuda, pycuda
    Running setup.py install for pycuda ... error
    ERROR: Command errored out with exit status 1:
     command: /home/nick/CryoSPARC/installation/cryosparc_worker/deps/anaconda/envs/cryosparc_worker_env/bin/python3.7 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-req-build-nwq5dreu/setup.py'"'"'; __file__='"'"'/tmp/pip-req-build-nwq5dreu/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-0io9t3rj/install-record.txt --single-version-externally-managed --compile --install-headers /home/nick/CryoSPARC/installation/cryosparc_worker/deps/anaconda/envs/cryosparc_worker_env/include/python3.7m/pycuda
         cwd: /tmp/pip-req-build-nwq5dreu/
    Complete output (99 lines):
    *************************************************************
    *** I have detected that you have not run configure.py.
    *************************************************************
    *** Additionally, no global config files were found.
    *** I will go ahead with the default configuration.
    *** In all likelihood, this will not work out.
    ***
    *** See README_SETUP.txt for more information.
    ***
    *** If the build does fail, just re-run configure.py with the
    *** correct arguments, and then retry. Good luck!
    *************************************************************
    *** HIT Ctrl-C NOW IF THIS IS NOT WHAT YOU WANT
    *************************************************************
    Continuing in 1 seconds...    
    /home/nick/CryoSPARC/installation/cryosparc_worker/deps/anaconda/envs/cryosparc_worker_env/lib/python3.7/distutils/dist.py:274: UserWarning: Unknown distribution option: 'test_requires'
      warnings.warn(msg)
    running install
    running build
    running build_py
    creating build
    creating build/lib.linux-x86_64-3.7
    creating build/lib.linux-x86_64-3.7/pycuda
    copying pycuda/_mymako.py -> build/lib.linux-x86_64-3.7/pycuda
    copying pycuda/elementwise.py -> build/lib.linux-x86_64-3.7/pycuda
    copying pycuda/compiler.py -> build/lib.linux-x86_64-3.7/pycuda
    copying pycuda/tools.py -> build/lib.linux-x86_64-3.7/pycuda
    copying pycuda/scan.py -> build/lib.linux-x86_64-3.7/pycuda
    copying pycuda/debug.py -> build/lib.linux-x86_64-3.7/pycuda
    copying pycuda/cumath.py -> build/lib.linux-x86_64-3.7/pycuda
    copying pycuda/reduction.py -> build/lib.linux-x86_64-3.7/pycuda
    copying pycuda/autoinit.py -> build/lib.linux-x86_64-3.7/pycuda
    copying pycuda/gpuarray.py -> build/lib.linux-x86_64-3.7/pycuda
    copying pycuda/_cluda.py -> build/lib.linux-x86_64-3.7/pycuda
    copying pycuda/curandom.py -> build/lib.linux-x86_64-3.7/pycuda
    copying pycuda/driver.py -> build/lib.linux-x86_64-3.7/pycuda
    copying pycuda/__init__.py -> build/lib.linux-x86_64-3.7/pycuda
    copying pycuda/characterize.py -> build/lib.linux-x86_64-3.7/pycuda
    creating build/lib.linux-x86_64-3.7/pycuda/gl
    copying pycuda/gl/autoinit.py -> build/lib.linux-x86_64-3.7/pycuda/gl
    copying pycuda/gl/__init__.py -> build/lib.linux-x86_64-3.7/pycuda/gl
    creating build/lib.linux-x86_64-3.7/pycuda/sparse
    copying pycuda/sparse/inner.py -> build/lib.linux-x86_64-3.7/pycuda/sparse
    copying pycuda/sparse/cg.py -> build/lib.linux-x86_64-3.7/pycuda/sparse
    copying pycuda/sparse/operator.py -> build/lib.linux-x86_64-3.7/pycuda/sparse
    copying pycuda/sparse/packeted.py -> build/lib.linux-x86_64-3.7/pycuda/sparse
    copying pycuda/sparse/__init__.py -> build/lib.linux-x86_64-3.7/pycuda/sparse
    copying pycuda/sparse/pkt_build.py -> build/lib.linux-x86_64-3.7/pycuda/sparse
    copying pycuda/sparse/coordinate.py -> build/lib.linux-x86_64-3.7/pycuda/sparse
    creating build/lib.linux-x86_64-3.7/pycuda/compyte
    copying pycuda/compyte/dtypes.py -> build/lib.linux-x86_64-3.7/pycuda/compyte
    copying pycuda/compyte/array.py -> build/lib.linux-x86_64-3.7/pycuda/compyte
    copying pycuda/compyte/__init__.py -> build/lib.linux-x86_64-3.7/pycuda/compyte
    running egg_info
    writing pycuda.egg-info/PKG-INFO
    writing dependency_links to pycuda.egg-info/dependency_links.txt
    writing requirements to pycuda.egg-info/requires.txt
    writing top-level names to pycuda.egg-info/top_level.txt
    reading manifest file 'pycuda.egg-info/SOURCES.txt'
    reading manifest template 'MANIFEST.in'
    warning: no files found matching 'doc/source/_static/*.css'
    warning: no files found matching 'doc/source/_templates/*.html'
    warning: no files found matching '*.cpp' under directory 'bpl-subset/bpl_subset/boost'
    warning: no files found matching '*.html' under directory 'bpl-subset/bpl_subset/boost'
    warning: no files found matching '*.inl' under directory 'bpl-subset/bpl_subset/boost'
    warning: no files found matching '*.txt' under directory 'bpl-subset/bpl_subset/boost'
    warning: no files found matching '*.h' under directory 'bpl-subset/bpl_subset/libs'
    warning: no files found matching '*.ipp' under directory 'bpl-subset/bpl_subset/libs'
    warning: no files found matching '*.pl' under directory 'bpl-subset/bpl_subset/libs'
    writing manifest file 'pycuda.egg-info/SOURCES.txt'
    creating build/lib.linux-x86_64-3.7/pycuda/cuda
    copying pycuda/cuda/pycuda-complex-impl.hpp -> build/lib.linux-x86_64-3.7/pycuda/cuda
    copying pycuda/cuda/pycuda-complex.hpp -> build/lib.linux-x86_64-3.7/pycuda/cuda
    copying pycuda/cuda/pycuda-helpers.hpp -> build/lib.linux-x86_64-3.7/pycuda/cuda
    copying pycuda/sparse/pkt_build_cython.pyx -> build/lib.linux-x86_64-3.7/pycuda/sparse
    running build_ext
    building '_driver' extension
    creating build/temp.linux-x86_64-3.7
    creating build/temp.linux-x86_64-3.7/src
    creating build/temp.linux-x86_64-3.7/src/cpp
    creating build/temp.linux-x86_64-3.7/src/wrapper
    creating build/temp.linux-x86_64-3.7/bpl-subset
    creating build/temp.linux-x86_64-3.7/bpl-subset/bpl_subset
    creating build/temp.linux-x86_64-3.7/bpl-subset/bpl_subset/libs
    creating build/temp.linux-x86_64-3.7/bpl-subset/bpl_subset/libs/python
    creating build/temp.linux-x86_64-3.7/bpl-subset/bpl_subset/libs/python/src
    creating build/temp.linux-x86_64-3.7/bpl-subset/bpl_subset/libs/python/src/object
    creating build/temp.linux-x86_64-3.7/bpl-subset/bpl_subset/libs/python/src/converter
    creating build/temp.linux-x86_64-3.7/bpl-subset/bpl_subset/libs/system
    creating build/temp.linux-x86_64-3.7/bpl-subset/bpl_subset/libs/system/src
    creating build/temp.linux-x86_64-3.7/bpl-subset/bpl_subset/libs/smart_ptr
    creating build/temp.linux-x86_64-3.7/bpl-subset/bpl_subset/libs/smart_ptr/src
    creating build/temp.linux-x86_64-3.7/bpl-subset/bpl_subset/libs/thread
    creating build/temp.linux-x86_64-3.7/bpl-subset/bpl_subset/libs/thread/src
    creating build/temp.linux-x86_64-3.7/bpl-subset/bpl_subset/libs/thread/src/pthread
    gcc -pthread -B /home/nick/CryoSPARC/installation/cryosparc_worker/deps/anaconda/envs/cryosparc_worker_env/compiler_compat -Wl,--sysroot=/ -Wsign-compare -fwrapv -Wall -O3 -DNDEBUG -fPIC -DBOOST_ALL_NO_LIB=1 -DBOOST_THREAD_BUILD_DLL=1 -DBOOST_MULTI_INDEX_DISABLE_SERIALIZATION=1 -DBOOST_PYTHON_SOURCE=1 -Dboost=pycudaboost -DBOOST_THREAD_DONT_USE_CHRONO=1 -DPYGPU_PACKAGE=pycuda -DPYGPU_PYCUDA=1 -DHAVE_CURAND=1 -Isrc/cpp -Ibpl-subset/bpl_subset -I/usr/local/cuda/include -I/home/nick/CryoSPARC/installation/cryosparc_worker/deps/anaconda/envs/cryosparc_worker_env/lib/python3.7/site-packages/numpy/core/include -I/home/nick/CryoSPARC/installation/cryosparc_worker/deps/anaconda/envs/cryosparc_worker_env/include/python3.7m -c src/cpp/cuda.cpp -o build/temp.linux-x86_64-3.7/src/cpp/cuda.o
    gcc: fatal error: cannot execute ‘cc1plus’: execvp: No such file or directory
    compilation terminated.
    error: command 'gcc' failed with exit status 1
    ----------------------------------------
ERROR: Command errored out with exit status 1: /home/nick/CryoSPARC/installation/cryosparc_worker/deps/anaconda/envs/cryosparc_worker_env/bin/python3.7 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-req-build-nwq5dreu/setup.py'"'"'; __file__='"'"'/tmp/pip-req-build-nwq5dreu/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-0io9t3rj/install-record.txt --single-version-externally-managed --compile --install-headers /home/nick/CryoSPARC/installation/cryosparc_worker/deps/anaconda/envs/cryosparc_worker_env/include/python3.7m/pycuda Check the logs for full command output.
  ------------------------------------------------------------------------
    Done.

@NKHO it looks like you’re missing a C++ compiler, required for installing pycuda. Please try installing g++. This will vary depending on your system. On Ubuntu/Debian, you can install it like this:

sudo apt-get install g++

Once you do that, try the following command from the cryosparc_worker folder:

bin/cryosparcw newcuda /usr/local/cuda

Hi @nfrasser, thanks for the help.

I managed to resolve the issue. Unfortunately I can’t remember exactly all of the steps I took but installing g++ may have been one of them in order to get pycuda to finall build. I ended up re-installing ubuntu just for the heck of it. The main problem, I think, that others may fail to consider if they ever encounter this issue is that running ./install … with an existing installation will not re-install CryoSPARC. I had to remove cryosparc_master and cryosparc_worker, unpack the original tar and run ./install again.

Could be useful to any noobs out there who try to take steps to fix their issue but continue to receive the error… make sure you you uninstall before trying to re-install! :slight_smile:

3 Likes