ModuleNotFoundError : No module named 'Crypto' (v3.0.0)

alburse · December 10, 2020, 8:23pm

Hi
We have been using cryosparc on a cluster. I was able to update my cryosparc from 2.15 to 3.0 without any issue. However whenever I submit a job to queue, the job is launched but it does not continue running. The job.log file gives this error

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "cryosparc_worker/cryosparc_compute/run.py", line 13, in init cryosparc_compute.run
ModuleNotFoundError: No module named 'Crypto'

I also tried a new installation and it is the same behavior.

Would you please help me solve this problem? Thank you
Alpay
Skiniotis Lab

nfrasser · December 10, 2020, 8:39pm

Hi @alburse, this indicates that the the dependency installation for the worker may not have completed all the the way through or can’t access them for some reason. Try this:

Via command line, navigate to either the cryosparc_worker or cryosparc2_worker, depending on your installation path
Reinstall dependencies with this command:

bin/cryosparcw forcedeps

Then try again. If that doesn’t work send me the output of this command from this command but take out the line that starts with export "CRYOSPARC_LICENSE_ID= before you post

bin/cryosparcw env

alburse · December 10, 2020, 9:19pm

Still the same

[alpays@sh02-07n01 /share/PI/yiorgo/alpay/modules/alpaysprograms/cryosparc/2.14/cryosparc2_worker]$ bin/cryosparcw env
export "CRYOSPARC_USE_GPU=true"
export "CRYOSPARC_CONDA_ENV=cryosparc_worker_env"
export "CRYOSPARC_DEVELOP=false"
...
export "CRYOSPARC_ROOT_DIR=/home/groups/yiorgo/alpay/modules/alpaysprograms/cryosparc/2.14/cryosparc2_worker"
export "CRYOSPARC_DIR=/share/PI/yiorgo/alpay/modules/alpaysprograms/cryosparc/2.12"
export "CRYOSPARC_PATH=/home/groups/yiorgo/alpay/modules/alpaysprograms/cryosparc/2.14/cryosparc2_worker/bin"
export "CRYOSPARC_LD_LIBRARY_PATH=/home/groups/yiorgo/alpay/modules/alpaysprograms/cryosparc/2.14/cryosparc2_worker/cryosparc_compute/blobio"
export "CRYOSPARC_CUDA_PATH=/share/software/user/open/cuda/10.0.130"
export "PATH=/home/groups/yiorgo/alpay/modules/alpaysprograms/cryosparc/2.14/cryosparc2_worker/bin:/home/groups/yiorgo/alpay/modules/alpaysprograms/cryosparc/2.14/cryosparc2_worker/deps/anaconda/envs/cryosparc_worker_env/bin:/home/groups/yiorgo/alpay/modules/alpaysprograms/cryosparc/2.14/cryosparc2_worker/deps/anaconda/condabin:/share/software/user/open/cuda/10.0.130/bin:/share/PI/yiorgo/alpay/modules/alpaysprograms/cryosparc/2.14/cryosparc2_master/deps/external/mongodb/bin:/share/PI/yiorgo/alpay/modules/alpaysprograms/cryosparc/2.14/cryosparc2_master/deps/anaconda/bin:/share/PI/yiorgo/alpay/modules/alpaysprograms/cryosparc/2.14/cryosparc2_master/bin:/share/software/user/open/libtiff/4.0.8/bin:/share/software/user/open/libjpeg-turbo/1.5.1/bin:/share/software/user/open/gcc/6.3.0/bin:/share/software/user/open/glib/2.52.3/bin:/share/software/user/open/x11/7.7/bin:/share/software/user/open/llvm/4.0.0/bin:/share/software/user/open/libxml2/2.9.4/bin:/share/software/user/open/fontconfig/2.12.4/bin:/share/software/user/open/freetype/2.8/bin:/share/software/user/open/libpng/1.2.57/bin:/usr/lib64/nvidia:/share/software/user/open/cuda/10.0.130/bin:/share/software/user/open/cuda/10.0.130/nvvm/bin:/share/software/user/open/cuda/10.0.130/NsightCompute-1.0:/share/software/user/srcc/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/dell/srvadmin/iSM/bin:/home/users/alpays/bin"
export "LD_LIBRARY_PATH=/home/groups/yiorgo/alpay/modules/alpaysprograms/cryosparc/2.14/cryosparc2_worker/cryosparc_compute/blobio:/share/software/user/open/cuda/10.0.130/lib64:/share/PI/yiorgo/alpay/modules/alpaysprograms/cryosparc/2.14/cryosparc2_master/lib:/share/software/user/open/libtiff/4.0.8/lib:/share/software/user/open/libjpeg-turbo/1.5.1/lib:/share/software/user/open/gcc/6.3.0/lib64:/share/software/user/open/gcc/6.3.0/lib/gcc/x86_64-pc-linux-gnu:/share/software/user/open/gcc/6.3.0/lib:/share/software/user/open/glib/2.52.3/lib:/share/software/user/open/libffi/3.2.1/lib64:/share/software/user/open/x11/7.7/lib:/share/software/user/open/libxkbcommon/0.9.1/lib64:/share/software/user/open/llvm/4.0.0/lib:/share/software/user/open/libxml2/2.9.4/lib:/share/software/user/open/fontconfig/2.12.4/lib:/share/software/user/open/freetype/2.8/lib:/share/software/user/open/libpng/1.2.57/lib:/share/software/user/open/zlib/1.2.11/lib:/usr/lib64/nvidia:/share/software/user/open/cuda/10.0.130/lib64:/share/software/user/open/cuda/10.0.130/nvvm/lib64:/share/software/user/open/cuda/10.0.130/extras/Debugger/lib64:/share/software/user/open/cuda/10.0.130/extras/CUPTI/lib64"
export "LD_PRELOAD="
export "PYTHONPATH=/home/groups/yiorgo/alpay/modules/alpaysprograms/cryosparc/2.14/cryosparc2_worker"
export "PYTHONNOUSERSITE=true"
export "CONDA_PREFIX=/home/groups/yiorgo/alpay/modules/alpaysprograms/cryosparc/2.14/cryosparc2_worker/deps/anaconda/envs/cryosparc_worker_env"

stephan · December 10, 2020, 9:58pm

Hi @alburse,

Can you try the following:

Via command line, navigate to either the cryosparc_worker or cryosparc2_worker , depending on your installation path
Edit the file bin/cryosparcw: change the line export PYTHONPATH="$CRYOSPARC_ROOT_DIR" to export PYTHONPATH=""

Then try running a job again

alburse · December 10, 2020, 10:26pm

Hi @stephan
Unfortunately, it is still same. I restarted and tried again too. We desperately want to try the new NU Refinement.

stephan · December 10, 2020, 10:32pm

Hi @alburse,

Can you re-do the worker installation again, but send us the full outputs of this command?

Via command line, navigate to either the cryosparc_worker or cryosparc2_worker , depending on your installation path
Reinstall dependencies with this command:
```
bin/cryosparcw forcedeps
```
Send us the output

alburse · December 10, 2020, 11:00pm

@stephan Here is the output

[alpays@sh02-07n01 /home/groups/yiorgo/alpay/modules/alpaysprograms/cryosparc/2.14/cryosparc2_worker]$ bin/cryosparcw forcedeps
Checking dependencies...
Forcing dependencies to be reinstalled...
  ------------------------------------------------------------------------
  Installing anaconda python...
  ------------------------------------------------------------------------
PREFIX=/home/groups/yiorgo/alpay/modules/alpaysprograms/cryosparc/2.14/cryosparc2_worker/deps/anaconda
Unpacking payload ...
Collecting package metadata (current_repodata.json): done
Solving environment: done

## Package Plan ##

  environment location: /home/groups/yiorgo/alpay/modules/alpaysprograms/cryosparc/2.14/cryosparc2_worker/deps/anaconda

  added / updated specs:
    - _libgcc_mutex==0.1=main
    - ca-certificates==2020.1.1=0
    - certifi==2020.4.5.1=py37_0
    - cffi==1.14.0=py37he30daa8_1
    - chardet==3.0.4=py37_1003
    - conda-package-handling==1.6.1=py37h7b6447c_0
    - conda==4.8.3=py37_0
    - cryptography==2.9.2=py37h1ba5d50_0
    - idna==2.9=py_1
    - ld_impl_linux-64==2.33.1=h53a641e_7
    - libedit==3.1.20181209=hc058e9b_0
    - libffi==3.3=he6710b0_1
    - libgcc-ng==9.1.0=hdf63c60_0
    - libstdcxx-ng==9.1.0=hdf63c60_0
    - ncurses==6.2=he6710b0_1
    - openssl==1.1.1g=h7b6447c_0
    - pip==20.0.2=py37_3
    - pycosat==0.6.3=py37h7b6447c_0
    - pycparser==2.20=py_0
    - pyopenssl==19.1.0=py37_0
    - pysocks==1.7.1=py37_0
    - python==3.7.7=hcff3b4d_5
    - readline==8.0=h7b6447c_0
    - requests==2.23.0=py37_0
    - ruamel_yaml==0.15.87=py37h7b6447c_0
    - setuptools==46.4.0=py37_0
    - six==1.14.0=py37_0
    - sqlite==3.31.1=h62c20be_1
    - tk==8.6.8=hbc83047_0
    - tqdm==4.46.0=py_0
    - urllib3==1.25.8=py37_0
    - wheel==0.34.2=py37_0
    - xz==5.2.5=h7b6447c_0
    - yaml==0.1.7=had09818_2
    - zlib==1.2.11=h7b6447c_3


The following NEW packages will be INSTALLED:

  _libgcc_mutex      pkgs/main/linux-64::_libgcc_mutex-0.1-main
  ca-certificates    pkgs/main/linux-64::ca-certificates-2020.1.1-0
  certifi            pkgs/main/linux-64::certifi-2020.4.5.1-py37_0
  cffi               pkgs/main/linux-64::cffi-1.14.0-py37he30daa8_1
  chardet            pkgs/main/linux-64::chardet-3.0.4-py37_1003
  conda              pkgs/main/linux-64::conda-4.8.3-py37_0
  conda-package-han~ pkgs/main/linux-64::conda-package-handling-1.6.1-py37h7b6447c_0
  cryptography       pkgs/main/linux-64::cryptography-2.9.2-py37h1ba5d50_0
  idna               pkgs/main/noarch::idna-2.9-py_1
  ld_impl_linux-64   pkgs/main/linux-64::ld_impl_linux-64-2.33.1-h53a641e_7
  libedit            pkgs/main/linux-64::libedit-3.1.20181209-hc058e9b_0
  libffi             pkgs/main/linux-64::libffi-3.3-he6710b0_1
  libgcc-ng          pkgs/main/linux-64::libgcc-ng-9.1.0-hdf63c60_0
  libstdcxx-ng       pkgs/main/linux-64::libstdcxx-ng-9.1.0-hdf63c60_0
  ncurses            pkgs/main/linux-64::ncurses-6.2-he6710b0_1
  openssl            pkgs/main/linux-64::openssl-1.1.1g-h7b6447c_0
  pip                pkgs/main/linux-64::pip-20.0.2-py37_3
  pycosat            pkgs/main/linux-64::pycosat-0.6.3-py37h7b6447c_0
  pycparser          pkgs/main/noarch::pycparser-2.20-py_0
  pyopenssl          pkgs/main/linux-64::pyopenssl-19.1.0-py37_0
  pysocks            pkgs/main/linux-64::pysocks-1.7.1-py37_0
  python             pkgs/main/linux-64::python-3.7.7-hcff3b4d_5
  readline           pkgs/main/linux-64::readline-8.0-h7b6447c_0
  requests           pkgs/main/linux-64::requests-2.23.0-py37_0
  ruamel_yaml        pkgs/main/linux-64::ruamel_yaml-0.15.87-py37h7b6447c_0
  setuptools         pkgs/main/linux-64::setuptools-46.4.0-py37_0
  six                pkgs/main/linux-64::six-1.14.0-py37_0
  sqlite             pkgs/main/linux-64::sqlite-3.31.1-h62c20be_1
  tk                 pkgs/main/linux-64::tk-8.6.8-hbc83047_0
  tqdm               pkgs/main/noarch::tqdm-4.46.0-py_0
  urllib3            pkgs/main/linux-64::urllib3-1.25.8-py37_0
  wheel              pkgs/main/linux-64::wheel-0.34.2-py37_0
  xz                 pkgs/main/linux-64::xz-5.2.5-h7b6447c_0
  yaml               pkgs/main/linux-64::yaml-0.1.7-had09818_2
  zlib               pkgs/main/linux-64::zlib-1.2.11-h7b6447c_3


Preparing transaction: done
Executing transaction: done
installation finished.
  ------------------------------------------------------------------------
    Done.
    anaconda python installation successful.
  ------------------------------------------------------------------------
  Extracting all conda packages...
  ------------------------------------------------------------------------
..................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
  ------------------------------------------------------------------------
    Done.
    conda packages installation successful.
  ------------------------------------------------------------------------
  Preparing to install all pip packages...
  ------------------------------------------------------------------------
Processing ./deps_bundle/python/python_packages/pip_packages/pycuda-2020.1.tar.gz
Processing ./deps_bundle/python/python_packages/pip_packages/scikit_cuda-0.5.3-py2.py3-none-any.whl
Skipping wheel build for pycuda, due to binaries being disabled for it.
Installing collected packages: scikit-cuda, pycuda
    Running setup.py install for pycuda ... done
Successfully installed pycuda-2020.1 scikit-cuda-0.5.3
  ------------------------------------------------------------------------
    Done.
    pip packages installation successful.
  ------------------------------------------------------------------------
  Main dependency installation completed. Continuing...
  ------------------------------------------------------------------------
Completed.
Currently checking hash for ctffind
Forcing reinstall for dependency ctffind...
  ------------------------------------------------------------------------
  ctffind 4.1.10 installation successful.
  ------------------------------------------------------------------------
Completed.
Currently checking hash for gctf
Forcing reinstall for dependency gctf...
  ------------------------------------------------------------------------
  Gctf v1.06 installation successful.
  ------------------------------------------------------------------------
Completed.
Completed dependency check.

stephan · December 10, 2020, 11:14pm

Hi @alburse,

Weird. Everything looks okay.

Can you do one more thing:
Edit cryosparc2_worker/bin/cryosparcw to add the following lines right before activate_conda_env() (line 51):

conda deactivate || echo "No conda env to deactivate"
unset CONDA_SHLVL
unset CONDA_PROMPT_MODIFIER
unset _CE_CONDA
unset CONDA_EXE
unset CONDA_PREFIX
unset CONDA_PYTHON_EXE
unset CONDA_DEFAULT_ENV
unset PYTHONPATH

Save the file and try running the job again.

alburse · December 10, 2020, 11:23pm

@stephan

Finally, the job started running after including those lines. Thank you so much @stephan. So it is related to PYTHONPATH being set somewhere.
Please let me know if you get a better understanding of it.

Alpay

stephan · December 11, 2020, 12:17am

Hi @alburse,

That’s great to hear. It must be that since the cryosparcw file is a bash script, it’s possible you have conda- related activations in your .bashrc or /etc/profile which are getting executed while this script is being called, corrupting cryoSPARC’s conda environment activation.

olibclarke · December 11, 2020, 1:36pm

@stephan maybe it would be worth having this line in the script by default? I can’t be the only one who has run into this multiple times - I have an existing conda environment and I update cryosparc and then run into a series of obscure errors and wonder what has gone wrong until I figure it out an hour later… maybe worth de-activating existing conda environments automatically for convenience?

alburse · December 11, 2020, 5:27pm

I actually do not have anything on the bashrc or bash_profile. I use a module system and my bash environment never loads something by default. Though, when I installed the master update, there was another PYTHONPATH set for another program I was using. It may be why I got this persistent PYTHONPATH error even when I forced to reinstall the worker. But I am sure any non linux geek may fall into this trap very easily. As @olibclarke said, if these lines do not have any disadvantages for using cryosparc, they can be included in the upcoming updates. I am sure it will fiddle with other program’s python path usage.