I run cryosparc on a cluster (CUDA 10.2 and CentOS7) but currently am unable to connect to localhost:3900 via firefox. Issues started after updating to 2.15 killing old supervisord processes after fresh install.
I get the error supervisorctl: command not found when I try to do any cryosparcm command. I cannot force update or backup and reinstall from scratch because I get the “supervisorctl: command not found” each time. I have updated yum, python, rebooted my gpu, etc, and pretty much everything I can think of.
Let me know if I should provide specific logs/details or if I could try something and report back.
I think it might be easier for you at this point to reinstall the cryosparc2_master package; it looks like something must’ve become corrupted, and reinstalling is the safest way to fix this (you won’t lose any data).
Hi @stephan,
Thanks for the reply. I am unable to successfully run cryosparcm backup due to supervisorctl command not being found. Is a reinstall still the best and safest option?
That’s right, my apologies. In this case, you can copy the entire database folder to a different location (and even compress it if you have to). Re-installing is still safe.
Is it possible if you can paste the entire output of the install script if you still have it? There may be something going wrong with the installation process that gets overlooked since an error usually doesn’t stop the entire installation.
No problem, here’s the output with a couple comments in it/redacted license ID. Thanks for looking it over:
(base) [user@strbiogpu1 cryosparc]$ ls
cryosparc2_master cryosparc2_master.tar.gz cryosparc2_worker cryosparc2_worker.tar.gz
(base) [user@strbiogpu1 cryosparc]$ rm -Rf cryosparc2_master
(base) [user@strbiogpu1 cryosparc]$ rm cryosparc2_master.tar.gz
(base) [user@strbiogpu1 cryosparc]$ export LICENSE_ID=<I should redact this right>
(base) [user@strbiogpu1 cryosparc]$ curl -L https://get.cryosparc.com/download/master-latest/$LICENSE_ID > cryosparc2_master.tar.gz
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- 0:00:02 --:--:-- 0
100 555M 100 555M 0 0 12.8M 0 0:00:43 0:00:43 --:--:-- 13.1M
(base) [user@strbiogpu1 cryosparc]$ ls
cryosparc2_master.tar.gz cryosparc2_worker cryosparc2_worker.tar.gz
(base) [user@strbiogpu1 cryosparc]$ tar -xf cryosparc2_master.tar.gz
(base) [user@strbiogpu1 cryosparc]$ cd cryosparc2_master/
(base) [user@strbiogpu1 cryosparc2_master]$ ./install.sh --license $LICENSE_ID --hostname strbiogpu1 --dbpath /data1/cryosparc2_database --port 39000
************ CRYOSPARC SYSTEM: MASTER INSTALLER **************
Installation Settings:
License ID : <I should redact this right>
Root Directory : /home/user/software/cryosparc/cryosparc2_master
Master Hostname : strbiogpu1
Database Directory : /data1/cryosparc2_database
Base HTTP Port Number : 39000
Developer Install : false
Operating System : Linux
Insecure Install : false
Standalone Install : false
Version : v2.15.0
******************************************************************
The cryoSPARC System master installation will host a database,
web application, and command layer for cryoSPARC, but will not
perform processing jobs.
Please read and confirm the above settings. The cryoSPARC database
will be stored at the database directory which is relative to the
root directory, unless provided as an absolute path.
You can change the database directory now by passing the --dbapth
parameter to this installer, or later via the command line.
Are the above settings correct?
1) Yes
2) No
#? 1
Continuing.
******************************************************************
Setting up hard-coded config.sh environment variables
******************************************************************
Installing all dependencies.
Checking dependencies...
Dependencies for python have changed - reinstalling...
------------------------------------------------------------------------
Installing anaconda python...
------------------------------------------------------------------------
PREFIX=/home/user/software/cryosparc/cryosparc2_master/deps/anaconda
installing: python-2.7.16-h9bab390_0 ...
Python 2.7.16 :: Anaconda, Inc.
installing: ca-certificates-2019.1.23-0 ...
installing: libgcc-ng-8.2.0-hdf63c60_1 ...
installing: libstdcxx-ng-8.2.0-hdf63c60_1 ...
installing: libffi-3.2.1-hd88cf55_4 ...
installing: ncurses-6.1-he6710b0_1 ...
installing: openssl-1.1.1b-h7b6447c_1 ...
installing: yaml-0.1.7-had09818_2 ...
installing: zlib-1.2.11-h7b6447c_3 ...
installing: libedit-3.1.20181209-hc058e9b_0 ...
installing: readline-7.0-h7b6447c_5 ...
installing: tk-8.6.8-hbc83047_0 ...
installing: sqlite-3.27.2-h7b6447c_0 ...
installing: asn1crypto-0.24.0-py27_0 ...
installing: certifi-2019.3.9-py27_0 ...
installing: chardet-3.0.4-py27_1 ...
installing: enum34-1.1.6-py27_1 ...
installing: futures-3.2.0-py27_0 ...
installing: idna-2.8-py27_0 ...
installing: ipaddress-1.0.22-py27_0 ...
installing: pycosat-0.6.3-py27h14c3975_0 ...
installing: pycparser-2.19-py27_0 ...
installing: pysocks-1.6.8-py27_0 ...
installing: ruamel_yaml-0.15.46-py27h14c3975_0 ...
installing: six-1.12.0-py27_0 ...
installing: cffi-1.12.2-py27h2e261b9_1 ...
installing: setuptools-41.0.0-py27_0 ...
installing: cryptography-2.6.1-py27h1ba5d50_0 ...
installing: wheel-0.33.1-py27_0 ...
installing: pip-19.0.3-py27_0 ...
installing: pyopenssl-19.0.0-py27_0 ...
installing: urllib3-1.24.1-py27_0 ...
installing: requests-2.21.0-py27_0 ...
installing: conda-4.6.14-py27_0 ...
installation finished.
------------------------------------------------------------------------
Done.
anaconda python installation successful.
------------------------------------------------------------------------
Preparing to install all conda packages...
------------------------------------------------------------------------
Downloading and Extracting Packages
##################################################################################################################################################################################### | 100%
all these were 100%
##################################################################################################################################################################################### | 100%
Preparing transaction: done
Verifying transaction: failed
#would this be the issue?
CondaMultiError: EnvironmentNotWritableError: The current user does not have write permissions to the target environment.
environment location: /opt/miniconda3
uid: 1000
gid: 1000
------------------------------------------------------------------------
Done.
conda packages installation successful.
------------------------------------------------------------------------
Preparing to install all pip packages...
------------------------------------------------------------------------
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7.
Processing ./deps_bundle/python/python_packages/pip_packages/Flask-JSONRPC-0.3.1.tar.gz
Processing ./deps_bundle/python/python_packages/pip_packages/Flask-PyMongo-0.5.1.tar.gz
Processing ./deps_bundle/python/python_packages/pip_packages/libtiff-0.4.2.tar.gz
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-req-build-dpzVPa/setup.py", line 77, in <module>
from numpy.distutils.core import setup, Extension
ImportError: No module named numpy.distutils.core
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-req-build-dpzVPa/
------------------------------------------------------------------------
Done.
pip packages installation successful.
------------------------------------------------------------------------
Main dependency installation completed. Continuing...
------------------------------------------------------------------------
Completed.
Currently checking hash for mongodb
Dependencies for mongodb have changed - reinstalling...
------------------------------------------------------------------------
mongodb 3.4.10 installation successful.
------------------------------------------------------------------------
Completed.
Completed dependency check.
************ CRYOSPARC MASTER INSTALLATION COMPLETE **************
CryoSPARC System master has now been installed.
Add bin directory to your ~/.bashrc ?
1) Yes
2) No
#? 2
To run cryosparc, be sure to add "/home/user/software/cryosparc/cryosparc2_master/bin" to your path
or add a symlink to "/home/user/software/cryosparc/cryosparc2_master/bin/cryosparcm" to your path.
#I already had this in my path
Please re-start your terminal shell to make the cryosparcm
command available.
Afterwards, you can start this master instance with:
$ cryosparcm start
To find out about other available commands, use:
$ cryosparcm help
In order to run processing jobs, you will also need to install
one or more cryoSPARC worker nodes and connect them to this
master node.
******************************************************************
(base) [user@strbiogpu1 cryosparc2_master]$ cryosparcm status
----------------------------------------------------------------------------
CryoSPARC System master node installed at
/home/user/software/cryosparc/cryosparc2_master
Current cryoSPARC version: v2.15.0
----------------------------------------------------------------------------
/home/user/software/cryosparc/cryosparc2_master/bin/cryosparcm: line 695: supervisorctl: command not found
You’re right, this part of the installation log seems to identify the issue:
CondaMultiError: EnvironmentNotWritableError: The current user does not have write permissions to the target environment.
environment location: /opt/miniconda3
uid: 1000
gid: 1000
It seems like the root conda environment was active while the installation was happening.
You will have to reinstall, but before you do, you must ensure that no other conda environments are active: