2.4.2 was working happily in Cluster mode (1 master, 6 workers) installed on CentOS 7.
Trying to update to 2.4.5 yesterday encountered exactly the same error - rolled-back to a backup and tried again today with 2.4.6 - here is the output:
[cryosparc@norfolk ~]$ cryosparcm update --check
CryoSPARC current version v2.4.2
update starting on Fri 30 Nov 10:24:02 UTC 2018
current version v2.4.2
new version v2.4.6
Update available!
[cryosparc@norfolk ~]$ cryosparcm update
CryoSPARC current version v2.4.2
update starting on Fri 30 Nov 10:24:06 UTC 2018
No version specified - updating to latest version.
=============================
Updating to version v2.4.6.
=============================
CryoSPARC is running.
Stopping cryosparc.
command_proxy: stopped
command_vis: stopped
webapp: stopped
command_core: stopped
database: stopped
Shut down
Downloading master update...
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- 0:00:01 --:--:-- 0
100 653M 100 653M 0 0 1443k 0 0:07:43 0:07:43 --:--:-- 1450k
Downloading worker update...
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
100 721M 100 721M 0 0 1462k 0 0:08:25 0:08:25 --:--:-- 1464k
Done.
Update will now be applied to the master installation,
followed by worker installations on other node.
Deleting old files...
Extracting...
Done.
Updating dependencies...
Checking dependencies...
Dependencies for python have changed - reinstalling...
------------------------------------------------------------------------
rm: cannot remove ‘/net/zeta/usr/mbu/software/cryosparc2/cryosparc2_master/deps/anaconda/lib’: Directory not empty
Installing anaconda python...
------------------------------------------------------------------------
ERROR: File or directory already exists: '/net/zeta/usr/mbu/software/cryosparc2/cryosparc2_master/deps/anaconda'
If you want to update an existing installation, use the -u option.
------------------------------------------------------------------------
Done.
anaconda python installation successful.
------------------------------------------------------------------------
Preparing to install all conda packages...
------------------------------------------------------------------------
/net/zeta/usr/mbu/software/cryosparc2/cryosparc2_master/deps_bundle_hashes/python/install_python.sh: line 14: conda: command not found
------------------------------------------------------------------------
Done.
conda packages installation successful.
------------------------------------------------------------------------
Preparing to install all pip packages...
------------------------------------------------------------------------
/net/zeta/usr/mbu/software/cryosparc2/cryosparc2_master/deps_bundle_hashes/python/install_python.sh: line 21: pip: command not found
------------------------------------------------------------------------
Done.
pip packages installation successful.
------------------------------------------------------------------------
Main dependency installation completed. Continuing...
------------------------------------------------------------------------
Completed.
Currently checking hash for mongodb
Dependencies for mongodb have not changed.
Completed dependency check.
Successfully updated master from version v2.4.2 to version v2.4.6.
Starting cryoSPARC System master process..
/net/zeta/usr/mbu/software/cryosparc2/cryosparc2_master/bin/cryosparcm: line 369: supervisorctl: command not found
Is there any way to force the dependency installer to run separately?
Here is my attempt:
[cryosparc@norfolk cryosparc2_master]$ cryosparcm deps
ERROR: ld.so: object '/net/zeta/usr/mbu/software/cryosparc2/cryosparc2_master/deps/anaconda/lib/libpython2.7.so.1.0' from LD_PRELOAD cannot be preloaded: ignored.
ERROR: ld.so: object '/net/zeta/usr/mbu/software/cryosparc2/cryosparc2_master/deps/anaconda/lib/libpython2.7.so.1.0' from LD_PRELOAD cannot be preloaded: ignored.
ERROR: ld.so: object '/net/zeta/usr/mbu/software/cryosparc2/cryosparc2_master/deps/anaconda/lib/libpython2.7.so.1.0' from LD_PRELOAD cannot be preloaded: ignored.
ERROR: ld.so: object '/net/zeta/usr/mbu/software/cryosparc2/cryosparc2_master/deps/anaconda/lib/libpython2.7.so.1.0' from LD_PRELOAD cannot be preloaded: ignored.
ERROR: ld.so: object '/net/zeta/usr/mbu/software/cryosparc2/cryosparc2_master/deps/anaconda/lib/libpython2.7.so.1.0' from LD_PRELOAD cannot be preloaded: ignored.
ERROR: ld.so: object '/net/zeta/usr/mbu/software/cryosparc2/cryosparc2_master/deps/anaconda/lib/libpython2.7.so.1.0' from LD_PRELOAD cannot be preloaded: ignored.
ERROR: ld.so: object '/net/zeta/usr/mbu/software/cryosparc2/cryosparc2_master/deps/anaconda/lib/libpython2.7.so.1.0' from LD_PRELOAD cannot be preloaded: ignored.
ERROR: ld.so: object '/net/zeta/usr/mbu/software/cryosparc2/cryosparc2_master/deps/anaconda/lib/libpython2.7.so.1.0' from LD_PRELOAD cannot be preloaded: ignored.
Checking dependencies...
Dependencies for python have not changed.
Currently checking hash for mongodb
Dependencies for mongodb have not changed.
Completed dependency check.
Okay, I moved the deps_hashes and re-ran cryosparcm deps and it seems to have made a difference - but now the service appears to start but hasn’t actually started the 39000 port
[cryosparc@norfolk ~]$ cryosparcm start
Starting cryoSPARC System master process..
CryoSPARC is not already running.
database: started
command_core: started
cryosparc command core startup complete.
command_vis: started
command_proxy: started
webapp: started
-----------------------------------------------------
CryoSPARC master started.
From this machine, access the webapp at
http://localhost:39000
From other machines on the network, access at
http://norfolk.maas:39000
Startup can take several minutes. Point your browser to the address
and refresh until you see the cryoSPARC web interface.
[cryosparc@norfolk ~]$ ss -tln
State Recv-Q Send-Q Local Address:Port Peer Address:Port
LISTEN 0 128 *:39003 *:*
LISTEN 0 50 *:39004 *:*
LISTEN 0 64 *:33440 *:*
LISTEN 0 128 *:6818 *:*
LISTEN 0 5 *:8649 *:*
LISTEN 0 64 *:37870 *:*
LISTEN 0 128 *:111 *:*
LISTEN 0 128 *:50641 *:*
LISTEN 0 128 *:22 *:*
LISTEN 0 128 127.0.0.1:631 *:*
LISTEN 0 128 *:39001 *:*
LISTEN 0 128 *:39002 *:*
LISTEN 0 128 127.0.0.1:6010 *:*
LISTEN 0 64 :::6556 :::*
LISTEN 0 64 :::38595 :::*
LISTEN 0 128 :::58187 :::*
LISTEN 0 128 :::111 :::*
LISTEN 0 128 :::22 :::*
LISTEN 0 128 ::1:631 :::*
LISTEN 0 64 :::46841 :::*
LISTEN 0 128 ::1:6010 :::*
[cryosparc@norfolk ~]$
Re-installing the dependencies:
[cryosparc@norfolk cryosparc2_master]$ mv deps_hashes deps_hashes.old
[cryosparc@norfolk cryosparc2_master]$ cryosparcm deps
Checking dependencies...
Dependencies for python have changed - reinstalling...
------------------------------------------------------------------------
Installing anaconda python...
------------------------------------------------------------------------
PREFIX=/net/zeta/usr/mbu/software/cryosparc2/cryosparc2_master/deps/anaconda
installing: python-2.7.14-h1571d57_29 ...
Python 2.7.14 :: Anaconda, Inc.
installing: ca-certificates-2017.08.26-h1d4fec5_0 ...
installing: conda-env-2.6.0-h36134e3_1 ...
installing: libgcc-ng-7.2.0-h7cc24e2_2 ...
installing: libstdcxx-ng-7.2.0-h7a57d05_2 ...
installing: libffi-3.2.1-hd88cf55_4 ...
installing: ncurses-6.0-h9df7e31_2 ...
installing: openssl-1.0.2n-hb7f436b_0 ...
installing: tk-8.6.7-hc745277_3 ...
installing: yaml-0.1.7-had09818_2 ...
installing: zlib-1.2.11-ha838bed_2 ...
installing: libedit-3.1-heed3624_0 ...
installing: readline-7.0-ha6073c6_4 ...
installing: sqlite-3.20.1-hb898158_2 ...
installing: asn1crypto-0.23.0-py27h7b0c810_0 ...
installing: certifi-2017.11.5-py27h71e7faf_0 ...
installing: chardet-3.0.4-py27hfa10054_1 ...
installing: enum34-1.1.6-py27h99a27e9_1 ...
installing: idna-2.6-py27h5722d68_1 ...
installing: ipaddress-1.0.18-py27h337fd85_0 ...
installing: pycosat-0.6.3-py27ha4109ae_0 ...
installing: pycparser-2.18-py27hefa08c5_1 ...
installing: pysocks-1.6.7-py27he2db6d2_1 ...
installing: ruamel_yaml-0.11.14-py27h672d447_2 ...
installing: six-1.11.0-py27h5f960f1_1 ...
installing: cffi-1.11.2-py27ha7929c6_0 ...
installing: setuptools-36.5.0-py27h68b189e_0 ...
installing: cryptography-2.1.4-py27h6697b16_0 ...
installing: wheel-0.30.0-py27h2bc6bb2_1 ...
installing: pip-9.0.1-py27ha730c48_4 ...
installing: pyopenssl-17.5.0-py27hcee3be0_0 ...
installing: urllib3-1.22-py27ha55213b_0 ...
installing: requests-2.18.4-py27hc5b0589_1 ...
installing: conda-4.3.31-py27_0 ...
installation finished.
------------------------------------------------------------------------
Done.
anaconda python installation successful.
------------------------------------------------------------------------
Preparing to install all conda packages...
------------------------------------------------------------------------
------------------------------------------------------------------------
Done.
conda packages installation successful.
------------------------------------------------------------------------
Preparing to install all pip packages...
------------------------------------------------------------------------
Processing ./deps_bundle/python/python_packages/pip_packages/Flask-JSONRPC-0.3.1.tar.gz
Processing ./deps_bundle/python/python_packages/pip_packages/Flask-PyMongo-0.5.1.tar.gz
Processing ./deps_bundle/python/python_packages/pip_packages/libtiff-0.4.2.tar.gz
Requirement already satisfied: Flask>=0.10 in ./deps/anaconda/lib/python2.7/site-packages (from Flask-JSONRPC==0.3.1)
Requirement already satisfied: PyMongo>=2.5 in ./deps/anaconda/lib/python2.7/site-packages (from Flask-PyMongo==0.5.1)
Requirement already satisfied: Werkzeug>=0.7 in ./deps/anaconda/lib/python2.7/site-packages (from Flask>=0.10->Flask-JSONRPC==0.3.1)
Requirement already satisfied: Jinja2>=2.4 in ./deps/anaconda/lib/python2.7/site-packages (from Flask>=0.10->Flask-JSONRPC==0.3.1)
Requirement already satisfied: itsdangerous>=0.21 in ./deps/anaconda/lib/python2.7/site-packages (from Flask>=0.10->Flask-JSONRPC==0.3.1)
Requirement already satisfied: click>=2.0 in ./deps/anaconda/lib/python2.7/site-packages (from Flask>=0.10->Flask-JSONRPC==0.3.1)
Requirement already satisfied: MarkupSafe in ./deps/anaconda/lib/python2.7/site-packages (from Jinja2>=2.4->Flask>=0.10->Flask-JSONRPC==0.3.1)
Skipping bdist_wheel for Flask-JSONRPC, due to binaries being disabled for it.
Skipping bdist_wheel for Flask-PyMongo, due to binaries being disabled for it.
Skipping bdist_wheel for libtiff, due to binaries being disabled for it.
Installing collected packages: Flask-JSONRPC, Flask-PyMongo, libtiff
Running setup.py install for Flask-JSONRPC ... done
Running setup.py install for Flask-PyMongo ... done
Running setup.py install for libtiff ... done
Successfully installed Flask-JSONRPC-0.3.1 Flask-PyMongo-0.5.1 libtiff-0.4.2
You are using pip version 9.0.1, however version 18.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
------------------------------------------------------------------------
Done.
pip packages installation successful.
------------------------------------------------------------------------
Main dependency installation completed. Continuing...
------------------------------------------------------------------------
Completed.
Currently checking hash for mongodb
Dependencies for mongodb have changed - reinstalling...
------------------------------------------------------------------------
mongodb 3.4.10 installation successful.
------------------------------------------------------------------------
Completed.
Completed dependency check.
“cryosparcm log webapp” shows nothing logged since 28th, which is when it was last successfully used.
For this part of problem; re-installing dependencies after removing them manually seems to have solved something.
I’ve made a separate post regarding the non-starting of port 39000