Can not install "Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-req-build-EsUMKo/"

robinbinchen · August 20, 2019, 4:00pm

I try to install cryosparc v2 as standalone on our lab new workstation. the system is Ubuntu 18.04.
however I found this error:

 ------------------------------------------------------------------------
  Preparing to install all pip packages...
  ------------------------------------------------------------------------
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7.
**Processing ./deps_bundle/python/python_packages/pip_packages/Flask-JSONRPC-0.3.1.tar.gz**
**Processing ./deps_bundle/python/python_packages/pip_packages/Flask-PyMongo-0.5.1.tar.gz**
**Processing ./deps_bundle/python/python_packages/pip_packages/libtiff-0.4.2.tar.gz**
**    Complete output from command python setup.py egg_info:**
**    Traceback (most recent call last):**
**      File "<string>", line 1, in <module>**
**      File "/tmp/pip-req-build-EsUMKo/setup.py", line 77, in <module>**
**        from numpy.distutils.core import setup, Extension**
**    ImportError: No module named numpy.distutils.core**
    
    ----------------------------------------
**Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-req-build-EsUMKo/**
  ------------------------------------------------------------------------
    Done.
    pip packages installation successful.
  ------------------------------------------------------------------------
  Main dependency installation completed. Continuing...
  ------------------------------------------------------------------------
  Completed.
  Currently checking hash for mongodb
  Dependencies for mongodb have changed - reinstalling...
  ------------------------------------------------------------------------
  mongodb 3.4.10 installation successful.
  ------------------------------------------------------------------------
  Completed.
  Completed dependency check. 

************ CRYOSPARC MASTER INSTALLATION COMPLETE **************

 CryoSPARC System master has now been installed. 
 Add bin directory to your ~/.bashrc ?
1) Yes
2) No
#? 1

 The following lines were appended to your ~/.bashrc:
  export PATH=/home/hendlab/CryoEM/cryosparc/cryosparc2_master/bin:$PATH

************ CRYOSPARC SYSTEM: STANDALONE INSTALLER **************

  The installer will now download, install and connect the worker
  package on this machine. The worker package will allow cryoSPARC
  to run compute jobs.

******************************************************************

 Starting cryoSPARC...

Starting cryoSPARC System master process..
CryoSPARC is already running.
If you would like to restart, use cryosparcm restart

******************************************************************

 Creating the initial cryoSPARC user...

**Traceback (most recent call last):**
**  File "/home/hendlab/CryoEM/cryosparc/cryosparc2_master/deps/anaconda/lib/python2.7/runpy.py", line 174, in _run_module_as_main**
**    "__main__", fname, loader, pkg_name)**
**  File "/home/hendlab/CryoEM/cryosparc/cryosparc2_master/deps/anaconda/lib/python2.7/runpy.py", line 72, in _run_code**
**    exec code in run_globals**
**  File "/home/hendlab/CryoEM/cryosparc/cryosparc2_master/cryosparc2_compute/client.py", line 12, in <module>**
**    import numpy as n**
**ImportError: No module named numpy**

can anyone help me to solve this?
Thank you!

stephan · August 21, 2019, 2:10pm

Hi @robinbinchen,

Is it possible if you can try reinstalling cryoSPARC?
You can do this by running:
cryosparcm update --override on the master node, then
bin/cryosparcw update --override on each worker node (inside the cryosparc2_worker dir)

More info here:

https://cryosparc.com/docs/reference/install/#forced-update

robinbinchen · August 21, 2019, 2:41pm

Hi @stephan
thank you for your suggestion.I reinstall the cryosparc and run the coomand
cryosparcm update -override and get the error：
line 446: supervisorctl: command not found
and the in the worker node
the bin/cryosparcw update -override get the error:
./cryosparcw: line 35: config.sh: No such file or directory

stephan · August 21, 2019, 2:46pm

Hi @robinbinchen,

At this point if might be better if you reinstall everything from scratch. Feel free to delete the cryosparc2_master and cryosparc2_worker directories and start from the first step.

robinbinchen · August 21, 2019, 3:24pm

Hi @stephan,
I reinstall the cryosparc2 again and it works. This time the master node install complete without error, however when it create the initial cryoSPARC user…

Creating user hendlab with email: sc4352@cumc.columbia.edu password: hendlab123*
*** client.py: command (http://hendlabGPU1:39002/api) did not reply within timeout of 300 seconds, attempt 1 of 3
*** client.py: command (http://hendlabGPU1:39002/api) did not reply within timeout of 300 seconds, attempt 2 of 3
*** client.py: command (http://hendlabGPU1:39002/api) did not reply within timeout of 300 seconds, attempt 3 of 3
Traceback (most recent call last):
  File "/home/hendlab/CryoEM/cryosparc/cryosparc2_master/deps/anaconda/lib/python2.7/runpy.py", line 174, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/home/hendlab/CryoEM/cryosparc/cryosparc2_master/deps/anaconda/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/home/hendlab/CryoEM/cryosparc/cryosparc2_master/cryosparc2_compute/client.py", line 82, in <module>
    cli = CommandClient(host, int(port))
  File "/home/hendlab/CryoEM/cryosparc/cryosparc2_master/cryosparc2_compute/client.py", line 33, in __init__
    self._reload()
  File "/home/hendlab/CryoEM/cryosparc/cryosparc2_master/cryosparc2_compute/client.py", line 61, in _reload
    system = self._get_callable('system.describe')()
  File "/home/hendlab/CryoEM/cryosparc/cryosparc2_master/cryosparc2_compute/client.py", line 49, in func
    r = requests.post(self.url, data = json.dumps(data, cls=NumpyEncoder), headers = header, timeout=self.timeout)
  File "/home/hendlab/CryoEM/cryosparc/cryosparc2_master/deps/anaconda/lib/python2.7/site-packages/requests/api.py", line 116, in post
    return request('post', url, data=data, json=json, **kwargs)
  File "/home/hendlab/CryoEM/cryosparc/cryosparc2_master/deps/anaconda/lib/python2.7/site-packages/requests/api.py", line 60, in request
    return session.request(method=method, url=url, **kwargs)
  File "/home/hendlab/CryoEM/cryosparc/cryosparc2_master/deps/anaconda/lib/python2.7/site-packages/requests/sessions.py", line 533, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/hendlab/CryoEM/cryosparc/cryosparc2_master/deps/anaconda/lib/python2.7/site-packages/requests/sessions.py", line 646, in send
    r = adapter.send(request, **kwargs)
  File "/home/hendlab/CryoEM/cryosparc/cryosparc2_master/deps/anaconda/lib/python2.7/site-packages/requests/adapters.py", line 516, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='hendlabgpu1', port=39002): Max retries exceeded with url: /api (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f94bdee9550>: Failed to establish a new connection: [Errno 111] Connection refused',))

And the status shows that

----------------------------------------------------------------------------
CryoSPARC System master node installed at
/home/hendlab/CryoEM/cryosparc/cryosparc2_master
Current cryoSPARC version: v2.9.0
----------------------------------------------------------------------------

cryosparcm process status:

command_core                     RUNNING   pid 3395, uptime 0:04:24
command_proxy                    RUNNING   pid 3426, uptime 0:04:20
command_vis                      RUNNING   pid 3414, uptime 0:04:22
database                         RUNNING   pid 3319, uptime 0:04:26
watchdog_dev                     STOPPED   Not started
webapp                           RUNNING   pid 3431, uptime 0:04:19
webapp_dev                       STOPPED   Not started

----------------------------------------------------------------------------

global config variables:

export CRYOSPARC_LICENSE_ID="7fc7c914-c061-11e9-b99e-53ff79edfbcf"
export CRYOSPARC_MASTER_HOSTNAME="hendlabGPU1"
export CRYOSPARC_DB_PATH="/home/hendlab/CryoEM/cryosparc/cryosparc2_database"
export CRYOSPARC_BASE_PORT=39000
export CRYOSPARC_DEVELOP=false
export CRYOSPARC_INSECURE=false

robinbinchen · August 22, 2019, 1:51am

@stephan
I reinstall again, and found the error changed. it shows for the worker node the install.sh didn’t exist.


************ CRYOSPARC SYSTEM: STANDALONE INSTALLER **************

  The installer will now download, install and connect the worker
  package on this machine. The worker package will allow cryoSPARC
  to run compute jobs.

******************************************************************

 Starting cryoSPARC...

Starting cryoSPARC System master process..
CryoSPARC is not already running.
database: started
command_core: started
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<title>500 Internal Server Error</title>
<h1>Internal Server Error</h1>
<p>The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application.</p>

command_vis: started
command_proxy: started
webapp: started
-----------------------------------------------------

CryoSPARC master started. 
 From this machine, access cryoSPARC at
    http://localhost:39000
 From other machines on the network, access cryoSPARC at
    http://hendlabGPU1:39000


Startup can take several minutes. Point your browser to the address
and refresh until you see the cryoSPARC web interface.

******************************************************************

 Creating the initial cryoSPARC user...

Creating user hendlab with email: sc4352@cumc.columbia.edu password: hendlab123*
Successfully created user account.

******************************************************************

 Installing the cryoSPARC worker package...

./install.sh: line 350: ./install.sh: No such file or directory

olibclarke · August 22, 2019, 1:44pm

Hi Robin - what was your install command? I got exactly this error when I accidentally mispecified the cryosparc worker path

Oli

robinbinchen · August 27, 2019, 2:14am

Hi @olibclarke and @stephan
I make a stupid mistake and type the wrong worker path. After I corrected it, I meet new issue when initial cryosparc user…

the command I use is

./install.sh  --standalone --license $LICENSE_ID --worker_path /home/hendlab/software/cryosparc2/cryosparc2_worker --cudapath /usr/local/cuda --ssdpath /mnt/SCRATCH/cryosparc_cache --initial_email "sc4352@cumc.columbia.edu" --initial_password hendlab123 --initial_name "hendlab" --insecure --dbpath /mnt/DATA1/cryosparc_database

the error show below:

 Starting cryoSPARC...

Starting cryoSPARC System master process..
CryoSPARC is already running.
If you would like to restart, use cryosparcm restart

******************************************************************

 Creating the initial cryoSPARC user...

Creating user hendlab with email: ww2238@columbia.edu password: hendlab123*
Traceback (most recent call last):
  File "/home/hendlab/software/cryosparc2/cryosparc2_master/deps/anaconda/lib/python2.7/runpy.py", line 174, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/home/hendlab/software/cryosparc2/cryosparc2_master/deps/anaconda/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/home/hendlab/software/cryosparc2/cryosparc2_master/cryosparc2_compute/client.py", line 83, in <module>
    print eval("cli."+command)
  File "<string>", line 1, in <module>
  File "/home/hendlab/software/cryosparc2/cryosparc2_master/cryosparc2_compute/client.py", line 57, in func
    assert False, res['error']
AssertionError: {u'message': u'OtherError: hendlabgpu1:39001: [Errno 111] Connection refused', u'code': 500, u'data': None, u'name': u'OtherError'}

and when I restart the cryosparc and found the error shown as below:

CryoSPARC is running.
Stopping cryosparc.
command_proxy: stopped
command_vis: stopped
webapp: stopped
command_core: stopped
Shut down
Starting cryoSPARC System master process..
CryoSPARC is not already running.
database: started
command_core: started
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<title>500 Internal Server Error</title>
<h1>Internal Server Error</h1>
<p>The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application.</p>

command_vis: started
command_proxy: started
webapp: started
-----------------------------------------------------

CryoSPARC master started. 
 From this machine, access cryoSPARC at
    http://localhost:39000
 From other machines on the network, access cryoSPARC at
    http://hendlabGPU1:39000


Startup can take several minutes. Point your browser to the address
and refresh until you see the cryoSPARC web interface.

when I try to update the worker node, it shows

bin/cryosparcw: line 35: config.sh: No such file or directory

it seems that the worker is totally not installed.

and in the web I can go to the login page, but it can not login, shows user no found.

olibclarke · August 29, 2019, 4:00pm

Hi Robin - have you checked the permissions on the master, worker and database directories? I believe the user installing cryosparc needs to be both owner and group for all three dirs, I have had cryptic errors from permissions problems in the past

Oli

robinbinchen · August 29, 2019, 4:26pm

Hi @olibclarke
I finally found the supervisord process didn’t close during the installation, I close it and found it install correctly.
Thank you so much

olibclarke · August 29, 2019, 5:54pm

ah fantastic, good to hear!!