Cryosparc starting error

Troubleshooting
1

Hi, I have an issue with cryosparcm start command

It went well until yesterday, but it suddenly doesn’t work today.
I didn’t turn off the computer or use the cryosparcm stop command.
I just closed the window tab yesterday and tried to connect to http://localhost:40000 today, but it didn’t work.
So I used the cryosparcm restart command and this error occurred.

Starting cryoSPARC System master process…
CryoSPARC is not already running.
database: ERROR (spawn error)
command_core: ERROR (spawn error)
cryosparc command core startup complete.
command_vis: started
command_proxy: ERROR (spawn error)
webapp: ERROR (spawn error)

CryoSPARC master started.
From this machine, access cryoSPARC at
http://localhost:40000

From other machines on the network, access cryoSPARC at
http://local02:40000

Startup can take several minutes. Point your browser to the address
and refresh until you see the cryoSPARC web interface.

cryosparcm_log_command_core

COMMAND CORE STARTED ===  2020-07-07 11:53:14.408843  ==========================
*** BG WORKER START
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "cryosparc2_command/command_core/__init__.py", line 223, in start
    app.run(host="0.0.0.0", port=port, threaded=True, passthrough_errors=False)
  File "/home/cryosparc_user/cryosparc2-40000/cryosparc2_master/deps/anaconda/lib/python2.7/site-packages/flask/app.py", line 841, in run
    run_simple(host, port, self, **options)
  File "/home/cryosparc_user/cryosparc2-40000/cryosparc2_master/deps/anaconda/lib/python2.7/site-packages/werkzeug/serving.py", line 1010, in run_simple
    inner()
  File "/home/cryosparc_user/cryosparc2-40000/cryosparc2_master/deps/anaconda/lib/python2.7/site-packages/werkzeug/serving.py", line 963, in inner
    fd=fd,
  File "/home/cryosparc_user/cryosparc2-40000/cryosparc2_master/deps/anaconda/lib/python2.7/site-packages/werkzeug/serving.py", line 806, in make_server
    host, port, app, request_handler, passthrough_errors, ssl_context, fd=fd
  File "/home/cryosparc_user/cryosparc2-40000/cryosparc2_master/deps/anaconda/lib/python2.7/site-packages/werkzeug/serving.py", line 699, in __init__
    HTTPServer.__init__(self, server_address, handler)
  File "/home/cryosparc_user/cryosparc2-40000/cryosparc2_master/deps/anaconda/lib/python2.7/SocketServer.py", line 417, in __init__
    self.server_bind()
  File "/home/cryosparc_user/cryosparc2-40000/cryosparc2_master/deps/anaconda/lib/python2.7/BaseHTTPServer.py", line 108, in server_bind
    SocketServer.TCPServer.server_bind(self)
  File "/home/cryosparc_user/cryosparc2-40000/cryosparc2_master/deps/anaconda/lib/python2.7/SocketServer.py", line 431, in server_bind
    self.socket.bind(self.server_address)
  File "/home/cryosparc_user/cryosparc2-40000/cryosparc2_master/deps/anaconda/lib/python2.7/socket.py", line 228, in meth
    return getattr(self._sock,name)(*args)
socket.error: [Errno 98] Address already in use

and this is cryosparc status

[cryosparc_user@local02 ~]$ cryosparcm status
----------------------------------------------------------------------------
CryoSPARC System master node installed at
/home/cryosparc_user/cryosparc2-40000/cryosparc2_master
Current cryoSPARC version: v2.15.0
----------------------------------------------------------------------------

cryosparcm process status:

app                              STOPPED   Not started
app_dev                          STOPPED   Not started
command_core                     FATAL     Exited too quickly (process log may have details)
command_proxy                    FATAL     Exited too quickly (process log may have details)
command_rtp                      STOPPED   Not started
command_vis                      STARTING  
database                         FATAL     Exited too quickly (process log may have details)
watchdog_dev                     STOPPED   Not started
webapp                           FATAL     Exited too quickly (process log may have details)
webapp_dev                       STOPPED   Not started

----------------------------------------------------------------------------

global config variables:

export CRYOSPARC_LICENSE_ID="xxxx"
export CRYOSPARC_MASTER_HOSTNAME="local02"
export CRYOSPARC_DB_PATH="/home/cryosparc_user/cryosparc2-40000/cryosparc2_database"
export CRYOSPARC_BASE_PORT=40000
export CRYOSPARC_DEVELOP=false
export CRYOSPARC_INSECURE=false

Localhost is connected but job errors occur.
Any help would be appreciated.

Thank you :smiley:

1 Like
2

Hi @KSJ, this looks like there are some zombie cryoSPARC services still running that were somehow disconnected from the supervisor process that manages them.

To fix this you may either restart the machine with hostname local02 or kill the processes manually.

Enter these commands in a terminal while logged into that machine:

ps -ax | grep supervisord
ps -ax | grep cryosparc

Look for the process ID of processes related to cryoSPARC at the beginning of each line of output.

Kill them with the kill command:

kill <pid>

More details here:

Then try cryosparcm start when you’re done.

1 Like
3

Hi, @nfrasser

Now it works!

Thank you for your help :innocent:

1 Like
4

Hi,
I am having a similar issue with cryosparc where the app and command_rtp options are not running (screenshot attached, this also has output of ps -ax | grep commands.

Screenshot 2021-08-06 at 11.36.20
Screenshot 2021-08-06 at 11.36.201116×1266 250 KB
). I am not able to run the icli interface also which I need to run a particle re-extract job). I have 2 datasets on this instance for which the processing is nearly finished, is there any way to solve this without disturbing/loosing any of the processed jobs.

Many thanks for your help.
Best,
Kapil