Update spawn error

I updated the cryosparc from 3.3.2 to 4.1.1 and get spawn errors. I made sure that no cryosparc jobs were orphaned and no mongod or supervisor files were in /tmp.

[cryosparc_user@xingcryoem cryosparc_user]$ ls /tmp
cryosparc-supervisor-4c4a4bf2c532bf2738317c6734588d17.sock
firefox
hsperfdata_tomcat
mongodb-39001.sock
pymp-2lVaqH
pymp-DfUqc1
pymp-fbeG20
pymp-yukEwF
rclone-cron.log
systemd-private-187902e8aef241af97b368b801496c00-bolt.service-H0eVvW
systemd-private-187902e8aef241af97b368b801496c00-chronyd.service-Tm9VYo
systemd-private-187902e8aef241af97b368b801496c00-colord.service-3ZUsm9
systemd-private-187902e8aef241af97b368b801496c00-cups.service-3zUVnV
systemd-private-187902e8aef241af97b368b801496c00-rtkit-daemon.service-Flb8E0
yxing2
[cryosparc_user@xingcryoem cryosparc_user]$ rm /tmp/cryosparc-supervisor-4c4a4bf2c532bf2738317c6734588d17.sock 
[cryosparc_user@xingcryoem cryosparc_user]$ rm /tmp/mongodb-39001.sock 
[cryosparc_user@xingcryoem cryosparc_user]$ cryosparcm stop
CryoSPARC is not already running.
If you would like to restart, use cryosparcm restart
[cryosparc_user@xingcryoem cryosparc_user]$ cryosparcm start
Starting cryoSPARC System master process..
CryoSPARC is not already running.
configuring database
    configuration complete
database: started
checkdb success
command_core: ERROR (spawn error)
[cryosparc_user@xingcryoem cryosparc_user]$ cryosparcm status
----------------------------------------------------------------------------
CryoSPARC System master node installed at
/mnt/ssd/cryosparc_user/software/cryosparc/cryosparc2_master
Current cryoSPARC version: v4.1.1
----------------------------------------------------------------------------

CryoSPARC process status:

app                              STOPPED   Not started
app_api                          STOPPED   Not started
app_api_dev                      STOPPED   Not started
app_legacy                       STOPPED   Not started
app_legacy_dev                   STOPPED   Not started
command_core                     BACKOFF   Exited too quickly (process log may have details)
command_rtp                      STOPPED   Not started
command_vis                      STOPPED   Not started
database                         RUNNING   pid 107267, uptime 0:00:14

----------------------------------------------------------------------------
An error ocurred while checking license status
Could not get license verification status. Are all CryoSPARC processes RUNNING?
[cryosparc_user@xingcryoem cryosparc_user]$ curl https://get.cryosparc.com/checklicenseexists/$LICENSE_ID
{"success": true}[cryosparc_user@xingcryoem cryosparc_user]$ 

The database comes up but no other jobs will spawn as seen in the status. Command core exited too quickly.

The log file is too long but here is the last few lines.

2023-01-12 15:57:57,423 COMMAND.BG_WORKER    background_worker    INFO     |  === STARTED === 
2023-01-12 15:57:57,423 COMMAND.CORE         run                  INFO     | === STARTED TASKS WORKER ===
 * Serving Flask app "command_core" (lazy loading)
 * Environment: production
   WARNING: This is a development server. Do not use it in a production deployment.
   Use a production WSGI server instead.
 * Debug mode: off
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/mnt/ssd/cryosparc_user/software/cryosparc/cryosparc2_master/cryosparc_command/command_core/__init__.py", line 211, in sta
rt
    app.run(host="0.0.0.0", port=port, threaded=True, passthrough_errors=False)
  File "/mnt/ssd/cryosparc_user/software/cryosparc/cryosparc2_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/site-pa
ckages/flask/app.py", line 990, in run
    run_simple(host, port, self, **options)
  File "/mnt/ssd/cryosparc_user/software/cryosparc/cryosparc2_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/site-pa
ckages/werkzeug/serving.py", line 1052, in run_simple
    inner()
  File "/mnt/ssd/cryosparc_user/software/cryosparc/cryosparc2_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/site-pa
ckages/werkzeug/serving.py", line 996, in inner
    srv = make_server(
  File "/mnt/ssd/cryosparc_user/software/cryosparc/cryosparc2_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/site-pa
ckages/werkzeug/serving.py", line 847, in make_server
    return ThreadedWSGIServer(
  File "/mnt/ssd/cryosparc_user/software/cryosparc/cryosparc2_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/site-pa
ckages/werkzeug/serving.py", line 740, in __init__
    HTTPServer.__init__(self, server_address, handler)
  File "/mnt/ssd/cryosparc_user/software/cryosparc/cryosparc2_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/sockets
erver.py", line 452, in __init__
    self.server_bind()
  File "/mnt/ssd/cryosparc_user/software/cryosparc/cryosparc2_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/http/se
rver.py", line 138, in server_bind
    socketserver.TCPServer.server_bind(self)
  File "/mnt/ssd/cryosparc_user/software/cryosparc/cryosparc2_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/sockets
erver.py", line 466, in server_bind
    self.socket.bind(self.server_address)
OSError: [Errno 98] Address already in use
2023-01-12 16:13:04,087 COMMAND.MAIN         start                INFO     |  === STARTED === 
2023-01-12 16:13:04,087 COMMAND.BG_WORKER    background_worker    INFO     |  === STARTED === 
2023-01-12 16:13:04,088 COMMAND.CORE         run                  INFO     | === STARTED TASKS WORKER ===
 * Serving Flask app "command_core" (lazy loading)
 * Environment: production
   WARNING: This is a development server. Do not use it in a production deployment.
   Use a production WSGI server instead.
 * Debug mode: off
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/mnt/ssd/cryosparc_user/software/cryosparc/cryosparc2_master/cryosparc_command/command_core/__init__.py", line 211, in sta
rt
    app.run(host="0.0.0.0", port=port, threaded=True, passthrough_errors=False)
  File "/mnt/ssd/cryosparc_user/software/cryosparc/cryosparc2_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/site-pa
ckages/flask/app.py", line 990, in run
    run_simple(host, port, self, **options)
  File "/mnt/ssd/cryosparc_user/software/cryosparc/cryosparc2_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/site-pa
ckages/werkzeug/serving.py", line 1052, in run_simple
    inner()
  File "/mnt/ssd/cryosparc_user/software/cryosparc/cryosparc2_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/site-pa
ckages/werkzeug/serving.py", line 996, in inner
    srv = make_server(
  File "/mnt/ssd/cryosparc_user/software/cryosparc/cryosparc2_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/site-pa
ckages/werkzeug/serving.py", line 847, in make_server
    return ThreadedWSGIServer(
  File "/mnt/ssd/cryosparc_user/software/cryosparc/cryosparc2_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/site-pa
ckages/werkzeug/serving.py", line 740, in __init__
    HTTPServer.__init__(self, server_address, handler)
  File "/mnt/ssd/cryosparc_user/software/cryosparc/cryosparc2_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/sockets
erver.py", line 452, in __init__
    self.server_bind()
  File "/mnt/ssd/cryosparc_user/software/cryosparc/cryosparc2_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/http/se
rver.py", line 138, in server_bind
    socketserver.TCPServer.server_bind(self)
  File "/mnt/ssd/cryosparc_user/software/cryosparc/cryosparc2_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/sockets
erver.py", line 466, in server_bind
    self.socket.bind(self.server_address)
OSError: [Errno 98] Address already in use
2023-01-12 16:13:09,342 COMMAND.MAIN         start                INFO     |  === STARTED === 
2023-01-12 16:13:09,343 COMMAND.BG_WORKER    background_worker    INFO     |  === STARTED === 
2023-01-12 16:13:09,344 COMMAND.CORE         run                  INFO     | === STARTED TASKS WORKER ===
 * Serving Flask app "command_core" (lazy loading)
 * Environment: production
   WARNING: This is a development server. Do not use it in a production deployment.
   Use a production WSGI server instead.
 * Debug mode: off
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/mnt/ssd/cryosparc_user/software/cryosparc/cryosparc2_master/cryosparc_command/command_core/__init__.py", line 211, in sta
rt
    app.run(host="0.0.0.0", port=port, threaded=True, passthrough_errors=False)
  File "/mnt/ssd/cryosparc_user/software/cryosparc/cryosparc2_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/site-pa
ckages/flask/app.py", line 990, in run
    run_simple(host, port, self, **options)
  File "/mnt/ssd/cryosparc_user/software/cryosparc/cryosparc2_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/site-pa
ckages/werkzeug/serving.py", line 1052, in run_simple
    inner()
  File "/mnt/ssd/cryosparc_user/software/cryosparc/cryosparc2_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/site-pa
ckages/werkzeug/serving.py", line 996, in inner
    srv = make_server(
  File "/mnt/ssd/cryosparc_user/software/cryosparc/cryosparc2_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/site-pa
ckages/werkzeug/serving.py", line 847, in make_server
    return ThreadedWSGIServer(
  File "/mnt/ssd/cryosparc_user/software/cryosparc/cryosparc2_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/site-pa
ckages/werkzeug/serving.py", line 740, in __init__
    HTTPServer.__init__(self, server_address, handler)
  File "/mnt/ssd/cryosparc_user/software/cryosparc/cryosparc2_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/sockets
erver.py", line 452, in __init__
    self.server_bind()
  File "/mnt/ssd/cryosparc_user/software/cryosparc/cryosparc2_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/http/se
rver.py", line 138, in server_bind
    socketserver.TCPServer.server_bind(self)
  File "/mnt/ssd/cryosparc_user/software/cryosparc/cryosparc2_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/sockets
erver.py", line 466, in server_bind
    self.socket.bind(self.server_address)
OSError: [Errno 98] Address already in use
2023-01-12 16:13:15,576 COMMAND.MAIN         start                INFO     |  === STARTED === 
2023-01-12 16:13:15,577 COMMAND.BG_WORKER    background_worker    INFO     |  === STARTED === 
2023-01-12 16:13:15,577 COMMAND.CORE         run                  INFO     | === STARTED TASKS WORKER ===
 * Serving Flask app "command_core" (lazy loading)
 * Environment: production
   WARNING: This is a development server. Do not use it in a production deployment.
   Use a production WSGI server instead.
 * Debug mode: off
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/mnt/ssd/cryosparc_user/software/cryosparc/cryosparc2_master/cryosparc_command/command_core/__init__.py", line 211, in sta
rt
    app.run(host="0.0.0.0", port=port, threaded=True, passthrough_errors=False)
  File "/mnt/ssd/cryosparc_user/software/cryosparc/cryosparc2_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/site-pa
ckages/flask/app.py", line 990, in run
    run_simple(host, port, self, **options)
  File "/mnt/ssd/cryosparc_user/software/cryosparc/cryosparc2_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/site-pa
ckages/werkzeug/serving.py", line 1052, in run_simple
    inner()
  File "/mnt/ssd/cryosparc_user/software/cryosparc/cryosparc2_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/site-pa
ckages/werkzeug/serving.py", line 996, in inner
    srv = make_server(
  File "/mnt/ssd/cryosparc_user/software/cryosparc/cryosparc2_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/site-pa
ckages/werkzeug/serving.py", line 847, in make_server
    return ThreadedWSGIServer(
  File "/mnt/ssd/cryosparc_user/software/cryosparc/cryosparc2_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/site-pa
ckages/werkzeug/serving.py", line 740, in __init__
    HTTPServer.__init__(self, server_address, handler)
  File "/mnt/ssd/cryosparc_user/software/cryosparc/cryosparc2_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/sockets
erver.py", line 452, in __init__
    self.server_bind()
  File "/mnt/ssd/cryosparc_user/software/cryosparc/cryosparc2_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/http/se
rver.py", line 138, in server_bind
    socketserver.TCPServer.server_bind(self)
  File "/mnt/ssd/cryosparc_user/software/cryosparc/cryosparc2_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/sockets
erver.py", line 466, in server_bind
    self.socket.bind(self.server_address)
OSError: [Errno 98] Address already in use
2023-01-12 16:22:19,484 COMMAND.MAIN         start                INFO     |  === STARTED === 
2023-01-12 16:22:19,485 COMMAND.BG_WORKER    background_worker    INFO     |  === STARTED === 
2023-01-12 16:22:19,486 COMMAND.CORE         run                  INFO     | === STARTED TASKS WORKER ===
 * Serving Flask app "command_core" (lazy loading)
 * Environment: production
   WARNING: This is a development server. Do not use it in a production deployment.
   Use a production WSGI server instead.
 * Debug mode: off
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/mnt/ssd/cryosparc_user/software/cryosparc/cryosparc2_master/cryosparc_command/command_core/__init__.py", line 211, in sta
rt
    app.run(host="0.0.0.0", port=port, threaded=True, passthrough_errors=False)
  File "/mnt/ssd/cryosparc_user/software/cryosparc/cryosparc2_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/site-pa
ckages/flask/app.py", line 990, in run
    run_simple(host, port, self, **options)
  File "/mnt/ssd/cryosparc_user/software/cryosparc/cryosparc2_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/site-pa
ckages/werkzeug/serving.py", line 1052, in run_simple
    inner()
  File "/mnt/ssd/cryosparc_user/software/cryosparc/cryosparc2_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/site-pa
ckages/werkzeug/serving.py", line 996, in inner
    srv = make_server(
  File "/mnt/ssd/cryosparc_user/software/cryosparc/cryosparc2_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/site-pa
ckages/werkzeug/serving.py", line 847, in make_server
    return ThreadedWSGIServer(
  File "/mnt/ssd/cryosparc_user/software/cryosparc/cryosparc2_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/site-pa
ckages/werkzeug/serving.py", line 740, in __init__
    HTTPServer.__init__(self, server_address, handler)
  File "/mnt/ssd/cryosparc_user/software/cryosparc/cryosparc2_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/sockets
erver.py", line 452, in __init__
    self.server_bind()
  File "/mnt/ssd/cryosparc_user/software/cryosparc/cryosparc2_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/http/se
rver.py", line 138, in server_bind
    socketserver.TCPServer.server_bind(self)
  File "/mnt/ssd/cryosparc_user/software/cryosparc/cryosparc2_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/sockets
erver.py", line 466, in server_bind
    self.socket.bind(self.server_address)
OSError: [Errno 98] Address already in use
2023-01-12 16:22:24,671 COMMAND.MAIN         start                INFO     |  === STARTED === 
2023-01-12 16:22:24,671 COMMAND.BG_WORKER    background_worker    INFO     |  === STARTED === 
2023-01-12 16:22:24,672 COMMAND.CORE         run                  INFO     | === STARTED TASKS WORKER ===
 * Serving Flask app "command_core" (lazy loading)
 * Environment: production
   WARNING: This is a development server. Do not use it in a production deployment.
   Use a production WSGI server instead.
 * Debug mode: off
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/mnt/ssd/cryosparc_user/software/cryosparc/cryosparc2_master/cryosparc_command/command_core/__init__.py", line 211, in sta
rt
    app.run(host="0.0.0.0", port=port, threaded=True, passthrough_errors=False)
  File "/mnt/ssd/cryosparc_user/software/cryosparc/cryosparc2_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/site-pa
ckages/flask/app.py", line 990, in run
    run_simple(host, port, self, **options)
  File "/mnt/ssd/cryosparc_user/software/cryosparc/cryosparc2_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/site-pa
ckages/werkzeug/serving.py", line 1052, in run_simple
    inner()
  File "/mnt/ssd/cryosparc_user/software/cryosparc/cryosparc2_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/site-pa
ckages/werkzeug/serving.py", line 996, in inner
    srv = make_server(
  File "/mnt/ssd/cryosparc_user/software/cryosparc/cryosparc2_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/site-pa
ckages/werkzeug/serving.py", line 847, in make_server
    return ThreadedWSGIServer(
  File "/mnt/ssd/cryosparc_user/software/cryosparc/cryosparc2_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/site-pa
ckages/werkzeug/serving.py", line 740, in __init__
    HTTPServer.__init__(self, server_address, handler)
  File "/mnt/ssd/cryosparc_user/software/cryosparc/cryosparc2_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/sockets
erver.py", line 452, in __init__
    self.server_bind()
  File "/mnt/ssd/cryosparc_user/software/cryosparc/cryosparc2_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/http/se
rver.py", line 138, in server_bind
    socketserver.TCPServer.server_bind(self)
  File "/mnt/ssd/cryosparc_user/software/cryosparc/cryosparc2_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/sockets
erver.py", line 466, in server_bind
    self.socket.bind(self.server_address)
OSError: [Errno 98] Address already in use
2023-01-12 16:22:30,939 COMMAND.MAIN         start                INFO     |  === STARTED === 
2023-01-12 16:22:30,940 COMMAND.BG_WORKER    background_worker    INFO     |  === STARTED === 
2023-01-12 16:22:30,940 COMMAND.CORE         run                  INFO     | === STARTED TASKS WORKER ===
 * Serving Flask app "command_core" (lazy loading)
 * Environment: production
   WARNING: This is a development server. Do not use it in a production deployment.
   Use a production WSGI server instead.
 * Debug mode: off
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/mnt/ssd/cryosparc_user/software/cryosparc/cryosparc2_master/cryosparc_command/command_core/__init__.py", line 211, in sta
rt
    app.run(host="0.0.0.0", port=port, threaded=True, passthrough_errors=False)
  File "/mnt/ssd/cryosparc_user/software/cryosparc/cryosparc2_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/site-pa
ckages/flask/app.py", line 990, in run
    run_simple(host, port, self, **options)
  File "/mnt/ssd/cryosparc_user/software/cryosparc/cryosparc2_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/site-pa
ckages/werkzeug/serving.py", line 1052, in run_simple
    inner()
  File "/mnt/ssd/cryosparc_user/software/cryosparc/cryosparc2_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/site-pa
ckages/werkzeug/serving.py", line 996, in inner
    srv = make_server(
  File "/mnt/ssd/cryosparc_user/software/cryosparc/cryosparc2_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/site-pa
ckages/werkzeug/serving.py", line 847, in make_server
    return ThreadedWSGIServer(
  File "/mnt/ssd/cryosparc_user/software/cryosparc/cryosparc2_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/site-pa
ckages/werkzeug/serving.py", line 740, in __init__
    HTTPServer.__init__(self, server_address, handler)
  File "/mnt/ssd/cryosparc_user/software/cryosparc/cryosparc2_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/sockets
erver.py", line 452, in __init__
    self.server_bind()
  File "/mnt/ssd/cryosparc_user/software/cryosparc/cryosparc2_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/http/se
rver.py", line 138, in server_bind
    socketserver.TCPServer.server_bind(self)
  File "/mnt/ssd/cryosparc_user/software/cryosparc/cryosparc2_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/sockets
erver.py", line 466, in server_bind
    self.socket.bind(self.server_address)
OSError: [Errno 98] Address already in use
[cryosparc_user@xingcryoem cryosparc_user]$ mv t.t command-core.txt
[cryosparc_user@xingcryoem cryosparc_user]$

You may want to try:

  1. cryosparcm stop
  2. Before deleting any “sock” files from /tmp, please check for orphaned CryoSPARC-related processes:
    ps xww | grep -e cryosparc -e mongo
  3. kill, but not kill -9, CryoSPARC-related processes that may still be running.
  4. Confirm there are no CryoSPARC-related processes left running:
    ps xww | grep -e cryosparc -e mongo
  5. delete cryosparc-supervisor-4c4a4bf2c532bf2738317c6734588d17.sock, mongodb-39001.sock from /tmp if they are still there.
  6. cryosparcm start
1 Like

thanks, this works. no kill -9.
kas

Hi satyshur,

Very soon, we are going to upgrade our cryosparc from v3.3.1 to v4.1.1. Other than the database stuff that you mentioned here, is there anything that we need to be concerned about or is the upgrade relatively smooth?