Command_core: ERROR (spawn error) for v4.3.0

Install, Configure and Update
1

Hi All, I got this error after updating to v4.3.0. Tried to kill all cryosparc related processes then restarted cryosparc but still had this error. Any suggestions? Thanks.

cryosparcm restart
CryoSPARC is running.
Stopping cryoSPARC
database: stopped
Shut down
Starting cryoSPARC System master process…
CryoSPARC is not already running.
configuring database
configuration complete
database: started
checkdb success
command_core: ERROR (spawn error)

2

Hi @donghuachen ,

Could you please report the output of cryosparcm log command_core?

Thanks!

3 
2023-08-09 09:28:59,496 start                INFO     |  === STARTED === 
2023-08-09 09:28:59,497 background_worker    INFO     |  === STARTED === 
2023-08-09 09:28:59,497 run                  INFO     | === STARTED TASKS WORKER ===
 * Serving Flask app "command_core" (lazy loading)
 * Environment: production
   WARNING: This is a development server. Do not use it in a production deployment.
   Use a production WSGI server instead.
 * Debug mode: off
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/data3/donghua/cryosparc/cryosparc_master/cryosparc_command/command_core/__init__.py", line 213, in start
    app.run(host="0.0.0.0", port=port, threaded=True, passthrough_errors=False)
  File "/data3/donghua/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/site-packages/flask/app.py", line 990, in run
    run_simple(host, port, self, **options)
  File "/data3/donghua/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/site-packages/werkzeug/serving.py", line 1052, in run_simple
    inner()
  File "/data3/donghua/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/site-packages/werkzeug/serving.py", line 996, in inner
    srv = make_server(
  File "/data3/donghua/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/site-packages/werkzeug/serving.py", line 847, in make_server
    return ThreadedWSGIServer(
  File "/data3/donghua/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/site-packages/werkzeug/serving.py", line 740, in __init__
    HTTPServer.__init__(self, server_address, handler)
  File "/data3/donghua/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/socketserver.py", line 452, in __init__
    self.server_bind()
  File "/data3/donghua/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/http/server.py", line 138, in server_bind
    socketserver.TCPServer.server_bind(self)
  File "/data3/donghua/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/socketserver.py", line 466, in server_bind
    self.socket.bind(self.server_address)
OSError: [Errno 98] Address already in use
4

Did you then (re-)confirm the absence of any CryoSPARC-related processes:

ps -eopid,ppid,cmd | grep -e cryosparc -e mongo

?
If your $CRYOSPARC_BASE_PORT or any of the subsequent 9 port numbers are inside the “local” port range
(cat /proc/sys/net/ipv4/ip_local_port_range)
the relevant service port may already be in use by an other process:

sudo ss -anp '( sport = :39002 or dport :39002 )'

would check if your command_core port is in use, assuming CRYOSPARC_BASE_PORT=39000.

1 Like
5

  1. ps -eopid,ppid,cmd | grep -e cryosparc -e mongo
    3125443 3122120 grep --color=auto -e cryosparc -e mongo

  2. cat /proc/sys/net/ipv4/ip_local_port_range
    32768 60999

  3. sudo ss -anp ‘( sport = :39102 or dport :39102 )’
    Netid State Recv-Q Send-Q Local Address:Port Peer Address:Port Process
    tcp LISTEN 129 128 0.0.0.0:39102 0.0.0.0:*
    tcp CLOSE-WAIT 220 0 ???.??.??.??:39102 ???.??.??.??:45716
    tcp CLOSE-WAIT 220 0 ???.??.??.??:39102 ???.??.??.??:45500
    tcp CLOSE-WAIT 220 0 ???.??.??.??:39102 ???.??.??.??:45526
    …

Please note that IP was replaced with ???.??.??.??.

6

Interesting. Now, does
sudo lsof -i :39102
show any PIDs that you could examine with
ps ww <PID>
?

7

When I do
sudo lsof -i :39102

nothing was shown.

8

Hi All, I restarted my workstation then updated cryosparc to v4.3.0 again. Now it is working. Thanks!

1 Like
9