Cryosparc v3.3.2 stalling at command_core: started

Install, Configure and Update
1

Hi All,
Cryosparc gave me an error of “Could not connect to command”. I stopped cryosparc, restarted the machine and on trying to restart it stalls at command_core: started.

If I control+C and Checked on cryosparcm status I get →

Current cryoSPARC version: v3.3.2
----------------------------------------------------------------------------

CryoSPARC process status:

app STOPPED Not started
app_dev STOPPED Not started
command_core RUNNING pid 28097, uptime 0:01:36
command_rtp STOPPED Not started
command_vis STOPPED Not started
database RUNNING pid 27988, uptime 0:01:38
liveapp STOPPED Not started
liveapp_dev STOPPED Not started
webapp STOPPED Not started
webapp_dev STOPPED Not started

----------------------------------------------------------------------------
***** client.py: command (http://xxx:39002/api) did not reply within timeout of 300 seconds, attempt 1 of 3**
***** client.py: command (http://xxx:39002/api) did not reply within timeout of 300 seconds, attempt 2 of 3**
***** client.py: command (http://xxx:39002/api) did not reply within timeout of 300 seconds, attempt 3 of 3**
An error ocurred while checking license status
Could not get license verification status. Are all CryoSPARC processes RUNNING?

I also tried with config.sh options Export Cryosparc_Insecure set to either true or false. it did not work.

I used firewall-cmd to add http to services → did not work.

here is part of the cryosparcm log command_core

Traceback (most recent call last):
** File “/home/cryosparc_user/software/cryosparc/cryosparc_master/cryosparc_compute/jobs/runcommon.py”, line 1844, in run_with_except_hook**
** run_old(args, kw)
** File “/home/cryosparc_user/software/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.7/threading.py”, line 870, in run*
** self._target(self._args, self._kwargs)
** File “/home/cryosparc_user/software/cryosparc/cryosparc_master/cryosparc_command/command_core/init.py”, line 218, in background_worker*
** last_audit_date = get_config_var(‘audit’, fail_notset=False, default={})**
** File “/home/cryosparc_user/software/cryosparc/cryosparc_master/cryosparc_command/command_core/init.py”, line 157, in wrapper**
** raise ServerError(s.getvalue(), code=400) from e**
flask_jsonrpc.exceptions.ServerError

Appreciate the help. Thanks

2

@nfrasser any insights with regard to that error.

3

@Bassem Has this cryoSPARC master functioned in the past? If so, have there been any changes to cryoSPARC software, operating system/configuration, or the network?
The firewall may still be preventing access to port 39002, and other cryoSPARC ports.
“Adding” http service with firewall-cmd might not resolve this issue, but might open port 80 instead.

4

Hi @wtempel,

This machine and cryoSPARC installation is new ~ a month old probably. During that month, it has been functional and we only needed to restart cryoSPARC once when the machine was rebooted. What would you suggest?

we are in contact with our IT, it seems that has been an incident with the data center but it preceded that fault by a day or two.

thanks,
bassem

5

A follow: university network system that involves us was not impacted. I appreciate if you have any insights @wtempel as to what to try.

6

Hi All,

I re-installed cryosparc and it started working again.

Of note, after deleting the cryosparc folders. i found that the license was not exported. i believe that might have been the problem. But i also do not know why would it not be exported in the first place, as cryosparc have been working for a month.

Thanks
Bassem

7

I am glad cryoSPARC is working again for you, sorry that it took re-installation.
The relevant record of the license id is assigned to the CRYOSPARC_LICENSE_ID variable and persistently stored in cryosparc_master/config.sh. $LICENSE_ID, which you may have used during installation, was not intended as a persistent licensid id record.

8

Hmmm…is there any info I can provide then to avoid this happening again or know what was the real reason behind the error? let me know.

9

Hi,

I am having exactly the same problem.

Cryosparc was working perfectly on my machine, and now is stalling at the startup:
“command_core: started”

I ma trying to reinstall but it does not solve the problem.

Any help is appreciated,
Thanks,

GIA

10

Reinstallation may also not be required. Please can you post the outputs of these commands

cryosparcm status | grep -v LICENSE
cryosparcm log database | tail -n 30
cryosparcm log command_core | tail -n 30
stat /abs/path/to/cryosparc_master/version # substitute correct path
cat /abs/path/to/cryosparc_master/version # substitute correct path
ps -eo user,pid,ppid,start,cmd | grep -e cryosparc_ -e mongo