Cryosparc cannot be started

Hello, we are running cryosparc v3.3.2 on HPC clusters as the single node mode. The installation was achieved by developing cryosparc into a singularity container. Previously, everything works well. In June, our cluster had one maintenance by updating some applications, such as Nvidia driver, singularity to appatainer, and etc. After the maintenance, cryosparc does not work. The issue is Firefox cannot render the page, so there is no login page. I ran cryosparcm status and cryosparcm env, below is the output:

CryoSPARC process status:

app                              RUNNING   pid 42084, uptime 0:01:37
app_dev                          STOPPED   Not started
command_core                     RUNNING   pid 41986, uptime 0:01:45
command_rtp                      RUNNING   pid 42017, uptime 0:01:42
command_vis                      RUNNING   pid 42011, uptime 0:01:43
database                         RUNNING   pid 41867, uptime 0:01:48
liveapp                          STOPPED   Not started
liveapp_dev                      STOPPED   Not started
webapp                           RUNNING   pid 42057, uptime 0:01:38
webapp_dev                       STOPPED   Not started

----------------------------------------------------------------------------
License is valid
----------------------------------------------------------------------------

global config variables:

export CRYOSPARC_LICENSE_ID=$(cat /cryosparc_license/license_id)
export CRYOSPARC_MASTER_HOSTNAME=$(hostname)
export CRYOSPARC_DB_PATH="/cryosparc_database"
export CRYOSPARC_BASE_PORT=39000
export CRYOSPARC_DEVELOP=false
export CRYOSPARC_INSECURE=true
export CRYOSPARC_CLICK_WRAP=true
CRYOSPARC_FORCE_USER=true

In command_core.log, I also noticed this error:

2023-07-11 14:54:26,562 COMMAND.CORE         run                  WARNING  | Failed to connect link: HTTP Error 502: Bad Gateway
2023-07-11 14:55:29,678 COMMAND.CORE         heartbeat_manager    ERROR    | HTTPSConnectionPool(host='get.cryosparc.com', port=443): Max retr
ies exceeded with url: /heartbeat/XXXXXXXXXX%20 (Caused by SSLError(OSError(107, 'Transport endpoint is not connecte
d')))

I hope you can give me some instructions how to troubleshooting and how to fix the issue.

Thank you,
Best,
Yucheng

Welcome to the forum @zhan4429
Please can you work with your HPC support team to ensure a successful outgoing connection from the container in which your CryoSPARC instance runs to https://get.cryosparc.com.
You may also consider updating your CryoSPARC software to the latest version. There have been many feature enhancements and bug fixes since v3.3.2.

@wtempel
Hi wtempel, thanks for your advice. Actually I am from the HPC support team. Unfortunately, I do not know much about cryosparc. I already tested the outgoing connection from container inside by the below command. The connection seems to be OK:

Singularity> curl https://get.cryosparc.com/checklicenseexists/$LICENSE_ID
{"success": true}

We also have the cryosparc v4 deployed on our cluster. But v4 has the same issue. I am kind of stuck here. So I really appreciate if your Cryosparc support team can give us some advice for troubleshooting.

Thank you,
Best,
Yucheng

Thanks for confirming the connection to https://get.cryosparc.com. In case of the deployed v4, please can you confirm

  • with cryosparcm status that app is running
  • in which context the browser is running (inside container or on the cluster node)
  • that the browser is also showing a blank page

[Added 2023-07-21] @zhan4429 May I suggest that, since you suspect that the issue equally affects v3 and v4 deployments, you collect the following diagnostic information on an affected v4 installation:

  1. the tgz file created with the command
    cryosparcm snaplogs
  2. the url including, if applicable, the port number that a user would inter into the browser address bar to access the CryoSPARC UI
  3. web browser debugging info (how?)

and email us the information. I will send you a direct message with the email address.

Specifically, which Singularity command should I use to run the container? Would you mind sending it to me? It would be really helpful as I’m having issues with that matter. Thanks