Can you set this variable to localhost and comment out (add a # in front of the line) export CRYOSPARC_HOSTNAME_CHECK=“dcc-cryosparc-05.rc.duke.edu”. Once that’s done, restart CryoSPARC (cryosparcm restart) and try again
I can, but we had to set that value because this server has 2 NICs. One network interface is where the CryoSPARC web server is connected to and the other NIC is plumbed into the subnet our Slurm cluster and where the GPU compute nodes that CryoSPARC uses for jobs is located. The CryoSPARC worker was unable to contact the master until we added that variable setting.
@jmnewton based on the logs, it seems two services (app at port 39000 and app_api at port 39006) are having trouble talking to each other.
Note that port 39006 used to host the CryoSPARC Live web application which has now been integrated into the main CryoSPARC application at 39000. The 39006 service now provides real-time subscriptions for the main web application’s server process and no longer needs to be accessible from a browser.
Do you have any authentication enabled in your Apache or proxy config for dcc-cryosparc-05.rc.duke.edu:39006 to allow browser access? Or perhaps forcing HTTPS on it? If so removing that config and restarting CryoSPARC should fix the issue.
If you’ve checked this and CryoSPARC still doesn’t load, could you send me the output of the following command from the master host?
curl "http://dcc-cryosparc-05.rc.duke.edu:39006"
Edit: Fixed curl URL
OK, here’s the output:
curl "http://dcc-cryosparc-05.rc.duke.edu:39006"
cryoSPARC Application APIOops sorry, gave you the wrong URL, it’s
curl "http://dcc-cryosparc-05.rc.duke.edu:39006"
By the way setting CRYOSPARC_MASTER_HOSTNAME=“localhost” and restarting throws an error:
ERROR: Re-run this command on the master node: localhost.
Alternatively, set CRYOSPARC_FORCE_HOSTNAME=true in cryosparc_master/config.sh to suppress this error.
If this error message is incorrect, set CRYOSPARC_HOSTNAME_CHECK to the correct hostname in cryosparc_master/config.sh.
Re: The localhost hostname error, try adding that line to you cryosparc_master/config.sh
export CRYOSPARC_FORCE_HOSTNAME=true
And restart.
Once you have that, try logging into the web interface again, wait for the web page loading to stall. Then can you send me the output of the following commands?
cryosparcm log app | tail -n 1000
cryosparcm log app_api | tail -n 1000
cryosparcm log command_core | tail -n 1000
Hi @jmnewton ,
Could you refresh the page while the ‘Network’ browser dev tools tab is open, select the /websocket line and paste a screenshot? Please make sure to wait a bit (shouldn’t report ‘pending…’). Example:
The first websocket capture uses the original cryosparc_master/config.sh and the second with the ‘export CRYOSPARC_MASTER_HOSTNAME=“localhost”’ change.
Hi @jmnewton ,
Thanks for these. The screenshots show the request is still in progress - we’d like to see the result of the request after it has returned. Please give it a minute or so.
The first screenshot indicates quite a few console errors - could you also send a screenshot of that over?
- Suhail
Hi @jmnewton ,
This is quite odd indeed. We haven’t been able to reproduce this issue. Could you either downgrade to v3.4 or stay on v4.0 and use the legacy application? You can start the process by running cryosparcm start app_legacy .
Is there any other information you can provide about your setup? Perhaps you can email us another set of recent logs.
- Suhail
We’re going to try to get our Security Office to allow http traffic if the researchers use a VPN context. We’ll then open the host firewall to that IP range and port 39000.
EDIT: fixed the NGINX issue, just needed to add some headers:
1 server {
2 listen 80;
3 server_name cryosparc.{hostname};
4
5 location / {
6 proxy_pass http://localhost:39000;
7 }
8
9 location /websocket {
10 proxy_pass http://localhost:39000;
11 proxy_http_version 1.1;
12 proxy_set_header Upgrade $http_upgrade;
13 proxy_set_header Connection "Upgrade";
14 proxy_set_header Host $host;
15 }
16 }
2 Likes
I got the proxy working! I had to add the following to my Apache conf file:
RewriteCond %{HTTP:Upgrade} =websocket [NC]
RewriteRule /(.*) ws://localhost:39000/$1 [P,L]
2 Likes
Hi @jmnewton and @posertinlab ,
That’s great to hear! To confirm, you’re both able to access and use the application after these changes?
- Suhail
Yes, the application is available.
1 Like
Able to access and use, no problem!
1 Like
Thanks for this, I’ve been breaking my head over this error for the past few days and I couldn’t figure it out!