Cryosparc installation and web proxy

Install, Configure and Update
1

I’m trying to install cryosparc v2.14.2 in standalone mode. The master reaches the public Internet through a HTTP proxy. During installation, the master appears to install OK, but cannot fully start. Output below. The installer does not continue with installing the worker.

How can cryosparc work with a proxy? curl https://get.cryosparc.com succeeds. However, cryosparcm start hangs.

************ CRYOSPARC SYSTEM: STANDALONE INSTALLER **************

  The installer will now download, install and connect the worker
  package on this machine. The worker package will allow cryoSPARC
  to run compute jobs.

******************************************************************

 Starting cryoSPARC...

Starting cryoSPARC System master process..
CryoSPARC is not already running.
database: started
command_core: started
2

Hi @mhucsf,

Is it possible if you can try starting each service individually?
So, first turn everything off: cryosparcm stop
Then run the commands in the following order:

cryosparcm start database
cryosparcm start command_core
curl localhost:39002  # you should see the output "Hello World from cryosparc command core."
curl localhost:39002/startup
cryosparcm start command_vis
cryosparcm start command_proxy
cryosparcm start webapp

If you didn’t see any output from curl localhost:39002, please post the output of the following commands:
cryosparcm log command_core and cryosparcm log database

3

Thanks for the guidance. I tried again with version 2.15.0, since it was released after I first made this post. Everything I mention from now on will be with that version.

curl localhost:39002 succeeds, but only after I unset http_proxy and unset https_proxy in Bash. Otherwise, the proxy tries to resolve localhost.

I see localhost mentioned in many places in the cryosparcm script. It surprises me that connections to localhost would be routed through our proxy, so I’ll work with our admins to verify the proxy is working as intended. The proxy runs Squid, and the clients simply have the previously mentioned environment variables pointing to the proxy’s IP number. Programs such as curl check for those specific environment variables. There’s no special proxy client program.

If it is indeed working as intended, how would you suggest I proceed? I can unset the proxy environment variables, but then I’d lose the ability to reach your license server.

4

Hi @mhucsf,

Do you have sudo access on the machine? If you do, you might be able to edit the /etc/hosts file to make sure that the line 127.0.0.1 localhost exists. You can also add the server’s hostname to this file (e.g., add 127.0.0.1. cryoem.server.com as a new line). This would avoid the machine having to query a DNS server to resolve the address of “localhost” or even its own hostname. I’m not too experienced in how proxies work but maybe that will work?

Otherwise I’d say:

  1. Unset proxy environment variables
  2. Start cryoSPARC: cryosparcm start
  3. Export your proxy environment variables
  4. Run any job to make sure cryoSPARC still works properly
5

/etc/hosts did indeed have an entry for localhost already.

The solution was to set an environment variable called NO_PROXY to include localhost and the FQDN hostname of the master. This environment variable had not been set on our cluster. Documentation: https://curl.haxx.se/docs/manpage.html.

With NO_PROXY set, the cryoSPARC installer succeeded.

Thanks for the guidance.

1 Like