2D Classification/Refinement failing due to incorrect CRYOSPARC_SSD_PATH

Hello!

I am running the latest version V4.1.2. of CryoSparc on the MRC LMB cluster.

Linux hal.lmb.internal 4.18.0-425.3.1.el8.x86_64 #1 SMP Tue Nov 8 14:08:25 EST 2022 x86_64 x86_64 x86_64 GNU/Linux
              total        used        free      shared  buff/cache   available
Mem:           3022         496         998           0        1527        2511
Swap:             3           0           3

Several days ago, I started noticing considerable web interface lags. Some of the functions are available (browsing projects, running queued jobs), but very often new jobs are not showing, I can’t build new ones, and deleted jobs are not being removed. Sometimes, it randomly works, but all actions are very slow.

E.g.:

2023-01-27 14:31:06,247 COMMAND.CORE         run                  INFO     | Received task layout_tree with 4 args and 0 kwargs
2023-01-27 14:32:34,381 COMMAND.CORE         run                  INFO     | Completed task in 88.13398480415344 seconds

This warning always shows up in :
2023-01-27 13:14:15,092 COMMAND.CORE         run                  WARNING  | Insecure mode - no SSL for external requests

Neither an update from 4.1.1 to 4.1.2 nor cryosparcm restart doesn't help. 

In addition, 2D classification and Non-Uniform refinement both keep failing. It might be a separate problem.(?) The error messages are:

Traceback (most recent call last):
  File "cryosparc_master/cryosparc_compute/run.py", line 96, in cryosparc_compute.run.main
  File "cryosparc_master/cryosparc_compute/jobs/refine/newrun.py", line 125, in cryosparc_compute.jobs.refine.newrun.run_homo_refine
  File "/lmb/home/agusach/software/cryosparc_worker/cryosparc_compute/particles.py", line 114, in read_blobs
    u_blob_paths = cache.download_and_return_cache_paths(u_rel_paths)
  File "/lmb/home/agusach/software/cryosparc_worker/cryosparc_compute/jobs/cache.py", line 119, in download_and_return_cache_paths
    compressed_keys = rc.cli.cache_request_check(worker_hostname, rc._project_uid, rc._job_uid, com.compress_paths(rel_paths))
  File "/lmb/home/agusach/software/cryosparc_worker/cryosparc_tools/cryosparc/command.py", line 104, in func
    with make_json_request(self, "/api", data=data) as request:
  File "/lmb/home/agusach/software/cryosparc_worker/deps/anaconda/envs/cryosparc_worker_env/lib/python3.8/contextlib.py", line 113, in __enter__
    return next(self.gen)
  File "/lmb/home/agusach/software/cryosparc_worker/cryosparc_tools/cryosparc/command.py", line 165, in make_request
    with urlopen(request, timeout=client._timeout) as response:
  File "/lmb/home/agusach/software/cryosparc_worker/deps/anaconda/envs/cryosparc_worker_env/lib/python3.8/urllib/request.py", line 222, in urlopen
    return opener.open(url, data, timeout)
  File "/lmb/home/agusach/software/cryosparc_worker/deps/anaconda/envs/cryosparc_worker_env/lib/python3.8/urllib/request.py", line 525, in open
    response = self._open(req, data)
  File "/lmb/home/agusach/software/cryosparc_worker/deps/anaconda/envs/cryosparc_worker_env/lib/python3.8/urllib/request.py", line 542, in _open
    result = self._call_chain(self.handle_open, protocol, protocol +
  File "/lmb/home/agusach/software/cryosparc_worker/deps/anaconda/envs/cryosparc_worker_env/lib/python3.8/urllib/request.py", line 502, in _call_chain
    result = func(*args)
  File "/lmb/home/agusach/software/cryosparc_worker/deps/anaconda/envs/cryosparc_worker_env/lib/python3.8/urllib/request.py", line 1383, in http_open
    return self.do_open(http.client.HTTPConnection, req)
  File "/lmb/home/agusach/software/cryosparc_worker/deps/anaconda/envs/cryosparc_worker_env/lib/python3.8/urllib/request.py", line 1358, in do_open
    r = h.getresponse()
  File "/lmb/home/agusach/software/cryosparc_worker/deps/anaconda/envs/cryosparc_worker_env/lib/python3.8/http/client.py", line 1348, in getresponse
    response.begin()
  File "/lmb/home/agusach/software/cryosparc_worker/deps/anaconda/envs/cryosparc_worker_env/lib/python3.8/http/client.py", line 316, in begin
    version, status, reason = self._read_status()
  File "/lmb/home/agusach/software/cryosparc_worker/deps/anaconda/envs/cryosparc_worker_env/lib/python3.8/http/client.py", line 277, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/lmb/home/agusach/software/cryosparc_worker/deps/anaconda/envs/cryosparc_worker_env/lib/python3.8/socket.py", line 669, in readinto
    return self._sock.recv_into(b)
socket.timeout: timed out

Sorry for so many problems.

Thanks!

Anastasia

Hello!
The problems seem to be resolved.
The slow interface was caused by the interaction between the cluster and the database location.

The 2D class problem was fixed by inserting the following line into the config.sh files:

export CRYOSPARC_SSD_PATH="/ssd/${SLURM_JOB_USER}-${SLURM_JOBID}"

Best wishes,
Anastasia