Our CryoSPARC installation on a Linux cluster was upgraded to 4.0.2 from 3.3 in late November. During the weekend, a cooling issue in the serverhall caused a sudden shutdown. After restarting CryoSPARC, things are not working correctly.
cryosparcm status gives this output:
----------------------------------------------------------------------------
CryoSPARC System master node installed at
/home/cryosparcuser/cryosparc/cryosparc_master
Current cryoSPARC version: v4.0.2
----------------------------------------------------------------------------
CryoSPARC process status:
app RUNNING pid 24622, uptime 0:03:34
app_api RUNNING pid 24660, uptime 0:03:32
app_api_dev STOPPED Not started
app_legacy STOPPED Not started
app_legacy_dev STOPPED Not started
command_core RUNNING pid 24517, uptime 0:03:49
command_rtp RUNNING pid 24572, uptime 0:03:41
command_vis RUNNING pid 24566, uptime 0:03:42
database RUNNING pid 24402, uptime 0:03:53
----------------------------------------------------------------------------
An error ocurred while checking license status
Could not get license verification status. Are all CryoSPARC processes RUNNING?
Occasionally, the “cryosparcm status” command finishes as it should.
The web interface is accessible, but it is not possible to start new jobs or do anything productive on the instance. When starting a new job, a pop-up message comes after some time saying “Unable to create job: Unknown 504 error”.
We have indications that these problems could be related to a problems we had with one of our file-servers in conjunction to the cooling failure. Some of our older projects are stored on that file server. command_core seems stuck, to trying to access files for one of the projects on that server. Is it possible to remove/archive/disconnect those projects to have the rest of the instance working properly?