cryoSPARC is surely running but cryosparcm status says not

Troubleshooting
1

Hi,

Our cryoSPARC is surely running, I can execute jobs correctly, but cryosparcm status command says not.

> cryosparcm status
----------------------------------------------------------------------------
CryoSPARC System master node installed at
/data2/cryosparcuser/cryosparc_master
Current cryoSPARC version: v3.2.0+210831
----------------------------------------------------------------------------

CryoSPARC is not running.

----------------------------------------------------------------------------

Thanks.
Kotaro

2

Hi, have you tried running

  • cryosparcm restart

and see the output then? Alternatively you can try:

  • cryosparcm stop and then cryosparcm start

But most importantly, do you have any issues with running the jobs?

1 Like
Updated from 3.3.2 to 4.1.1 and now cannot start cryoSPARC -- getting EADDRINUSE in logs
3

Hi @kttn,

This usually happens when the .sock file that supervisord uses to keep track of cryosparc processes is deleted. This file lives in the /tmp folder, so if that folder was wiped out for any reason (quite common), you’ll see this happen.
Theres no way to reconnect to the living cryoSPARC instance, so the next thing to do is to kill off the running, zombied cryoSPARC instance, and start a fresh one.

ps -ax | grep “supervisord” (kill only the process that is running from your cryosparc install)
ps -ax | grep “cryosparc” (kill all the matching processes related to your cryosparc instance)
ps -ax | grep “mongod” (kill only the process running your cryosparc database)

e.g. kill 82681

Then start cryoSPARC again: cryosparcm start

Database: ERROR (spawn error)
User not found after moving database
4

Is there a way to specify where the supervisord sock file is placed?

5

Hi @karcaw,

Yes, there is, using the CRYOSPARC_SUPERVISOR_SOCK_FILE environment variable.
The default value is /tmp/cryosparc-supervisor-$root_dir_hash.sock
where root_dir_hash is $(echo -n $CRYOSPARC_ROOT_DIR | md5sum | awk '{print $1}').
You can append this variable to cryosparc_master/config.sh. Once you’ve saved the file, restart cryoSPARC: cryosparcm stop then cryosparcm start

Location of *.sock
6

Hi, thank you for the inputs.

@stephan
Is there any risk in keep using the zombied (by deletion of the sock file) cryoSPARC instance?
This situation is hard to recognize, as I hadn’t noticed this until I manually executed cryosparcm status command.

7

Hi @kttn,

You won’t run into any problems unless someone tries to turn on cryoSPARC- this will create two instances that are trying to make changes to the same database (which might become a bigger problem).
Also, you won’t be able to update the instance.

Database: ERROR (spawn error)
8

Ok, I understand. Thank you @stephan!

9

I often has this happening, that cryosparcm status returns CryoSPARC is not running. for example at the moment. But I can see a .sock file in /tmp which seems like the one you describe in this thread. It is named cryosparc-supervisor-xxxxxxxxxxxxxxxxxxxxxxxxxx.sock and the timestamp is from just a few minutes ago. There is only one instance running on this computer.

Could there be other reasons for cryosparcm not recognising that there is an instance running?

10

Does the time stamp coincide with the time at which CryoSPARC was restarted?
What are the outputs of these commands

cryosparcm status
ps -eo user:16,pid,ppid,start,cmd | grep -e cryosparc_ -e mongo
ls -l /tmp/cryosparc*.sock /tmp/mongo*.sock