Delay on cryosparcm start

Install, Configure and Update
1

Hello cryosparc Team,

Version used: 4.0.3

We face some issues during cryosparcm start in the last few versions on one of our machines.
The command checkdb is executed, but the previous start database ports did not start, yet.

For example:

$ cryosparcm start

Starting cryoSPARC System master process..
CryoSPARC is not already running.
configuring database
    configuration complete
database: started
exception: connect failed
ERROR: Could not check database configuration - is the database running?
Start the database with the following command:

    /opt/development/cryosparc_3.1/cryosparc_master/bin/cryosparcm restart

Then re-run checkdb

I could fix this issue by adding the same code used after start command_core

    while ! python -m bin.minicurl http://$CRYOSPARC_MASTER_HOSTNAME:$CRYOSPARC_MONGO_PORT --auth; do
        sleep 1
    done

I added this to every start database call in cryosparcm in

  • start)
  • backup)
  • restore)
  • changeport)

Now additionally, we have a similar delay issue in cryosparcm start

$ cryosparcm start

Starting cryoSPARC System master process..
CryoSPARC is not already running.
configuring database
    configuration complete
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/opt/development/cryosparc_3.1/cryosparc_master/cryosparc_compute/database_management.py", line 68, in configure_mongo
    mongod.wait(5)
  File "/opt/development/cryosparc_3.1/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.7/subprocess.py", line 1019, in wait
    return self._wait(timeout=timeout)
  File "/opt/development/cryosparc_3.1/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.7/subprocess.py", line 1645, in _wait
    raise TimeoutExpired(self.args, timeout)
subprocess.TimeoutExpired: Command '['mongod', '--dbpath', '/home/stabrin/CRYOSPARC', '--port', '39001', '--oplogSize', '64', '--replSet', 'meteor', '--nojournal']' timed out after 5 seconds

Again, I was able to fix this by increasing the timeout from 5 to 60 seconds.

Would it be possible to include those changes into cryosparc core?
I needed to make those changes manually for the last versions, but it is painful ^^
Additionally, it breaks the cryosparcm update procedure and does no longer update the worker automatically in a single-workstation setup.

Thank you for your help :slight_smile:
Best,
Markus

1 Like
2

Welcome to the forum @mstabrin, and thank you for reporting this problem and your workaround. I will send you a direct message to request some logs so we may understand why the delays occur in the first place.

3

I have the same error when doing backup. Just doing the backup command again after lets it run normally.

cryosparcuser@kermit103:~$ cryosparcm backup --dir=/home/cryosparcuser

Backing up to /home/cryosparcuser/cryosparc_backup_2022_11_15_08h45.archive

CryoSPARC is not already running.

Starting the database in case it's not already running.
Starting cryoSPARC database
database: started
exception: connect failed
ERROR: Could not check database configuration - is the database running?
Start the database with the following command:

    /home/cryosparcuser/cryosparc_master/bin/cryosparcm restart

Then re-run checkdb

cryosparcuser@kermit103:~$ cryosparcm backup --dir=/home/cryosparcuser

Backing up to /home/cryosparcuser/cryosparc_backup_2022_11_15_08h46.archive

CryoSPARC is running.
Starting the database in case it's not already running.
Starting cryoSPARC database
database: ERROR (already started)
    database configuration is OK.

Executing mongodump.

MongoDB shell version v3.6.23
connecting to: mongodb://kermit103:39001/meteor?authSource=admin&gssapiServiceName=mongodb
Implicit session: session { "id" : UUID("XXX") }
MongoDB server version: 3.6.23
{
        "acknowledged" : true,
        "insertedId" : ObjectId("XXX")

cheers
Kilian

4

Hello everybody,

I just updated to version 4.1 and the problem appears to be gone for now!
Thank you :smiley:

Best,
Markus