What to do when Database fills up disk?

Data Management
1

For several years multiple users have been using a single Cryosparc instance on our group system consisting of a master node with several workers. Over time the database has grown to close to a TB and it will soon fill up the disk. The question now becomes if we can shrink the database (running CS v5.0) as many projects have been detached or deleted, if we should simply copy it onto a larger disk and continue using it as is, or if we should start a new database from scratch. Any suggestions on what might be the best course of action is welcomed.

2

I have been able to detach projects which allows me to transfer them from one drive to another. Then you can reattach them in their new location as described in this guide:

3

@Pielhaas Have you already looked at Guide: Reduce Database Size (v4.3+) | CryoSPARC Guide?

4

Yes, I had a look at this. It reduced the database only by a less than a GB.

5

Prior to compacting the database, have you checked that those database records of detached projects that are no longer needed have been deleted from the database using the Delete Project from Database project action?

image
image474×488 20.4 KB

6

This database dates back to Crysparc v3 and I am not sure how projects have been removed in the past. Most recent projects have been deleted as suggested.

7

Both options are plausible.
If you choose the option

copy it onto a larger disk and continue using it as is,

be aware of the validity and consistency considerations (mentioned in Database migration from v4.5.3 to 4.5.7 - #2 by wtempel).

For the

option, are you considering installation and operation of an additional CryoSPARC instance, or the replacement of the existing CryoSPARC instance?

8

It would be a replacement of the current instance.

9

In this case, the choice between the “larger disk” and “from scratch” would depend on how many of the existing projects are still of interest. If the number is small, you may prefer to build a new database from scratch:

  1. detach the “interesting” projects
  2. preserve outputs of the commands
    cryosparcm user list
cryosparcm resources
    for reference later
  3. stop CryoSPARC
  4. redefine CRYOSPARC_DB_PATH inside cryosparc_master/config.sh to point to a suitable, empty directory
  5. start CryoSPARC
  6. re-create CryoSPARC users and re-connecting worker lanes, with reference to command outputs preserved earlier
  7. attach the “interesting” projects