Extremely large collection-*.wt file

[arohou]

Hello,

Would appreciate some guidance. We’re running low on disk space on our local database disk. One of the collection*.wt files is extremely large (516G, representing ~90% of storage used). Is there a way to work out what it causing this to be so large?

In general, is there a way to measure the “footprint” of each project in the database?

Cheers,
Alexis

[stephan]

Hi @arohou,

There’s not really a way to get the size of the database objects that correspond to all information in a project (without a complicated query, at least), but there is a way to see which collections inside the MongoDB database are taking up the most space. There is also a command you can run to release space back to the OS (MongoDB doesn’t do this automatically when documents are deleted).
See this post:

Repair MongoDB Database & Reduce Database Size

Hi @Rajan, It is possible to reduce the size of the MongoDB database by using the compact command, but please note this is a blocking command; cryoSPARC will not work while this command is running. MongoDB does not release storage space back to the OS when you delete documents, which is why the storage size remains the same. First, create a backup of your database. Then, in a shell on the master node: cryosparcm restart eval $(cryosparcm env) mongo localhost:$CRYOSPARC_MONGO_PORT # will c…

[arohou]

Thanks @stephan I shall try the compaction.

Do you have any tips in general on identifying, among a long list of projects, which are likely to have bigger footprints on the database?

[stephan]

Hi @arohou,

The collections in mongo that would take up the most amount of space are events and fs.files. Each job that’s created in a project stores its plots and text logs in those collections. There is not really an easy way to get the size of a query in the version of MongoDB that we’re running.