Files on scratch directory

Hardware and Performance
1

Hi,

We noticed that CryoSparc is putting a lot of data on the scratch directory that is never erased (we have over 30TB there). Is it safe to just erase them? What are they not automatically erased at the end of a job?

Thanks,
Best,
Nicolas

2

Hi @ncoudray,

It is definitely safe to delete those files if no jobs are currently using them. cryoSPARC will automatically manage the “scratch” directory and free up space when other jobs request to use it. If you’d like to limit how much of the “scratch” directory cryoSPARC uses, reconfigure your node’s properties to have an “SSD Quota”. See here: https://guide.cryosparc.com/processing-data/tutorials-and-case-studies/tutorial-ssd-particle-caching-in-cryosparc#advanced-parameters

3

Thanks @stephan
I have a few other questions about it:

1- What would be the best way to do it on a cluster? Workers hostnames are different at each run. 2- If we want to limit to 10 TB for example, should we just do:

bin/cryosparcw connect --worker <worker_hostname> --update --ssdquota 10000

3- In this command, if we don’t specify anything for the other options, will they be reset to some default values, or will they keep whichever parameters we initially set during the installation?

4- Also, should we stop “cryosparcm stop” first and then restart it or not?

5- And next time we do an update of CryoSparc, will it keep this settings, or should we run the command again?

6- Files stored on the scratch are all temporary and used only during the run of that particular job, right? So, if we want to manually remove all files older than 6 months, could we safely do:

find /gpfs/scratch/our_directory/ -type f -mtime +180 -exec rm {} ;

or would you recommend something else to clean the scratch?

Thanks again,
Best