Deleting data from store-v2

Data Management
1

Is it safe to delete data in “/scratch/user/instance_xxxx:yyyyy/store-v2” (xxxx is cluster name and yyyyy is port)?

I believe the data is accumulated there from “cache on SSD“. How to properly delete these? if the “store-v2” folder has any other data too, what is the best way to remove it safely without affecting projects?

2

Welcome to the forum @ashwinuday .
When no jobs are running on the CryoSPARC instance and /scratch/user/instance_xxxx:yyyyy/ does not hold any data other than particles cached by CryoSPARC), it is safe to remove the links/ and store-v2/ directories together. Please keep in mind that manual deletion should not be required under normal circumstances, as CryoSPARC should manage deletions from cache automatically.

3

Thank you for your response @wtempel

I understand it should clear up regularly. But in my case it had increased upto 16TB and was taking up space required for processing.

Is there any way to cap it at a lower storage?

Thanks.

4

This suggests that storage is shared between project directories and particle cache, which would pose two problems:

  1. Caching may shorten the lifespan of underlying storage devices. If cache storage is shared with project directory storage, a cache device failure may lead to the loss of project data.
  2. The benefit of caching depends on cache storage being significantly faster than and, therefore, distinct from project directory storage.

Please review the --ssdquota parameter for node targets and cache_quota_mb setting for cluster targets.