Handling of scratch space

Data Management
1

Dear cryoSPARC developers,

scratch space is usually disk space temporarily needed if memory is not big enough. The needed scratch space should be released after the job has finished. As also other users have pointed out, cryoSPARC occupies all allowed scratch space and only releases some of it if there is not enough space left for the current job.
This creates problems when sharing scratch space with other programs on a SLURM cluster. Since cryoSPARC effectively takes away all of its allowed scratch space, I have to balance the scratch space quota for cryoSPARC against the left scratch space needed by other programs, which is quite delicate.

To my knowledge, cryoSPARC is the only program that uses all of its allowed scratch space instead of releasing it when jobs have finished.

Is it possible to tell cryoSPARC to immediately release any needed scratch space after a job has finished?
If not, could you please add such an option to cryoSPARC’s handling of scratch space?
The alternative would be to write a cron job that deletes older files on the scratch disk, which I don’t like, honestly.

Best regards,

Dirk.

2

@dirk The CRYOSPARC_SSD_CACHE_LIFETIME_DAYS parameter controls the persistence of cached data. After modifying cryosparc_master/config.sh, please restart cryoSPARC.

3

Hi Dirk,

You may also find the --ssdquota <ssd_quota_mb> and --ssdreserve <ssd_reserve_mb> configuration options useful.

Best,

Ryan

4

@wtempel
Many thanks! I will reduce the lifetime of cached files using this environment variable.

@RyanFeathers
Hi Ryan! Thanks - this is how I control the scratch space quota for cryoSPARC.

Best regards,

Dirk.