Configuring Per-User SSD Cache for CryoSPARC Workers Connected Directly to the Master

Install, Configure and Update
1

Dear CS Team,

We are a large CryoSPARC installation with approximately 25-30 users sharing an SSD cache of around 50 TB. Currently, each job takes a long time to check whether the required particles are already in the cache, as it has to scan the entire 50 TB cache.

To address this issue, we would like to configure a dedicated cache directory for each CryoSPARC user, rather than for each Unix user.

In the worker configuration, we currently have the following options:

export CRYOSPARC_LICENSE_ID="xx"
export CRYOSPARC_SSD_PATH="/scratch/CS/"
export CRYOSPARC_CACHE_NUM_THREADS=16
export CRYOSPARC_CACHE_LOCK_STRATEGY="master"

Is there a way to access the CryoSPARC username associated with each job on workers config.sh connected directly to the master, similar to how we access it through the {{ cryosparc_username }} variable in SLURM script?

Ideally, we would like to dynamically set CRYOSPARC_SSD_PATH based on the CryoSPARC username so that each user has a dedicated cache directory, regardless of the Unix account under which the worker process runs.

Something like this where ${cryosparc_username} reflects the name from CS.

export CRYOSPARC_SSD_PATH="/scratch/CS/${cryosparc_username}"

Any guidance on how to achieve this would be greatly appreciated.

Best regards,

Rajan

2

If you’re using slurm, it should be possible to add

export CRYOSPARC_SSD_PATH="/scratch/CS/{{ cryosparc_username }}
before
{{ run_cmd }}
to the job script and just remove this line from config.sh

3

Thanks @bsobol but as I wrote above we need it for workers connected directly to master and not via SLURM.