Directory and not node specific name for ssd cache folder

Arunabh · September 20, 2026, 4:58pm

Hi!

I run cryosparc on our university cluster on which cryosparc is not continuously run, so I run it through slurm for a wall time of 7 days. However, the ssd cache is named instance_<$HOSTNAME>:port, which is different each time I /start/restart cryosparc, and hence the files get copied in a new folder every time. It is possible to name the folder instance_<$USER>:port for ssd cache? Constraining –nodelist works, but it depends upon availability.

Thank you!

wtempel · September 21, 2026, 10:54pm

@Arunabh There may be workarounds, depending on how the GPU worker lanes of your CryoSPARC instance are implemented.
Please can you post the output of the command

cryosparcm resources

Arunabh · September 22, 2026, 5:44pm

Target Property Value

short_jobs Type cluster
Host short_jobs
Lane short_jobs
Worker /project/pkgs/cryosparc_worker/bin/cryosparcw
qdel command scancel {{ cluster_job_id }}
qinfo command sinfo
qstat command squeue -j {{ cluster_job_id }}
qsub command sbatch {{ script_path_abs }}
send command {{ command }}
script_tpl #!/bin/bash
#SBATCH -J cryosparc_{{ project_uid }}_{{ job_uid }}
#SBATCH --ntasks {{ num_cpu }}
#SBATCH --partition=standard
#SBATCH --time=6:00:00
#SBATCH -A grouper_id
{{ run_cmd }}
Cache Path /scratch/myuserid

short GPU jobs Type cluster
Host short GPU jobs
Lane short GPU jobs
Worker /project/pkgs/cryosparc_worker/bin/cryosparcw
qdel command scancel {{ cluster_job_id }}
qinfo command sinfo
qstat command squeue -j {{ cluster_job_id }}
qsub command sbatch {{ script_path_abs }}
send command {{ command }}
script_tpl #!/bin/bash
#SBATCH -J cryosparc_{{ project_uid }}_{{ job_uid }}
#SBATCH --ntasks {{ num_cpu }}
#SBATCH --nodes=1
#SBATCH --partition=gpu
#SBATCH --gres=gpu
#SBATCH --mem={{ (ram_gb*2000)|int }}M
#SBATCH --output={{ job_dir_abs }}/slurm.out
#SBATCH --error={{ job_dir_abs }}/slurm.err
#SBATCH --time=1:30:00
#SBATCH -A grouper_id
{{ run_cmd }}
Cache Path /scratch/myuserid

4 GPUs Type cluster
Host 4 GPUs
Lane 4 GPUs
Worker /project/pkgs/cryosparc_worker/bin/cryosparcw
qdel command scancel {{ cluster_job_id }}
qinfo command sinfo
qstat command squeue -j {{ cluster_job_id }}
qsub command sbatch {{ script_path_abs }}
send command {{ command }}
script_tpl #!/bin/bash
#SBATCH -J cryosparc_{{ project_uid }}_{{ job_uid }}
#SBATCH --ntasks {{ num_cpu }}
#SBATCH --nodes=1
#SBATCH --partition=gpu
#SBATCH --gres=gpu:4
#SBATCH --mem={{ (ram_gb*2000)|int }}M
#SBATCH --output={{ job_dir_abs }}/slurm.out
#SBATCH --error={{ job_dir_abs }}/slurm.err
#SBATCH --time=24:00:00
#SBATCH -A grouper_id
{{ run_cmd }}
Cache Path /scratch/myuserid

GPU-MIG Type cluster
Host GPU-MIG
Lane GPU-MIG
Worker /project/pkgs/cryosparc_worker/bin/cryosparcw
qdel command scancel {{ cluster_job_id }}
qinfo command sinfo
qstat command squeue -j {{ cluster_job_id }}
qsub command sbatch {{ script_path_abs }}
send command {{ command }}
script_tpl #!/bin/bash
#SBATCH -J cryosparc_{{ project_uid }}_{{ job_uid }}
#SBATCH --ntasks {{ num_cpu }}
#SBATCH --partition=gpu-mig
#SBATCH --gres=gpu:1
#SBATCH --mem={{ (ram_gb*1000)|int }}M
#SBATCH --output={{ job_dir_abs }}/slurm.out
#SBATCH --error={{ job_dir_abs }}/slurm.err
#SBATCH --time=6:00:00
#SBATCH -A grouper_id
{{ run_cmd }}
Cache Path /scratch/myuserid

multi_proc Type cluster
Host multi_proc
Lane multi_proc
Worker /project/pkgs/cryosparc_worker/bin/cryosparcw
qdel command scancel {{ cluster_job_id }}
qinfo command sinfo
qstat command squeue --job {{ cluster_job_id }}
qsub command sbatch {{ script_path_abs }}
send command {{ command }}
script_tpl #!/bin/bash
#SBATCH -J CS_{{ project_uid }}_{{ job_uid }}
#SBATCH --ntasks=640
#SBATCH --nodes=10
#SBATCH --partition=parallel
#SBATCH --time=12:00:00
#SBATCH -A grouper_id
{{ run_cmd }}
Cache Path /scratch/myuserid
Cache Reserve 10.00 GB

RTX3090 Type cluster
Host RTX3090
Lane RTX3090
Worker /project/pkgs/cryosparc_worker/bin/cryosparcw
qdel command scancel {{ cluster_job_id }}
qinfo command sinfo
qstat command squeue -j {{ cluster_job_id }}
qsub command sbatch {{ script_path_abs }}
send command {{ command }}
script_tpl #!/bin/bash
#SBATCH -J cryosparc_{{ project_uid }}_{{ job_uid }}
#SBATCH --ntasks {{ num_cpu }}
#SBATCH --nodes=1
#SBATCH --partition=interactive
#SBATCH --gres=gpu:rtx3090:1
#SBATCH --mem={{ (ram_gb*2000)|int }}M
#SBATCH --output={{ job_dir_abs }}/slurm.out
#SBATCH --error={{ job_dir_abs }}/slurm.err
#SBATCH --time=2:00:00
#SBATCH -A grouper_id
{{ run_cmd }}
Cache Path /scratch/myuserid

GPU-POD Type cluster
Host GPU-POD
Lane GPU-POD
Worker /project/pkgs/cryosparc_worker/bin/cryosparcw
qdel command scancel {{ cluster_job_id }}
qinfo command sinfo
qstat command squeue -j {{ cluster_job_id }}
qsub command sbatch {{ script_path_abs }}
send command {{ command }}
script_tpl #!/bin/bash
#SBATCH -J cryosparc_{{ project_uid }}_{{ job_uid }}
#SBATCH --ntasks {{ num_cpu }}
#SBATCH --nodes=1
#SBATCH --partition=gpu
#SBATCH --gres=gpu:a100:4
#SBATCH -C gpupod
#SBATCH --mem={{ (ram_gb*2000)|int }}M
#SBATCH --output={{ job_dir_abs }}/slurm.out
#SBATCH --error={{ job_dir_abs }}/slurm.err
#SBATCH --time=30:00:00
#SBATCH -A grouper_id
{{ run_cmd }}
Cache Path /scratch/myuserid

Ampere_single Type cluster
Host Ampere_single
Lane Ampere_single
Worker /project/pkgs/cryosparc_worker/bin/cryosparcw
qdel command scancel {{ cluster_job_id }}
qinfo command sinfo
qstat command squeue -j {{ cluster_job_id }}
qsub command sbatch {{ script_path_abs }}
send command {{ command }}
script_tpl #!/bin/bash
#SBATCH -J cryosparc_{{ project_uid }}_{{ job_uid }}
#SBATCH --ntasks {{ num_cpu }}
#SBATCH --nodes=1
#SBATCH --partition=gpu
#SBATCH --gres=gpu:1
#SBATCH --mem={{ (ram_gb*2000)|int }}M
#SBATCH --output={{ job_dir_abs }}/slurm.out
#SBATCH --error={{ job_dir_abs }}/slurm.err
#SBATCH --time=6:00:00
#SBATCH --constraint=“a6000|a40|a100”
#SBATCH -A grouper_id
{{ run_cmd }}
Cache Path /scratch/myuserid

Ampere-x4 Type cluster
Host Ampere-x4
Lane Ampere-x4
Worker /project/pkgs/cryosparc_worker/bin/cryosparcw
qdel command scancel {{ cluster_job_id }}
qinfo command sinfo
qstat command squeue -j {{ cluster_job_id }}
qsub command sbatch {{ script_path_abs }}
send command {{ command }}
script_tpl #!/bin/bash
#SBATCH -J cryosparc_{{ project_uid }}_{{ job_uid }}
#SBATCH --ntasks {{ num_cpu }}
#SBATCH --nodes=1
#SBATCH --partition=gpu
#SBATCH --gres=gpu:8
#SBATCH --mem={{ (ram_gb*2000)|int }}M
#SBATCH --output={{ job_dir_abs }}/slurm.out
#SBATCH --error={{ job_dir_abs }}/slurm.err
#SBATCH --time=0:10:00
#SBATCH --constraint=“a6000|a40|a100”
#SBATCH -A grouper_id
{{ run_cmd }}
Cache Path /scratch/myuserid

Thanks!

Arunabh

wtempel · September 30, 2026, 3:36pm

Thanks for clarifying that you are using scheduler lanes of type cluster.
Am I assuming correctly that

the /scratch/myuserid is shared between cluster nodes?
you have confirmed that /scratch/myuserid storage enables significantly faster read access than does the storage for CryoSPARC project directories?

A potential workaround could be to create a symbolic link

named according to the current CryoSPARC master hostname
pointing to the directory named according to the old CryoSPARC master hostname

every time you start CryoSPARC master services on a different cluster node:

cd /scratch/myuserid
ln -s instance_original_hostname\:port instance_current_hostname\:port

Arunabh · October 2, 2026, 7:14pm

Yes, the scratch is a common SSD Weka-based storage shared between cluster nodes. individual nodes have their own local ssd scratch, but its tricky to use and clean them after the job because the only way to enter into those nodes is through OoD.
I tested a GPFS based HDD storage (project directory space) and the SSD Weka scratch (SSD cache space) for job run speeds, by cloning a NU-refine job. For this I used RTX6000pro GPU (GPU-MIG partitioned 1/4th, so 24GB available VRAM) GPU-MIG queue. Interestingly, for a boxsize/pxsize of 360/0.857 for 80K particles, the run finished faster on GPFS (HDD) than Weka (SSD) (23m 40s vs 29m44s). The GSFSC res. is off by 0.02, but I assume that that’s a random seed problem.

I think for my purposes, I can just run jobs without SSD cache.

Curiously, it could be a quick solution, but if I simply rename the folder, it works!

Thank you for prompting to compare process speeds with/without the cache.

Regards,

Arunabh