Memory specification in slurm script

[asif]

Hi,

I am trying to submit the job from the master node to worker nodes through slurm script. When I hard code the cluster_script with mem, no. of cpu and no.gpu everything works fine.
When I use the following in the script

#SBATCH -c {{ num_cpu }}
#SBATCH --gres=gpu:{{ num_gpu }}
#SBATCH --mem={{ ram_gb|int }}G

The no.of cpu and gpu parameter get translated to the slurm job submission script as mentioned in cryosparc gui but memory defaults to the max memory of the master node even when I specify 100 or 200 in the GUI.

Is it something wrong that I am doing here? Is there a way to specify other slurm parameters like account, partition etc in cryosparc GUI instead of hard coding in the cluster_script.sh

Thanks,
Asif.

[wtempel]

@asif

but memory defaults to the max memory of the master node even when I specify 100 or 200 in the GUI.

The match with the “max memory of the master node” may be a coincidence, and depend on job type.
What was the job type, and what is “max memory of the master node”.
Custom values set for certain reserved variables, like ram_gb, will not be effective:

'Cluster submission script variables' ignored

Hi @robukowski, The variables ram_gb, num_gpu, and num_cpu are actually meant to be unchangeable values set by CryoSPARC when submitting jobs to the cluster representing the job’s estimated required resources to run. Any custom variables using these names will have their values overwritten. However, it is still possible to modify the requested resources in a cluster submission script using custom variables. Below is a SLURM cluster configuration where the value for the amount of RAM required …

[asif]

Hello Wtempel,

Sorry for the late response. Got involved with other stuff, back again to Slurm script, I see that cryosparc decides the resource requirements based on the job type and the job size.

But what happens if the nodes through the slurm does not satisfy the resource requirement. For example, cryosparc needs 2 gpus but the nodes have only one.

Is there a way to have the time for the slurm job(#SBATCH --time) defined as a custom variable?

Thanks,
Asif.

[ccgauvin94]

@asif

But what happens if the nodes through the slurm does not satisfy the resource requirement. For example, cryosparc needs 2 gpus but the nodes have only one.

In my experience, SLURM won’t launch the job because it won’t be able to find a partition with the requested resources. However, regarding GPUs specifically, I’m not aware of a job that requires 2. Any jobs that can use more than one let the user specify the number of GPUs.

@asif

Is there a way to have the time for the slurm job(#SBATCH --time) defined as a custom variable?

Yes, just make up a new variable. For instance, I’m using

#SBATCH --time {{ time_hr_min_sec }}

[wtempel]

@ccgauvin94

Yes, just make up a new variable. For instance, I’m using

… as described here.