Setting up cryosparc to send jobs into multiple nodes!

I have cryosparc installed and setup to run on gpu node of our cluster setup. But there are other nodes available and I want to be able to select the node to queue the job to. Since not all jobs require gpu, it would be more efficient to send each job to a different partition. How do I set this up?

Hi @73km,

I’d like to understand you issue in a bit more detail. Have you currently setup cryoSPARC on a one cluster and you’re looking to split up the cluster into specific nodes, or do you already have your nodes as separate workers?

Addy.

So in our cluster setup there are multiple partitions/nodes. Over a hundred CPU nodes and only 2 GPU nodes. While setting up cryosparc, I set partition=gpu [for the gpu node] because that’s where cuda drivers were and cryosparc can make use of gpu. But I realize not all jobs require GPUs. So now I want to be able to select which partition to send the job to based on the requirement. In tutorial videos, I see option to select different lanes while queuing jobs. It would be nice if I can make those lanes option with different partitions we have.

Hi @73km,
Do you use slurm on your cluster?
if you have some lane definition for the GPU partition, just copy that and modify:

  • The cluster_info.json file so that “name” is different (that will be the name of your new line)
  • The default cluster_script.sh so that the parameter
    #SBATCH --partition=your_gpu_slurm_partition
    points to your new partition (your_cpu_partition).

Then run cryosparcm cluster connect and your new lane should be visible on your web.
You will still need to choose the lane when you launch the job…
Best,
Juan

1 Like

Thanks! It worked! I wish I could put multiple of these cluster scripts so that I can monitor them simultaneously. It seems I can put only one file at a time.

Hi @73km,

If I’m understanding your post correctly, you’d like to have a copy of each set of cluster integration scripts.
You can do the following:
Create a folder called cryosparc_cluster_scripts adjacent to cryosparc_master and cryosparc_worker. Inside this folder, create a new folder for each partition (you can run cryosparcm cluster dump <cluster name> inside each folder to get cryoSPARC to write out the two files for that cluster integration):

-  cryosparc_cluster_scripts
  - partition1
    - cluster_info.json
    - cluster_script.sh
  - partition2
    - cluster_info.json
    - cluster_script.sh
  - partition3
    - cluster_info.json
    - cluster_script.sh
 

When you run cryosparcm cluster connect, it reads the two files required from your current working directory. Whenever you want to make modifications to any of the scripts, navigate to the folder of the partition you want to modify, modify the files, then run the commands.
This way you don’t have to keep modifying a single set of files in order to update each configuration.