Install on shared GPU cluster (h4h, Toronto, Canada)

open

#1

I’m working with Zhibin from the Bioinformatics and HPC Core at UHN (Toronto, Canada) to see if we can install cryosparc on h4h

  • info on UHN’s h4h slurm cluster: https://1drv.ms/w/s!AinqViGDqMQDiFsDGm9YthNkfpvD
  • h4h has one GPU node with 3 GPUs for general use
  • can not set up http
  • no ssh to cluster compute nodes
  • can not have a web server running all the time on the cluster node because the user will not be able to access it
  • software now allowed to schedule jobs unless it can go through the cluster’s scheduler.
  • cluster IT staff can not allow a webapp/server running all the time on the cluster and they will not open the port to the internet. Cluster uses have to submit jobs to the cluster themselves.

#2

Hey @Geoffrey,

Submitting to a SLURM cluster is fully supported. The only understandable confusion here is where to run the master node, since you aren’t able to run processes on the login node. Technically, you should be able to run the master node on your own laptop (and submit jobs to the cluster), but the main requirement is that the storage layer is shared across the master and worker nodes. This can be done by potentially mounting the group directory (/cluster/projects) onto your laptop, but I doubt the cluster admins would allow that since it’s not even mounted on the login node. Another option would be to set up some sort of “Edge Node” that is located in the same network as the cluster- this would just be another VM that can submit to the cluster, but again shares the storage layer. Either way, it might be worth talking to the cluster admins in this case.