Optimizing Queuing System

Features and Functionality
1

Hi

It’s really great that cryosparc has a inbuild queuing system.

I have 4x 4-GPU System in one lane. At the moment when a job is scheduled it seams to fill first GPU-System 1 then GPU-System 2 and so on. This gives a lot of load on GPU-System 1 and GPU-System 4 is just idle.

So i would like to suggest the following optimization:

  • When a job is schedule it should check which GPU-System has the most free resources and schedule it to this host.
  • Jobs within the same Project should run as much as possible on the same GPU-System to make use of the cached data

Thanks

2

Thanks for your suggestions. You may want to consider the following alternatives to a single CryoSPARC scheduler lane with multiple worker nodes:

  1. a lane with a connected cluster, where the cluster is managed by an external workload manager, like SLURM
  2. connected worker nodes distributed over several scheduler lanes (see cryosparcw connect parameters
    --lane, --newlane)
  3. overriding the scheduler, queuing directly to a GPU