Queue to instance

Install, Configure and Update
1

I wonder if it would be possible to have a general, node-agnostic queue that users could submit jobs to in addition to the lane queues. I find that users will often queue to the default node without checking GPU availability, and/or won’t check which jobs are currently using GPUs. For example, maybe all nodes are full, but one is running global CTF refinements (~1 hour) and the other is running local refinements at high resolution (~8 hours). These scenarios result in one node having a long queue and the other being empty.

There are some jobs for which one node might be better than another, but it would be nice to be able to submit to a queue that’s “Wherever this will run first”.

2

It almost seems that you are describing a “cluster”-type lane or a collection of such lanes with a carefully configured cluster backend. CryoSPARC already provides some “plumbing” for that. Beyond that, of course, there remains the not insignificant task of the appropriate cluster-side configuration:

  • selection of an external (to CryoSPARC) workload manager
  • configuration of that workload manager

How is your instance currently configured:
cryosparcm cli "get_scheduler_lanes()"
cryosparcm cli "get_scheduler_targets()"
?