Hi,
I am considering setting up cryoSPARC on our university HPC. The idea it to make it accessible to many users. Ideally there should be a single web server running for all the users.
It is not clear though if all the jobs sent to the queuing system (SLURM) will be submitted by a single user from the point of view of the queuing system. This would be an issue. If 10 people submit a job simultaneously and the queuing system sees 10 jobs submitted by a single HPC user, it will not allow the 10 jobs to run simultaneously. On the other hand if it would see each job as being submitted by a different user it would allow all 10 jobs to run simultaneously.
Can you please clarify the situation and let me know how you recommend dealing with such a case?
Thanks
Benoit
Welcome to the forum @bzuber.
As you already suspected, jobs submitted through the web UI are counted against the quota of the system user that owns the cryoSPARC instance. Cluster resources and quota need to be allocated according to the combined needs of all cryoSPARC users. You may still capture information about the individual cryoSPARC users for accounting via the job_creator or cryosparc_username submission script variables. Such accounting would require a custom add-on to your existing cluster management software.
Alternatively, you could run a separate instance for each user and allocate resources as usual, but that would entail foregoing the project-sharing facilities implemented in cryoSPARC.
Thank you for your explanations @wtempel. I’ll think a bit more about the options and discuss with our cluster managers which one they prefer.
Running a single instance for all CryoSPARC users has other implications (besides Slurm quotas) such as file access from a single source to all user data.
We run CryoSPARC with a job scheduler and we’re moving from individual user instances to PI-group instances (4-5 users under the same CS instance) and avoid a single user accessing all cryoEM data. You can run multiple CryoSPARC instance using the same server/host by changing the port range CryoSPARC uses in the cryosparc_master/config file.
Just my two cents, hope it helped.