Advice (wanted) for cluster administrators


I’ve read the Instructions but they don’t seem to offer the advice I’m looking for.

I am a staff member at Texas A&M University. My title is kind of strange but I am basically a cluster administrator for the purpose of this discussion. We use Open OnDemand as our clusters’ web interface and SLURM as the scheduler. A user has asked me to install Cryosparc for his group of (I assume) multiple users. This is something we normally do on behalf of our users, both as a support service and to reduce redundant disk usage.

Due to University policy, all files and processes on the cluster must be owned by a user. We are not allowed to have robot accounts or shared accounts. When reading the Cryosparc instructions, it seems to me that the only feasible option is to have each user own their own cryosparc instance, or perhaps a group of users can share their instance as long as one of them actually owns it. Question: can multiple cryosparc instances (owned by different users) share a (partial) common installation area to reduce disk usage and setup time?

Due to our own practices, we are reluctant to allow the master process to run on a login node or a server node. We have a large number of users, so our policies about long-running processes on login nodes have to be very strict, and we don’t allow users access to our server nodes at all. It seems to me that the only feasible option is to start the cryosparc master process on a compute node and hope that the runtime is long enough for all the jobs. Question: how long does a master process need to run for it to be worth it?

Our group (of staff) are also researchers and one of the tasks we are looking into is ways to make the Open OnDemand platform more practically useful for research universities and their users. We have already integrated a few different research computing frameworks into our OnDemand build. It seems natural to me that Cryosparc would be a good fit for this strategy and worth giving a chance.

I saw a few months ago several of you on here were discussing ways to improve Cryosparc to make it more friendly to clusters, especially regarding user permissions and the separation of instance configurations from the central installation. Is there any news about this I should know about? Is there any other advice you would give me before I jump directly into the deep end?

Richard Lawrence

Hi @rarensu,

Sorry for the late reply.

Unfortunately, no, each cryoSPARC instance will need to have its own installation directory.

It depends on the users- each job in cryoSPARC can take anywhere from 1 second all the way to 12 hours depending on the size of data being processed and type of job (the master needs to be running in order for jobs to work). On top of that, if users ever want to examine their results, the master has to be turned on. It’s not difficult to turn on and off cryoSPARC as you use it, but if the cluster has a cap on the runtime of jobs, it may be difficult to manage them.

Is it possible to set up an edge node separate from the cluster that has access to the cluster management system?

Unfortunately, there isn’t. We’re hoping to work on this part soon!