Hi,
We’re working with HPC on optimising Cryosparc that is running on our cluster. We were wondering if it is standard practice to disable transparent hugepages on the cluster and whether there are any risks or downsides to doing this?
The other question I had was what is the average database size that is stored in the cryosparc directory? Currently ours is hovering around 2TB.
Many thanks,
Andrea
We run CryoSPARC on a shared cluster where many applications benefit from enabled THP, so we kept it enabled. It caused stability issues only on version 4.6.0. 4.6.2 and 4.7.X are, as far as I know, as stable as pre-4.6.
On the other hand, there are reports (on this forum) that in some cases CS benefits from disabled THP in terms of performance. As CS is only a small percentage of jobs and GPU-time on our cluster, we didn’t investigate it deeper.
As for the DB size - sky is the limit and it only depends on the number of projects and jobs. In our setup, we have separate CS instance (so also DB) for every user or small research group (around 40 actively used instances). The largest single-user DB now is over 200GB, medium ones are usually 50-100 GB.
Our DB is just under 3TB in size.
I think the interesting point would be database size for how many projects/jobs run? And do you see a reduction in database size when projects are detached?
Our largest database is currently ~350GB (25 projects, some with 50 jobs, some with ~2,000…), next largest is ~100GB.
Detaching or archiving projects alone doesn’t remove anything from the database. You will also have to use Delete Project from Database option and/or perform mongodb compacting.
Correct. Please find details in the CryoSPARC guide.