I have a project that contains 1600 jobs already. At some point, cryosparc interface became quite slow, such that any interaction takes a few seconds to complete. This includes adding jobs, adding or removing input slots, etc. This only happens in this project, not the other ones which contain far fewer jobs (<500). Is there something I can do to help the situation?
As far as I know, this is a scaling limitation with MongoDB. We have hundreds of projects and thousands of jobs and experience it as well.
Perhaps there is a way to rebuild the database indices and get some performance back. Also, storing the cryoSPARC database on a fast SSD seems to help somewhat.
1 Like
Hi @rbehrouzi, @DanielAsarnow,
Thanks for bringing this up. While ensuring MongoDB runs on an SSD will improve performance, it should easily be able to handle very large cryoSPARC instances. Within the cryoSPARC web application we are making a ton of updates to ensure the interface continues to be quick in scenarios where an instance has hundreds of projects and tens of thousands of jobs. This is done in a variety of ways: incrementally loading information in chunks, caching queries that take long to load, etc. These updates are scheduled for an upcoming release (no exact date yet!).
- Suhail
2 Likes