Effect of adding indexes to MongoDB?

Troubleshooting
1

Hi,

I recently ran into some issues with a ‘Curate Exposures’ job on a session with 60k+ movies. I think it was running into default RAM limits for MongoDB for queries that require sorting. It sounds like adding some specific indices would help, but I wasn’t sure if cryoSPARC would like adding this type of behind the scenes tweaking, ie. because of impacts elsewhere like slower inserts and updates. I’d be interested in dev thoughts on tweaking MongoDB behind the scenes to try and optimize query performance or limitations like the one I ran into.

For now I just split the dataset in 8 and ran 8 separate curate jobs.

Cheers!
-Craig

2

Hi Craig, thanks for digging into this issue, it’s good that you have a workaround until we can provide a permanent fix. We’ll be taking a look at addressing this for a future release, will post an update here when we have one!