Cache Management Causing RAM Saturation and Job Freezes

Install, Configure and Update
1

Hello CryoSPARC Community,

I am currently experiencing an issue with CryoSPARC where the buffer/cache fills up the RAM very quickly during jobs like Refinement, leading to job freezes. When I run free -h, it shows that my RAM is completely filled (250G), which necessitates frequent manual cache clearing to continue my work.

To mitigate this, I’ve set up an automated script to clear the cache every hour using the following crontab entry:

0 * * * * /bin/bash -c 'free -g && sync && echo 3> /proc/sys/vm/drop_caches && echo " && free -g'

While this provides a temporary solution, it’s not ideal as it requires constant intervention and could potentially disrupt other processes.

System Specifications:

  • Operating System: Rocky Linux release 8.9 (Green Obsidian)
  • GPU: 4x NVIDIA GeForce RTX 4090
  • CryoSPARC Version: 4.4.1
  • RAM: 250G

Questions and Requests:

  1. Has anyone encountered a similar issue? If so, how did you resolve it?
  2. Is there a more efficient way to manage the cache within CryoSPARC to prevent RAM saturation?

I’m eager to find a more sustainable solution to this issue and would greatly appreciate any advice or suggestions.

Best
Wen

2

Welcome to the forum @tosyl

The cache to which you refer is expected to be managed by the operating system, and is distinct from the particle cache, which is managed by CryoSPARC.
What is the output of the command

cat /sys/kernel/mm/transparent_hugepage/enabled

?

3

Thank you for your prompt response!
Here is the output of the command:

[always] madvise never
4

Set it to never… it’ll speed up a lot of jobs as well if running them simultaneously.

:slight_smile:

5

Thanks! But transparent hugepages have already been disabled.

[always] madvise never
6

indicates that transparent_hugepage has not been disabled. The expected output on a computer with disabled transparent_hugepage should look like

always madvise [never]

(discussion).

7

As @wtempel points out, [always] means enabled. Set it to [never].

1 Like
8

Sorry for misunderstanding! I’m definitely going to give it a try. Appreciate the help!

1 Like