Hello,
I was examining the cryosparc shell script in the cryosparc/bin directory and saw that if you run cryosparc tuning (As root or with sudo) all it does is place the following lines in root’s crontab
* * * * * sync && echo 3 > /proc/sys/vm/drop_caches
@reboot nvidia-smi -pm 1
So every minute the computer is flushing file system buffers and then it frees pagecache, dentries, and inodes. Why? This will cause an overall decrease in system performance because this is dumping the cache the kernel has built up. This makes sense to do before you start benchmarking disk IO but not every minute.
The other portion of the tuning process makes sense but I am puzzled why the first line is included.
Hi @clil16,
We found that clearing the caches regularly significantly improved performance on our systems because otherwise, the OS would cache particle stack files in RAM as they were read (as part of the normal file cache in case the same files need to be read again) but as soon as RAM would fill up with this background cache, there was a large performance hit for allocating new memory or reading new data off disk, presumably because the system had to suffer lots of TLB misses during the background act of clearing out the RAM cache when RAM was full. At least that’s my guess - I’m not completely clear with the inner workings of the Linux kernel and this observation was almost entirely empirical. Not clearing the cache often led to identical experiments taking double or more wall time. And as far as I am aware, we have not found any performance degradation with these tuning options enabled.
Does your experience differ? We’d be glad to know, especially if you have other suggestions on how systems can be tuned for this workload.
Thanks,
Ali