Queue message [Errno 24] Too many open files:

Hi @Pielhaas,

I apologize for the delay in responding to you on this issue!

We did make a change in v4.6 that keeps more file descriptors open. We are still looking into how we can make this easier for users in the future, but in the meantime here is some information that may allow you to work around the problem:

There are several things that limit the number of open file descriptors on Linux. First there is the system-wide maximum number of file descriptors. This is usually a very large number and I doubt it would be the problem in this case, but just to be sure, I recommend checking it

cat /proc/sys/fs/file-max

Then there are two per-process limits on the number of open file descriptors: the so-called hard limit and the soft limit. We only need to be concerned with the hard limit, because CryoSPARC automatically sets the soft limit as high as it is allowed to. You can check what the hard limit is by running the following command while logged in as the cryosparc user.

ulimit -Hn

If this is a low number (e.g. 1024), it could be causing problems. The hard limit can be raised by following the instructions here. I would recommend setting both the hard and soft limits to whatever the following command outputs

cat /proc/sys/fs/nr_open

Hopefully this helps. One user who tried these instructions on AWS reported that they also needed to make changes to two files in /etc/security/limits.d: 00_all_limits.conf, and 01_efa.conf, since those were interfering with the values in /etc/security/limits.conf, so that might be something to be aware of depending on your Linux distribution.

–Harris

1 Like