Rtx4090 suddenly became 10 times slower

Install, Configure and Update
1

Hi All,

We recently got some new rtx4090s GPUs. After installing cryosparc and updating to the newest version everything was extremely quick. Most jobs would finish ~5 times faster than on a 2080 for example. However, out of a sudden the 4090s choked and are now even slower than the 2080s (especially for 2D classification, ab-initio, 3DVA and NU refinement; homogenous refinement seems to take roughly the same time). I was wondering if anyone else has experienced such an issue before. These are the drivers we have (compatible with CUDA 11.8):


>+-----------------------------------------------------------------------------+
| NVIDIA-SMI 520.56.06    Driver Version: 520.56.06    CUDA Version: 11.8     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA GeForce ...  On   | 00000000:41:00.0 Off |                  Off |
|  0%   49C    P2    71W / 450W |    555MiB / 24564MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
|   1  NVIDIA GeForce ...  On   | 00000000:61:00.0 Off |                  Off |
|  0%   45C    P2    36W / 450W |    670MiB / 24564MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

Any ideas as to why this may be happening will be greatly appreciated!

Daniel

2

Verify that you have enough free space for the database.
We encountered a similar behavior when we ran out of disk space for the database.

3

I thought this may be the issue and even moved the database to a new location but unfortunately for us it didn’t resolve the problem, in both directories I have plenty of free space.