CUDA Error & get_gpu_info

Install, Configure and Update
1

Hi, I’m having an issue with CUDA 11.8.
When I tried to use the filament tracer, I got an error.

The Error code was like blow:

Traceback (most recent call last):
  File "cryosparc_master/cryosparc_compute/run.py", line 129, in cryosparc_master.cryosparc_compute.run.main
  File "cryosparc_master/cryosparc_compute/jobs/template_picker_gpu/run.py", line 56, in cryosparc_master.cryosparc_compute.jobs.template_picker_gpu.run.run
  File "cryosparc_master/cryosparc_compute/jobs/template_picker_gpu/run.py", line 99, in cryosparc_master.cryosparc_compute.jobs.template_picker_gpu.run.do_pick
  File "cryosparc_master/cryosparc_compute/jobs/template_picker_gpu/run.py", line 347, in cryosparc_master.cryosparc_compute.jobs.template_picker_gpu.run.do_pick
  File "/home/sang/sw/cryosparc/cryosparc_worker/cryosparc_compute/skcuda_internal/fft.py", line 112, in __init__
    self.handle = cryosparc_gpu.gpufft_get_plan(
RuntimeError: cuda failure (driver API): cuCtxGetDevice(&device)
-> [unknown error code]

and I found another error:

(base) sang@DESKTOP-6FUA02T:~/sw/cryosparc/cryosparc_master$ ./bin/cryosparcm cli "get_gpu_info()"
None

And I share my workstation info as below:

single workstation

  1. nvidia-smi
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 520.56.06    Driver Version: 522.30       CUDA Version: 11.8     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA GeForce ...  On   | 00000000:01:00.0  On |                  N/A |
|  0%   42C    P8    39W / 215W |    602MiB /  8192MiB |      1%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|    0   N/A  N/A       386      G   /Xwayland                       N/A      |
+-----------------------------------------------------------------------------+
  1. nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2022 NVIDIA Corporation
Built on Wed_Sep_21_10:33:58_PDT_2022
Cuda compilation tools, release 11.8, V11.8.89
Build cuda_11.8.r11.8/compiler.31833905_0

I’ll be waiting for your reply.

2

Welcome to the forum @sang.

Please can you post the outputs of these commands:

nvidia-smi --query-gpu=name --format=csv
uname -a
cat /etc/*release
3

Thank you for reply. @wtempel

The output is:

(base) sang@DESKTOP-6FUA02T:/mnt/c/Users/user$ nvidia-smi --query-gpu=name --format=csv

name
NVIDIA GeForce RTX 2070 SUPER

(base) sang@DESKTOP-6FUA02T:/mnt/c/Users/user$ uname -a

Linux DESKTOP-6FUA02T 5.15.167.4-microsoft-standard-WSL2 #1 SMP Tue Nov 5 00:21:55 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

(base) sang@DESKTOP-6FUA02T:/mnt/c/Users/user$ cat /etc/*release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=22.04
DISTRIB_CODENAME=jammy
DISTRIB_DESCRIPTION="Ubuntu 22.04.5 LTS"
PRETTY_NAME="Ubuntu 22.04.5 LTS"
NAME="Ubuntu"
VERSION_ID="22.04"
VERSION="22.04.5 LTS (Jammy Jellyfish)"
VERSION_CODENAME=jammy
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=jammy
4

Thanks @sang for posting this additional information.

We have not tested running CryoSPARC under WSL2, but a user may have succeeded with an older version of CryoSPARC (Patch CTF failing with many errors including cuda version mismatch - #2 by reza092).

This amount of VRAM is smaller than the currently published minimum, which may cause certain GPU-enabled CryoSPARC jobs to fail.

5

It has been resolved via downgrading to 4.3.1 version.

I really appreciated it.

Good Luck with everything. :slight_smile: