2D class - "2 errors detected in the compilation of "/tmp/tmpxft_00002cbf_00000000-6_kernel.cpp1.ii"

Troubleshooting
1

I got the following error when performing 2D Classification. I run Patch CTF, Blob picking, inspection, and extraction using the same node without any GPU errors. Any idea what is going wrong now?

2

Hi @stavros, could you please paste the full traceback? Thanks

4

Sure:

[CPU: 9.65 GB] Traceback (most recent call last):
File “cryosparc2_compute/jobs/runcommon.py”, line 1678, in run_with_except_hook
run_old(*args, **kw)
File “cryosparc2_master/cryosparc2_compute/engine/cuda_core.py”, line 128, in cryosparc2_compute.engine.cuda_core.GPUThread.run
File “cryosparc2_master/cryosparc2_compute/engine/cuda_core.py”, line 129, in cryosparc2_compute.engine.cuda_core.GPUThread.run
File “cryosparc2_worker/cryosparc2_compute/engine/engine.py”, line 1082, in cryosparc2_compute.engine.engine.process.work
File “cryosparc2_worker/cryosparc2_compute/engine/engine.py”, line 645, in cryosparc2_compute.engine.engine.EngineThread.backproject
File “cryosparc2_worker/cryosparc2_compute/engine/cuda_kernels.py”, line 1656, in cryosparc2_compute.engine.cuda_kernels.backproject
File “cryosparc2_master/cryosparc2_compute/engine/cuda_core.py”, line 380, in cryosparc2_compute.engine.cuda_core.context_dependent_memoize.wrapper
File “cryosparc2_worker/cryosparc2_compute/engine/cuda_kernels.py”, line 1562, in cryosparc2_compute.engine.cuda_kernels.get_backproject_kernels
File “/home/stavros/apps/cryosparc/cryosparc2_worker/deps/anaconda/lib/python2.7/site-packages/pycuda/compiler.py”, line 291, in init
arch, code, cache_dir, include_dirs)
File “/home/stavros/apps/cryosparc/cryosparc2_worker/deps/anaconda/lib/python2.7/site-packages/pycuda/compiler.py”, line 254, in compile
return compile_plain(source, options, keep, nvcc, cache_dir, target)
File “/home/stavros/apps/cryosparc/cryosparc2_worker/deps/anaconda/lib/python2.7/site-packages/pycuda/compiler.py”, line 137, in compile_plain
stderr=stderr.decode(“utf-8”, “replace”))
CompileError: nvcc compilation of /tmp/tmphLiw6P/kernel.cu failed
[command: nvcc --cubin -arch sm_30 -I/home/stavros/apps/cryosparc/cryosparc2_worker/deps/anaconda/lib/python2.7/site-packages/pycuda/cuda kernel.cu]
[stderr:
kernel.cu(50): error: identifier “__ldg” is undefined

kernel.cu(154): error: identifier “__ldg” is undefined

2 errors detected in the compilation of “/tmp/tmpxft_00000f6d_00000000-6_kernel.cpp1.ii”.

5

Hi @stavros,

What model of GPU are you using? From the error message, it looks like the GPU is quite old - cryoSPARC requires compute capability 3.5 or higher.

Harris

1 Like
6

Hello @hsnyder
We are using a system with both a GTX680, and RTX2080. Nvidia-smi outputs the following:

NVIDIA-SMI 418.40.04 Driver Version: 418.40.04 CUDA Version: 10.1

If I only select the RTX2080 GPU then I get the following:

CompileError: nvcc preprocessing of /tmp/tmpaVnN_X.cu failed
[command: nvcc --preprocess -arch sm_75 -I/home/stavros/apps/cryosparc/cryosparc2_worker/deps/anaconda/lib/python2.7/site-packages/pycuda/cuda /tmp/tmpaVnN_X.cu --compiler-options -P]
[stderr:
nvcc fatal : Value ‘sm_75’ is not defined for option ‘gpu-architecture’
]

7

Hi @stavros,

This error looks like cryoSPARC isn’t using CUDA 10. You can recompile cryoSPARC to use CUDA 10 by:

Navigate to cryosparc2_worker
eval $(./bin/cryosparcw)
./bin/cryosparcw newcuda <path-to-CUDA-10>

More details here

1 Like
8

When I proceed to do that I get the following:

WARNING: Requirement ‘deps_bundle/python/python_packages/pip_packages/pycuda-2019.1.tar.gz’ looks like a filename, but the file does not exist

Processing ./deps_bundle/python/python_packages/pip_packages/pycuda-2019.1.tar.gz

ERROR: Could not install packages due to an EnvironmentError: [Errno 2] No such file or directory: ‘/home/stavros/apps/cryosparc/cryosparc2_worker/deps_bundle/python/python_packages/pip_packages/pycuda-2019.1.tar.gz’

and now nothing works.

I tried solution here:

It completes installation, but I still get the same error.