Issues with cryosparc2 compiled with cuda version 8

open

#1

ok, next problem… running cryosparc compiled with cuda 8 actually is causing problems.

In running 2D classification, I get this error

Traceback (most recent call last):
  File "cryosparc2_worker/cryosparc2_compute/run.py", line 78, in cryosparc2_compute.run.main
  File "cryosparc2_worker/cryosparc2_compute/jobs/class2D/run.py", line 310, in cryosparc2_compute.jobs.class2D.run.run_class_2D
  File "cryosparc2_worker/cryosparc2_compute/engine/engine.py", line 923, in cryosparc2_compute.engine.engine.process
  File "cryosparc2_worker/cryosparc2_compute/engine/engine.py", line 933, in cryosparc2_compute.engine.engine.process
  File "cryosparc2_worker/cryosparc2_compute/engine/cuda_core.py", line 126, in cryosparc2_compute.engine.cuda_core.allocate_gpu
  File "/opt/cryosparc2/cryosparc2_worker/deps/anaconda/lib/python2.7/site-packages/pycuda/gpuarray.py", line 549, in fill
    func = elementwise.get_fill_kernel(self.dtype)
  File "<decorator-gen-13>", line 2, in get_fill_kernel
  File "/opt/cryosparc2/cryosparc2_worker/deps/anaconda/lib/python2.7/site-packages/pycuda/tools.py", line 430, in context_dependent_memoize
    result = func(*args)
  File "/opt/cryosparc2/cryosparc2_worker/deps/anaconda/lib/python2.7/site-packages/pycuda/elementwise.py", line 496, in get_fill_kernel
    "fill")
  File "/opt/cryosparc2/cryosparc2_worker/deps/anaconda/lib/python2.7/site-packages/pycuda/elementwise.py", line 161, in get_elwise_kernel
    arguments, operation, name, keep, options, **kwargs)
  File "/opt/cryosparc2/cryosparc2_worker/deps/anaconda/lib/python2.7/site-packages/pycuda/elementwise.py", line 147, in get_elwise_kernel_and_types
    keep, options, **kwargs)
  File "/opt/cryosparc2/cryosparc2_worker/deps/anaconda/lib/python2.7/site-packages/pycuda/elementwise.py", line 75, in get_elwise_module
    options=options, keep=keep)
  File "/opt/cryosparc2/cryosparc2_worker/deps/anaconda/lib/python2.7/site-packages/pycuda/compiler.py", line 291, in __init__
    arch, code, cache_dir, include_dirs)
  File "/opt/cryosparc2/cryosparc2_worker/deps/anaconda/lib/python2.7/site-packages/pycuda/compiler.py", line 254, in compile
    return compile_plain(source, options, keep, nvcc, cache_dir, target)
  File "/opt/cryosparc2/cryosparc2_worker/deps/anaconda/lib/python2.7/site-packages/pycuda/compiler.py", line 78, in compile_plain
    checksum.update(preprocess_source(source, options, nvcc).encode("utf-8"))
  File "/opt/cryosparc2/cryosparc2_worker/deps/anaconda/lib/python2.7/site-packages/pycuda/compiler.py", line 55, in preprocess_source
    cmdline, stderr=stderr)
CompileError: nvcc preprocessing of /tmp/tmp_CN9lp.cu failed
[command: nvcc --preprocess -arch sm_70 -I/opt/cryosparc2/cryosparc2_worker/deps/anaconda/lib/python2.7/site-packages/pycuda/cuda /tmp/tmp_CN9lp.cu --compiler-options -P]
[stderr:
nvcc fatal   : Value 'sm_70' is not defined for option 'gpu-architecture'
]

Looks like there are flags incompatible with cuda8, i.e. sm_70?
Where are compiler options set for cuda libraries?
Thanks


#2

I found a previous topic that addresses this:

However, I followed all the instructions to point cryosparc to the right cuda libraries, I check that the paths were correct, uninstalled pycuda and resinstalled with the appropriate paths, but still… same error.

config file in cryosparc2_worker/config.sh also points to cuda 8 libraries. So where is conflict? I can’t find it.


#3

Hi @ekellogg,

This error arises when new GPU’s are used with older versions of CUDA.