ExecError: error invoking nvcc --preprocess

Troubleshooting
1

i upgraded my centos 7.3 to the latest packages and also cryosparc to todays version.

according to this forum i did:
cryosparc stop
eval $(cryosparc env)
pip uninstall pycuda
pip install pycuda --no-cache-dir

  • here it gets an error with cannot find cuda.h

then i did (because your tool needs it):
pip install pycuda

then:
export CFLAGS=-I/usr/local/cuda-8.0/targets/x86_64-linux/include; export LDFLAGS=-L/usr/local/cuda-8.0/targets/x86_64-linux/lib; cryosparc reinstallpycuda
and it finished without problem.

when we run a cryosparc experiment i get the following error after a while:

File “/home/opt/cryosparc/anaconda2/lib/python2.7/site-packages/pytools/prefork.py”, line 59, in call_capture_output
_ % (" ".join(cmdline), e))_
ExecError: error invoking ‘nvcc --preprocess -arch sm_61 -I/home/opt/cryosparc/anaconda2/lib/python2.7/site-packages/pycuda/cuda /tmp/tmp6k3KC9.cu --compiler-options -P’: [Errno 2] No such file or directory

i have a gtx1080 with sm61.
why does the software need to compile cuda stuff when its running?

2

Hi @wlugmayr,

We use PyCUDA internally which is a python binding around CUDA that performs just-in-time compilation of kernels as they are invoked. The install process doesn’t actually compile anything, all compilation happens the first time a kernel is run (then the compiled binaries are cached).

Perhaps you can add those two CFLAGS and LDFLAGS exports to the bash profile of the linux user that is running cryosparc? Then stopping and starting again should all work.

Please let us know if that works.
Thanks,
Ali

3

hi,

i put it into my .bashrc and it did not help.

also when installing pycuda (not from cache), it is compiling and has the same error. this is why i came to the idea to add the flags to the environment. so the pycuda with cryosparc pip stops without any problem. so why must it be recompiled again?

cheers,
wolfgang

4

hi,
do you have a solution yet?
otherwise nobody from our team can use your software!!!
cheers,
wolfgang

5

Hi @wlugmayr,

Thanks for your patience - we’re pretty swamped at the moment. It’s difficult to debug the system settings and environment remotely but can you try the following:

logged in as the Linux user that runs cryosparc:

which nvcc
(this should point to /usr/local/cuda/bin/nvcc)
eval $(cryosparc env)
which nvcc
(this should still point to the correct nvcc location)
nvcc --version
(this should report a version number)
echo $LD_LIBRARY_PATH
(this should contain /usr/local/cuda/lib64)
which pip
(this should be the pip inside the cryosparc installation folder)
echo $PYTHONPATH
(this should be empty unless you have some other software installed under this user that sets it)
cryosparc stop
pip uninstall pycuda
pip install pycuda --no-cache-dir
cryosparc start

the above is not very different from what you’ve already done, so I’m not sure what clues will be found. As far as I know we’ve never seen the cannot find cuda.h error before, nor had to set CFLAGS or LDFLAGS for installing pycuda. Generally in our tests having PATH and LD_LIBRARY_PATH set properly is all that is required.

An alternative is to reinstall cryosparc as a different linux user, then backup the database from your current instance and restore it in the new installation (this will require some thinking to make sure the bulk directory paths remain consistent)

Ali

6