Python dependency for v 2.15/'Homogeneous Refinement (NEW!)'?

Install, Configure and Update
1

I am unable to run a Homogeneous Refinement (NEW!) job using either my current 2.15 installation nor when I previously built the job using version 2.14; Homogeneous Refinement (Legacy) completes fine.

The job’s error log (pasted below) suggests that it may have something to do with the original installation of cryosparc, which contains and points to python 2.7. I have python 3.7 installed on the workstation for cryosparc_user/anacanda3 but the cryosparc worker references python 2.7. See error log below, which I think implicates the wrong or unusable python reference, but I really don’t know. Is it necessary to uninstall cryosparc and reinstall with newest python, or is it possible to update python in the cryosparc deps directory?

[CPU: 2.65 GB]   Traceback (most recent call last):
  File "cryosparc2_compute/jobs/runcommon.py", line 1685, in run_with_except_hook
    run_old(*args, **kw)
  File "/home/cryosparc_user/software/cryosparc/cryosparc2_worker/deps/anaconda/lib/python2.7/threading.py", line 754, in run
    self.__target(*self.__args, **self.__kwargs)
  File "cryosparc2_worker/cryosparc2_compute/engine/newengine.py", line 1691, in cryosparc2_compute.engine.newengine.process.work
  File "cryosparc2_worker/cryosparc2_compute/engine/newengine.py", line 1731, in cryosparc2_compute.engine.newengine.process.work
  File "cryosparc2_worker/cryosparc2_compute/engine/newengine.py", line 879, in cryosparc2_compute.engine.newengine.EngineThread.compute_resid_pow
  File "cryosparc2_worker/cryosparc2_compute/engine/newcuda_kernels.py", line 3326, in cryosparc2_compute.engine.newcuda_kernels.compute_resid_pow
  File "cryosparc2_worker/cryosparc2_compute/engine/cuda_core.py", line 362, in cryosparc2_compute.engine.cuda_core.context_dependent_memoize.wrapper
  File "cryosparc2_worker/cryosparc2_compute/engine/newcuda_kernels.py", line 3279, in cryosparc2_compute.engine.newcuda_kernels.get_compute_resid_pow_kernel
  File "/home/cryosparc_user/software/cryosparc/cryosparc2_worker/deps/anaconda/lib/python2.7/site-packages/pycuda/compiler.py", line 291, in __init__
    arch, code, cache_dir, include_dirs)
  File "/home/cryosparc_user/software/cryosparc/cryosparc2_worker/deps/anaconda/lib/python2.7/site-packages/pycuda/compiler.py", line 254, in compile
    return compile_plain(source, options, keep, nvcc, cache_dir, target)
  File "/home/cryosparc_user/software/cryosparc/cryosparc2_worker/deps/anaconda/lib/python2.7/site-packages/pycuda/compiler.py", line 137, in compile_plain
    stderr=stderr.decode("utf-8", "replace"))
CompileError: nvcc compilation of /tmp/tmpZxoJ3B/kernel.cu failed
[command: nvcc --cubin -arch sm_61 -I/home/cryosparc_user/software/cryosparc/cryosparc2_worker/deps/anaconda/lib/python2.7/site-packages/pycuda/cuda kernel.cu]
[stderr:
kernel.cu(131): error: identifier "__shfl_down_sync" is undefined

kernel.cu(256): error: identifier "__shfl_down_sync" is undefined

kernel.cu(318): error: identifier "__shfl_down_sync" is undefined

kernel.cu(382): error: identifier "__shfl_down_sync" is undefined

kernel.cu(569): error: identifier "__ballot_sync" is undefined

5 errors detected in the compilation of "/tmp/tmpxft_0000118b_00000000-7_kernel.cpp1.ii".
]
2 Likes
2

Hi @earturo, this error indicates that your CUDA installation is out of date. The newest releases of cryoSPARC require recent CUDA features (version ≥9.2 and ≤10.2) for some newer jobs like new Homogeneous Refinement and 3D Variability. Some jobs may still work with old CUDA but most jobs will drop support for older CUDA versions in upcoming releases of cryoSPARC.

I suggest upgrading to a newer CUDA version like 10.2 (note that cryoSPARC doesn’t support CUDA 11 yet).

cryoSPARC comes with its own installation of Python 2.7, which is why you are seeing the version mismatch.

Hope that helps,

Nick

3

Thanks! I’ll look into your advice regarding a CUDA upgrade. And I guess these refinement jobs can still run on cryosparc with python 2.7?

4

That’s right, no need to change anything about your Python installation

5

Resolved the issue by reinstalling cryosparc and providing the correct CUDA path. There was CUDA 8.0 also installed on the computer and there were other non-cryosparc related issues on the computer. All of these could have caused the conflict and the error. All of these were fixed by the reinstallation.