Pycuda error “__shfl_down_sync” during Homogenous Refinement (NEW)

Troubleshooting
1

Hi all,

I just upgraded to v2.12 to try out the new homogenous refinement and the job had the following error:

CompileError: nvcc compilation of /tmp/tmpVTXdYy/kernel.cu failed
[command: nvcc --cubin -arch sm_61 -I/Data_HDD2/Programs/packages/cryosparc_v2/cryosparc2_worker/deps/anaconda/lib/python2.7/site-packages/pycuda/cuda kernel.cu]
[stderr:
kernel.cu(131): error: identifier “__shfl_down_sync” is undefined

kernel.cu(256): error: identifier “__shfl_down_sync” is undefined

kernel.cu(318): error: identifier “__shfl_down_sync” is undefined

kernel.cu(382): error: identifier “__shfl_down_sync” is undefined

kernel.cu(569): error: identifier “__ballot_sync” is undefined

5 errors detected in the compilation of “/tmp/tmpxft_00005552_00000000-7_kernel.cpp1.ii”.
]

Could you let me know how to fix this problem? Let me know if additional information is needed.

PS: this is one of our old gpu workstations, which is still running cuda 8.0.

Thanks,
Wei

2

I just updated to v2.12 and get the same error when I run local CTF refinement or on the fly CTF refinement within new homogeneous refinement.

I am also on Cuda 8.0.

-Aaron

3

Hi @user123, @wxh180,

You might have to install a newer version of CUDA. Is this possible?

4

Hi @stephan,
Yes. It is working well on our new GPU workstations with CUDA10. For this one, we probably will keep it at this old CUDA version to use some legacy softwares.
Thanks,
Wei

5

Hi @wxh180,

Thanks for the update. It seems like the new CTF Refinement code is only compatible with CUDA 10+. Is it possible to have multiple CUDA installations on the same machine?

6

Thanks for the tip! Must be time to update.
-Aaron