Template Picker error: cufftAllocFailed

open

#1

Hi,

When I select 3+ 2D classes to use for the Template Picker I get this error:

Traceback (most recent call last): File “cryosparc2_worker/cryosparc2_compute/run.py”, line 78, in cryosparc2_compute.run.main File “cryosparc2_worker/cryosparc2_compute/jobs/template_picker_gpu/run.py”, line 112, in cryosparc2_compute.jobs.template_picker_gpu.run.run File “cryosparc2_worker/cryosparc2_compute/jobs/template_picker_gpu/run.py”, line 218, in cryosparc2_compute.jobs.template_picker_gpu.run.run File “/CryoSparc/CryoSparcV2/cryosparc2_worker/deps/anaconda/lib/python2.7/site-packages/skcuda/fft.py”, line 126, in init onembed, ostride, odist, self.fft_type, self.batch) File “/CryoSparc/CryoSparcV2/cryosparc2_worker/deps/anaconda/lib/python2.7/site-packages/skcuda/cufft.py”, line 741, in cufftMakePlanMany cufftCheckStatus(status) File “/CryoSparc/CryoSparcV2/cryosparc2_worker/deps/anaconda/lib/python2.7/site-packages/skcuda/cufft.py”, line 116, in cufftCheckStatus raise e cufftAllocFailed

I do not get this error when I use 3 or less 2D classes for the Template picker.

What can I do to fix this issue?


#2

Hi @chana this is unfortunate - we routinely use the particle picker with dozens of classes. Can you report the size of the micrographs, the size of the particles, the pixel size, your GPU hardware info, etc?
We recently fixed a bug (out in the last release) where K3 data was causing out-of-memory errors that were avoidable. I think in another thread you reported that you’re already on the latest version?


#3

Similar issue. K3 data cannot be processed unless binned. Also have an older data set with
px of 8964 13176 that has the same error. Even 2x binning this data has the same error. Everthing else appears fine. Running latest 2.5 with CUDA 10


#4

I observed that images that have this problem give the following info
“Computing cc with FFT size 4096” just before memory error. Might explain the problem
This is regardless of the particle size given or the actual image size. Even if binned 3 fold the same FFT size of 4096 shows up.
All other template picking routines have FFT sizes listed in the 100s.


#5

Hi all, this has been fixed - we had an optimization that was kicking in too aggressively (trying to process multiple templates at a time regardless of the FFT sizes and GPU memory) that has been turned off. To be released in the next release (soon)


#6

Thanks…hoping to use soon


#7

I still observe this error after the v2.8 update.


#8

Hi @MHB,

What GPU(s) are you using?


#9

We are using Geforce GTX 1080


#10

Also still same problem in 2.9.