Template Picking For K3 Data

Particle Picking
1

Hello All,

I am attempting to work through a data set with K3 images(11520x8184 pixels) . My particles are pretty tricky to pick, but cryosparc2 helped when I had K2 data. Now that I have taken K3 data. The workstation has 128 GB RAM and 4 1080Ti, with 48 cores. The template picker is failing to process this data and I cant figure out why. It works when I run it with a single template, but crashed with 2 or more with the following error:

Two templates given —>
Traceback (most recent call last):
File “cryosparc2_worker/cryosparc2_compute/run.py”, line 78, in cryosparc2_compute.run.main
File “cryosparc2_worker/cryosparc2_compute/jobs/template_picker_gpu/run.py”, line 111, in cryosparc2_compute.jobs.template_picker_gpu.run.run
File “cryosparc2_worker/cryosparc2_compute/jobs/template_picker_gpu/run.py”, line 208, in cryosparc2_compute.jobs.template_picker_gpu.run.run
File “/home/cryosparc2/cryosparc2_worker/deps/anaconda/lib/python2.7/site-packages/skcuda/fft.py”, line 115, in init
onembed, ostride, odist, self.fft_type, self.batch)
File “/home/cryosparc2/cryosparc2_worker/deps/anaconda/lib/python2.7/site-packages/skcuda/cufft.py”, line 222, in cufftPlanMany
cufftCheckStatus(status)
File “/home/cryosparc2/cryosparc2_worker/deps/anaconda/lib/python2.7/site-packages/skcuda/cufft.py”, line 110, in cufftCheckStatus
raise cufftExceptions[status]
cufftAllocFailed

Three or more templates given —>
Traceback (most recent call last):
File “cryosparc2_worker/cryosparc2_compute/run.py”, line 78, in cryosparc2_compute.run.main
File “cryosparc2_worker/cryosparc2_compute/jobs/template_picker_gpu/run.py”, line 111, in cryosparc2_compute.jobs.template_picker_gpu.run.run
File “cryosparc2_worker/cryosparc2_compute/jobs/template_picker_gpu/run.py”, line 192, in cryosparc2_compute.jobs.template_picker_gpu.run.run
File “/home/cryosparc2/cryosparc2_worker/deps/anaconda/lib/python2.7/site-packages/pycuda/gpuarray.py”, line 210, in init
self.gpudata = self.allocator(self.size * self.dtype.itemsize)
MemoryError: cuMemAlloc failed: out of memory

I guess this issue is caused by the size of the new micrographs but I am not sure about that. Something I noticed is the micrographs appear ‘padded’ to make them square but this makes them 2-3x bigger (judging by the display in cryosparc2). I would like to avoid binning my micrographs before importing them into cryosparc but maybe that can fix the problem. Any help would be great !

Cheers
Adam

2

I’m running into the exact same issue. K3 data on a 2 1080Ti and 64GB RAM system fails the same way if I use more than 2 templates (MemoryError: cuMemAlloc failed: out of memory). This is in cryosparc2 2.5.0, have tried installing the worker with cuda 8, 9 and 10. Runs with 1 or 2 templates but as Adam says, the micrographs are padded on all sides so they are squared. Has anyone else ran into this and resolved it?

Thanks!
Miguel

3

Hi @mbr, @alyokom,

Thank you for reporting this. We’re currently looking into it!

4

@mbr @alyokom Can you confirm that in both cases, the data that failed was K3 super-resolution data?

5

Correct. My data was super resolution.

6

I had imported motion corrected sums produced by motioncor2 that are bin2 (4092 x 5760).
Interestingly, today another K3 dataset that’s a little different (5456 x 7680 and ~1000 micrographs instead of ~2000) worked fine: I imported it the same way and although the micrographs are still padded to a square, I auto-picked using 15 templates on the same machine without any errors.

7

As of v2.8.0, Patch Motion, Patch CTF and Particle Extraction jobs are optimized to be able to process K3 super resolution movies (~65 frames) on a 1080Ti or better.