Reference based motion correction failed - Cuda issue?

Sure. Sorry, this is going to be a bit long…

Could have been clearer - in CryoSPARC, no advantage.

In RELION they do - I’ve pushed box sizes up to 1,480 pixels (with no Xorg or other GPU-utilising programs running) and successfully completed 3D refinement converging at 0.02 degrees - however, CryoSPARC has some other limits (due to PyFFTW and other code I think) which means that 1,100 pixel boxes (or very close) are the maximum I’ve had run successfully in CryoSPARC - even 1,122 pixel would crash (don’t ask me how much time I’ve spent experimenting with this… :rofl:)

If I was after STA, A6000s would be a good choice (our tomo box right now is their older sibling, the RTX8000 48GB). If using RELION and boxes <1,450 pixels, A6000s again are a great choice. If you want to go 1,500+ pixels, you need to think about going the CPU codepath in RELION… or the 80GB GPUs if money truly is no object! But from my estimates the 80GB GPUs will top out around 1,600-1,700 pixels anyway.

The largest box I’ve tested and successfully reconstructed - which is going to be great fun to deposit to EMDB - was 1,800 pixels and each half-set MPI process was eating ~270GB system RAM during later steps of the 3D refinement. Memory requirements quickly get out of hand after that.

Since the AVX-512 code in RELION was - if I remember correctly - written by Intel, I need to see if RELION can be compiled to well use the AVX-512 hardware in AMDs new Zen 4 (now) / Zen 5 (next year) or if so much of it comes from the Intel compiler and/or Intel still do their old tricks and deliberately de-optimise for anything not-Intel. And/or how GCC 13/14 handle it. But no time.

As it is, if shopping for a huge-box-system right now, I’d look at a dual-socket Emerald Rapids setup with 2-4TB of RAM and probably quad 24GB GPUs so that you can use GPUs for smaller box situations, then switch to AVX-512 code for the final refinements on fully unbinned particles.

Directly related to these two threads:

Sorry for the potential confusion around my earlier comment. :frowning_face:

1 Like