NU-refine fail - no heartbeat

Hi all,

Trying NU-refine on a big particle (~2.5 mda) that is very long and filament like. I picked a box size (1260 px, @ 0.959 A/px) that is large enough to see the repeating unit, but there are units outside the focal region. Because of this, the resolution on the outside unit is very low, while the middle is very good (2.6 A).



The NU-refinement starts iteration 0 but quickly fails, very likely running out of ram. Theres 120 gb. I am pushing for high resolution so down sampling doesn’t seem like a good option. Is there anything that can be tuned to make it run, even if it takes a long time?
Thanks

My experience with that box size is that you will need possibly > 250 GB RAM. Make sure Local and Global CTF optimizations are off, although if it’s failing that quickly, you probably haven’t gotten there yet.

1 Like

Thanks - thought so. Any idea of a box size that would work with 120 gb?

I believe we’ve gotten ~800 to run successfully.

Are you sure you are running out of RAM and not VRAM? What card are you using and what error do you get when it fails? With a 24GB card I know we have run at least 900px box size refinements, I think up to 1200px but would need to check that

Cheers
Oli

Could be that. Its a 3090 and when the job fails cryosparc locks up and kicks me out. After rebooting computer and restarting cryosparc there is this error only.

Job is unresponsive - no heartbeat received in 60 seconds.

I think ill just have to downsample.

That sounds more like a system RAM issue - if it’s running out of VRAM you will generally see the job die with a CuMemAlloc error message.

1 Like

Thanks for the insight. This is why I’m trying to downsample the particle in the other thread.

Hi @mjmcleod64 ! That’s an interesting-looking structure there, and it looks like the filament-like structure is somewhat flexible (there is map smearing visible). Unfortunately it sounds like you are running into issues with VRAM during your non-uniform refinement most likely because your box size is pretty big.

If you are looking to get fine details about the unit within your filament-like particle, In order to over-come the VRAM issue you are encountering, I suggest an approach where you refine 1 unit to high resolution, then separately refine more units together at lower resolution, to allow you to see the relative arrangement of the units. If you like, you can then go on to combine multiple copies of your high resolution units into a frankenstein/composite map.

To get a high resolution structure of one unit:

  1. Make sure that a single unit is in the centre of the refined volume box. If it is not, you can recenter by inputting your volume and particles and run the Volume align job, using ChimeraX to locate a reasonable centre position. Note that ChimeraX reports coordinates in Å, and CryoSPARC uses coordinates in pixels, so you need to include that conversion.
  2. Re-extract these particles with a large enough box for just a single unit: You can try this at 0.959 Å/pix as the box size required should be quite a bit smaller than your current box. If this still runs out of VRAM during NU-refinement, you need to decide if you prefer to crop in real space, or to Fourier crop (both may reduce the maximum achievable resolution but if you are at 2.6 Å, as @olibclarke mentioned you could afford to Fourier crop a little without hitting nyquist).
  3. Perform Non-Uniform refinement using your newly extracted particles and your recentred volume.
  4. Perform particle subtraction to include just the density for 1 unit
  5. Perform local refinement

To get a lower resolution map of multiple units:

  1. Extract separately your filament (however many units you wish to include) with a large box size and Fourier crop to a box size that your GPU can handle.
  2. Perform Homogeneous or Non-Uniform refinement
  3. You can then go on to combine multiple copies of your single unit by aligning them in your lower-resolution multi-unit map using ChimeraX or Phenix to make a composite/frankenstein map.
3 Likes