Advices for large volume 3D alignment

Utilities
1

Hi everyone,

I’m trying to align multiple 3D volumes (different refinement metaparameters) to a reference volume with correct chirality and a known pdb model fitting it, to check manually which metaparameters give better details in the map.

Whereas this approach worked perfectly for 320 px box on a different project, for a larger box (640 px) on another project I get:

Traceback (most recent call last):
  File "cryosparc_worker/cryosparc_compute/run.py", line 93, in cryosparc_compute.run.main
  File "cryosparc_worker/cryosparc_compute/jobs/align_3D/run.py", line 71, in cryosparc_compute.jobs.align_3D.run.run_align_3D
  File "cryosparc_worker/cryosparc_compute/jobs/align_3D/run.py", line 146, in cryosparc_compute.jobs.align_3D.run.align3D
  File "/opt/cryosparc/cryosparc_worker/cryosparc_compute/align3D_gpu.py", line 121, in align_volumes
    errs = compute_align_errs(rs,ts,gfV1,gfV2,radwn,cuda_dev=cuda_dev,stream=stream)
  File "/opt/cryosparc/cryosparc_worker/cryosparc_compute/align3D_gpu.py", line 30, in compute_align_errs
    gpuarray.empty((N_R,N_T_aligned),n.float32))
  File "/opt/cryosparc/cryosparc_worker/deps/anaconda/envs/cryosparc_worker_env/lib/python3.7/site-packages/pycuda/gpuarray.py", line 210, in __init__
    self.gpudata = self.allocator(self.size * self.dtype.itemsize)
pycuda._driver.MemoryError: cuMemAlloc failed: out of memory

which is obviously a memory issue.
Is there a way I could keep the alignment within CryoSPARC with the current box size, or should I switch to other tools? And are there any tools that can do it in automated fashion – so far I’ve done only semi-manual ChimeraX volume alignment, which seems tedious for ~10 volumes🥲

2

if this could be recorded as a feature request: it could be super convenient if the “Align 3D” had an option to up/down sample all the input maps to a necessary size, then align and apply the transformation matrices to the original maps. It would solve both the above mentioned topic problem, as well as my other issue with maps being of a slightly different size and hence me running “Volume tools” for each of them separately every time🥲

2 Likes
3

Have you tried Situs?

4

you’re referring to this tool?

If yes – no, I haven’t tried it. It’s unclear from the user guide, how’d one use it to align 2 volumes onto each other – could you perhaps navigate me to it?

5

I thought you could use the matchpt program to make a point cloud representation of maps and then align them, but maybe one of the targets has to be a PDB model after all. Chimera(X) also has a global search mode for fitmap, to avoid the manual placement, that might work for you as well.

6

@team are there any news on this?

I would like to align two maps and update particle alignments. Even an RTX 3090 (24GB VRAM) cannot handle alignment of maps with box size 720px.

How to about this?

7

Two ideas:

  1. For such a large particle, alignments from a 1.5x or 2x downsample might be just as good, or keep a reasonably Nyquist relative your obtained resolution. (E.g at 0.8 A/px 720px by 1.5 is 3.2A Nyquist and only 480px). If necessary do the alignment this way and then re-refine one of the jobs at full resolution.

  2. You could align the volumes in Chimera(X) and use another tool like subparticles.py (or probably Oxford localrec) to transform the particles using that matrix

1 Like
8

Hi Daniel,

Those are two very suggestions, but I have been inclined to not take it. The 3D volumes to align are very high-resolution (almost hiting Nyquist) and I need a garanteee that the aligment is going to correspond to the quality of the data to not hinder incorrect interpretation on the next phase. But I have not tried that path yet… and to be honest neither did I try cryoSPARC’s “Align 3D maps” job, so I don’t know how it behaves and how good it performs. @DanielAsarnow do you have any experience with it?

Perhaps the @team could explain a bit of the strategy behind this job?

Regarding option 2, it is something I did not explore yet, but it has more steps and requires at least 3 different pieces of software and the workflow in between which i wanted to avoid. Still I am willing to try this less immediate workflow when I have time. I am always happy to use pyem :slight_smile:

Have a great weekend!

9

Probably the alignment can be at much lower resolutions than you think and still produce the same high-resolution structure. E.g. limiting alignments to 12 A can often still produce a 3 A or better structure. Then once you have an aligned reference, you can re-run a local refinement of the full size data with an initial low-pass resolution just a bit lower than the target res. The map will stay aligned.

If you just want your refinement output to align with another map, you can just directly use that other map (or a manually aligned copy of the output map) as the reference, too (with a global search).

But if you haven’t actually tried the align maps tool than what is the issue you’re running into?

10

Hi @AndreGraca et al.! Just bumping this thread to mention that there is a (relatively new) option in the align 3D job to also update particle alignments.

Potential workflow for larger volumes: 1) downsample volumes, 2) run align 3D with Update particle alignments turned on (supply an original resolution particle group for each aligned map), 3) run homogeneous reconstruction only on each of the particle sets to recover the higher resolution maps.

Valentin

1 Like
11

Hi Daniel (@DanielAsarnow)! Seems like I missed reading your reply until I got a notification regarding Valentin (@vperetroukhin) reply. Thank you to both of your for your replies, I will address each of you individually, although there are common points.

Daniel (@DanielAsarnow), am trying indeed going that way with a local refinement.

By the time I last replied to you I had not yet downsampled the volume, so the problem was still the same:

Now I have aligned the maps based on 2x volume downsampling.

Valentin (@vperetroukhin), indeed I was from the very beginning trying the update particle alignments with the Align 3D Maps job. It is very easy to use this tool to very quickly align in one go several volumes and corresponding particles to a reference volume. It is a great tool, however when starting from very high-resolution maps ~2Å, it roughly superimposes the maps, but not a perfect alignment. For instance, check the following figure:

image
image871×649 167 KB

The output of the Align 3D Maps job are the volumes in this figure (downsampled 2x!), reference map in blue and two maps aligned in yellow and grey.

I wish to have obtained a better alignment, but I am not sure this is possible for high-res volumes using this tool. What I did not try is to lowpass filter the maps before using Align 3D Maps (I only resampled to the box size to half of its original volume), exactly the workflow you suggested.

Regarding the last step of your workflow, Valentin,

I am running a local refinement instead of an homogenous refinement because the particle set is of very low-contrast and I have experienced that other refinements will drive the reconstruction to lower resolutions than the original 3D reconstructions.

I will play a little bit more with this and let you know the results, if you will.

Thank you!
André

12

@AndreGraca ok, thanks for the detailed write up! The current Align 3D algorithm definitely leaves some room for improvement – we’ve added this to our near-term roadmap.

1 Like
13

Hi all,

CryoSPARC v4.5, released May 7 2024, now has improvements to the align 3D algorithm to make it more robust, along with changes to the input and output processing; more details are available in the job’s info page.

In addition, the behaviour of Align 3D has been updated:

  • Volumes connected to the “Maps to align” input can contain multiple slots, and each of these will be transformed according to the optimal alignments found by the job
  • Specifically, a volume’s map slot is the one used for alignment – this can be from a downsampled job
  • All other connected slots (map_half_A, map_half_B, map_sharp, etc.) are transformed according to the optimal alignments, but are not independently aligned

If you run into memory issues, you can try downsampling the volumes connected to the map slot, and connecting the non-downsampled input to any of the auxiliary inputs (for example, map_filtered as shown in the screenshot below). All of the connected auxiliary inputs will be present in the outputs.

Screen Shot 2024-05-07 at 12.24.09 PM
Screen Shot 2024-05-07 at 12.24.09 PM712×1210 68 KB

Best,
Michael

3 Likes
14

How to align multiple 3D volumes (different refinement metaparameters) to a reference volume or model with correct chirality and a known pdb model fitting it?

15

Hi @iram,

You can connect multiple maps to an align3D job and they will all be aligned to the reference provided. In the case that you want to use a PDB model as a reference, you would first need to convert it to a volume using the molmap command in ChimeraX, and then import that volume to your project.

All the best,
Kye

1 Like