Alignments degrade after re-extracting particles

[cbeck]

Hi everyone!

After reading some posts on this forum on how to determine an adequate box size for a given particle, I concluded that my particle was too close to the edge of the box, which was probably cutting off delocalized high-resolution signal and stopping me from reaching a higher resolution.

After performing a Local Refinement (default parameters, box size 360, cropped to 256), I used Volume Align Tools to re-center the masked region to the center of the box (I connected the particles, volume, and mask from the previous local refinement). Next, I re-extracted the particles with a larger box size of 480, Fourier-cropped to 360 (recenter using aligned shifts ON). I then performed another Local Refinement (default parameters) on the re-extracted particles using the re-centered volume and mask to see if the resolution improved.

However, I noticed that the angle and shift changes were suddenly all over the place after the first iteration of local refinement, and I’m having trouble understanding why.

These are the alignment changes from the final iteration of Local Refinement before re-extracting particles:

These are the alignment changes from the first iteration of Local Refinement after re-extracting particles:

and the second iteration:

Performing a Homogeneous Reconstruction Only after re-extracting also results in a significantly worse map with a resolution of ~23 A, instead of the previous 3.17 A…

If I’m understanding this previous discussion correctly, Volume Alignment Tools will adjust alignments3D/shift as necessary and leave location/center_x_frac and location/center_y_frac unchanged. Since I activated recenter using aligned shifts in the Extraction Job, am I right in thinking that the new location/center_x_frac and location/center_y_frac values will be equal to the previous values plus the previous alignments3D/shift? I would also think that the new alignments3D/shift would be close to 0. I don’t think the poses should have changed after re-extracting, so I’m not really sure why the alignment shifts drift so drastically after re-extracting the particles, and the same goes for the alignment angles.

Right now, the only thing I can think of to do would be to perform another global alignment, but this isn’t desirable since I spent a lot of effort reaching these current alignments. I’d really appreciate any insights on how the alignment information changes after aligning and re-extracting particles and what I might be doing wrong.

Cheers,
cbeck

[olibclarke]

Hi @cbeck

This shouldn’t happen - reconstruction after subparticle extraction should give comparable resolution to whatever the masked resolution in the subparticle would have been in the global refinement. Not sure exactly what has gone wrong in this instance, but definitely this is not what we have seen. Perhaps double check that you had “recenter using aligned shifts” enabled during extraction?

Cheers
Oli

[cbeck]

Hi Oli,

The extraction job definitely has “Recenter using aligned shifts” enabled. Here are all the parameters I used:
Extraction box size: 480
Fourier crop to box size: 360
Save results in 16-bit floating point
Recenter using aligned shifts: On

Reconstructing after Volume Align Tools gives a slightly worse but similar resolution to the local refinement - 3.35 A vs 3.17 A. However, reconstructing after the extraction job gives a resolution of 24.18 A, so I guess something must be going wrong with the extraction step…

I’ve definitely re-extracted particles before and have never seen the alignments degrade like this, but this is also the first time I’ve re-extracted with a larger box. I’ll experiment with re-running the extraction job, but I’d appreciate any insights anyone might have.

Best,
cbeck

[cbeck]

I’ve tried a few steps to troubleshoot this, but I haven’t had any luck so far.

The re-extraction job originally had multiple sets of particles from different local refinements. I then used the Particle Sets Tool to filter the extraction job using the UIDs for one of the local refinements and used these re-extracted particles for a second local refinement. I wondered if something was going wrong with the Particle Sets Tool, so this time, I tried re-extracting from just one of the local refinements and directly proceeding to Homogeneous Reconstruction Only, but I’m still getting the same issue.

I also wondered if something was wrong with the source micrographs, because I used micrographs that came from an Exposure Group Utilities job, which I used to group exposures by their beam tilt. However, even if I use the original micrographs that weren’t assigned to exposure groups, I still get the same issue.

[kstachowski]

Hi @cbeck,

We have tried to recapitulate the issue that you are encountering, but have mostly failed to do so with two different targets and various box sizes, and particles/volume shifting. One thing we did find (and would be expected to produce poor results) is when using a static mask in homogeneous reconstruct that is not correctly aligned to the shifted particles, especially in cases where the physical box size is changed. Please see below:

In this workflow, I have performed the same jobs you mentioned (local refinement → volume alignment tools → extraction → reconstruct), but when I use the unshifted mask from the local refinement, I get a reconstructed volume that is at 18A. If i use the mask that was shifted with the particles in volume alignment tools, then I get the expected result. Performing a local refinement using the mask, volume, and particle output of J210 would also result in a poor quality map.

Therefore we have a couple questions/requests:

• Can you verify that your masks are properly aligned to the particles?
• What version of CS are you using?
• Did you perform RBMC prior to the local refinements?

If everything is correct with the masks, would you be able to provide us with the following:

• Local ref map and mask
• Homogeneous Reconstruction map and mask (if used)
• A picture of your workflow in tree view with jobs selected so I/O lines are highlighted.

Best,
Kye

[cbeck]

Thanks for your response, Kye!

To answer your questions, I’m using v4.6, and I did not perform RBMC. The map and mask for the Local Refinement both come from the Volume Alignment Tools job, and I verified that they’re properly aligned by opening them in ChimeraX. Unfortunately, I can’t show the actual the maps and masks in this post, but I’d be happy to send that to you separately.

Here is an annotated picture of the workflow. The map thumbnails are redacted.

The Local Refinement (outlined in blue, J92) is selected. It uses the particles from the intersection of the Extraction job (outlined in green, J87) and the Volume Align Tools job (outlined in yellow, J86). It also uses the volume and mask from the Volume Align Tools job (purple line).

The Homogeneous Reconstruction (outlined in purple, J121) does not use a mask.

I’ve done some more troubleshooting, and I think I’ve pinned the problem down to the extraction job. Please refer to the following workflow:

Performing Homogeneous Reconstruction Only (purple, J120) directly after the initial local refinement results in a reasonable reconstruction. However, skipping the Volume Align Tools and directly performing any type of re-extraction still severely degrades the alignments.

• Yellow (J111): Larger box size with recentering
• Green (J116): Same box size with recentering
• Blue (J118): Same box size without recentering

I don’t understand why extracting using the same box size and without recentering would change the alignments at all, but the alignments clearly degrade.

I should note that re-extracting the particles has never been a problem outside of this situation. In a separate dataset that I’m currently processing (also with v4.6), re-extracting with a larger box size didn’t degrade the alignments at all.

Please let me know if you need any further information!

Best,
cbeck

[olibclarke]

Just to narrow it down a bit more ( I’m sure Kye will have more informed suggestions)

1. Have you tried using the other (CPU-based) extraction job, and does that give the same results?
2. Are the auto-calculated masks during reconstruction sensible?
3. Have you tried extracting without Fourier cropping?

I would expect it to work regardless, but just to try to isolate the issue maybe these tests might be informative? Puzzling…

EDIT: The one other thing I can think of - are you extracting from the same set of micrographs (not re-imported from e.g. relion)? And do the particles look centered in the problematic extraction jobs?

[cbeck]

Thank you for the suggestions, Oli!

1. I tried the CPU-based extraction job as you suggested, and it give the same results
   2.The auto-calculated mask during reconstruction is poor, presumably because it’s calculated off the reconstructed volume. Based off your suggestion, I tried supplying the mask that I used for the Local Refinement before re-extracting. Although the reported resolution improved from 19-25 A to 15 A, the quality of the actual reconstruction hasn’t changed. As before, at high thresholds, the map looks like an aggressively low-pass filtered version of the original map, but at low thresholds, the map looks weirdly elongated along one axis. I’ve verified that the auto-generated mask and the mask that I’ve supplied are both positioned correctly by opening them in ChimeraX
2. I tried extracting without Fourier cropping, and still have the same issue.

Yes, I’m extracting the from the same set of micrographs in cryoSPARC.

So based off the troubleshooting I’ve done so far, it seems like I’ve narrowed the problem down to the actual extraction job. Everything looks fine before re-extracting, and reconstructing the map directly after the initial Local Refinement or directly after the Volume Align Tools results in a good reconstruction. However, after any type of Extraction job (GPU or CPU, Fourier cropping or not, larger box size or same box size), the alignments degrade. Based off the bimodal distribution in the alignment changes plot in my original post, the extraction job appears to preserve the alignments for half the particles, but completely scrambles the alignments for the other half…

For now, I can sidestep this issue by doing a global refinement after re-extracting followed by a Local Refinement. This allows me to achieve the same resolution that I attained prior to re-extracting, but I’m really confused why the extraction job is causing issues for this dataset and not any of the others I’ve processed. If I have time, I might try to inspect the .cs file for the Local Refinement and see if there’s anything weird with the population of particles with scrambled alignments. Maybe they all come from some corrupted batch of micrographs? I’d really appreciate any other suggestions for what I could look for.

Cheers,
cbeck

[kstachowski]

Hi @cbeck,

Thanks for taking time to send over all of the necessary files for us to try to nail down what is going wrong. Currently, we cannot identify any issues related to particle pose or positions that would be leading to this error, nor can we reproduce this error on three other datasets internally. Since performing a NU-ref post extraction appears to solve the problem, we will consider this issue resolved for now. If you, or others, see this happen again, please feel free to post here.

Best,
Kye