Combining subparticle extraction with signal subtraction?

Hi,

We would like to perform partial signal subtraction in combination with subparticle extraction. That is, to extract subparticles of a larger assembly (in a smaller, local box) while performing partial signal subtraction for the rest of the assembly. Could this workflow be supported in cryoSPARC?

I think allowing cropping of particles around an arbitrary center (e.g. supplied using Volume Alignment Tools) should do the trick, if signal subtraction is initially performed in the larger box, but currently this is not possible (only cropping around the center of the box is supported via the Downsample job). Alternatively, performing signal subtraction at the point of particle extraction in the local box could also work.

Cheers
Oli

4 Likes

The way I did it in pyem is that the original particle box has to be large enough that the desired cropped box at the subparticle center doesn’t overlap the edge. This is pretty much what Oli describes (though it doesn’t matter if subtraction is in the large box, or the small box, the latter being slightly cheaper).

Cryosparc could potentially support going back to the micrographs to avoid the initial size limitation. The big box particles would be before any subparticle expansion so it’s not that terrible but it does potentially require more time/storage if those initial particles have to be re-extracted in a huge box.

4 Likes

(though it doesn’t matter if subtraction is in the large box, or the small box, the latter being slightly cheaper)

I think it does matter? Or at least, it is important that a projection of the “big box” volume is what is subtracted (whether it is subtracted from particles that are in a small or large box), because otherwise signal outside the bounds of the small box (which in certain orientations will be behind or in front of the subparticle) will not be subtracted.

Right - but after that, you only need the part you’re going to subtract. The subtraction can be on small boxes.

1 Like

Hi all,

Thanks for the discussion – we looked this over and for the purpose of cropping around arbitrary centers, we will likely add an option to “Downsample Particles” that simply allows for re-centering the particles prior to downsampling and/or cropping. Specifically, the downsample job will apply the 2D shifts stored in alignments3D/shift to the particle images (in Fourier space, so there would be a “wrap-around” effect). It will then re-write these particles out to disk and set the alignments3D/shift to zero.

To use this to crop around an arbitrary center, one would first use Volume Alignment Tools to re-center particles around the desired subparticle center, and then pass these to downsample particles while enabling the re-centering parameter, optionally cropping to reduce the box size. Would this adequately address the use-cases discussed above?

The only caveat is as @DanielAsarnow mentioned – since the shifted box could potentially overlap the initial edge of the particles (depending on the crop size), this is something to pay attention to. Minor wrap-around with small shifts are likely to be tolerated okay by CryoSPARC (indeed wrap-around is induced during BnB alignment search while searching over shifts), but large wrap-arounds might be undesirable. To make this clearer, we could print some simple diagnostics (e.g. the maximum shift value across the particles) and/or possibly add a switch to fail if the wrap exceeds a given amount? If the wrap-around is excessive, one would have to go back to the micrographs, extract at a larger realspace box size, then repeat subtraction and Volume Alignment Tools, then finally run downsampling with re-centering.

Note that if you have access to the micrographs (and aren’t interested in signal subtraction), cropped subparticle extraction is natively possible by combining Volume Alignment Tools with Extract from Micrographs with re-centering enabled. I believe signal subtraction is the only case where we have to crop from the input particles and can’t go back to the micrographs.

3 Likes

Hi @mmclean, I think this would work! So just to clarify, the workflow would be:

  1. Use volume alignment tools to extract centered on the subparticle, using a sufficiently large box to capture the entire particle (only caveat here is that this might be a huge box if it is a small subparticle on the periphery of a large particle, which may cause memory issues)

  2. Reconstruct using the particles from 1, to obtain a reconstruction including the entire particle, centered on the subparticle.

  3. Subtract using the particles from 1, volume from 2, and appropriate mask

  4. Use downsample to crop around subparticle

This seems like it should work - the only issue may be the large box (up to 2x the original box) required for step 1, and whether this will then fit in memory for the reconstruction step?

EDIT:

Actually I think I have this wrong. I think the actual workflow would be:

  1. Re-extract in large enough box to contain the particle, plus room for later cropping of subparticle;
  2. Reconstruct
  3. Subtract
  4. Volume alignment tools to center on subparticle
  5. Crop using downsample.

Still requires large box, but not as large as I first thought

2 Likes

Yes, this is what I had in mind too. In step 5 just ensuring that the to-be-added re-centering parameter is activated. At the end of the day, you would end up with aligned subparticles, cropped around the region of interest & with undesired density subtracted away, that could be used for downstream local refinements, 3D Var, 3D class, etc. :slight_smile:

Best,
Michael

1 Like

Hi guys, this discussion is great, and think the suggestions would work nicely.
Just passing to know if this re-centering button/feature is already implemented (or maybe is planned(?)).
Best,

GermĂĄn

Yes. We expect this to be included in the next release, date tbd.

1 Like

(cryoSPARC 4.4.1)

Did someone try the new ‘Recenter using aligned shifts’ in the ‘Downsample’ job already?

I used the ‘Vol Align Tool’ to shift the particle orientations and then want to crop the particles around the new center position.

My impression is that the job ignores the shift and crops the particles around the center. Using the ‘alignments3D’ output from the ‘Downsample’ job in another particle extraction (for sanity check) just extracts particles in the midpoint of the particle, so somehow the shift is not correctly applied or lost?

1 Like

Hi @shuber,

Thanks for reporting this. Can you confirm the parameters of the problematic downsample particles job?

We have encountered a bug while trying to replicate this where recentering and cropping simultaneously in the same downsample particles job. In the current implementation, the particles are first cropped around the center, and then shifted to the desired location. Since shifts are done via wrap-around, this means that if the shifts are larger than the initial crop size, that information will be lost.

If the problematic downsample job was done with both cropping and re-centering enabled, could you try the following steps:

  1. Run a first downsample particles job with only the “Re-center using aligned shifts” parameter activated, ensuring that alignments3D are present in the input particles connection.
  2. Run a second downsample particles job on the outputs of 1), with both “Crop / pad to box size” and “Fourier crop to box size” parameters set to the desired box size

and let me know if this resolves the discrepancy?

We have triaged this bug.
Best,
Michael

Hi @mmclean,

I can confirm that I indeed used both recentering and cropping simultaneously when I encountered the bug.

I tried the steps you suggested with (1) recentering and (2) cropping in seperate Downsample jobs, and the bug disappeared. I will work around the problem for now with this workflow.

Thank you for your help!

Best,
Stefan

1 Like

Dear @shuber,

This bug has now been fixed as of CryoSPARC v4.5, released today. Downsample Particles now should work with simultaneous re-centering and real space cropping within the same job, and the workaround indicated above should no longer be necessary. (This is because cropping is now directly done around the new center coordinates, rather than cropping first and re-centering after).

Best,
Michael

2 Likes