We would like to perform partial signal subtraction in combination with subparticle extraction. That is, to extract subparticles of a larger assembly (in a smaller, local box) while performing partial signal subtraction for the rest of the assembly. Could this workflow be supported in cryoSPARC?
I think allowing cropping of particles around an arbitrary center (e.g. supplied using Volume Alignment Tools) should do the trick, if signal subtraction is initially performed in the larger box, but currently this is not possible (only cropping around the center of the box is supported via the Downsample job). Alternatively, performing signal subtraction at the point of particle extraction in the local box could also work.
The way I did it in pyem is that the original particle box has to be large enough that the desired cropped box at the subparticle center doesnât overlap the edge. This is pretty much what Oli describes (though it doesnât matter if subtraction is in the large box, or the small box, the latter being slightly cheaper).
Cryosparc could potentially support going back to the micrographs to avoid the initial size limitation. The big box particles would be before any subparticle expansion so itâs not that terrible but it does potentially require more time/storage if those initial particles have to be re-extracted in a huge box.
(though it doesnât matter if subtraction is in the large box, or the small box, the latter being slightly cheaper)
I think it does matter? Or at least, it is important that a projection of the âbig boxâ volume is what is subtracted (whether it is subtracted from particles that are in a small or large box), because otherwise signal outside the bounds of the small box (which in certain orientations will be behind or in front of the subparticle) will not be subtracted.
Thanks for the discussion â we looked this over and for the purpose of cropping around arbitrary centers, we will likely add an option to âDownsample Particlesâ that simply allows for re-centering the particles prior to downsampling and/or cropping. Specifically, the downsample job will apply the 2D shifts stored in alignments3D/shift to the particle images (in Fourier space, so there would be a âwrap-aroundâ effect). It will then re-write these particles out to disk and set the alignments3D/shift to zero.
To use this to crop around an arbitrary center, one would first use Volume Alignment Tools to re-center particles around the desired subparticle center, and then pass these to downsample particles while enabling the re-centering parameter, optionally cropping to reduce the box size. Would this adequately address the use-cases discussed above?
The only caveat is as @DanielAsarnow mentioned â since the shifted box could potentially overlap the initial edge of the particles (depending on the crop size), this is something to pay attention to. Minor wrap-around with small shifts are likely to be tolerated okay by CryoSPARC (indeed wrap-around is induced during BnB alignment search while searching over shifts), but large wrap-arounds might be undesirable. To make this clearer, we could print some simple diagnostics (e.g. the maximum shift value across the particles) and/or possibly add a switch to fail if the wrap exceeds a given amount? If the wrap-around is excessive, one would have to go back to the micrographs, extract at a larger realspace box size, then repeat subtraction and Volume Alignment Tools, then finally run downsampling with re-centering.
Note that if you have access to the micrographs (and arenât interested in signal subtraction), cropped subparticle extraction is natively possible by combining Volume Alignment Tools with Extract from Micrographs with re-centering enabled. I believe signal subtraction is the only case where we have to crop from the input particles and canât go back to the micrographs.
Hi @mmclean, I think this would work! So just to clarify, the workflow would be:
Use volume alignment tools to extract centered on the subparticle, using a sufficiently large box to capture the entire particle (only caveat here is that this might be a huge box if it is a small subparticle on the periphery of a large particle, which may cause memory issues)
Reconstruct using the particles from 1, to obtain a reconstruction including the entire particle, centered on the subparticle.
Subtract using the particles from 1, volume from 2, and appropriate mask
Use downsample to crop around subparticle
This seems like it should work - the only issue may be the large box (up to 2x the original box) required for step 1, and whether this will then fit in memory for the reconstruction step?
EDIT:
Actually I think I have this wrong. I think the actual workflow would be:
Re-extract in large enough box to contain the particle, plus room for later cropping of subparticle;
Reconstruct
Subtract
Volume alignment tools to center on subparticle
Crop using downsample.
Still requires large box, but not as large as I first thought
Yes, this is what I had in mind too. In step 5 just ensuring that the to-be-added re-centering parameter is activated. At the end of the day, you would end up with aligned subparticles, cropped around the region of interest & with undesired density subtracted away, that could be used for downstream local refinements, 3D Var, 3D class, etc.
Hi guys, this discussion is great, and think the suggestions would work nicely.
Just passing to know if this re-centering button/feature is already implemented (or maybe is planned(?)).
Best,
Did someone try the new âRecenter using aligned shiftsâ in the âDownsampleâ job already?
I used the âVol Align Toolâ to shift the particle orientations and then want to crop the particles around the new center position.
My impression is that the job ignores the shift and crops the particles around the center. Using the âalignments3Dâ output from the âDownsampleâ job in another particle extraction (for sanity check) just extracts particles in the midpoint of the particle, so somehow the shift is not correctly applied or lost?
Thanks for reporting this. Can you confirm the parameters of the problematic downsample particles job?
We have encountered a bug while trying to replicate this where recentering and cropping simultaneously in the same downsample particles job. In the current implementation, the particles are first cropped around the center, and then shifted to the desired location. Since shifts are done via wrap-around, this means that if the shifts are larger than the initial crop size, that information will be lost.
If the problematic downsample job was done with both cropping and re-centering enabled, could you try the following steps:
Run a first downsample particles job with only the âRe-center using aligned shiftsâ parameter activated, ensuring that alignments3D are present in the input particles connection.
Run a second downsample particles job on the outputs of 1), with both âCrop / pad to box sizeâ and âFourier crop to box sizeâ parameters set to the desired box size
I can confirm that I indeed used both recentering and cropping simultaneously when I encountered the bug.
I tried the steps you suggested with (1) recentering and (2) cropping in seperate Downsample jobs, and the bug disappeared. I will work around the problem for now with this workflow.
This bug has now been fixed as of CryoSPARC v4.5, released today. Downsample Particles now should work with simultaneous re-centering and real space cropping within the same job, and the workaround indicated above should no longer be necessary. (This is because cropping is now directly done around the new center coordinates, rather than cropping first and re-centering after).
