We are just starting to test out 3D-Flex, very exciting!
A couple of queries:
Is 3D-Flex compatible with prior local refinement? E.g. can I take the output of a local refinement, and look at residual flexibility/heterogeneity in the domain afterwards using 3D-Flex?
What is the recommended way to handle symmetric systems? Symmetry expansion and local refinement? Or refine with higher symmetry, but then treat in C1 for the purposes of 3D-Flex?
Cheers
Oli
EDIT:
So far seems to be working with symmetry expanded particles (and improved resolution/detail after flex-reconstruct vs C1); Flex-refine jobs after local refinement have so far only yielded latent coordinates that seem to correspond to rigid rotations of the entire masked region. Perhaps parameter tweaking is required.
Relatedly - how does Flex Data Prep handle data that has been treated with Volume Alignment Tools? In this case, the center of the local reconstruction is not at the center of the particle image. Does Flex data prep crop around the sub-particle (as would happen if one re-extracted with re-centering) or around the center of the particle image? I think probably the latter, but presume that means 3D-flex is not compatible with data treated in this way?
I can comment on your second question: 3D Flex Prep does the latter – it doesn’t crop around the sub-particle, it takes the particle images as they are on disk and crops around the center of the images, then downsamples. For treating data that has been aligned to a sub-particle via Volume Alignment Tools, you could re-extract particles with re-centering enabled, and with a large enough box size to ensure that none of the original protein is “cut off” if any of the particles. (This way the center of the re-extracted images would coincide with the center of the sub-particles.) Note that since the sub-particle is likely not centered at the COM of the protein, this means that slightly larger box sizes are probably needed to do this.
Hi @olibclarke,
How about your result of 3D-Flex following symmetry expansion and local refinement? I face the same situation. Could you please share any ideas? Thanks!
We have no additional updates, but the current workaround to use 3D Flex on arbitrarily re-centered particles would be to use Volume Alignment Tools followed by Re-extraction. We have noted to make 3D Flex work on particles that have been re-centered in Volume Alignment Tools but not re-extracted.
One possible workaround I can think of - the Downsample tool now supports cropping around an arbitrary center (instead of just the box center), so I goess one could go Downsample (cropping around subparticle center → 3D Flex Data prep → 3D Flex Train?
I believe this would also work as well! The main caveat being that if the shift induced by re-centering is significant, downsample accounts for the edges of the images that get shifted out of the FOV by wrapping around, so for large changes to the shift (relative to the box size), I’d likely still recommend re-extracting if possible.
Ah yes – the re-centering in downsample is done in the same way as how shifts are applied to particle images during orientation search (which for computational reasons is via using the Fourier Shift Theorem, which has the wrap-around effect).
But perhaps we could reconsider – would adding noise make more sense? Perhaps it would be important to match the power of any added noise to the power of the particle images?
I guess if the box is big enough it’s fine, but if the box is tight one could run into trouble, having protein features reappearing on the other side of the box - not sure what the best answer is.
I guess it is a pretty niche situation at the end of the day so maybe it isn’t worth worrying about - it might be worth adding a tooltip indicating that this is what is going on though, as the current tooltip is a bit confusing:
Ah I see the confusion – in the case that you want to re-center and expand (pad) the real-space particles (via setting the crop parameter to something larger than the box size), the particles will be mean-padded and then shifted. I’ll make a note to clarify in the re-center parameter option that re-centering is done via the wrap-around method.
3D Flex is compatible with locally refined particles and symmetry expanded particles, provided that the input particles and volume are centered on the region of interest.
Starting from a consensus refinement, let’s consider the general case of performing symmetry expansion, then local refinement on a subregion of the volume. The steps might look something like this:
Obtain a consensus refinement, a set of aligned particles, and a mask covering the region of interest in the structure
Run a Symmetry Expansion job with the appropriate symmetry group
Connect the symmetry expanded particles to a Volume Alignment Tools job, along with the volume and mask.
Activate “Re-center to mask center of mask”
Run a Local Refinement job using volume, particles, and mask from step 3.
At this point, if you would like to perform 3D Flex downstream (i.e. focused on the masked region that was refined in step 4), the following step is highly recommended. This ensures that the center of the structure is as close as possible to the center of the particle images, allowing you to use a smaller training / crop box size in Flex Data Prep:
Run an Extract from Micrographs with “Recenter using aligned shifts” activated
Alternatively, a Downsample Particles job can be run, but this is only recommended if the original micrographs are inaccessible, or if using subtracted particles and you’d like 3D Flex to continue operating on subtracted particles
Now, the particles from step 5 and volume from step 4 can be connected to a Flex Data Prep job, and the volume + mask from step 4 can be connected to a Flex Mesh Prep job.
Note that combining symmetry expansion and local refinement with 3D Flex in a workflow like this, is one of many ways to treat flexible, symmetric structures. It is also a valid workflow to omit the symmetry expansion step – for example, our Case Study on Continuous and Discrete Heterogeneity processes a C3 symmetric dataset, but doesn’t use symmetry expansion or local refinement prior to 3D Flex.
I will try this out. A little question already: After symmetry expansion and before recentering I already did a local refinement of the protomer of interest (yielding a high resolution). I reextracted at a boxsize of 400 px after re-centering (initial boxsize 576 px) with “recenter using aligned shifts” enabled. A box size large enough to extract the whole protein complex. I did a sanity check using “homolog. reconstruct only” which led to a terrible resolution of 19 Å. What could have gone wrong? It is very much related to Alignments degrade after re-extracting particles - #8 by cbeck
If I perform local refinement after re-centering on mask center, before re-extracting, the resolution increases from 2.46 to 2.39 Å. If I re-extract these particles at 400 px the "homolog. reconstruct only” gives a resolution of 22 Å and local refinement with the re-centered mask + volume (also cropped to 400 px using volume tools) gives a resolution of 6.8 Å.
If I re-extract these particles at the original size of 576 Å, "homolog. reconstruct only” gives a resolution of 26 Å and local refinement with the re-centered mask + volume gives a resolution of 3.72 Å.
There is a potential workaround of doing another NU-refinement before the local refinement after re-extraction but initial tests don’t recreate the same resolution as before, which forces me to re-optimise a mask, refinement parameters.
I think the problem lies in the re-extraction, it must have messed up the alignments. I use rbmc-ed particles from 2 combined datasets, could there be some information loss? What would happen if I don’t re-extract but just use the cropping of the flex data prep on the re-centered local refinement?
Sorry for the bombardement… and thanks in advance!
I am unsure of what your previous processing steps are, but given the following sets of jobs:
(A)
local refinement after re-centering on mask center, before re-extracting, the resolution increases from 2.46 to 2.39 Ă….
(B)
"homolog. reconstruct only” gives a resolution of 22 Å and local refinement with the re-centered mask + volume (also cropped to 400 px using volume tools) gives a resolution of 6.8 Å.
(C)
re-extract these particles at the original size of 576 Å, "homolog. reconstruct only” gives a resolution of 26 Å and local refinement with the re-centered mask + volume gives a resolution of 3.72 Å.
Can you send over a Workflow JSON file for the following sets of jobs?
A → B
A → C
Workflows can be generated by multi-selecting the set of jobs, right clicking, and selecting “Create Workflow”, and providing a name for it. Then you can export these files and send via DM. If you have any issues, please let me know.
