Hi,
What is the best workflow to perform local motion correction of subparticles (either reference-based or otherwise)?
Ideally, I would like to calculate trajectories from the full particle (which has more signal, and more stable trajectories), but then extract matched sets of subparticles using the improved local alignment parameters and dose weighting.
Currently, the best way I can think of to do this is to align/extract in an oversized box, reconstruct, then use volume alignment tools to recenter, and crop around that center using the downsample tool. However this is (1) very space/time inefficient and (2) fails completely above a certain box size (1126), where reconstruction of the whole particle is currently not possible in cryosparc.
Is there a better way to do this currently?
In a future version, would it be possible to allow a single full-particle reference to be paired with a symmetry expanded particle set after volume alignment tools, to allow for pairing of the sets of subparticles with the appropriate full reference? In either local motion or RBMC?
Cheers
Oli
1 Like
Hi @olibclarke,
Sorry for the delayed response! We haven’t tried this ourselves, but think your workflow should work. As you mentioned, it will likely be space and time inefficient, but to our knowledge would probably be the only way to currently approach this. What box size would you need to use to achieve this for your dataset?
We tried to be careful about units and normalization in our formulation of RBMC, so in principle using hyperparameters from the full particle to motion correct the sub-particles should be okay. Moreover, it seems like hyperparameters that are “in the ballpark” of right tend to give reasonable results. That said, less signal is fundamentally harder to align, so I don’t know if that effect would manifest to a significant degree. If you do attempt to do this, please let us know how it goes.
Additionally, we have noted a feature request for sub-particles and symmetry expanded mates in RBMC.
All the best,
Kye
1 Like
Hi Kye,
The problem is that the native box size I would need for this would be on the order of 1400 pixels - and I would then need to refine before cropping the subparticles out, as otherwise the shifts will be inaccurate (due to drift during RBMC, as discussed elsewhere).
I guess I could run RBMC, downsample 2x & refine to get “good enough” alignments, then crop out subparticles from the original RBMC particles using the alignments from the downsampled refinement? as otherwise refinement will fail in the original box size…
But there is another problem, I think - in order to start RBMC, I need to provide a matching set of particles & volume - but I can’t reconstruct from 1400px particles as I will run out of GPU RAM. I guess I can just extract the particles in that size, and crop/pad the volume from a smaller box size to match? EDIT: actually just tried this, and it requires one additional step - because Vol Tools only works on one map (not all the passthrough maps), separate vol tools jobs need to be run for the full map and both half maps, and then the outputs recombined using the lower level slots. EDIT2:… but when I try this, it doesn’t work 
see screenshot
Cheers
Oli
EDIT3: I tried a different approach - performing the pre-RBMC reconstruction directly in a reduced box size - but this fails with EWS correction on:
Traceback (most recent call last):
File "cryosparc_master/cryosparc_compute/run.py", line 115, in cryosparc_master.cryosparc_compute.run.main
File "cryosparc_master/cryosparc_compute/jobs/refine/newrun.py", line 1174, in cryosparc_master.cryosparc_compute.jobs.refine.newrun.run_homo_reconstruct
File "cryosparc_master/cryosparc_compute/engine/newengine.py", line 3279, in cryosparc_master.cryosparc_compute.engine.newengine.reconstruct
File "cryosparc_master/cryosparc_compute/engine/newengine.py", line 3292, in cryosparc_master.cryosparc_compute.engine.newengine.reconstruct
File "cryosparc_master/cryosparc_compute/engine/newengine.py", line 482, in cryosparc_master.cryosparc_compute.engine.newengine.EngineThread.setup_ews_correct
AssertionError: TODO must check box size convention for ewald sphere radius calculation
Hey @olibclarke,
We are working on this internally but wanted to see if you were able to get this working.
Was this request spurred by your recent RyR preprint?
Best,
Kye
1 Like
Great! It actually wasn’t that, but it would be useful there too, absolutely, for further improving resolution in the corner subparticles 
And no, wasn’t able to get it working
Happy to share data if needed!
1 Like
Hi @olibclarke,
It turns out that currently, this kind of workflow can be done, though it is not as streamlined as it could be, and we appreciate you bringing it to our attention! I have been able to piece together a workflow that performs sub-particle RBMC on EMPIAR-10793:
- After obtaining a homogeneous particle stack, I extracted particles at a box size of 800 px and F-cropped to 600 px
- Using the RBMC job, I calculated hyperparameters, experimental dose weights, and motion trajectories for the full particle.
- I symmetry expanded the particles with the correct symmetry operator
- I created a mask for the subparticle and used Volume Alignment Tools to shift the subparticles to the center of the box
- I used the job
Apply Trajectories with the hyperparameters/dose weights/trajectories from the RBMC of the full particles, and then the particles/volume/mask outputs from the Volume Alignment Tools job.
- Lastly, I used a downsample job to real-space crop the particles to a more manageable box size before moving onto local refinement/3D-classification.
Note that for the experimental dose weights to be applied correctly to subparticles, you must currently use the same physical box size that was used for calculation of the hyperparameters and experimental dose weights of the full particle (ie no downsampling or cropping in the Apply Trajectories job unless you removed the experimental dose weights). Alternatively, you can omit the EDWs as an input to Apply Trajectories and this will then apply a standard model-based dose weighting curve, which in part defeats the purpose of using RBMC. This is why the downsample job with cropping is recommended at the end, to save on disc space and computational expense.
Cheers,
Kye
3 Likes
Thanks Kye this is very helpful, thanks for figuring it out!
Between steps 2 and 3, did you re-refine with the RBMC-improved full-particles? If not, would this be possible (to improve alignments prior to local refinement), or not recommended? I guess it doesn’t matter, as the trajectories are already baked in at that point?
Also, did you compare local refinements for the RBMC-corrected & cropped subparticles and those extracted from the original micrographs - did you see the expected improvements in resolution?
1 Like
Hi @olibclarke,
I did not, as I only had RBMC do the hyperparameter and dose weight calculations and not actually perform the motion correction.
My goal was to try to limit the storage space overhead with as few extractions as possible but still obtain a comparable result to show that the workflow can be done. In theory, you should be able to use RBMC’d particles as inputs.
I did compare results, they were about the same. I think any improvement in the map quality will be target/dataset dependent.
Best,
Kye
2 Likes
Thanks heaps Kye! We will test on some of our cases, very helpful to have this available!
2 Likes
Hi Kye, one more query about step 2 - you mentioned that you just ran this up to the hyperparameter estimation stage, and then used apply trajectories. I thought the trajectories were calculated after hyperparameter estimation? Are the trajectories actually generated during hyperparameter estimation, and only applied during the motion-correct particles step?
