Hi all,
In my current workflow I need remove some unwanted particles after refinement based on the tilt angle. I will first export the particles in the refinement job, and then remove unwanted particles with the help of cryosparc tools. The result particles will be re-imported with “Import result group” job. However, when I run new refinement jobs with these particles, they must be re-cached to SSD.
Since I don’t make any changes for the particle images, the particles I re-import here should be on the SSD already. Can these re-imported particles be processed directly, just like those accepted particles from “select 2D classes” or “remove duplicate particles”?
In addition, I do this because I’m processing filament structures. And I find sometimes after 3D refinement, remove particles with large tilt angles is a good way to remove garbage. Currently in “helix refine” we can limit the tilt angle. Is it possible to remove particles with large tilt angle directly in this job?
Particle import is expected to trigger renewed caching of the newly imported particles. You may circumvent the re-importation by instead saving your modified particles using the Project.save_external_result() method. This example shows the method “in action”.
My colleague will separately respond regarding
Hi @ScientificWitchery ,
Currently you cannot use any built-in methods to remove particles with large tilt via CryoSPARC. However, this is possible using CryoSPARC Tools. You can obtain a subset of a particle dataset with particles with tilt greater than max_tilt removed by using the following Python script.
import os
import numpy as n
from cryosparc_tools.cryosparc.tools import CryoSPARC
from cryosparc_compute import geometry
cs = CryoSPARC(
license="xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
host="...", # YOUR HOST HERE
base_port=40000,
email="example@address.com", # YOUR EMAIL HERE
password="password" # YOUR PASSWORD HERE
)
assert cs.test_connection()
max_tilt = 5 # desired max tilt here
project = cs.find_project('PX') # desired project id here
job = project.find_job('JY') # desired job id here
particles_dset = job.load_output('particles') # particles from desired job
poses = particles_dset['alignments3D/pose']
def get_tilts(poses):
""" Get the tilt angles from 3D particle poses """
Rs = geometry.expmaps(poses)
projdirs = Rs[...,2]
EAs = geometry.projdir_to_EA(projdirs)
tilts = 90.0 - n.rad2deg(EAs[:,1])
return tilts
tilts = get_tilts(poses)
# plot tilt histogram
from matplotlib import pyplot as plt
plt.hist(tilts, bins=100)
plt.show()
# subset particles with tilts less than max_tilt
tilt_mask = n.abs(tilts) < max_tilt
particles_dset_subset = particles_dset.mask(tilt_mask)
# Use the project.save_external_result() method with the example specified arguments to save the subset dataset back to
# CryoSPARC for further analysis. The result will be a new “External job” in CryoSPARC in the same workspace as the
# original job.
project.save_external_result('WX', particles_dset_subset, 'particle', f"tilt_less_than_{str(max_tilt)}" )
Make sure to replace the license ID, host, email, password, with that relevant to your instance, as well as max_tilt, and PX, JY, WZ with the project/job/workspace numbers with the desired project/job/workspace.
We’ve made a note that tilt is a useful attribute for particle-based curation!
Best,
Michael
3 Likes
A modified version of this script might be useful for removing particles with unexpectedly large deviations from mean micrograph defocus.
After Patch CTF, I don’t necessarily want to discard micrographs with large defocus ranges (this will happen with gold or junk in the frame), but I would like to flag particles that have very large deviations from the mean (as they are probably on gold or junk). Would this be easy to do with cryosparc tools?
How do you intend to use the flag during downstream processing?
I don’t mean literally flag, sorry - select would be a better choice of words.
I would like to select particles based on the difference between the defocus at that particle position, and the overall mean defocus of the micrograph.
The hypothesis is that large differences in defocus are likely due to junk, gold, crystalline ice or carbon, so by selecting these particles I would be able to test this hypothesis and get rid of them if necessary.
Thank you @wtempel ! I used dataset.save() method before. I will try it later.
Thank you @mmclean !
Tilt angle may be an efficient attribute for particle curation during helical reconstruction. It is expected only side view can be found. But I often find some particles are assigned with large tilt angles, like this figure:
Remove particles with large tilt angles could improve the resolution. 2D classification can also remove these bad particles, but it is not very efficient.
1 Like