Problem with local refinement with NCS features

I have a complex with identical-looking domains A & B, which are not related by symmetry. So to refine a combined A & B, I did the following:

  • J1 Align the entire complex from particle set 1 with X particles (domain A appears much better)
  • J2 Focus refinement of domain A
  • J3 Subtract domain A from particle set 1 (now we have subtracted particle set)
  • J4 Focus refinement of domain B using the subtracted particle set
  • J5 Using Volume Alignment Tool to Align J4 volume & particle of domain B to the volume from J2 containing focused refined domain A
  • Now I combine particles from J2 and J5 for refinement together (2 X particles in total with X particles from J2 and X signal subtracted particles from J5). However, the local refinement only recognizes 1/2 of the particle set, only X finally.

Is there anyway to do this without re-extracting particles?

Hi @builab,

Thanks for the question, this is an interesting case. Can you clarify what you mean by the final local refinement only recognizing 1/2 of the particle set? Could you show the full inputs & outputs tab to the final local refinement?


Here’s the input

J405 = signal subtracted particles having 164,759 particles.
J408 & J409 are re-extracted particles (frustrated to try different ways to get the full particles count) having 164,759 particles in total as well.

So, it is supposed to be 329k particles in total. However, This is the output showing only 164,759 particles

It always recognizes 164,759 particles in total along the way.

Hi @builab,

Thanks for this, and apologies for the delayed response. Based on your described workflow, we’ve identified what’s happening. When creating subtracted particles in the particle subtraction job, the unique ID’s (UID’s) of the particles are unmodified from the inputs. This is so that a correspondence can be maintained between the subtracted particle dataset and their originals.

(Side note: this correspondence is needed in a few cases, one example being:

  • Following a Local Refinement on subtracted particles with a Homogeneous Reconstruction Only, where you want to use the original image data (blob input). In this case, the reconstruct only job will need as input: the low-level alignments3D output of the local refinement on subtracted particles, together with the blob and ctf outputs from the original particles.)

However, the workflow you’ve described is important and it should be supported. The main detail needed to get this to work is to ensure that the UID’s of the subtracted particles and original particles are different (disjoint), so that the final refinement will identify them as different particle sets. As you alluded to, you can get around this in CryoSPARC by re-extracting the original particles from J2 (which will regenerate their UID’s), then combining these with the subtracted particles from J5 into a single local refinement. But this method would waste space in writing out all of the particles again.

You can use CryoSPARC Tools to do this. See this link for usage info, including how to connect to a CryoSPARC instance in a script. The example script below will take the outputs of a specified job (e.g. J5), reassign the particles’ UID’s, then output the reassigned particles to a new “External” job in the specified workspace – just specify the info at the top of the script. These particles can then be combined with the original particles from J2.

import numpy as n
from import CryoSPARC
from cryosparc.dataset import Dataset

# Connect to your CryoSPARC Instance: see
cs = CryoSPARC(

puid = "<your project ID here>" # e.g. P100
wuid = "<your workspace ID here>" # e.g. W10
source_particle_juid = "<job UID of source particles here>" # e.g. J5

# =================================================

project = cs.find_project(puid)
job = project.create_external_job(wuid, title="Import Reassigned UID Subtracted Particles")
job.connect("subtracted_particles", source_particle_juid, "particles")
job.add_output("particle", "reassigned_uid_particles", slots=['blob', 'ctf', 'alignments3D'])

particles = job.load_input("subtracted_particles", ['blob', 'ctf', 'alignments3D'])
out_particles = particles.copy()

if len(n.intersect1d(particles['uid'], out_particles['uid'])) == 0:
    print("No UID collisions found")
    print("UID collisions found!")

job.alloc_output('reassigned_uid_particles', out_particles)
job.save_output('reassigned_uid_particles', out_particles)

The workflow you’ve described is a great example of how to use Volume Alignment Tools together with Particle Subtraction to handle NCS. To support this without having to go through this UID-reassigning workaround, we’ve triaged this as something to support and are thinking about the simplest way to implement this!

EDIT: Amending my reply – re-extraction will not regenerate particles’ UIDs, therefore re-extracting the original particles won’t circumvent this issue. The only way to do this is via manual particle re-assignment, using the provided script.



This is a fantastic answer. Thanks a lot @mmclean . I will test it and let you know here that it works. Just a bit busy so perhaps next week.

1 Like

This process is also called local symmetry, localized reconstruction, or multibody refinement in EM programs - it can be also be done using Relion tools, pyem scripts, or localrec. (Just for information).

@mmclean would it not be easier to simply export and import the second particle set?

Also - I think the simplest way to have the two local refinements in register is to disable transforming the rotation center back from the fulcrum coordinate. I believe that will be equivalent to how I do it with pyem (e.g. in this paper).

@builab You can also use these local refinements to start careful focused classifications - if there are different states they can be tracked back to the source particles to check for correlation.

1 Like

Localized rec in Relion works but it is very inconvenient if you work with a huge assembly and the feature of interest is off-center. Due to the polishing routine, off-center features of a huge assembly require new extraction of huge box size & must be polished again.

Export and import definitely work.

@DanielAsarnow Thanks for the suggestion, I will try that.

You don’t need to polish, though, and then they are the same, no? You must not be polishing the cryoSPARC results either?

PS you could also do a combined refinement in Relion by exporting and combining both particle sets - if they are not aligned they can be using a transformation matrix from ChimeraX’s fitmap command and the --transform argument for or

If you start everything with Relion, you had to polish it. Unfortunately, a lot of projects started with Relion with motioncorr2 not patch motion tracking in cryosparc.

Thanks for the suggestion. It is definitely easier to be able to use with chimeraX fitmap command.

Why do you “have to” polish it? The resolution was worse? Maybe a low number of patches / no -InFmMotion?

Always worse in our case without polishing. Perhaps due to the low number of patches (6x4). I never tried so much. But our sample is not a typical single particle. It is a huge filament with a very heterogenous distribution.

If you look at Patch Motion job output, it usually chooses a quite high number of patches. For motioncor2 I usually use 8x6 and -InFmMotion 1. The average relationship across my datasets is Relion/Motioncor2 +0.2Ă… vs. cryosPARC, polishing -0.2 Ă… vs. cryoSPARC. Occasionally polishing fails and it gets worse though.