"Remove duplicates" remove particles that are not duplicates


After running homo refinement, I did the remove duplicates to check if there is any duplicates in my final dataset. It turned out that remove duplicates remove particles that are clearly not duplicates. The result is attached. Does anyone have an idea what could cause this problem?


What parameters did you use for remove duplicates, and what is the scale on the image?

I set the minimum separation distance to 20Å, shift key to alignment3D and scored field to pick_stats/ncc_score. The diameter of the white circle is ~250Å.


Definitely something odd going on. Where did the alignments3d come from?

The alignments 3D originated from Homo refinement with re-extracted unbinned particles.

I’ve noticed that when I ran remove duplicates after Homo refinement with binned particles (bin4), without re-extraction, remove duplicates yields reasonable results. It appears that something went wrong during the re-extraction process.

@Ty22 Thanks for your post. Please let us know if you would be willing to share some data with the Structura team so that we may arrangements a transfer of the following data:

  • From the job directory of the homogeneous refinement job:
    • JX_{last_iteration_number}_particles.cs
    • JX_particles.csg
    • JX_passthrough_particles.cs
  • From the job directory of the upstream Extract from Micrographs job that the particles came from (the problematic re-extraction job):
    • JY_micrographs.csg
    • picked_micrographs.cs
    • JY_passthrough_micrographs.cs
  • Job reports for the aforementioned jobs

I’m currently encountering the same issue. After a Template picking and extraction with 4x binning, I ran a “Remove Duplicates” job and got the same pattern you shared on my micrographs. Can maybe someone help with this?

Thanks :slight_smile: