"Remove duplicates" remove particles that are not duplicates

Particle Picking
1

Hello,

After running homo refinement, I did the remove duplicates to check if there is any duplicates in my final dataset. It turned out that remove duplicates remove particles that are clearly not duplicates. The result is attached. Does anyone have an idea what could cause this problem?

Thanks

Remove_duplicates_results
Remove_duplicates_results2750×1755 1.42 MB

2

What parameters did you use for remove duplicates, and what is the scale on the image?

3

I set the minimum separation distance to 20Å, shift key to alignment3D and scored field to pick_stats/ncc_score. The diameter of the white circle is ~250Å.

Thanks,
Tianyang

4

Definitely something odd going on. Where did the alignments3d come from?

5

The alignments 3D originated from Homo refinement with re-extracted unbinned particles.

I’ve noticed that when I ran remove duplicates after Homo refinement with binned particles (bin4), without re-extraction, remove duplicates yields reasonable results. It appears that something went wrong during the re-extraction process.

6

@Ty22 Thanks for your post. Please let us know if you would be willing to share some data with the Structura team so that we may arrangements a transfer of the following data:

  • From the job directory of the homogeneous refinement job:
    • JX_{last_iteration_number}_particles.cs
    • JX_particles.csg
    • JX_passthrough_particles.cs
  • From the job directory of the upstream Extract from Micrographs job that the particles came from (the problematic re-extraction job):
    • JY_micrographs.csg
    • picked_micrographs.cs
    • JY_passthrough_micrographs.cs
  • Job reports for the aforementioned jobs
7

Hi,
I’m currently encountering the same issue. After a Template picking and extraction with 4x binning, I ran a “Remove Duplicates” job and got the same pattern you shared on my micrographs. Can maybe someone help with this?

Thanks :slight_smile:

8

Welcome to the forum @InesBenhammouche.
Would you be willing to share the following data with the CryoSPARC developers:

  1. job reports for the Extract and Remove Duplicates jobs.

  2. from the job directory of the Extract job, files

    • J99_micrographs.csg
    • picked_micrographs.cs
    • J99_passthrough_micrographs.cs

    (replace J99 with the actual job ID of the Extract job.)

If you can share the files with us on your preferred data sharing platform, please email us the link. If needed, we can alternatively provide upload instructions. I will send you a private message with the appropriate email address.