Particles coordinates lost in extracted_particles.cs upon extraction

Dear All,
I recently exported a set of particles from cryosparc to be re-extracted into relion4 to run polishing. By using csparc2star.py with the particles.cs from refinement as “base.cs” and the extracted_particles.cs from the last extraction before the same refinement to get rlnCoordinate as “aux.cs” (referring to the following command).

csparc2star.py base.cs [aux.cs>] output.star

This combined with some scripting allowed me to successfully re-extract a fraction of the particles in relion with prior information on the position and alignment. Nonetheless, a lot of particles were missing. After investigation, it appears that the extracted_particles.cs (from the last extraction before refinement) contains only information for a part of the particles ( I see that by converting the extracted_particles.cs in a star file and then counting lines of the generated star file less the header). Interestingly, the passthrough_particles.cs contain all particles (assessed the same way) and cryosparc reextracted all the particles without incomplete Micrographs.

Is there an explanation why the “extracted_particles.cs” contain information for only a part of the particles or might these lines get lost upon cs to star conversions?

Even though these coordinates information are lost in the “extracted_particles.cs” extraction is successful in cryosparc so I assume this information might be stored somewhere else. Any Idea about how to get it back to generate a star file with all my particles including rlnCoordinates ?

Thank you for your help and advice,
Julien

extracted_particles.cs should contain information for all extracted particles. The observed difference may be due to the omission of particles that were not extracted due to being too close to the edge of the source exposure. Metadata for such “edge” particles may be handled differently in other software packages.

Hi wtempel, Based on what you said it’s most probably a problem occurring during extraction. It can’t be induced by the omission of particles on the edge since particles missing in the file were still counted as extracted in the job. Based on that I reran the full extraction and got a consistent extracted_particles.cs file with all partic
Best,
Julien

You may eliminate this possibility by determining the array dimensions of the cs files without conversion.
In CryoSPARC v4.1+, you can use CryoSPARC Tools. For older versions of CryoSPARC, please see this page of the guide.
If you think this is a bug in CryoSPARC, would you be willing to share with us the job reports and extracted_particles.cs files for a pair of job where extracted_particles.cs was consistent and inconsistent with the logged counts?

How many particles are we talking about here? It’s normal for an extraction to have fewer output than input particles, because the ones that overlap the edges of the micrographs are discarded.

Hi Wtempel, I removed the inconsistent extraction when I saw the new one worked fine. If I face it again I will share the job report and extracted_particles.cs files. Thank you for the cryosparc Tools information, I’ll use that from now.
Best,
Julien