Picking coordinates transfer error csparc -> relion for K2 data

Troubleshooting
1

Hi,
I routineously use cryosparc for particle picking, 2D class and initial 3D refinement from F3EC images, convert the data to star (thx Daniel) and re-extract them with relion.
Now I had some K2 datasets in hand which all failed for some reason: the coordinates of the picked particles are off.
Did anybody else observe this behavior? I see no reason, unless the odd image dimensions of K2 micrographs cause something weird.

Screenshot
Screenshot1334Ă—787 694 KB

Best,
tarek

2

Hi @tarek,

This is strange - cryosparc stores pick locations as a fraction of the micrograph size (in each dimension). Perhaps @DanielAsarnow can comment on how micrograph .cs files are converted?

3

It’s pretty simple:

df[star.Relion.COORDX] = cs[u'location/center_x_frac']
df[star.Relion.COORDY] = cs[u'location/center_y_frac']
df[star.Relion.COORDS] = np.round(df[star.Relion.COORDS] *
    cs['location/micrograph_shape'][:, ::-1]).astype(np.int)

The only thing that can go wrong is if location/micrograph_shape is wrong or the order isn’t consistent. (Note the ::-1 as it’s z, y, x).

4

In that case I guess micrograph_shape is causing my problem as I doubt the order would suddenly become a problem for K2 data exclusively.
What is meant exactly by that? My data is K2 super-res (7676 x 7420).
#edit
just realized that the dimensions are exchanged in the import micrographs tab compared to spider/eman2.
micrograph_blob/shape : [7420 7676]
This would match your note concerning the order…
Any ideas?

5

@apunjani I also noticed that a template I created using:

x, y = np.meshgrid(np.arange(-32,32), np.arange(-32,32))
r = np.sqrt(x**2 + y**2)
sigma = 4.0
g = np.exp(-r**2 / (2*sigma**2)) / (np.sqrt(2 * np.pi * sigma**2))

Is a bit off-center when imported. What’s the center pixel in cryosparc? Is it not arr[box/2, box/2]?

6

some investigation this morning solved my problem:
the order of the coordinates is wrong, which becomes relevant when micrograph dimensions are not symmetric.
giving micrograph_blob/shape : [X Y]
post-conversion correction can be made by: (coordx/shape X)*shape Y and (coordy/shape Y)*shape X
this will lead to the centered particles.
it would be great if you would change the conversion function in pyem to obsolete this “fix” @DanielAsarnow
I am wondering that nobody was suffering from this before…

7

@tarek it doesn’t affect reconstructions, or imported coordinates. You say switching X and Y “fixes” the coordinates, but I’ve gone over this before and found that the coordinates aren’t right regardless of the convention, when they are actually displayed in relion_display_gui. I left it the way it is because it was closer than the other way during these tests. (I also tried a few possible alternate conventions). I also think the convention in cryoSPARC may have changed at some point, and might not agree between movies (3D) and micrographs (2D). There’s also another field, micrograph_blob/shape, but right now I’m just looking at the location one.

For example with a command like:
relion_display --coords J31/imported/stack_0495_2x_SumCorr.star --i J31/imported/stack_0495_2x_SumCorr_T.mrc --angpix 1.2156 --lowpass 20 --particle_radius 100 --pick --scale 0.2 &
are the particles really in the right place and centered?

If @apunjani gives us the conventions, or we’re very sure we have it figured out, I can update csparc2star.py one last time. The official export feature should be released soon, so if it’s not a trivial edit I may decide to skip it.

8

For example with a command like:
relion_display --coords J31/imported/stack_0495_2x_SumCorr.star --i J31/imported/stack_0495_2x_SumCorr_T.mrc --angpix 1.2156 --lowpass 20 --particle_radius 100 --pick --scale 0.2 &
are the particles really in the right place and centered?

yes, after conversion they are prefectly centered. I am only speaking about imported micrographs. Never tried movies, since I pre-align them with MC2.

Here is a small “fix” if someone wants to try it in the shell for a super-res K2 micrograph.

awk '{print $1,$2,$3,7420*$4/7676,7676*$5/7420,$6,$7,$8,$9,$10,$11,$12,$13,$14,$15,$16,$17,$18,$19,$20}' csparc_coords.star > csparc_coords_transform.star

9

@tarek I updated csparc2star.py with a few changes:

  • Default behavior is according to @tarek’s observations
  • Use --swapxy to swap the axes when converting coordinates
  • Use --boxsize N with N the refinement box size, if the refinement box was different than the original particle box
  • Angle representation conversions accelerated about 100x
1 Like