Re-extracting in Relion just picks random noise

I have done the initial processing in CryoSPARC (including patch-motion corection and patch ctf) and want to polish those particles in RELION. I’ve exported the CryoSPARC particles and converted them using including the --inverty option. In RELION the movies were processed and are motioncorrected and ctf estimated, from those I now want to re-extract.

I’ve adjusted the micrograph path in the file (interestingly Relion changed every . in the Micrograph in the name to _ ) and made sure the opticsGroup is matching.

My file looks like this:

# version 30001


_rlnVoltage #1 
_rlnImagePixelSize #2 
_rlnSphericalAberration #3 
_rlnAmplitudeContrast #4 
_rlnOpticsGroup #5 
_rlnImageSize #6 
_rlnImageDimensionality #7 
_rlnOpticsGroupName #8 
  300.000000     1.050000     2.700000     0.100000            1          384            2 opticsGroup1 

# version 30001


_rlnImageName #1 
_rlnMicrographName #2 
_rlnCoordinateX #3 
_rlnCoordinateY #4 
_rlnAngleRot #5 
_rlnAngleTilt #6 
_rlnAnglePsi #7 
_rlnOriginXAngst #8 
_rlnOriginYAngst #9 
_rlnDefocusU #10 
_rlnDefocusV #11 
_rlnDefocusAngle #12 
_rlnPhaseShift #13 
_rlnCtfBfactor #14 
_rlnOpticsGroup #15 
_rlnRandomSubset #16 
_rlnClassNumber #17 
_rlnGroupNumber #18 
_rlnNormCorrection #19 
_rlnLogLikeliContribution #20 
_rlnMaxValueProbDistribution #21 
_rlnNrOfSignificantSamples #22 
_rlnImageOriginalName #23 
000001@J34/extract/000002298445352262704_2024-05-13_20.42.57_40S-1v2_87-5_00012_30.0_1-12_patch_aligned_doseweighted_particles.mrcs         MotionCorr/job015/frames/2024-05-13_20_42_57_40S-1v2_87-5_00012_30_0_1-12.mrc         1426.000000  1985.000000   115.120843    69.853913    99.233130    12.431996    13.481996 12339.794922 12281.777344   264.055359     0.000000     0.000000            1            2            1            1     0.606181 6.187094e+05     0.457129            5 000001@Subtract/job003/Particles/subtracted_rank1_opticsgroup1.mrcs 

and my micrograph file looks like this:

# version 30001


_rlnOpticsGroupName #1 
_rlnOpticsGroup #2 
_rlnMicrographOriginalPixelSize #3 
_rlnVoltage #4 
_rlnSphericalAberration #5 
_rlnAmplitudeContrast #6 
_rlnMicrographPixelSize #7 
opticsGroup1            1     1.050000   300.000000     2.700000     0.100000     1.050000 

# version 30001


_rlnCtfPowerSpectrum #1 
_rlnMicrographName #2 
_rlnMicrographMetadata #3 
_rlnOpticsGroup #4 
_rlnAccumMotionTotal #5 
_rlnAccumMotionEarly #6 
_rlnAccumMotionLate #7 
MotionCorr/job015/frames/2024-05-13_15_57_27_40S-1v2_86-1_00001_30_0_1-1_PS.mrc MotionCorr/job015/frames/2024-05-13_15_57_27_40S-1v2_86-1_00001_30_0_1-1.mrc MotionCorr/job015/frames/            1    42.934522    18.514962    24.419560 

However, when I extract my particles and run a 2D classification to confirm I don’t get any particles:


My initial thought was that something has to be off with the coordinate system, therefore I compared the coordinates from an ice crystal between a CryoSPARC processed micrograph and a Relion processed micrograph and the are the same (when taking in account that from CryoSPARC to Relion the Y-axis needs to be inverted)

Do you know what could’ve gone wrong during conversion? I tried the same protocol on a different dataset and there it worked (different microscope and no classification in Relion before tring to re-extract).


How did you compare them?

There must be some difference that slipped past then? You probably need --flipy as well btw, as the micrographs are upside down in Relion vs. cryoSPARC motion correction (in addition to the convention flip).

Are you using the latest pyem and your steps look like Method 0 from pyem wiki?

Thank you for the suggestions, and sorry for the late reply!

How did you compare them?

I downloaded the same micrograph one processed with CryoSPARC and one processed with Relion and opened them with ChimeraX. There I checked the coordinates for multiple distinct features. The x coordinate was the same, the y coordinate was different. After I manually inverted the y coordinate in Relion however, it was also the same value.

There must be some difference that slipped past then? You probably need --flipy as well btw, as the micrographs are upside down in Relion vs. cryoSPARC motion correction (in addition to the convention flip).

This I just tried (–flipy as an addition to --inverty when running and id ended up with no improvement as the 2D classes still look like the particles are extracted in the wrong position:

Are you using the latest pyem

Right now I’m using the pyem version from last october which I should update. Do I need a full re-installation for iit to work? I haven’t bothered with it yet because so far I didn’t have any troubles.

steps look like Method 0 from pyem wiki?

My steps are more or less the same: I exported the particles from CryoSPARC (previously without --flipy but the latest try was with --flipy) and re-extracted them from the Relion MotionCorr Job. The refinement did not work and there I tried 2D classification in order to see what’s wrong with the particles, the result is what you see above. Also I did no use your function --strip-uid and --micrograph-path and just edited them into the file before re-extracting.

There are some additional dependencies like starfile, otherwise you can just update pyem itself. I don’t remember when various things were fixed last year. Sounds like you have an old version where --inverty doesn’t do anything since you’re saying manually flipping the coordinates works.

Oh sorry, I think there was a misunderstanding: --inverty does work, I just manually did the inversion to see if --inverty gives the correct y-value.
However, I’ll update pyem and give it another go!

Here’s how it should work:

  • If you open the same micrograph cryosparc/patch motion and RelionCorr versions, the Relion one should be upside down as compared to cryosparc (or .tif unaligned sum).
  • cryosparc coordinate origin in bottom left, normalized coordinates
  • Relion coordinate origin in top left, integer pixel coordinates
  • ergo when different motion correction is used, and extraction is done in Relion, the two flips cancel (--inverty)
  • note cryosparc flips coordinates on import (i.e. Relion coords are imported correctly), thus --inverty is also needed to cycle coordinates in and out of cryosparc with unchanged values
  • so no invert argument is required when cryosparc Extract Particles and relion Extract are to be used on the same micrographs (original behavior since cryosparc didn’t have movie processing at first)
  • the size of the micrograph is taken from the cryosparc metadata not the new micrographs so they will require adjustment if the binning is different
  • if the images are upside down but we use the same particle orientations and defocus angles they will be wrong, --flipy corrects the particle metadata so that relion_reconstruct will work on the converted particles (if you change motion correction both flip metadata and invert coords will be needed!)
  • instead one could import the RelionCorr micrographs in cryosparc, run patch motion, import the particles and use Patch Extract to fix the particle CTFs, then extract, refine, and finally export with no invert arg - this is the safest method approach
  • --flip should not significantly affect 2D classification results

You should always be able to flip back and forth and one way will work…also there is --invertx but I don’t think you can need it unless you had vertical movies and they got rotated a certain way.

Thank you for that suggestion! I’ll try that the next couple of days and hopefully it’ll work!

FYI I did just have a case going from cryosparc to Relion MotionCorr, --flipy alone was the correct choice. (Relion 5).