Bayesian Polishing of cryosparc-processed particles

closed

#1

Hi!

I have run in a roadblock in switching cryosparc and relion for the purpose of performing different processing steps on the same dataset. In particular, I am struggling to produce the correct star files as inputs for Bayesian Polishing. Please see my case below; I would really appreciate if anyone has suggestions about good workflows or scripts to make the transition workable.

Example scenario:
I import, patch motion correction, patch ctf estimation, particle picking and extraction, 2D classification, ab initio reconstruction and homogenous refinement in cryosparc. I find it to be a fast and robust way of getting an first pass reconstruction and good particle set.

Then, I use cspar2star.py (thanks @DanielAsarnow !) to export the stack from the .csv produced from the refinement and, with minor adjustments, import the particles into relion to harness the power of its 3D classifier, 3D auto-refine, beam tilt correction / ctf-refine.

Now, I would like to perform Bayesian Polishing, which requires (nominally, please correct me if I am wrong):
(1) a micrograph star file that points to motion-corrected mrc files with associated metadata files (is it possible to generate these for the cryosparc patch motion correction outputs?)
(2) a particle star file (e.g. from CtfRefine or 3DRefine)
(3) a PostProcess star file

Is it possible to use the cryosparc motion corrected micrographs in any way? Or is it easier to re-run the motion correction through the relion GUI, and then edit star files to point to the relion MotionCorr outputs rather than to the cryosparc ones? If so, will particle coordinates be compatible between the two outputs?

Thank you for reading and thank you for all past/future advice!

Stefan


#2

I think you will need to re run motion correction through the relion GUI, as you say - the alignment data used in Patch Motion is not saved anywhere AFAIK, at least not in a format that relion can read. Re-extraction of particles should work fine, but easy enough to test, just run a class2D straight after re-extraction to confirm all is good.

Cheers
Oli


#3

Thanks Oli! Will give it a shot asap.
I figured that if there was a solution, you’d be the one to have it… :slight_smile:


#4

I’ve tried everything that comes to mind, with no luck…

I’m dealing with energy-filtered K3-acquired tiffs. If I compare the mrcs output from motioncorrection in cryosparc and relion, they are in the same orientation (no rotaion of mirroring).

I’ve noted csparc2star.py converts the cryosparc metadata (either from homogeneous refinement or class select) to a star file that reports X,Y coordinates that do not fit onto the image dimesions unless they are inverted (@DanielAsarnow, I’d be happy to share some images if have interest in replicating the issue) . With the axes inverted, I have tried extracting with all combinations of flipped x adn y axes (none, x-only, y-only and both x and y axes flipped), with no success.

I am wondering if this is an issue with how cryosparc or pyem handles energy-filtered K3 tiffs?


#5

Use --swapxy . Cheers.


#6

Thank you for the advice; this equivalent to swapping _rlnCoordinateX with _rlnCoordinateY in the star file, right?


#7

No, it’s not (and as you already found that produces incorrect results). Feel free to look inside if you want to know how it works.


#8

Hi all,
Thanks @DanielAsarnow and @olibclarke for clarifications. CryoSPARC Patch Motion does in fact write out motion trajectories (as plain numpy array files in the job output dir) but Relion or other programs will not know how to read these.

Also the reason for the --swapxy (I believe - looking through pyem code) is that in cryoSPARC, we use the convention that 2D/3D arrays are always stored in C-order, with the fastest axis being the “x” axis, and we use a right hand coordinate system. That means that 2D arrays are stored with their slow (first) axis being Y, and second (fast) axis being X. Therefore the “shape” of a 2D array will be [ny,nx] and a 3D array shape will be [nz,ny,nx].
Vectors that describe a position (in 2D or 3D) also follow the right-hand convention, meaning that they are stored in standard geometry order, (x,y,z). So particle locations are stored as (x,y) pairs, while the shape of the micrograph is stored as [ny,nx] to maintain the conventions.


#9

@apunjani That’s all what I expected, but folks reported swapped coordinates sometimes and sometimes not (and at a certain point I swapped whether --swapxy was set true or set false on actually swapping). It seems the difference is whether or not movies or micrographs are originally imported, but I felt it was easier to add the option and try both than to figure it out. :slight_smile: