Import Relion particles with link to micrographs

lizellelubbe · February 11, 2021, 8:18am

Hi,
I am trying to import particles from Relion but have been unsuccessful despite reading previous posts regarding this on the discussion forum. I would like to use the results from Relion Refine as input for Non-uniform refine in CS.

I am using cryoSPARC version 3.1.0 and Relion version 3.1

These were the steps I followed:

Import particles from Relion Refine job (run_data.star) using particle meta path only without link to micrographs or specifying any other parameters. This contained some warnings that psize_A, rlnDetectorPixelSize and rlnMagnification were not read.
I therefore tried to import particles with links to micrographs. First imported motion corrected micrographs from different gridsquare directories:

micrographs data path:
/home/cryosparc/lbbliz002/bi24039-1/CHPC/EMU_relion_raw_2019/MotionCorr/job015/__/raw/GridSquare_7115*/Data/FoilHole_*.mrc
specified pixel size (after bin2x during motioncorr) of 1.06apix
voltage of 300
Cs of 2.7
total exposure of 43

These were imported succesfully with psize_A reported of 1.06 and example filename of:
J10/imported/FoilHole_8065276_Data_7120079_7120081_20191008_0757_fractions.mrc

I next tried to import the particles with links to these micrographs but failed. Could anyone please explain what I am doing wrong? I tried different ‘length of path prefix to cut’ values but still failed to link them. I don’t think I have to specify the suffix values in this case.
(imported micrographs were dropped into input micrographs and ignore raw data option was enabled. No length of cut was specified in this case nor any microscope overrides)

This is what my run_data.star file looks like:

version 30001

data_optics

loop_
_rlnOpticsGroupName #1
_rlnOpticsGroup #2
_rlnMtfFileName #3
_rlnMicrographOriginalPixelSize #4
_rlnVoltage #5
_rlnSphericalAberration #6
_rlnAmplitudeContrast #7
_rlnImagePixelSize #8
_rlnImageSize #9
_rlnImageDimensionality #10
opticsGroup1 1 mtf_k3_standard_300kV_FL2.star 0.530000 300.000000 2.700000 0.100000 1.060000 500 2

version 30001

data_particles

loop_
_rlnCoordinateX #1
_rlnCoordinateY #2
_rlnAutopickFigureOfMerit #3
_rlnImageName #4
_rlnMicrographName #5
_rlnOpticsGroup #6
_rlnCtfMaxResolution #7
_rlnCtfFigureOfMerit #8
_rlnDefocusU #9
_rlnDefocusV #10
_rlnDefocusAngle #11
_rlnCtfBfactor #12
_rlnCtfScalefactor #13
_rlnPhaseShift #14
_rlnAngleRot #15
_rlnAngleTilt #16
_rlnAnglePsi #17
_rlnOriginXAngst #18
_rlnOriginYAngst #19
_rlnClassNumber #20
_rlnNormCorrection #21
_rlnLogLikeliContribution #22
_rlnMaxValueProbDistribution #23
_rlnNrOfSignificantSamples #24
_rlnGroupName #25
_rlnRandomSubset #26
_rlnGroupNumber #27
4413.000000 770.000000 -2.05747 000001@Extract/job840/“doubleunderscore”/raw/GridSquare_7115372/Data/FoilHole_8065276_Data_7120079_7120081_20191008_0757_fractions.mrcs MotionCorr/job015/__/raw/GridSquare_7115372/Data/FoilHole_8065276_Data_7120079_7120081_20191008_0757_fractions.mrc 1 3.213273 0.203067 18864.640625 18662.230469 72.225395 0.000000 1.000000 0.000000 61.153502 78.351764 -131.04003 -0.35262 -0.89826 1
0.292398 1.232844e+06 0.113141 23 group_4 1 1

olibclarke · February 11, 2021, 8:27am

Hi,

This might not be it, but I think you need to move or copy your star file to the relion root dir, rather than leaving it in the job dir - otherwise the relative paths will not resolve.

Cheers
Oli

lizellelubbe · February 11, 2021, 8:38am

Thanks for the quick reply @olibclarke!

Unfortunately still got the same error after copying the star file to the relion project directory. Does the fact that the example source and query appear to match mean that cutting of the paths is not necessary when importing? Do you have any other ideas that I can try?

BTW should I be worried about the warning messages for the star file? Is there a way to manually set these params?

olibclarke · February 11, 2021, 8:50am

Yeah that’s the other thing I meant to suggest - have you tried manually setting the pixel size? you can do so when you import (in the section “Microscope Parameter Overrides” - you don’t need to set them all, just setting pixel size is fine)

lizellelubbe · February 11, 2021, 9:09am

Okay manually setting pixel size got rid of those warnings but it still fails to link the micrographs

olibclarke · February 11, 2021, 9:23am

Have you double checked that the pattern you use for importing catches all micrographs from all grid squares? does the number of micrographs imported match the number of unique micrographs referenced in the star file?

lizellelubbe · February 11, 2021, 10:54am

Ah you’re right! I accidentally put a ‘5’ in the pattern and there were 3 gridsquares with a ‘6’ that got skipped. Such a silly mistake. The import and linking has worked - thank you!

I want to now use these particles for NU refinement but it fails if I use the imported refined map from Relion as volume without specifying a mask. Should I first do a homogeneous refinement in Cryosparc before proceeding to NU? Should the mask for NU be created from this map?

lizellelubbe · February 11, 2021, 11:03am

I just tried the ab initio reconstruction job and it failed. I used the default settings for ab initio except for setting number of particles to 9500.

Did something go wrong with the particle import although there was no error anymore?
This was the output from particle import:

olibclarke · February 11, 2021, 11:12am

Hmm yes that should work not sure why it is failing

You could also just try importing without linking to mics? (remembering to manually set pixel size) - the only reason to link to mics is if you want to re-extract within cryosparc

lizellelubbe · February 11, 2021, 3:05pm

Okay, the import without link to mics worked and ab initio is now running. Thanks for all the help Oli - I am a real Cryosparc newbie! But the program looks very cool
Just one last quick question, the options that are given in the job builder are very limited while there are many other parameters in the input tab as the job is running. How would I set these parameters prior to job submission?

spunjani · February 11, 2021, 3:17pm

@lizellelubbe, please toggle the ‘Advanced mode’ switch on the top of the Job Builder to see advanced parameters:

Also described in the guide: https://guide.cryosparc.com/processing-data/user-interface-and-usage-guide/create-and-build-jobs#default-and-spec-parameters

Thanks!

lizellelubbe · February 11, 2021, 3:20pm

Thanks that’s great! I’ll follow the guide

chenx469 · May 4, 2021, 9:34pm

Hi, @lizellelubbe,

Did you do more modification when running ab initio using the imported particle without link to mics? I still have the same error message as in your post.

Import job was complete and looks fine.

My run_data.star file is

version 30001

data_optics

loop_
_rlnOpticsGroupName #1
_rlnOpticsGroup #2
_rlnMtfFileName #3
_rlnMicrographOriginalPixelSize #4
_rlnVoltage #5
_rlnSphericalAberration #6
_rlnAmplitudeContrast #7
_rlnImagePixelSize #8
_rlnImageSize #9
_rlnImageDimensionality #10
opticsGroup1 1 …/MTF_K3_200kV_standard/MTF_K3_200kV.star 0.470000 200.000000 2.700000 0.100000 0.940000 200 2

version 30001

data_particles

loop_
_rlnCoordinateX #1
_rlnCoordinateY #2
_rlnAutopickFigureOfMerit #3
_rlnImageName #4
_rlnMicrographName #5
_rlnOpticsGroup #6
_rlnCtfMaxResolution #7
_rlnCtfFigureOfMerit #8
_rlnDefocusU #9
_rlnDefocusV #10
_rlnDefocusAngle #11
_rlnCtfBfactor #12
_rlnCtfScalefactor #13
_rlnPhaseShift #14
_rlnAngleRot #15
_rlnAngleTilt #16
_rlnAnglePsi #17
_rlnOriginXAngst #18
_rlnOriginYAngst #19
_rlnClassNumber #20
_rlnNormCorrection #21
_rlnLogLikeliContribution #22
_rlnMaxValueProbDistribution #23
_rlnNrOfSignificantSamples #24
_rlnGroupName #25
_rlnGroupNumber #26
_rlnRandomSubset #27

Do you have any suggestion? Thank you very much.

Xiang

lizellelubbe · May 5, 2021, 4:48pm

Hi @chenx469,

Sorry that you are running into this error message. I got it to work by copying the star file to the relion directory (just one up from its original location in eg. Refine3D) and used that path in the Particle meta path field during import to cryosparc. I also specified the pixel size under Microscope Parameter Overrides. Have you tried that?

I have since managed to run import with link to micrographs and now routinely use this since it allows re-extraction within cryosparc.
I did this by running an import micrographs job with micrographs data path (/path/to/Reliondirectory/MotionCorr/job0XX/__/raw/GridSquare*/Data/FoilHole*.mrc), pixel size, voltage, Cs and exposure specified. Then, I run import particles as before but with link to that import micrographs job.

Hope this helps and that you solve your issue soon!

chenx469 · May 5, 2021, 5:02pm

Hi, @lizellelubbe,

I did not copy the star file to the relion directory and it must be the reason that I got trouble. Thank you so much for your quick reply. I will test it as you suggested.

Xiang