Cryosparc particles to Relion back to cryosparc

Hi,

I did all processing on cryosparc and obtained a ~3.7A map with ~800K particles.

I used:
csparc2star.py X.cs Y.star
… to generate the star file for Classification and Refinement on Relion and after processing I have obtained a better map although Relion’s resolution estimate is lower (~3.98A).

I wanted to re-import the set of particles from Relion into cryosparc for CTF/local refinement. Cannot figure out how to do this…

On the import particle job, I am not sure which all fields I need to fill and also with what files. The particle meta path/data path - what files should I use? The Relion job folder has the run_data.star with the particle information but I cannot seem to figure out the rest very clearly.

Any help here would be very appreciated!

Hi @schow,

You should use the run_data.star file for the Particle meta path.

For the Particle data path, you should leave it blank if the paths in the star file are relative to the star file location. Otherwise, you should set the data path to the folder that contains all the particle .mrc files.

@spunjani Thank you for your reply.

I did this and ran the particle input job. It completed and the # of particles were correct but there were a bunch of warnings (see below).

I used the particles from above and tried to start a local refinement job with volume/mask from an earlier cryosparc job and got an error - it seems some essential information is missing in the run_data.star and I’m not sure how to put it back. Any suggestions?

Warnings during import particle job:

[CPU: 551.1 MB]  Loaded star file with 684804 items
[CPU: 551.1 MB]  Fields loaded from star file:  ['rlnVoltage', 'rlnSphericalAberration', 'rlnAmplitudeContrast', 'rlnOpticsGroup', 'rlnImageSize', 'rlnImagePixelSize', 'rlnImageDimensionality', 'rlnImageName', 'rlnAngleRot', 'rlnAngleTilt', 'rlnAnglePsi', 'rlnOriginXAngst', 'rlnOriginYAngst', 'rlnDefocusU', 'rlnDefocusV', 'rlnDefocusAngle', 'rlnPhaseShift', 'rlnCtfBfactor', 'rlnRandomSubset', 'rlnClassNumber', 'rlnGroupNumber', 'rlnNormCorrection', 'rlnLogLikeliContribution', 'rlnMaxValueProbDistribution', 'rlnNrOfSignificantSamples']
[CPU: 551.1 MB]  **Warning**: The input star file does not contain rlnDetectorPixelSize and rlnMagnification, and parameter psize_A was not set. Output will not contain data for particles.
[CPU: 551.1 MB]  --------------------------------------------------------------
[CPU: 551.1 MB]  Compiling CTF information...
[CPU: 551.3 MB]  **Warning**: The input star file does not contain rlnCoordinateX and rlnCoordinateY and rlnMicrographName. Output will not contain pick locations for particles.
[CPU: 551.3 MB]  **Warning:** Output will not contain particle orientations since parameter 'psize_A' was not set.
[CPU: 551.3 MB]  --------------------------------------------------------------
[CPU: 551.3 MB]  Particle information has now been imported for 684804 particles, creating outputs...
[CPU: 575.8 MB]  --------------------------------------------------------------
[CPU: 575.8 MB]  Loaded 684804 particles.
[CPU: 575.8 MB]    Common fields: 
[CPU: 575.8 MB]            ctf/accel_kv :  {300.0}
[CPU: 575.8 MB]               ctf/cs_mm :  {2.7}
[CPU: 575.8 MB]        ctf/amp_contrast :  {0.1}
[CPU: 575.8 MB]        ctf/exp_group_id :  {1, 2}
[CPU: 209.5 MB]  --------------------------------------------------------------
[CPU: 209.5 MB]  Compiling job outputs...
[CPU: 209.5 MB]  Updating job size...
[CPU: 209.5 MB]  Exporting job and creating csg files...

Error when I tried to start local refinement from the imported particles:

[CPU: 82.1 MB]   Traceback (most recent call last):
  File "cryosparc_worker/cryosparc_compute/run.py", line 52, in cryosparc_compute.run.main
  File "/home/cryosparc_user/software/cryosparc/cryosparc2_worker/cryosparc_compute/jobs/runcommon.py", line 847, in check_default_inputs
    assert False, 'Non-optional inputs from the following input groups and their slots are not connected: ' + missing_inputs + '. Please connect all required inputs.'
AssertionError: **Non-optional inputs from the following input groups and their slots are not connected**: particles.blob, particles.alignments3D. Please connect all required inputs.

Hi @schow,

Sorry for the inconvenience- the alignments3D results are missing because they weren’t imported.

Can you clear the job, put it back in “Building” mode (by either clicking the “building” button on the job card or selecting the job, then pressing “B”, and set the Pixel Size (Angstrom) parameter? There is currently a bug in v3.1 where the pixel size isn’t read from the .star file: Relion 3.1 import - rlnImagePixelSize

Thank you for your reply @stephan

I tried to set the pixel size and got an error (see below). One question here: The particles are all binned/2 so when I enter the pixel size I should use the binned pixel size right (1.38) as opposed to the unbind (0.69).

Error during particle import:
[CPU: 206.7 MB] Importing particles from /data2/UPenn_pH8_KCl_lowmag/P4/Refine3D/job104/run_data.star
[CPU: 206.7 MB] File extension is star
[CPU: 206.7 MB] Importing star file.
[CPU: 559.9 MB] --------------------------------------------------------------
[CPU: 559.9 MB] Loaded star file with 684804 items
[CPU: 559.9 MB] Fields loaded from star file: [‘rlnVoltage’, ‘rlnSphericalAberration’, ‘rlnAmplitudeContrast’, ‘rlnOpticsGroup’, ‘rlnImageSize’, ‘rlnImagePixelSize’, ‘rlnImageDimensionality’, ‘rlnImageName’, ‘rlnAngleRot’, ‘rlnAngleTilt’, ‘rlnAnglePsi’, ‘rlnOriginXAngst’, ‘rlnOriginYAngst’, ‘rlnDefocusU’, ‘rlnDefocusV’, ‘rlnDefocusAngle’, ‘rlnPhaseShift’, ‘rlnCtfBfactor’, ‘rlnRandomSubset’, ‘rlnClassNumber’, ‘rlnGroupNumber’, ‘rlnNormCorrection’, ‘rlnLogLikeliContribution’, ‘rlnMaxValueProbDistribution’, ‘rlnNrOfSignificantSamples’]
[CPU: 559.9 MB] --------------------------------------------------------------
[CPU: 559.9 MB] Reading particle data locations…
[CPU: 559.9 MB] Reading rlnImageName to get indices and paths…
[CPU: 844.4 MB] Searching for linked data files…
[CPU: 844.4 MB] Warning: Could not find file J14/extract/10002699502160694249_FoilHole_13654662_Data_13539109_13539111_20201213_181157_fractions_patch_aligned_doseweighted_particles.mrcs, searching within /data2/UPenn_pH8_KCl_lowmag/P4/Refine3D/job104. Ignoring these images. Will not warn again.
[CPU: 844.4 MB] Warning: There are 5123 files referenced by the input star file that could not be found.
[CPU: 844.4 MB] Traceback (most recent call last):
File “cryosparc_master/cryosparc_compute/run.py”, line 84, in cryosparc_compute.run.main
File “/home/cryosparc_user/software/cryosparc/cryosparc2_master/cryosparc_compute/jobs/imports/run.py”, line 141, in run_import_particles
assert False, “Files referenced by the input star file are missing.”
AssertionError: Files referenced by the input star file are missing.

The mrcs/mrc files are not in the folder that cryosparc is searching in. How can I direct to the correct folder?

Thank you in advance!

Ok so I changed the particle data path to project to the directory which contained all the cryosparc jobs that I had done previously and which also had the .mrcs files. The particles loaded correctly and it seems that I need to use the pixel size that I have been using for my reconstructions (so bin/2). Can you confirm if this is correct?

The particles get displayed and look correct. I was also able to launch the local refinement job. There were a couple of warnings display during the import - is this okay?

Message during particle import job:
[CPU: 551.4 MB] Loaded star file with 684804 items
[CPU: 551.4 MB] Fields loaded from star file: [‘rlnVoltage’, ‘rlnSphericalAberration’, ‘rlnAmplitudeContrast’, ‘rlnOpticsGroup’, ‘rlnImageSize’, ‘rlnImagePixelSize’, ‘rlnImageDimensionality’, ‘rlnImageName’, ‘rlnAngleRot’, ‘rlnAngleTilt’, ‘rlnAnglePsi’, ‘rlnOriginXAngst’, ‘rlnOriginYAngst’, ‘rlnDefocusU’, ‘rlnDefocusV’, ‘rlnDefocusAngle’, ‘rlnPhaseShift’, ‘rlnCtfBfactor’, ‘rlnRandomSubset’, ‘rlnClassNumber’, ‘rlnGroupNumber’, ‘rlnNormCorrection’, ‘rlnLogLikeliContribution’, ‘rlnMaxValueProbDistribution’, ‘rlnNrOfSignificantSamples’]
[CPU: 551.4 MB] --------------------------------------------------------------
[CPU: 551.4 MB] Reading particle data locations…
[CPU: 551.4 MB] Reading rlnImageName to get indices and paths…
[CPU: 835.9 MB] Warning: Parameter particle_blob_path was set, which overrides rlnImageName despite the latter being present in the input star file.
[CPU: 835.9 MB] Parameter particle_blob_path was set and is a directory, so will be used as the search base for finding referenced data paths in the star file.
[CPU: 835.9 MB] Searching for linked data files…
[CPU: 836.0 MB] --------------------------------------------------------------
[CPU: 836.0 MB] Compiling CTF information…
[CPU: 846.3 MB] Warning: The input star file does not contain rlnCoordinateX and rlnCoordinateY and rlnMicrographName. Output will not contain pick locations for particles.
[CPU: 846.3 MB] --------------------------------------------------------------
[CPU: 846.3 MB] Compiling particle pose information…
[CPU: 846.3 MB] Converting euler angles…
[CPU: 846.3 MB] Converting rlnOriginXAngst to pixels using parameter psize_A
[CPU: 976.6 MB] --------------------------------------------------------------
[CPU: 976.6 MB] Particle information has now been imported for 684804 particles, creating outputs…
[CPU: 976.6 MB] Found references to 5123 unique data files
[CPU: 978.5 MB] Import paths were unique at level -1
[CPU: 978.5 MB] Example imported relative path:
J266/imported/10002699502160694249_FoilHole_13654662_Data_13539109_13539111_20201213_181157_fractions_patch_aligned_doseweighted_particles.mrcs
[CPU: 979.0 MB] Reading MRC file headers to check shape…
[CPU: 1.09 GB] --------------------------------------------------------------
[CPU: 1.09 GB] Loaded 684804 particles.
[CPU: 1.09 GB] Common fields:
[CPU: 1.09 GB] ctf/accel_kv : {300.0}
[CPU: 1.09 GB] ctf/cs_mm : {2.7}
[CPU: 1.09 GB] ctf/amp_contrast : {0.1}
[CPU: 1.09 GB] ctf/exp_group_id : {1, 2}
[CPU: 1.09 GB] blob/psize_A : {1.38}
[CPU: 1.09 GB] blob/shape : [180 180]
[CPU: 1.09 GB] --------------------------------------------------------------
[CPU: 1.09 GB] Making plots…