Csparc2star.py converted .star file does not have rlnCoordinateX and rlnCoordinateY of particle

Troubleshooting
1

Hi all,

I tried to covert .cs file to .star file using the following command, which failed to calculate the rlnCoordinateX and rlnCoordinateY of particles.

When I use “–copy-micrograph-coordinates” it showed the following error.
“Coordinates merge key: None”

I was trying to re-extract the particles using relion3.0 and it failed due to the missing information of rlnCoordinateX and rlnCoordinateY.

csparc2star.py cryosparc_P1_J65_005_particles.cs cryosparc_P1_J65_005_184k.star --loglevel debugDetected CryoSPARC 2+ .cs file
Creating particle DataFrame from recarray
Directly copied fields: rlnDefocusAngle, rlnDetectorPixelSize, rlnCtfFigureOfMerit, rlnSphericalAberration, rlnAmplitudeContrast, rlnCtfMaxResolution, rlnVoltage, rlnDefocusU, rlnPhaseShift, rlnDefocusV, rlnImageName, rlnMagnification
Converting DEFOCUSANGLE from degrees to radians
Converting PHASESHIFT from degrees to radians
Assigning parameters 2D classes or single 3D class
Changing RANDOMSUBSET to 1-based index
Changing CLASS to 1-based index
Converting Rodrigues coordinates to Euler angles
Converted Rodrigues coordinates to Euler angles
Output fields: rlnDefocusAngle, rlnDetectorPixelSize, rlnCtfFigureOfMerit, rlnSphericalAberration, rlnAmplitudeContrast, rlnCtfMaxResolution, rlnVoltage, rlnDefocusU, rlnPhaseShift, rlnDefocusV, rlnImageName, rlnMagnification, rlnOriginX, rlnOriginY, rlnAngleRot, rlnAngleTilt, rlnAnglePsi, rlnClassNumber, rlnRandomSubset

2

You didn’t provide any source of coordinates. Give an original star file with coordinates to --copy-micrograph-coordinates or a cryoSPARC passthrough file to --passthrough.

3

Hi Daniel,

I tried earlier and here is the log info.

csparc2star.py cryosparc_P1_J65_005_particles.cs cryosparc_P1_J65_005_184k.star --passthrough passthrough_particles.cs --loglevel debug
Detected CryoSPARC 2+ .cs file
Reading passthrough file
Particle passthrough detected
Concatenating passthrough fields: alignments2D/split, alignments2D/shift, alignments2D/pose, alignments2D/psize_A, alignments2D/error, alignments2D/error_min, alignments2D/resid_pow, alignments2D/slice_pow, alignments2D/image_pow, alignments2D/cross_cor, alignments2D/alpha, alignments2D/weight, alignments2D/pose_ess, alignments2D/shift_ess, alignments2D/class_posterior, alignments2D/class, alignments2D/class_ess, location/micrograph_uid, location/micrograph_path, location/micrograph_shape, location/center_x_frac, location/center_y_frac
Creating particle DataFrame from recarray
Directly copied fields: rlnDefocusAngle, rlnDetectorPixelSize, rlnCtfFigureOfMerit, rlnSphericalAberration, rlnAmplitudeContrast, rlnMicrographName, rlnCtfMaxResolution, rlnVoltage, rlnDefocusU, rlnPhaseShift, rlnDefocusV, rlnImageName, rlnMagnification
Converting normalized particle coordinates to absolute
Converted particle coordinates from normalized to absolute
Converting DEFOCUSANGLE from degrees to radians
Converting PHASESHIFT from degrees to radians
Collecting particle parameters from most likely classes
Columns must be same length as key
A passthrough file may be required (check inside the cryoSPARC 2+ job directory)
Columns must be same length as key
Traceback (most recent call last):
File “/opt/pyem/csparc2star.py”, line 42, in main
df = metadata.parse_cryosparc_2_cs(cs, passthrough=args.passthrough, minphic=args.minphic)
File “/opt/pyem/pyem/metadata.py”, line 349, in parse_cryosparc_2_cs
[cs[names[c]][i] for i, c in enumerate(cls)]))
File “/home/spuser/.local/lib/python2.7/site-packages/pandas/core/frame.py”, line 3116, in setitem
self._setitem_array(key, value)
File “/home/spuser/.local/lib/python2.7/site-packages/pandas/core/frame.py”, line 3138, in _setitem_array
raise ValueError(‘Columns must be same length as key’)
ValueError: Columns must be same length as key

csparc2star.py cryosparc_P1_J65_005_particles.cs cryosparc_P1_J65_005_184k.star --copy-micrograph-coordinates goodparticles_BoxNet2Mask_20180918.star
Traceback (most recent call last):
File “/opt/pyem/csparc2star.py”, line 103, in
sys.exit(main(parser.parse_args()))
File “/opt/pyem/csparc2star.py”, line 68, in main
df = star.smart_merge(df, coord_star, fields=fields, key=key)
File “/opt/pyem/pyem/star.py”, line 89, in smart_merge
s2 = s2.set_index(key, drop=False)
File “/home/spuser/.local/lib/python2.7/site-packages/pandas/core/frame.py”, line 3909, in set_index
level = frame[col]._values
File “/home/spuser/.local/lib/python2.7/site-packages/pandas/core/frame.py”, line 2688, in getitem
return self._getitem_column(key)
File “/home/spuser/.local/lib/python2.7/site-packages/pandas/core/frame.py”, line 2695, in _getitem_column
return self._get_item_cache(key)
File “/home/spuser/.local/lib/python2.7/site-packages/pandas/core/generic.py”, line 2489, in _get_item_cache
values = self._data.get(item)
File “/home/spuser/.local/lib/python2.7/site-packages/pandas/core/internals.py”, line 4124, in get
raise ValueError(“cannot label index with a null key”)
ValueError: cannot label index with a null key

4

What job type are you converting? Did you originally pick outside cryosparc and import a star file?

5

I picked the particles using Warp (.star) and then imported into the cryosparc.
After 2D classification and 3D classification, I ran the homo-refine and then I exported particles using csparc2star.py.
I used Warp .star file (containing rlnCoordinateX and rlnCoordinateY) for --copy-micrograph-coordinates.
Is it because csparc2star.py cannot handle .star file from Warp?

Uhnsoo

6

It doesn’t matter where the .star files come from as long as they are correct and use the standard field names.

You should be able to use --copy-micrograph-coordinates. Please post the last line of your original and converted star files for comparison (I just want to see the rlnImageName, rlnImageOriginalName, and rlnMicrographName fields). tail -1 file.star is the easiest way (or -2 if there’s an extra blank line at the end).