This does work fine with the latest version of cryoSPARC for me, I think you are either passing the wrong passthrough file, or are giving the arguments out of order. The option arguments with -- should either be all after, or all before the positional input and output arguments. Also, you should use the passthrough_particles_all_classes.cs file with heterogeneous refinement jobs.
Since it seems to work for you, here is a recent example of mine. with cryosparc 2.4.0, pyem v0.3 (cloned on 2018/10/08).
I have combined in total 4 different “select 2D classes” jobs into one homogeneous refinement job.
Each tree was created by going down the road from importing micrographs, picking, extracting… to 2D classification.
user@machine:/data/scratch/user/2018-10-26$ /home/user/pyem/csparc2star.py --loglevel debug --passthrough P10/J65/passthrough_particles.cs P10/J65/cryosparc_P10_J65_005_particles.cs csparc_P10_J65.star
Detected CryoSPARC 2+ .cs file
Reading passthrough file
Particle passthrough detected
Concatenating passthrough fields: alignments2D/split, alignments2D/shift, alignments2D/pose, alignments2D/psize_A, alignments2D/error, alignments2D/error_min, alignments2D/resid_pow, alignments2D/slice_pow, alignments2D/image_pow, alignments2D/cross_cor, alignments2D/alpha, alignments2D/weight, alignments2D/pose_ess, alignments2D/shift_ess, alignments2D/class_posterior, alignments2D/class, alignments2D/class_ess, location/micrograph_uid, location/micrograph_path, location/micrograph_shape, location/center_x_frac, location/center_y_frac, pick_stats/ncc_score, pick_stats/power, pick_stats/template_idx, pick_stats/angle_rad
/home/user/pyem/pyem/util/util.py:74: FutureWarning: Numpy has detected that you may be viewing or writing to an array returned by selecting multiple fields in a structured array.
This code may break in numpy 1.16 because this will return a view instead of a copy – see release notes for details.
joint[:, offset:offset + size] = a.view(np.uint8).reshape(n, size)
Creating particle DataFrame from recarray
Directly copied fields: rlnDefocusAngle, rlnDetectorPixelSize, rlnCtfFigureOfMerit, rlnSphericalAberration, rlnAmplitudeContrast, rlnMicrographName, rlnCtfMaxResolution, rlnVoltage, rlnDefocusU, rlnPhaseShift, rlnDefocusV, rlnImageName, rlnMagnification
Converting normalized particle coordinates to absolute
Converted particle coordinates from normalized to absolute with subpixel origin
Converting DEFOCUSANGLE from degrees to radians
Converting PHASESHIFT from degrees to radians
Collecting particle parameters from most likely classes
Columns must be same length as key
A passthrough file may be required (check inside the cryoSPARC 2+ job directory)
Columns must be same length as key
Traceback (most recent call last):
File “/home/user/pyem/csparc2star.py”, line 42, in main
df = metadata.parse_cryosparc_2_cs(cs, passthrough=args.passthrough, minphic=args.minphic)
File “/home/user/pyem/pyem/metadata.py”, line 326, in parse_cryosparc_2_cs
[cs[names[c]][i] for i, c in enumerate(cls)]))
File “/home/user/miniconda2/lib/python2.7/site-packages/pandas/core/frame.py”, line 3116, in setitem
self._setitem_array(key, value)
File “/home/user/miniconda2/lib/python2.7/site-packages/pandas/core/frame.py”, line 3138, in _setitem_array
raise ValueError(‘Columns must be same length as key’)
ValueError: Columns must be same length as key
Hi,
I wonder if you ever solved the problem. I am getting the same problem.
Best,
Chiara
Unfortunately not. My guess is that if the particles originate from multiple individual import jobs the traceback gets lost during refinement.
My workaround is to immediately export particle coordinates after select 2D, re-extract with relion and reimport the combined stack into cryosparc.
This can be used for homogeneous refinement with appropriate export of alignment parameters.
However, in some cases I observe considerable amounts of misaligned particles during further (external) classifications. This seems to be more pronounced in cryosparc than in other suites, therefore double check your results.
I am using a simple import so theoretically it should work. I was wondering if it is just a cryosparc version problem. I am also using 2.4. I have tried both the heterogeneous refinement and the particle selection outputs. The particles extract well with relion but when I try to do a refinement it tells me that I am using helical particles.
I find it very strange.
Hi,
Anybody find the particles coordinates in star file converted from passthrough.cs (in the refine jobs) got wrong? Once using relion to re-extract particles, it complained extractParticlesFromOneFrame ERROR: particles lies completely outside micrograph. And it’s true, the coordinates are larger than the micrographs dimension value.
Though I can try to re-extract particles with unbinned pixels in cryoSPARC then I can using extracted particles to refine or class3D in relion. Just in my occasion, relion re-extract particles working more faster and maybe the memory is not enough, cryoSPARC extract bigger box (>500) usually stops with no error sign, I have to change to a smaller box size to run to the end.
Use --swapxy. Also use --boxsize if your refinement volume is a different size from the particles.
Cheers.
Sounds good, I would try. Thanks!
If this is still a lingering issue, here is a script to export only the particle coordinates from cryosparc to star files that can be imported in relion. https://github.com/tribell4310/reliosparc
1 Like