Importing particle.star (from relion 3.1) fails in cryosparc V3.0.1

[Hormel]

Hi,

I got a problem when importing particle.star file (from relion 3.1) using cryoparc V3.0.1. The error is shown as follows:
…
License is valid.

Running job on master node

[CPU: 66.0 MB] Project P1 Job J49 Started

[CPU: 66.0 MB] Master running v3.0.1, worker running v3.0.1

[CPU: 66.1 MB] Running on lane default

[CPU: 66.1 MB] Resources allocated:

[CPU: 66.1 MB] Worker: xxxx

[CPU: 66.1 MB] --------------------------------------------------------------

[CPU: 66.1 MB] Importing job module for job type import_particles…

[CPU: 206.5 MB] Job ready to run

[CPU: 206.7 MB] ***************************************************************

[CPU: 206.7 MB] Importing particles from /xxxx/xxxxx/xxxx/Select/job004/particles.star

[CPU: 206.7 MB] File extension is star

[CPU: 206.7 MB] Importing star file.

[CPU: 224.9 MB] --------------------------------------------------------------

[CPU: 224.9 MB] Loaded star file with 56710 items

[CPU: 224.9 MB] Fields loaded from star file: [‘rlnImageName’, ‘rlnMicrographName’, ‘rlnCoordinateX’, ‘rlnCoordinateY’, ‘rlnAngleRot’, ‘rlnAngleTilt’, ‘rlnAnglePsi’, ‘rlnOriginXAngst’, ‘rlnOriginYAngst’, ‘rlnDefocusU’, ‘rlnDefocusV’, ‘rlnDefocusAngle’, ‘rlnPhaseShift’, ‘rlnCtfBfactor’, ‘rlnRandomSubset’, ‘rlnClassNumber’, ‘rlnOpticsGroup’, ‘rlnGroupNumber’, ‘rlnNormCorrection’, ‘rlnLogLikeliContribution’, ‘rlnMaxValueProbDistribution’, ‘rlnNrOfSignificantSamples’]

[CPU: 224.9 MB] --------------------------------------------------------------

[CPU: 224.9 MB] Reading particle data locations…

[CPU: 224.9 MB] Reading rlnImageName to get indices and paths…

[CPU: 249.3 MB] Warning: Parameter particle_blob_path was set, which overrides rlnImageName despite the latter being present in the input star file.

[CPU: 249.3 MB] Parameter particle_blob_path was set and is a directory, so will be used as the search base for finding referenced data paths in the star file.

[CPU: 249.3 MB] Searching for linked data files…

[CPU: 249.3 MB] --------------------------------------------------------------

[CPU: 249.3 MB] Compiling CTF information…

[CPU: 252.8 MB] Warning: No micrographs were connected as inputs, so correspondences cannot be found, so output will not contain pick locations for particles.

[CPU: 252.8 MB] --------------------------------------------------------------

[CPU: 252.8 MB] Compiling particle pose information…

[CPU: 252.8 MB] Converting euler angles…

[CPU: 252.8 MB] Converting rlnOriginXAngst to pixels using parameter psize_A
[CPU: 269.6 MB] Traceback (most recent call last):
File “cryosparc_master/cryosparc_compute/run.py”, line 84, in cryosparc_compute.run.main
File “/home/cryosparc_user/software/cryosparc/cryosparc_master/cryosparc_compute/jobs/imports/run.py”, line 307, in run_import_particles
poseds.data[‘shift’][idx] = make_shift_rln(starrow)
ValueError: could not broadcast input array from shape (2,56710) into shape (2)

Any comments and suggestions wil be greatly appreciated!

Thanks,
Bin

[dzy]

Have you solved the problem?
I encountered the same error.

[Hormel]

Yes, I got clues from the previous posts.

Importing symmetry expanded particles from Relion 3.1 doesn't work [bug]

Hi, Recent versions of cryosparc have the capacity to import particles with orientation data intact from a star file. This is necessary for working with symmetry expanded particle sets. However, if the offsets are specified in Angstroms, (_rlnOriginXAngst, _rlnOriginYAngst), rather than pixels, it fails, as it is looking for _rlnOriginX/_rlnOriginY. The former is what is now written by Relion 3.1. Would it be possible to add support for import of this column type? I can make it work by just m…

Import particle.star from Relion-3.1 to cryosparc

Hi Oli, thanks very much for your reply. I did notice that CS didn’t read my data_optics table, which is odd. After including those manually, it did import despite two warnings : [CPU: 250.5 MB] star file fields: [CPU: 250.5 MB] Index([u’rlnImageName’, u’rlnMicrographName’, u’rlnCoordinateX’, u’rlnCoordinateY’, u’rlnAnglePsi’, u’rlnOriginX’, u’rlnOriginY’, u’rlnMagnification’, u’rlnDetectorPixelSize’, u’rlnDefocusU’, u’rlnDefocusV’, u’rlnDefocusAngle’, u’rlnPhaseShift’,…

Solved the problem by
(1) manually changing " _rlnOriginXAngst, _rlnOriginYAngst" to " _rlnOriginX , _rlnOriginY"
(2) modifying the values in these two columns by being divided by pixel size using awk

[ajian]

I follow your suggestion by dividing, still not working, please see below
License is valid.

Running job on master node

[CPU: 66.1 MB] Project P2 Job J43 Started

[CPU: 66.1 MB] Master running v3.0.1, worker running v3.0.1

[CPU: 66.2 MB] Running on lane default

[CPU: 66.2 MB] Resources allocated:

[CPU: 66.2 MB] Worker: localhost

[CPU: 66.2 MB] --------------------------------------------------------------

[CPU: 66.2 MB] Importing job module for job type import_particles…

[CPU: 205.5 MB] Job ready to run

[CPU: 205.5 MB] ***************************************************************

[CPU: 205.5 MB] Importing particles from /data/cryoem/HeR_Nb/20201207_9_300kv/Polish/job023/shiny.star

[CPU: 205.5 MB] File extension is star

[CPU: 205.5 MB] Importing star file.

[CPU: 250.9 MB] --------------------------------------------------------------

[CPU: 250.9 MB] Loaded star file with 213811 items

[CPU: 250.9 MB] Fields loaded from star file: [‘rlnImageName’, ‘rlnMicrographName’, ‘rlnCoordinateX’, ‘rlnCoordinateY’, ‘rlnAngleRot’, ‘rlnAngleTilt’, ‘rlnAnglePsi’, ‘rlnOriginX’, ‘rlnOriginY’, ‘rlnDefocusU’, ‘rlnDefocusV’, ‘rlnDefocusAngle’, ‘rlnPhaseShift’, ‘rlnCtfBfactor’, ‘rlnOpticsGroup’, ‘rlnRandomSubset’, ‘rlnClassNumber’]

[CPU: 250.9 MB] --------------------------------------------------------------

[CPU: 250.9 MB] Reading particle data locations…

[CPU: 250.9 MB] Reading rlnImageName to get indices and paths…

[CPU: 333.4 MB] Searching for linked data files…

[CPU: 333.4 MB] Traceback (most recent call last):
File “cryosparc_master/cryosparc_compute/run.py”, line 84, in cryosparc_compute.run.main
File “/home/lrss/softwares/cryosparc/cryosparc_master/cryosparc_compute/jobs/imports/run.py”, line 120, in run_import_particles
all_raw_paths = sorted(list(set(blobdf[‘path’])))
TypeError: ‘<’ not supported between instances of ‘NoneType’ and ‘str’

[Hormel]

@ajian I recommend you check the modified star file using gedit. AWK command should not change the original format of the file (separate space, etc). It looked like it has been introduced some typeerrors.

[alfredo07]

Hi Rashed,
could you please share the command you use to change the values of the _rlnOriginXAngst, _rlnOriginYAngst ? I am facing the same problem than you.

Best,
Alfredo

[Hormel]

@alfredo07
Here is my procedure:

(1) remove the header info in the star file.

(2) apply the following commands:

awk ‘{split($0, a, FS, seps); a[8]=a[8]/0.89; for (i=1;i<=NF;i++) printf("%s%s", a[i], seps[i]); print “”}’ input.star > output1.star

(0.89 is the pixel size of my image data, 8 refers to the 8th column)

awk ‘{split($0, a, FS, seps); a[9]=a[9]/0.89; for (i=1;i<=NF;i++) printf("%s%s", a[i], seps[i]); print “”}’ output1.star > output2.star

(9 refers to the 9th column)

(3) copy back the original header info to output2.star, and then change ‘rlnOriginXAngst’, ‘rlnOriginYAngst’ to ‘rlnOriginX’, ‘rlnOriginY’ in the header.

Now it should be ready for cryosparc V3.0.1.

[Hormel]

You may encounter error if the version of awk in your computer is too old. In such case, upgrade to gawk should work.

[alfredo07]

Many thanks @Hormel . It worked with gawk and correcting some symbols that were copy/past wrong