Cryosparc to relion

Import and Export
1

Dear All,

I am experiencing some issues exporting 2D classification results back into relion where the particles were originally extracted. I am using the latest relion, pyem, conda, and cryosparc.

The command:
/data/CRYOSPARC/pyem/csparc2star.py --copy-micrograph-coordinates /data/XXX/Extract/job41/particles.star /data/CRYOSPARC/USERS/Omid/P1/J154/particles_selected.cs from_cryosparc.star

I now get the following error:
Traceback (most recent call last):
File “/data/CRYOSPARC/pyem/csparc2star.py”, line 28, in
from pyem import metadata
File “/data/CRYOSPARC/pyem/pyem/ init .py”, line 3, in
from . import metadata
File “/data/CRYOSPARC/pyem/pyem/metadata.py”, line 401
ptdf = util.dataframe_from_records_mapped(pt, {**general, **micrograph})

I removed the top part (data_optics section) as suggested in the forums in order to import it into cryosparc. I assume that pyem will re-import this info back from cryosparc.

I saw your recent post (Csparc2star.py rlnMicrographName error) as well on this issue but it was not clear to me what the command should be to get around this problem or if this is the same problem at all.

Best,
Omid

2

Hi All,

After reinstalling miniconda and pyem I get another error but I think it is related:

/data/CRYOSPARC/miniconda3/pyem/csparc2star.py --copy-micrograph-coordinates /data/XXX/movies_raw/Sep06_collect/Extract/job41/particles.star /data/CRYOSPARC/USERS/Omid/P1/J154/particles_selected.cs from_cryosparc.star
Traceback (most recent call last):
File “/data/CRYOSPARC/miniconda3/pyem/csparc2star.py”, line 28, in
from pyem import metadata
File “/data/CRYOSPARC/miniconda3/pyem/pyem/init.py”, line 3, in
from . import metadata
File “/data/CRYOSPARC/miniconda3/pyem/pyem/metadata.py”, line 23, in
from . import star
File “/data/CRYOSPARC/miniconda3/pyem/pyem/star.py”, line 28, in
from pyem.util import natsort_values
File “/data/CRYOSPARC/miniconda3/pyem/pyem/util/init.py”, line 3, in
from .util import *
File “/data/CRYOSPARC/miniconda3/pyem/pyem/util/util.py”, line 28, in
from … import vop
File “/data/CRYOSPARC/miniconda3/pyem/pyem/vop/init.py”, line 3, in
from .vop import *
File “/data/CRYOSPARC/miniconda3/pyem/pyem/vop/vop.py”, line 22, in
from pyfftw.interfaces.numpy_fft import rfftn
File “/data/CRYOSPARC/miniconda3/lib/python3.7/site-packages/pyfftw/init.py”, line 43, in
from . import interfaces
File “/data/CRYOSPARC/miniconda3/lib/python3.7/site-packages/pyfftw/interfaces/init.py”, line 237, in
from . import scipy_fftpack
File “/data/CRYOSPARC/miniconda3/lib/python3.7/site-packages/pyfftw/interfaces/scipy_fftpack.py”, line 65, in
from scipy.fftpack import (dct, idct, dst, idst, diff, tilbert, itilbert,
ImportError: cannot import name ‘_fftpack’ from ‘scipy.fftpack’ (/data/CRYOSPARC/miniconda3/lib/python3.7/site-packages/scipy/fftpack/init.py)

Omid

3

Dear @DanielAsarnow,

I moved the post from github to here since more people might see it. Although I never solved the issue fully, I did make the symbolic link as suggested at the bottom of the installation wiki. So now the environment seem to be reading in correctly and csparc2star works on its own without any inputs. The issue was related to reading in the correct python.

However I am still getting an error:

(pyem) [omid@c105445 Sort1]$ /data/CRYOSPARC/pyem/csparc2star.py --copy-micrograph-coordinates /data/XXX/movies_raw/Sep06_collect/Extract/job041/particles.star /data/CRYOSPARC/USERS/Omid/P1/J154/particles_selected.cs from_cryosparc.star --loglevel debug
Detected CryoSPARC 2+ .cs file
Reading primary file
Classification parameters not found
Directly copied fields: ucsfImagePath, rlnDetectorPixelSize, ucsfImageIndex, ucsfUid, rlnMagnification
Defocus values not found
Traceback (most recent call last):
File “/data/CRYOSPARC/pyem/csparc2star.py”, line 106, in
sys.exit(main(parser.parse_args()))
File “/data/CRYOSPARC/pyem/csparc2star.py”, line 62, in main
glob(args.copy_micrograph_coordinates)), join=“inner”)
File “/data/CRYOSPARC/miniconda2/envs/pyem/lib/python3.7/site-packages/pandas/core/reshape/concat.py”, line 255, in concat
sort=sort,
File “/data/CRYOSPARC/miniconda2/envs/pyem/lib/python3.7/site-packages/pandas/core/reshape/concat.py”, line 301, in init
objs = list(objs)
File “/data/CRYOSPARC/pyem/csparc2star.py”, line 61, in
(star.parse_star(inp, keep_index=False, augment=True) for inp in
File “/data/CRYOSPARC/pyem/pyem/star.py”, line 282, in parse_star
df = pd.read_csv(starfile, skiprows=ln, delimiter=’\s+’, header=None, nrows=nrows)
File “/data/CRYOSPARC/miniconda2/envs/pyem/lib/python3.7/site-packages/pandas/io/parsers.py”, line 685, in parser_f
return _read(filepath_or_buffer, kwds)
File “/data/CRYOSPARC/miniconda2/envs/pyem/lib/python3.7/site-packages/pandas/io/parsers.py”, line 463, in _read
data = parser.read(nrows)
File “/data/CRYOSPARC/miniconda2/envs/pyem/lib/python3.7/site-packages/pandas/io/parsers.py”, line 1154, in read
ret = self._engine.read(nrows)
File “/data/CRYOSPARC/miniconda2/envs/pyem/lib/python3.7/site-packages/pandas/io/parsers.py”, line 2059, in read
data = self._reader.read(nrows)
File “pandas/_libs/parsers.pyx”, line 881, in pandas._libs.parsers.TextReader.read
File “pandas/_libs/parsers.pyx”, line 896, in pandas._libs.parsers.TextReader._read_low_memory
File “pandas/_libs/parsers.pyx”, line 950, in pandas._libs.parsers.TextReader._read_rows
File “pandas/_libs/parsers.pyx”, line 937, in pandas._libs.parsers.TextReader._tokenize_rows
File “pandas/_libs/parsers.pyx”, line 2132, in pandas._libs.parsers.raise_parser_error
pandas.errors.ParserError: Error tokenizing data. C error: Expected 10 fields in line 41, saw 16

4

Hi All,

Problem is solved. I was using the wrong python and not the original particle stack from relion because I made a new copy with the optics removed, but did not specify that particle stack, which was causing the last error.

Omid

5

Great, glad you figured it out!