Export selected classes as MRC stack? (v2) [Feature request]

Hi,

I would like to be able to export selected classes (templates) as an mrc stack, rather than as a cs file (which I don’t know how to convert to an mrc). This would be useful for using these class averages outside of cryosparc, e.g. for picking in relion.

Cheers
Oli

2 Likes

This should do it, no? :slight_smile:

The same seems to exist for a Select Template job

Cheers

Juraj

Thanks Juraj! Yep that should do the trick - just didn’t realize “blob” meant mrc haha… :slight_smile: Weird that that file is not in the corresponding project directory though.

Cheers
Oli

Actually that’s not exactly right on closer inspection - clicking on that (for selected classes) downloads a stack of all the classes, not just the selected classes - it doesn’t generate a stack of the selected templates.

Cheers
Oli

Basically, Class2D jobs make stacks as MRCs, but Select2D jobs do not.

Cheers
Oli

I see. But then I would re-label this as a bug and not a feature request. If I ask for a blob of something called “templates_selected” and get back the input templates, I doubt that’s the intended behavior.

Cheers
J

Hi @olibclarke, @xeniorn,

You’re right - this is not the correct behaviour.
Actually what happens is that all raw data (i.e. blobs) are stored in mrc format. But metadata (including file path and index within a file, in the case of particles or templates) is stored in a .cs file which is just a numpy tabular format. The select2D job therefore does not create a new .mrc file, just a new .cs file that points to the original .mrc file but only has rows for the selected subset of templates. We will change the behaviour so that select2D does output its own .mrc file that can be downloaded.

1 Like

Hello all,

As you may have noticed, this feature is now implemented as of v2.2.0.

Best,
Ali H.

1 Like

Hi,

Coming back to this topic. Did the behaviour change again? In the most recent version (2.14.2) I again get the full template stack, even if I download only selected templates. I can see that in the project directory, the templates_selected.cs file points to the index of the selected 2d classes in the full stack, but it would be great if also a new mrc stack with the selected templates would be created. If not by default, maybe when selecting to ‘export’ the job? Thanks a lot.

Cheers,
Simon

2 Likes

I’m using same version (2.14.2) as well.

Can I learn how to download mrcs file from cryosparc’s “Select 2D classes”? Even with cryosparc tutorial set, I don’t know how to do it. Once I have mrcs file, I can easily convert it into image (tif) file using eman2.

I need to extract/download selected particles after 2-d classification. Therefore, I clicked download button and blob in “particles_selected”

However, downloaded particles_selected.cs is a cryosparc file rather than mrcs.

I converted this cs file into star by pyem, and this star file has mrc and location information of 8,281 number of particles, but I’m not sure how to extract mrcs/image file from this star file.

relion_preprocess used to generate mrcs from star file, but it appears (at least to me) to have many version dependent incompatibility issues around format/options.

If I click download button in “templates_selected”,
Screen Shot 2020-04-29 at 5.50.38 PM
I could download mrc file, but this seems averaged volume rather than individually selected particles.

Hi dnamkr,

The mrc file of the templates you can download is simply a stack of all the 2d classes (a volume is basically a stack of 2d images). Most likely your mrc viewer (eman2) of choice interprets the downloaded stack as a 3D volume. For the desired display as individual 2D classes with eman2 you can simply rename the downloaded mrc file to mrcs and eman2 will then automatically interpret the file as a stack of 2d images rather than a 3D volume. Hope this helps.

Cheers,
Simon

1 Like

Hello simon.

Yes, I knew this already (change mrc to mrcs and convert to image files) as Steven Ludtke suggested. It worked very well for my other sets. However, I didn’t think of applying to here as well.

Thanks to your advice, I could generate image files now.

However, for a blob (mrc) downloaded from
Screen Shot 2020-05-01 at 1.22.50 PM

didn’t work as intended as Oli and you reported in 2018 and recently.

I downloaded cryosparc_P71_J30_020_class_averages.mrc and changed to cryosparc_P71_J30_020_class_averages.mrcs and converted to images files. However, it generated 50 image files (Templates selected : 18, Templates excluded : 32). It is supposed to generate only 18 image files.

hi

looks like this is still a bug, even in the latest ver. Bummer. I just want to get the selected vs excluded in 2 separate files! pretty pls fix this bug? I’ll start a new thread over in bugs.

1 Like

Hi @orangeboomerang,

The select2D job outputs indices of the selected templates for subsequent jobs but doesn’t duplicate the template .mrc files themselves - we will change this behaviour in the next version.
For now, you can use the following script to split up the templates:

First, note the project (PXX) and job (JXX) number of the select 2D job. Then:
cryosparcm icli
will get you into the interactive cryoSPARC shell.

import os
from cryosparc2_compute.blobio import mrc
dset = rc.load_output_group_direct('PXX','JXX','templates_selected')
pdir = rc.get_project_dir_abs('PXX')
hdr, data = mrc.read_mrc(os.path.join(pdir, dset.data['blob/path'][0]))
psize = dset.data['blob/psize_A'][0]
selected_mask = n.zeros(len(data), n.bool)
selected_mask[dset.data['blob/idx']] = True
selected_data = data[selected_mask]
rejected_data = data[~selected_mask]
mrc.write_mrc('./selected.mrcs', selected_data, psize)
mrc.write_mrc('./rejected.mrcs', rejected_data, psize)

This will create the two separate .mrcs files that you can use.

3 Likes

I tried this, but my 3rd line attempt (dset = rc.load_output_group_direct(‘P10’,‘J401’,‘templates_selected’)) resulted in
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
in ()
----> 1 dset = rc.load_output_group_direct(‘P10’,‘J401’,‘templates_selected’)

NameError: name 'rc' is not defined

I have recently encountered this problem using cryoSPARC V4.0.1
The original solution provided by apunjani in this thread did not work for me, below the mildly modified version which did (changed original lines 2 & 7):

import os
import numpy as np
from cryosparc_compute.blobio import mrc
dset =rc.load_output_group_direct('PXX','JXX','templates_selected')
pdir = rc.get_project_dir_abs('PXX')
hdr, data = mrc.read_mrc(os.path.join(pdir, dset.data['blob/path'][0]))
psize = dset.data['blob/psize_A'][0]
selected_mask = np.zeros(len(data), dtype=bool)
selected_mask[dset.data['blob/idx']] = True
selected_data = data[selected_mask]
rejected_data = data[~selected_mask]
mrc.write_mrc('/Path/to/Folder/selected.mrcs', selected_data, psize)mrc.write_mrc('./rejected.mrcs', rejected_data, psize)

Noticeably, I had to enter an actual path, ‘./selected.mrcs’ did not produce any errors but no mrcs file was created either.