How can I use a particle subset identified in RELION but use the particle blobs form a cryosparc extraction job

mvorlaender · November 15, 2024, 10:32am

Hi, I have an issue witht he following workflow. Here is what I did

Refine binned particles (binned particle set A) in cryosparc and convert to RELION star file
Classify binned particles in RELION (using binned particles to make relion run faster)
In parallel re-extact particle set A in cryosparc in a bigger box (unbinned particle set A)
Then, re-import a subset of the binned particle set A from RELION to cryosparc
Now I would like to use the Relion particle subset A, but with the bigger box size that I extracted in cryosparc. I have done this by replacing the particle blob of the relion subset with the particle blob of the cryosparc re-extract job in a Homogenous Reconstruction job. However, the reconstruction job fails with the following Warning and error:

Warning: Could not find matching connection slots for input group particles, connection index 0. This indicates the inner connection slots come from two different datasets with different unique IDs. Please check the particles input group's connection slots.

Traceback (most recent call last):
  File "cryosparc_master/cryosparc_compute/run.py", line 116, in cryosparc_master.cryosparc_compute.run.main
  File "cryosparc_master/cryosparc_compute/jobs/refine/newrun.py", line 984, in cryosparc_master.cryosparc_compute.jobs.refine.newrun.run_homo_reconstruct
  File "/home/svc_cryosparc/cryosparc_worker/cryosparc_compute/jobs/common.py", line 756, in create_missing_fields_in_dataset
    currfields = dataset.fields()
AttributeError: 'NoneType' object has no attribute 'fields'

Details on Cryosparc to relion conversion:

csparc2star.py particles.cs passthrough_particles.cs --inverty
Then I relinked the cryosparc particles from the extract jobs from .mrcs to .mrc so that RELION can read the particle stacks.

Details on relion to cryosparc import

I connected the particle.star file from relion Select subset job and the cryosparc source exposures, and used the following settings :

This produces warnings as in the screenshot, but Homogenous reconstruction, or re-extracting those particles in cryosparc work fine

So my question is, how can I a particle subset identified in RELION but use the particle blobs form a cryosparc extraction job?

Thanks in advance for your help, I hope I made myself clear.
Best,
Matthias

leetleyang · November 15, 2024, 12:50pm

Hi,

One way you could go about this:

Perform a Particle Sets Tool > Intersect routine between [cs_set_1] and [rln_subset_1] to generate [cs_subset_1].
Perform a second Particle Sets Tool > Intersect routine between [cs_subset_1] and [cs_set_2] to generate [cs_subset_1_unbinned].

You should then be able to use the result group, [cs_subset_1_unbinned], as is for downstream jobs.

Cheers,
Yang

mvorlaender · November 15, 2024, 1:11pm

Hi Yang,
thanks a lot for the suggestion. I tried that, but the Intesection of the two particle sets is 0. I tried intersecting on UID and path (with and without "Ignore leading UID), but no luck.
In my cryosparc to relion conversion script I replace relative paths with absolute paths so that I can launch relion form where it suits me. Changing absolute paths back to relative paths also did not help.

Here is the log from the star file import, in case that helps:

Compiling particle location information...

[CPU:  338.6 MB  Avail:  50.68 GB]
  Attempting to find corresponding filenames in rlnMicrographName and connected input exposures..

[CPU:  343.1 MB  Avail:  50.68 GB]
  Example source exposure filename: 

[CPU:  343.1 MB  Avail:  50.68 GB]
     FoilHole_6713877_Data_6713691_36_20241106_153323_EER_patch_aligned_doseweighted.mrc

[CPU:  346.0 MB  Avail:  50.68 GB]
  Example query exposure filename: 

[CPU:  346.0 MB  Avail:  50.68 GB]
     FoilHole_6713877_Data_6713691_36_20241106_153323_EER_patch_aligned_doseweighted.mrc

[CPU:  348.6 MB  Avail:  50.67 GB]
--------------------------------------------------------------

[CPU:  348.6 MB  Avail:  50.67 GB]
Compiling particle pose information...

[CPU:  348.6 MB  Avail:  50.68 GB]
  Converting euler angles..

[CPU:  348.6 MB  Avail:  50.68 GB]
  Converting rlnOriginXAngst to pixels using parameter psize_A

[CPU:  350.4 MB  Avail:  50.67 GB]
--------------------------------------------------------------

[CPU:  350.4 MB  Avail:  50.67 GB]
Particle information has now been imported for 11608 particles, creating outputs...

[CPU:  350.8 MB  Avail:  50.67 GB]
Found references to 6465 unique data files

[CPU:  353.8 MB  Avail:  50.71 GB]
Import paths were unique at level -1

[CPU:  353.8 MB  Avail:  50.71 GB]
Example imported relative path:
 J166/imported/003877001545062605423_FoilHole_6713877_Data_6713691_36_20241106_153323_EER_patch_aligned_doseweighted_particles.mrcs

[CPU:  353.8 MB  Avail:  50.71 GB]
Reading MRC file header to check shape...

[CPU:  359.3 MB  Avail:  50.71 GB]
--------------------------------------------------------------

[CPU:  359.3 MB  Avail:  50.71 GB]
Validating outputs...

[CPU:  359.3 MB  Avail:  50.71 GB]
All expected outputs were produced.

[CPU:  359.3 MB  Avail:  50.71 GB]
--------------------------------------------------------------

[CPU:  359.3 MB  Avail:  50.71 GB]
Loaded 11608 particles.

[CPU:  359.3 MB  Avail:  50.71 GB]
  Common fields: 

[CPU:  359.3 MB  Avail:  50.71 GB]
          ctf/accel_kv :  {300.0}

[CPU:  359.3 MB  Avail:  50.71 GB]
             ctf/cs_mm :  {2.7}

[CPU:  359.3 MB  Avail:  50.71 GB]
      ctf/amp_contrast :  {0.1}

[CPU:  359.3 MB  Avail:  50.71 GB]
      ctf/exp_group_id :  {1}

[CPU:  359.3 MB  Avail:  50.71 GB]
          blob/psize_A :  {4.130906}

[CPU:  359.3 MB  Avail:  50.71 GB]
            blob/shape :  [128 128]

[CPU:  359.3 MB  Avail:  50.71 GB]
--------------------------------------------------------------

[CPU:  359.3 MB  Avail:  50.70 GB]
Making plots...

Cheers, matthias

leetleyang · November 15, 2024, 1:16pm

Hi,

Path is probably the parameter that you want. However, in order for the subsequent string-matching to line up, you’ll need to edit the star file to reference the original .mrc particle stack.

Cheers,
Yang

mvorlaender · November 15, 2024, 1:23pm

Awesome, that did the trick! Thanks, much appreciated!