Particles not mapped onto correct micrographs after re-importing particles from star file

Import and Export
1

Hi all,

I exported a job and converted to star file using pyem (for use in relion)

When I re-import selected particles back to cryosparc (while connecting to source micrographs), the particle locations are all mapped onto a single micrograph (the last micrograph out of ~8000).

The particles have their x and y coords, but they don’t map to their correct micrograph.

Where might I be going wrong?

Below are lines from the star file and screenshots from the cryosparc import job

Many thanks!

…
Cryosparc version: 4.5.3

pyem (v0.5) command:
csparc2star.py Px/exports/jobs/Jx_nonuniform_refine_new/Jx_particles/Jx_particles_exported.cs from_csparc.star --inverty

Star file after pyem:

data_optics

loop_
_rlnVoltage #1
_rlnImagePixelSize #2
_rlnSphericalAberration #3
_rlnAmplitudeContrast #4
_rlnOpticsGroup #5
_rlnImageSize #6
_rlnImageDimensionality #7
_rlnOpticsGroupName #8
300.000000 0.820000 2.700000 0.100000 1 300 2 opticsGroup1
300.000000 0.820000 2.700000 0.100000 2 300 2 opticsGroup2
300.000000 0.820000 2.700000 0.100000 3 300 2 opticsGroup3
300.000000 0.820000 2.700000 0.100000 4 300 2 opticsGroup4
300.000000 0.820000 2.700000 0.100000 5 300 2 opticsGroup5
300.000000 0.820000 2.700000 0.100000 6 300 2 opticsGroup6
300.000000 0.820000 2.700000 0.100000 7 300 2 opticsGroup7
300.000000 0.820000 2.700000 0.100000 8 300 2 opticsGroup8
300.000000 0.820000 2.700000 0.100000 9 300 2 opticsGroup9
300.000000 0.820000 2.700000 0.100000 10 300 2 opticsGroup10
300.000000 0.820000 2.700000 0.100000 11 300 2 opticsGroup11
300.000000 0.820000 2.700000 0.100000 12 300 2 opticsGroup12
300.000000 0.820000 2.700000 0.100000 13 300 2 opticsGroup13
300.000000 0.820000 2.700000 0.100000 14 300 2 opticsGroup14
300.000000 0.820000 2.700000 0.100000 15 300 2 opticsGroup15
300.000000 0.820000 2.700000 0.100000 16 300 2 opticsGroup16
300.000000 0.820000 2.700000 0.100000 17 300 2 opticsGroup17
300.000000 0.820000 2.700000 0.100000 18 300 2 opticsGroup18
300.000000 0.820000 2.700000 0.100000 19 300 2 opticsGroup19
300.000000 0.820000 2.700000 0.100000 20 300 2 opticsGroup20
300.000000 0.820000 2.700000 0.100000 21 300 2 opticsGroup21
300.000000 0.820000 2.700000 0.100000 22 300 2 opticsGroup22
300.000000 0.820000 2.700000 0.100000 23 300 2 opticsGroup23
300.000000 0.820000 2.700000 0.100000 24 300 2 opticsGroup24
300.000000 0.820000 2.700000 0.100000 25 300 2 opticsGroup25
300.000000 0.820000 2.700000 0.100000 26 300 2 opticsGroup26
300.000000 0.820000 2.700000 0.100000 27 300 2 opticsGroup27
300.000000 0.820000 2.700000 0.100000 28 300 2 opticsGroup28
300.000000 0.820000 2.700000 0.100000 29 300 2 opticsGroup29
300.000000 0.820000 2.700000 0.100000 30 300 2 opticsGroup30
300.000000 0.820000 2.700000 0.100000 31 300 2 opticsGroup31
300.000000 0.820000 2.700000 0.100000 32 300 2 opticsGroup32
300.000000 0.820000 2.700000 0.100000 33 300 2 opticsGroup33
300.000000 0.820000 2.700000 0.100000 34 300 2 opticsGroup34
300.000000 0.820000 2.700000 0.100000 35 300 2 opticsGroup35
300.000000 0.820000 2.700000 0.100000 36 300 2 opticsGroup36
300.000000 0.820000 2.700000 0.100000 37 300 2 opticsGroup37
300.000000 0.820000 2.700000 0.100000 38 300 2 opticsGroup38
300.000000 0.820000 2.700000 0.100000 39 300 2 opticsGroup39
300.000000 0.820000 2.700000 0.100000 40 300 2 opticsGroup40
300.000000 0.820000 2.700000 0.100000 41 300 2 opticsGroup41
300.000000 0.820000 2.700000 0.100000 42 300 2 opticsGroup42

data_particles

loop_
_rlnImageName #1
_rlnMicrographName #2
_rlnCoordinateX #3
_rlnCoordinateY #4
_rlnAngleRot #5
_rlnAngleTilt #6
_rlnAnglePsi #7
_rlnOriginXAngst #8
_rlnOriginYAngst #9
_rlnDefocusU #10
_rlnDefocusV #11
_rlnDefocusAngle #12
_rlnPhaseShift #13
_rlnCtfBfactor #14
_rlnOpticsGroup #15
_rlnRandomSubset #16
_rlnClassNumber #17
000001@J441/extract/000000893081524113219_FoilHole_17320353_Data_17259383_17259385_20240818_070824_fractions_particles.mrc J9/imported/000000893081524113219_FoilHole_17320353_Data_17259383_17259385_20240818_070824_fractions.mrc 4606 3231 -83.289825 61.893661 104.485977 12.415313 7.034062 11722.744141 11599.829102 -47.498547 0.000000 0.000000 15 1 1
000002@J441/extract/000000893081524113219_FoilHole_17320353_Data_17259383_17259385_20240818_070824_fractions_particles.mrc J9/imported/000000893081524113219_FoilHole_17320353_Data_17259383_17259385_20240818_070824_fractions.mrc 4090 2644 102.185600 75.366180 23.391327 8.107643 -0.960938 11825.352539 11702.437500 -47.498547 0.000000 0.000000 15 2 1
000003@J441/extract/000000893081524113219_FoilHole_17320353_Data_17259383_17259385_20240818_070824_fractions_particles.mrc J9/imported/000000893081524113219_FoilHole_17320353_Data_17259383_17259385_20240818_070824_fractions.mrc 1316 1155 139.922684 111.851341 -64.307793 6.572813 -10.493438 12151.963867 12029.048828 -47.498547 0.000000 0.000000 15 1 1
000004@J441/extract/000000893081524113219_FoilHole_17320353_Data_17259383_17259385_20240818_070824_fractions_particles.mrc J9/imported/000000893081524113219_FoilHole_17320353_Data_17259383_17259385_20240818_070824_fractions.mrc 1842 942 -62.174458 79.620575 50.845787 -2.421562 15.720938 12135.361328 12012.446289 -47.498547 0.000000 0.000000 15 2 1
000005@J441/extract/000000893081524113219_FoilHole_17320353_Data_17259383_17259385_20240818_070824_fractions_particles.mrc J9/imported/000000893081524113219_FoilHole_17320353_Data_17259383_17259385_20240818_070824_fractions.mrc 2207 1064 -58.675297 112.319984 100.841301 19.718437 14.952188 12104.006836 11981.091797 -47.498547 0.000000 0.000000 15 1 1

Screenshot 2024-09-28 at 7.49.26 pm
Screenshot 2024-09-28 at 7.49.26 pm1920×1128 158 KB

Screenshot 2024-09-28 at 9.36.35 pm
Screenshot 2024-09-28 at 9.36.35 pm1920×634 106 KB

2

Welcome to the forum @Jkegg.
Please can you post the outputs of the commands

cryosparcm cli "get_job('P99', 'J466', 'job_type', 'version', 'instance_information', 'status', 'params_spec')"
grep ^0 /path/to/from_csparc.star | awk '{print $2}' | sort -u | wc -l
grep ^0 /path/to/from_csparc.star | awk '{print $2}' | sort -u | tail -n 5

where you replace P99 with the actual project ID.

3

Thanks for your help @wtempel
Here are the outputs:

{‘_id’: ‘66fcd11755ddc30380614494’, ‘instance_information’: {‘available_memory’: ‘903.03GB’, ‘cpu_model’: ‘Intel(R) Xeon(R) Gold 6230R CPU @ 2.10GHz’, ‘ofd_hard_limit’: 524288, ‘ofd_soft_limit’: 1024, ‘physical_cores’: 52, ‘platform_architecture’: ‘x86_64’, ‘platform_node’: ‘m3t101’, ‘platform_release’: ‘5.14.0-284.25.1.el9_2.x86_64’, ‘platform_version’: ‘#1 SMP PREEMPT_DYNAMIC Wed Aug 2 14:53:30 UTC 2023’, ‘total_memory’: ‘1006.60GB’, ‘used_memory’: ‘93.82GB’}, ‘job_type’: ‘import_particles’, ‘params_spec’: {‘alignments3D_exists’: {‘value’: True}, ‘blob_exists’: {‘value’: True}, ‘ctf_exists’: {‘value’: True}, ‘enable_validation’: {‘value’: True}, ‘location_exists’: {‘value’: True}, ‘particle_blob_path’: {‘value’: ‘/fs04/scratch2/ws92/josh/20240926_IFs_3d/J441/extract/’}, ‘particle_meta_path’: {‘value’: ‘/fs04/scratch2/ws92/josh/20240926_IFs_3d/from_csparc.star’}, ‘query_cut_prefix’: {‘value’: 0}, ‘query_cut_suff’: {‘value’: 0}, ‘remove_leading_uid’: {‘value’: True}, ‘source_cut_prefix’: {‘value’: 0}, ‘source_cut_suff’: {‘value’: 0}}, ‘project_uid’: ‘P20’, ‘status’: ‘completed’, ‘uid’: ‘J561’, ‘version’: ‘v4.5.3’}
3645
J9/imported/018431161878794715209_FoilHole_17292575_Data_17259383_17259385_20240817_221608_fractions.mrc
J9/imported/018432294214653504101_FoilHole_17292518_Data_17259380_17259382_20240817_221734_fractions.mrc
J9/imported/018438006823327373216_FoilHole_17292744_Data_17259383_17259385_20240817_235442_fractions.mrc
J9/imported/018442950121432382276_FoilHole_17292670_Data_17259380_17259382_20240817_235529_fractions.mrc
J9/imported/018445802500026611032_FoilHole_18302670_Data_17259383_17259385_20240818_133257_fractions.mrc