I’m trying to make it easy for crYOLO users to import their picked coordinates into cryoSPARC.
I followed the suggestions in this thread and created a STAR file as described in the thread (an image will follow in the next post, I’m not yet allowed to post more then one image ^^).
However, when we try to import this star file into cryoSPARC we run into errors:
Thanks for providing these micrographs, I’ll take a look at all the details here and let you know exactly what fields are required in the .star file to allow particle locations to be imported from crYOLO into CryoSPARC v4.
Looking at your .star file, that should be all the information you need: rlnMicrographName, rlnCoordinateX, and rlnCoordinateY.
The reason your job failed is because of the warning:
Warning: No micrographs were connected as inputs, so correspondences cannot be found, so output will not contain pick locations for particles.
Then connect the imported micrographs to this Import Particles job so that the job can find the correspondences between the particle locations and the micrographs. Try that out and let me know if it works.
We got it to work. So the problem was, when you import motion corrected micrographs they get a 22-letter-prefix. But the path in my star file does not have this prefix so it was not able to match it.
We needed to set the cut-prefix option in cryoSPARC to make it work.
Thats great!! In the next version of crYOLO it will write an star file that is compatible with cryoSPARC.
Follow up question:
As you might know, crYOLO is also supporting filaments. Are they handled any differently? Do I need to provide a special format for them?
Filament picks can be imported if, in the particle star file, each particle pick has an associated rlnHelicalTubeID field that maps the particle to an ID that specifies which filament it belongs to (unique for each filament). Is such a field generated by crYOLO during the tracing procedure? CryoSPARC also can read: rlnAnglePsiPrior (the in-plane rotation angle of the filament, which could be estimated during picking), rlnHelicalRise (the helical rise in Angstroms, if known), and rlnNrHelicalAsymUnits (the number of unique asymmetric units per particle, if known). These are mapped to data fields in the filament result in the Import Particles job. The imported micrographs and their star file shouldn’t need any modifications from the standard case.
It’s also possible to import filaments without any of these fields, but it is preferable to at least have the rlnHelicalTubeID because this can be used downstream in Helical Refinement for proper gold-standard particle splits.
Is there any other filament-related information generated by crYOLO that could be useful when importing filament picks into CryoSPARC?
Right now the format is start-end - however, I will add a format that fits for cryosparc. rlnAnglePsiPrior is calculated internally and can be provided.
Is there any other filament-related information generated by crYOLO that could be useful when importing filament picks into CryoSPARC?
I’ve a confidence value for each and every pick (also for the single particle picks). Can cryoSPARC make use of this?
Best,
Thorsten
Edit:
This is how a filament star file for cryosparc would look like:
Filament picks can be imported if, in the particle star file, each particle pick has an associated rlnHelicalTubeID field that maps the particle to an ID that specifies which filament it belongs to (unique for each filament).
So far, the IDs are only unique for one micrographs, not across all filaments on all micrographs. Is that a problem?
The filament star file looks valid, but right now we do assume rlnHelicalTubeID is unique across the entire dataset when converting it to CryoSPARC’s filament_uid. Do you know if in standard RELION helical datasets rlnHelicalTubeID is only guaranteed to be unique within a single micrograph? If so, we should probably update our import code…
Re: the confidence value – we could certainly read this into CryoSPARC. Currently for other particle picking software like Topaz, we read confidence values into our pick_stats results so that particles can be thresholded/subset by that confidence value in our Inspect Picks job (it overwrites the “normalized cross correlation” value that we normally compute in our in-house template/blob picking algorithms). Which rln field would this confidence value be written to?
Do you know if in standard RELION helical datasets rlnHelicalTubeID is only guaranteed to be unique within a single micrograph? If so, we should probably update our import code…
I’m trying to find out and let you know.
Which rln field would this confidence value be written to?
Hm, right now its not written in any STAR file by crYOLO. Its only written in the crYOLO cbox format. However, I would guess that _rlnAutopickFigureOfMerit is probably the right choice?
Do you know if in standard RELION helical datasets rlnHelicalTubeID is only guaranteed to be unique within a single micrograph? If so, we should probably update our import code…
Takanori Nakane just answered it on the CCPEM mailing list:
This all is perfect, and thank you very much for seeking out the helical tube ID clarification! I’ve made a note to:
allow imports of rlnAutopickFigureOfMerit into our pick_stats/ncc_score so that imported particles from crYOLO, RELION, etc., can be thresholded by the FOM in CryoSPARC’s Inspect Particle Picks job
update our import code to correct the assumptions made on the rlnHelicalTubeID field
These changes should be upcoming in a future patch to CryoSPARC v4.
regarding these features, are there any updates on importing crYOLO picked particles into CryoSPARC.
I have managed to import the particle locations from crYOLO into CryoSPARC, however, I was not able to take advantage of the figure of merit using the Inspect Particle Picks Job in CryoSPARC.
This is how my particle location file looks like (cryosparc.star file):
Is there any specifc parameter or aspect to consider in order to employ this thresholding of the figure of merit. While one can in crYOLO itself take control of the figure of merit, as far as I am aware it is not possible to do this so conveniently while looking at the picks on the micrographs and how the picking is updated in real time.
As of CS v4.1, the rlnFOM values should be transposed from a star file into the pick_stats/ncc_score of the particles.cs file. How are you picking these particles and importing them into CS? Using an external results job and crYOLO via CS-tools? Or through the crYOLO GUI and importing a particle stack? Or another approach?
I am picking the particles using the napari-boxmanager, a crYOLO plugin for picking and visualizing the picking results. After crYOLO has picked the particles it outputs a star file dedicated to importing the particle locations into cryosparc (cryosparc.star). It is of the format shown in my post above.
I then import these particle locations using the “Import Particle Stack” Job from CS. The particle meta path leads to the cryosparc.star file. The parameter Ignore raw data, Ignore pose data and Remove leading UID in input exposure file name are checked.
Furthermore I checked the following parameters:
Enable Strict Checking
Particle pick locations (locations)
Particle filament data (filament)
Particle pick statistics (pick_stats)
Then I connect the imported micrographs (the one on which I performed the picking in crYOLO) to this “Import Particle Stack” Job and run the Job.
The Job runs and imports the particle picks. However, only when I connect the outputs of this “Import Particle Stack” Job with a “Manual Picker” Job I am able to inspect the picking. The “Inspect Picks” Job does not show the picks on the micrographs let alone provide the score thresholds.
So I am not sure how I can control this, while I am already quite happy of being able to successfully import the particle locations into CS, it would be helpful to have real time control over the threshold. Since this is something CS provides I wanted to take advantage of this.
I hope this information helps, maybe I am missing something. Please let me know if you need any other information regarding my particle picking workflow.
Same error…
