I recently learned on this forum how to export results groups from a cryoSPARC live session, which was very helpful because I previously didn’t know how to keep the exposures connected to their metadata and thus had to rerun motion correction and CTF estimation after transferring the movies to my own workstation. However, the .xml files don’t seem to be included as part of the result group.
This isn’t too much of a problem, because the .xml files can be separately imported pretty easily with the 'Import Beam Shifts" job. Still, would it be possible to include .xml files as part of the “Live Export Exposures job” so that all of the metadata gets package together at the same time?
Thanks for the feature request. Since exposure group clustering based on beam shift requires having a representative sampling of the beam shifts that exist in the full dataset, it’s not highly amenable to use in Live (one would have to wait until they have enough movies collected to be able to cluster them into different exposure groups). Because of this, the recommended way to do exposure group clustering is after live data collection, as you noted:
Export exposures from the Live session
Run “Import Beam Shifts” pointing to the original data directory
Run “Exposure Group Utilities” in the cluster&split mode, with the exported particles from Live connected
Import of XML files in Live would eliminate step 2, which could simplify the above; we’ve noted the request to import beam shift data from XML files in CryoSPARC Live
Hi,
I am using CryoSPARC version 4.6.2 and have a dataset collected using SmartScope on SerialEM. The file names are stamped with a timestamp instead of the tilt angle. To address this, I extracted image shift values from the MDOC files (SerialEM metadata), converted them into XML format, and successfully ran the “Import Beam Shifts” job. (Result output beamshift cluster plot).
However, when I ran the “Exposure Group Utilities” job, I did not see the colorful exposure group clustering plot in the output log. Am I missing something?
Attached parameters I am using for exposure group utilites.
The parameters look correct – could you show us the outputs of the job? As well, do you notice in the event log, if you click “Show from top” and then scroll through it, do you see any colourful clustering plots?
I have one more question. I found some MDOC files with image shift values in the e-05 scale, so I modified the MDOC-to-XML conversion script to interpret e-05 values as 00.000000. Is this the correct approach?
Good question! I am not actually sure. But what I can say is that the absolute values of the beam shifts once they are imported to CryoSPARC are not used aside from generating this clustering. So if they are all off by a scale factor 1e-5, it doesn’t actually matter (as of CryoSPARC v4.6.2 and all versions before; it’s possible this could change in the future). In other words, the units of the beam shift are not really interpreted further by CS.
Edit: @Ravi.Thakkar369 my apologies! I realized I interpreted your message wrong… setting the values to 0 exactly will have a different effect than what I described above
assuming all beam shift values are of order 10^(-5), I think you would want your script to keep the values as they are and not set them all to 0. If they are all set to 0, CryoSPARC will see beam shifts of 0 and thus will just put all exposures into one cluster. But if they are kept as they are, clustering will work as shown in your plot above because the relative differences between them are significant.
if only some beam shift values are of order 10^(-5) and the rest are of order ~1 as your plot shows, then it may be fine to set them to zero because their scale is so small relative to the rest of your exposures’ beam shifts. But in either case, I’d suggest leaving the values as they are to avoid confusion if its possible
