Dear collegues,
Could you please promt how to set up the beam shift files correctly?
when I used the same location as my eer files or not using anything in this field I get the error
Thank you in advance
Sincrerely,
Dmitry
Hi @Dmitry.
Did you already review the tutorial?
Are the xml and eer stored side-by-side in the same directory?
Please can you post as text the filenames of an example eer file and the corresponding xml file.
1 Like
Hello @wtempel,
Yes, I read it, thank you. And I believe there is a space for improvement in the description/example of the Prefix/Suffix part.
Yes, XML and EER are in the same directory.
I managed to run the Import Movies protocol correctly, or at least I hope so.
Briefly – the Prefix/Suffix topic is about letting the program cut the appropriate number of characters in the name of EER and the corresponding XML files.
So, in my example, I have the following files:
FoilHole_7211057_Data_6115600_36_20231107_093040_EER.eer — EER file
FoilHole_7211057_Data_6115600_36_20231107_093040.xml — XML file
Both files are equal from the beginning, where the name starts with FoilHole, meaning that 0 characters are different for EER and XML files.
In other words, the Prefix is 0 for both EER and XML files.
Then, those 2 files differ only in the name after the numbers _093040.
In other words – they differ by Suffix.
The goal is for both files to be equal in character length.
- From the EER file program needs to remove _EER.eer = 8 characters
- From XML file .xml = 4 characters.
In practice, that means that in the Import Movies protocol below XML import one should activate the Import Beam Shift Values from XML Files slider (becomes green). In the opened part below, one needs to place the following values
Additionally, one can specify the EPU XML metadata path. Usually, this path coincides with location of EER movie files. In that case it can be left empty since the program will check for the location of XML files in EER location directory.
I hope this information can be useful to someone.
Thank you, @wtempel.
Kind regards,
Dmitry
hello @wtempel,
Although the import of the movies finished without warning I see no Beam shift scatter plot in the import movies protocol.
Could you please advise, where shall I look for it?
Thank you.
Kind regards,
Dmitry
Hi Dmitry, after importing the beam shifts, you need to use Exposure Group Utilities to perform the clustering and assign the groups.
1 Like
Hello @olibclarke ,
So, I am a bit confused since in the Exposre Group Utilities one needs to set up the cluster number.
And the cluster number can be derived from the beam tilt scatter plot that (how I understood) can be obtained after import step.
Or did I get it wrong?
Thank you!
Kind regards,
Dmitry
Ah gotcha - yes, you should get a plot like the one you describe I think. I haven’t actually done this at the Import Movies step, only later with the Import Beam Shift job (which does show this plot).
I imagine it should also show during Import Movies with import beam tilts activated, but i haven’t tested this.
You could always do a dummy run of Exposure Group Utilities with say 10 clusters, and then re-run after inspecting the plot.
1 Like
Thank you, Oli, @olibclarke
I will do so.
My current issue is that I don’t see this plot anywhere in Import Movies protocol.
At the same time, I have no error in the Import Movies protocol during import of EER+ XML (AFIS).
So either
a) I did something wrong 
b) this plot is located somewhere in the Import Movies, and I don’t see it.
c) a Bug.
Kind regards,
Dmitry
There’s no mention anywhere in the log of it importing beam tilts? Maybe stupid question, but did you click “show from top” and look from the start of the log?
If you try exposure group utilities, at least that should tell you whether the beam tilts were imported successfully
1 Like
Hello Oli,
You are right, indeed – the window sorting fixed the issue 
.
Maybe it is a good idea for the CS developers’ team to add this window to the resulting one, as it works with other protocols (non-uniform 3d refinement, etc).
May I ask for your expertise/advice on my resulting beam shift scatter plot?
a) There is no direct cluster information present in the beam shift scatter plot. One must count the clusters to determine the number of clusters. Is that correct?
If so, it can be another possible improvement for the CS team to have the resulting number of clusters written in the output result file.
b) My beam shift scatter plot looks different from the tutorial one. Have you ever seen such a distribution? Is that normal, or may it trigger a concern?
Many thanks.
Sincerely,
Dmitry
Hi Dmitry,
Yes you have to count them. I think this is something best done manually - in your case the number of clusters is visually obvious, but in other cases (e.g. collecting using Leginon) it may not be so obvious, so I’m not sure automated cluster counting is the way to go.
I am not super familiar with EPU but the plot looks fine to me. I think the one in the tutorial was with multiple shots per hole, hence the ring-like clusters.
Cheers
Oli
2 Likes
Dear Oli, @olibclarke and colleagues,
in the CS manual it is written that "The output exposures are now ready for downstream processing, including motion correction, CTF estimation, and particle picking. "
Does that mean that the output of Exposure groups can be used as input for the new round of motion correction, CTF estimation and particle picking?
Also isn’t the clustering taken into account (if used as an input) when running the Homogenious (Non-uniform) refinement, when global CTF refinement is enabled?
Thank you.
Kind regards,
Dmitry
That’s fine. It looks like one shot per hole, on an R1.2/1.3 grid? With variance in how holes were identified in EPU, and with the standard 7x7+5+5+5+5 on the edges for a total of 69 beam shift groups.
Unless using special grids (e.g.: 0.6/1 or 2/2) or playing around with extremely large datasets, it’s usually a fairly safe bet to set 69 beam shift groups as the effect of beam tilt within the same hole (if using multiple shots per hole) is almost zero, and even then you need to be in the <1.3A range…
What you would usually do when processing. I normally split both exposures and particle stacks immediately prior to global CTF refinement the first time. Splitting particle sets by optics group is easier in CryoSPARC than in RELION (where I make sure to have optics groups set before I extract particles from the full dataset)…
If split, yes, clustering it taken into account. However, just because you import beam shift groups doesn’t mean that they’ve been clustered into groups - that has to be done manually.
1 Like
hello @rbs_sci,
yes, it is R1.2/1.3 grid. Indeed - 69 beam shift groups was found.
Thank you for sharing your expertise.
Kind regards,
Dmitry
1 Like