Merging EER datasets with different number of frames

Hello all,

I have a question about the EER format. If I collect two datasets of the same sample, with the same dose, but with different amount of frames, how is that dealt with in cryoSPARC? Can I actually combine the two datasets? More generally about EER format, what happens to residual frames if they can’t be divided into equal number of fractions? Is that data lost or can cryoSPARC deal with having different number of frames in each fraction?

Thanks!

Hi masper!

These are good questions about the EER format! It’s slightly different from a normal image format, so let’s make some comparisons to, say, a “regular” TIFF movie.

The EER format

In a TIFF movie, set up your microscope to record, say, a one second movie, split into fifty frames. Your movie will therefore have fifty frames, each of which is an image. Each pixel in one of these frames tells you how many electrons hit that pixel during that part of the 1 second movie. Consider a 2x2 section of the first frame:

This tells us that, during the first 20 ms of the movie (1 second movie / 50 frames), the pixels had the following electron counts:

  • (0, 0): 1
  • (1, 0): 2
  • (0, 1): 0
  • (1, 1): 1

You can see that the same image can be encoded as a table or as an image. The EER format skips the image and just saves the table. An advantage of this is that modern cameras have a very fast frame rate – much faster than you’d want to record your movie. This means we can get very precise information about when an electron hit the detector. The EER table might look like this:

Time (ms) Pixel (coord)
2.00 (1, 0)
3.02 (1, 1)
10.85 (1, 0)
19.20 (0, 0)

This would produce the same image if you added together all the events in the first 20 ms, like we did for the TIFF frame. But, if you later decide you need a higher temporal precision, you could split your 1 second movie into 100 frames instead, or 60, or however many you like. The table would then be used to produce what those frames would look like, taking only the events that happened during the appropriate time period. These images that you make out of the table are called “fractions”, but are analogous to frames in the traditional movie sense.

I hope that makes it clear what happens with an EER movie when you request a certain number of fractions from your movie. There aren’t extra frames that aren’t used – the table for the whole movie is divided into that number of equal fractions, which are then used to produce images that can be aligned by Patch Motion Correction or other techniques.

Can you combine two datasets

If the total dose is the same, you can split the movies into the same number of frames and combine them without issue.

If the total dose is the different then when you split the movies into the same number of fractions, each frame will have a different dose. This will be okay in most cases – the only job that will present problems will be Reference Based Motion Correction.

I hope that helps!

2 Likes

That’s kind of a critical point… :wink:

1 Like

It’s true, you’re leaving a bit of resolution on the table without Reference Based Motion Correction. If that’s a critical part of your workflow, the important things are

  • dose per frame, and
  • number of frames

So if you have two EER datasets with different total dose, you could fractionate them such that the dose per frame is the same, then discard the final frames from the dataset with more dose until the number of frames is also the same.

For example, if you have one dataset with 50 e/Å2 and another with 40 e/Å2, you could fractionate them into 50 and 40 frames, respectively, then discard the final 10 frames from the former. This would leave you with two datasets, both with 1 e/Å2/frame and 40 frames.

2 Likes

@rposert Thank you very much for the detailed explanation! Super helpful!

1 Like