ValueError during Micrograph Denoiser

Troubleshooting
1

I am trying to go back to use the Cryosparc Micrograph Denoiser job on a dataset that I had previously stared processing in v 4.4.1 (we now have version 4.6.0). In order to do this, I initially started by trying out the instructions listed here: Job: Micrograph Denoiser (BETA) | CryoSPARC Guide. However, although the training part of the Micrograph Denoiser job completes, when it starts “denoising of input micrographs”, it then throws an error:

Screenshot 2024-10-10 at 13.43.05
Screenshot 2024-10-10 at 13.43.052120×212 42.8 KB

This happens even if I try to import the movies, patch motion correct, and CTF correct from scratch or I try various subset sizes or inputs (from CTF, from motion correction, from manual curation, etc.). This happens with this dataset on a single workstation as well as on our cluster instance (both v 4.6.0, data folder was copied from the workstation to the cluster).

I have been able to determine that it is dataset specific as there is another dataset that I first denoised in cryosparc v. 4.5.3 and which also worked again in 4.6.0 (new job, without cloning) on the cluster instance. Others have also used denoising on our workstations before. The headers of the motion corrected images seem to show all the same dimensions (determined using IMOD “header”) and the dimensions of the movies match that of the gain reference so there are no obvious shape mismatches in the source data. They are binned by 2 (Output F-crop factor: 1/2) during patch motion correction.

The dimensions in the error don’t match the dimensions stated at the beginning of the job event log so I am having a bit of trouble troubleshooting what is causing the error…whether it is a particular subset of images in the dataset or something generally about the images?
From the event log:

[CPU:  547.8 MB  Avail: 249.45 GB] Loaded info for 100 micrographs
[CPU:  552.0 MB  Avail: 249.45 GB] Frame shape in  : [4092 5760], psize_in : 0.3353999853134155
[CPU:  552.0 MB  Avail: 249.45 GB] Frame shape out : [455 640], psize_out: 3.0185998678207397
[CPU:  552.0 MB  Avail: 249.45 GB] Frame shape crop: [443 628] (only for training)

Has anybody encountered this error before with Micrograph Denoiser or have any troubleshooting suggestions? I do not have access to command line commands on the cluster instance, but I do on the workstation instance if any further information is needed. Thank you in advance for your help!

2

Hi @leighk! Welcome to the forum!

I notice in the log messages you posted that the pixel size is 0.34 Å. Since you F-cropped by 2, this would mean that your data was collected with a pixel size of 0.17 Å. Is this correct?

3

Hi @rposert,

I didn’t collect the original data, but yes, the pixel size I was given by the microscopist was 0.1677 Å/pix (super-resolution). The other dataset I mention in my post where the denoising did work (that I redid as a test) had a raw nominal pixel size of 0.26 Å/pix. I hope that helps!

4

Hi @leighk! Thanks for confirming the pixel size for me. Unfortunately, the Micrograph Denoiser will not work with very high magnifications right now. We have recorded this issue.

5

Hi @rposert,

Thank you for clarifying that limitation. Could you tell me what the smallest pixel size is that Micrograph Denoiser can handle?

6

Hi @leighk!

The reason this is failing is that micrographs are downsampled to a 3 Å pixel size before denoising, then denoised in 512 x 512 pixel patches. In your case, downsampling the micrograph to 3 Å results in a micrograph that’s only 455 x 640 pixels, which is too small for a single patch.

So, for this issue, I believe the smallest pixel size that will work is (512 px * 3 Å/px) / (smallest detector dimension). In your case, this is 512 * 3 / 4092 = 0.376 Å/px (i.e., ~0.19 Å/px superres). However, this is still a much higher magnification than we tested with, so I cannot guarantee that this would currently work either.

7

Hi @rposert,

Thank you for the explanation! :slight_smile:

1 Like