Select 2D Classes runs out of memory

[stefan]

Hi,

I am running a successful 2D classification on a particle stack that was imported as a single mrcs file (might be a relevant detail?), but I get the following error when trying to select any of the 2D classes.

[CPU: 76.99 GB]  Traceback (most recent call last):
   File "cryosparc2_master/cryosparc2_compute/run.py", line 78, in cryosparc2_compute.run.main
   File "cryosparc2_compute/jobs/select2D/run.py", line 262, in run
 _, particles_to_plot = mrc.read_mrc(os.path.join(proj_dir_abs, particles_dset_exclude.data['blob/path'][0]))
   File "cryosparc2_compute/blobio/mrc.py", line 114, in read_mrc
 data = read_mrc_data(file_obj, header, start_page, end_page, out)
   File "cryosparc2_compute/blobio/mrc.py", line 77, in read_mrc_data
 data = n.fromfile(file_obj, dtype=dtype, count= num_pages * ny * nx).reshape(num_pages, ny, nx)
 MemoryError

I have not had this issue before. I’m running verision 2.13.2.

[stephan]

Hey @stefan,

This is a known bug, thank you for reporting! The problem, as you predicted, arises from the code trying to load the entire particle stack into memory. You can see a similar post here:

Select 2D fails when using particles from relion_preprocess

Hi @Jason, @pgoetz, This turns out to be a problem with how the Select2D job works. When plotting the particles you see in the “Overview” tab, we read in the entire particle stack because we assume it came from one micrograph (so not too many). In this case, we might have to be less lazy and only open the particle images we need so we don’t use the entire system’s memory! For the time being, is it possible if you can split up the particle stack into smaller .mrc files? We’ll fix this and update…

Which includes a link to another discussion forum post where I attached a patch file.
Let me know if it works for you!

[stefan]

Hi @stephan,

Works like a charm. Thank you and sorry for spamming with a new topic!

Cheers,
Stefan