Import Micrographs broken in 2.12?

jcfromme · November 29, 2019, 9:08pm

I just updated our installation to 2.12 and now Import Micrographs appears to be broken, giving this error message:

Traceback (most recent call last):
File “cryosparc2_master/cryosparc2_compute/run.py”, line 78, in cryosparc2_compute.run.main
File “cryosparc2_compute/jobs/imports/run.py”, line 534, in run_import_movies_or_micrographs
if params[‘skip_header_check’]:
KeyError: ‘skip_header_check’

I’m confused because there is no option to Skip Header Check in the Import Micrographs job, like there is in the Import Movies job

stephan · November 29, 2019, 10:16pm

Hi @jcfromme,

Nice catch- if you run cryosparcm update again you’ll get the fix (v2.12.2). Sorry about that!

jcfromme · November 30, 2019, 12:45pm

Hi @stephan,
Thanks for this - but now in 2.12.2, there is a new error:

Traceback (most recent call last):
File “cryosparc2_master/cryosparc2_compute/run.py”, line 78, in cryosparc2_compute.run.main
File “cryosparc2_compute/jobs/imports/run.py”, line 595, in run_import_movies_or_micrographs
if params[‘override_exp_group_id’] == None:
KeyError: ‘override_exp_group_id’

This may be a similar issue, because there is no “Override Exposure Groups Id” box in Import Micrographs?

drawoliver · November 30, 2019, 4:38pm

I am also having the exact same problem with 2.12 (which was updated today)

Traceback (most recent call last): File “cryosparc2_master/cryosparc2_compute/run.py”, line 78, in cryosparc2_compute.run.main File “cryosparc2_compute/jobs/imports/run.py”, line 595, in run_import_movies_or_micrographs if params[‘override_exp_group_id’] == None: KeyError: ‘override_exp_group_id’

Any solution please?

NB: Downgrading to v2.11.0 and importing the (same) pre-motion corrected micrographs worked fine.

user123 · November 30, 2019, 8:11pm

I get this error, too. Unable to import any mics with the above error.

CBEN · December 1, 2019, 3:17am

I too am getting this error. We are running v2.12.2. I’ll double check that. If I find a solution I will get back to you all.

ABehal2020 · December 1, 2019, 4:05am

Downgrading - thank you @drawoliver - worked for us as well.

CBEN · December 1, 2019, 9:58pm

Yes. It seems that downgrading to v2.11 worked.

scaiola · December 2, 2019, 7:45am

Got the same error when importing particles.

Did you find a solution ?

stephan · December 2, 2019, 2:51pm

Hi @scaiola,

Can you post the error you received for import particles?

The import micrographs fix should be released today, as well as a few other fixes.

scaiola · December 2, 2019, 3:28pm

When I tried to import a particles.star from an extract job of Relion I got that :

Traceback (most recent call last):
File “cryosparc2_master/cryosparc2_compute/run.py”, line 78, in cryosparc2_compute.run.main
File “cryosparc2_compute/jobs/imports/run.py”, line 52, in run_import_particles if params[‘override_exp_group_id’] == None:
KeyError: ‘override_exp_group_id’

stephan · December 2, 2019, 3:32pm

Hi @scaiola,

I believe your builder is out of sync with the application. Please restart cryosparcm restart then run cryosparcm cli "refresh_job_types()"

mplockhartuom · December 3, 2019, 1:49pm

Hi @stephan

I’ve been running into some problems with importing Micrographs in this version but also 2.12.2 as a whole.

System

User: cryosparc_user (with access to stored data)
System: Ubuntu 16.04 LTS
Cuda: 9.1 (tried 10.1 but received errors regarding libs as discussed in >> 3D Variability Analysis Errors - v2.9.0
GPUs: 2x TitanV 1x 1080 Founders
CPU: Threadripper 1950x

Micrographs

I get the import error with this (even after following the above refresh etc):

Movies

When running full frame correction or patch motion correction I get the error below when it get to the end of the run.

Full motion

Patch motion

As you can see it appears the motion correction has actaully run but has failed to complete properly.

Movies are gain corrected ‘.mrc’ collected in linear mode with 79 frames with a total dose of 80 e.

As far as I can tell doing this with ungain corrected ‘.tiff’ files works.

Any help would be greatly appreciated.

EDIT:

I’ve just had this fail on ‘*.tif’ files too.

EDIT2:
I’ve only had this work with the Relion test data.

alexmag · December 3, 2019, 5:38pm

I still have the same problem.

DanielAsarnow · December 4, 2019, 8:58pm

We have it as well…

crescalante · December 4, 2019, 9:09pm

Also have same problem. Has anybody found a solution?

DanielAsarnow · December 4, 2019, 9:14pm

I can confirm that a forced update, restart, and job list refresh doesn’t work. Has anyone tried a full reinstall?

crescalante · December 4, 2019, 9:21pm

I did same, didn’t work also. Have not tried full reinstall.

jcfromme · December 4, 2019, 9:29pm

@stephan said they were working on a variety of bug-fixes, including this one, but they have not yet released another update.

stephan · December 4, 2019, 11:39pm

Hi @jcfromme, @crescalante, @DanielAsarnow, @alexmag, @mplockhartuom, @scaiola, @CBEN, @ABehal2020, @user123, @drawoliver,

This bug has been (actually) fixed! Update to cryoSPARC v2.12.4 (cryosparcm update) to get the latest fixes, which include:

New:

Exposure Groups Utility Updates
- Create exposure groups using regular expressions and separators
- Test your exposure group creation strategy
3D Variability updates to default parameters: create better results without tweaking

Fixed:

Multiprocessing jobs (Patch CTF, Patch motion, Full-frame motion) fail with abnormal termination errors
Import micrographs and import movies function incorrectly with exposure groups
Imported jobs with inputs cause the tree layout to fail or throw an error
3D Variability display "intermediate" mode outputs are incorrectly scaled
3D Variability display "cluster" mode outputs contain no particles