I originally imported motion-corrected micrographs into cryoSPARC, and from these did CTF correction, particle picking and classification. I have now imported the raw .mrcs movies into cryoSPARC, hoping to perform a local motion correction job. Is it possible to link these movies to the originally imported motion-corrected micrographs, as currently I get the following error when trying a local motion correction job:
Traceback (most recent call last):
File "cryosparc2_worker/cryosparc2_compute/run.py", line 49, in cryosparc2_compute.run.main
File "cryosparc2_compute/jobs/runcommon.py", line 823, in check_default_inputs
assert False, 'Non-optional inputs from the following input groups and their slots are not connected: ' + missing_inputs + '. Please connect all required inputs.'
AssertionError: Non-optional inputs from the following input groups and their slots are not connected: movies.background_blob, movies.rigid_motion. Please connect all required inputs.
It is possible to do this by first manually linking the unique movie IDs to the micrograph IDs and doing full-frame motion correction on the movies first.
Immediately after running the Import Movies job, log in to the machine where cryosparc2_master is installed via Terminal and start the interactive CLI:
cryosparcm icli
Enter the following commands, substituting the example paths with the project paths on your local file system. For example, if your project is at /home/cryosparcuser/projects/P3 and your Import Micrographs job is J1, use /home/cryosparcuser/projects/P3/J1/imported_micrographs.cs for the first path:
from cryosparc2_compute import dataset
micrographs = dataset.Dataset().from_file('/path/to/micrographs_job/imported_micrographs.cs')
movies = dataset.Dataset().from_file('/path/to/movies_job/imported_movies.cs')
movies.data['uid'] = micrographs.data['uid']
movies.to_file('/path/to/movies_job/imported_movies.cs')
This links the imported movies to the imported micrographs by giving each movie the same unique ID as the micrographs. Note this assumes the imported movies were imported in the same order as the micrographs (i.e., have similar file names)
Exit the interactive CLI with Ctrl + D
In cryoSPARC, start a “Full-frame motion correction (M)” job with the imported movies as input.
Create a “Local motion correction (multi)” job. Specify the micrographs from the full-frame motion correction job for the “movies” input slot and your particle picks for the “particles” input slot. Start the job.
Hi Nick,
Thank you for your help!
However I am still running into issues. I am working with three data sets that I am combining. I did the above uid modification and global motion correction seperately for all three datasets. I then tried combining all three with particles from later in my workflow for local motion correction. This ran for a couple of days and then came back as failed. Weirdly, if I do random subsets of 10 micrographs for each data set using the same particle stack the local motion corr job works. The job also fails when I try entire datasets individually.
I find this error at the end of the job.log file:
Traceback (most recent call last):
File "cryosparc2_worker/cryosparc2_compute/run.py", line 82, in cryosparc2_compute.run.main
File "cryosparc2_worker/cryosparc2_compute/jobs/motioncorrection/run_local.py", line 408, in cryosparc2_compu
te.jobs.motioncorrection.run_local.run_local_motion_correction_multi
File "cryosparc2_compute/dataset.py", line 473, in from_data
assert all(data['uid']), "Given data must have a uid field"
TypeError: 'NoneType' object has no attribute '__getitem__'
========= main process now complete.
========= monitor process now complete.
Why would this not be the case for the random subsets jobs that worked?
Hi @maxm, it looks like this happens because there are some micrographs without any particle picks. This should be fixed in the latest version of cryoSPARC v2.15, out today. Can you try updating and see if that fixes the issue?
If you can, try local motion correction on a smaller subset of micrographs that includes some without any picks, and if that works then the full run should work as well.
