I used topaz (separate from cryosparc, because it is not working in our version of cryosparc) to pick particles from a small subset of my data, and now I am trying to get those particles back into cryosparc so they can be classified. I am trying to use the Import Particle Stack job and I am encountering the following error:
[CPU: 240.3 MB] Importing particles from /mnt/disks/data-1/20240506_cryosparc_arctica/topaz-train/extract1-converted.star
[CPU: 240.4 MB] File extension is star
[CPU: 240.4 MB] Importing star file.
[CPU: 247.7 MB] --------------------------------------------------------------
[CPU: 247.7 MB] Loaded star file with 154512 items
[CPU: 247.7 MB] Fields loaded from star file: ['rlnAutopickFigureOfMerit', 'rlnMicrographName', 'rlnCoordinateX', 'rlnCoordinateY']
[CPU: 247.7 MB] --------------------------------------------------------------
[CPU: 247.7 MB] Reading particle data locations...
[CPU: 247.7 MB] Parameter particle_blob_path was set and is a directory, so will be used as the search base for finding referenced data paths in the star file.
[CPU: 247.7 MB] Searching for linked data files...
[CPU: 248.0 MB] Traceback (most recent call last):
File "/home/xxxx/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.7/site-packages/pandas/core/indexes/base.py", line 2895, in get_loc
return self._engine.get_loc(casted_key)
File "pandas/_libs/index.pyx", line 70, in pandas._libs.index.IndexEngine.get_loc
File "pandas/_libs/index.pyx", line 101, in pandas._libs.index.IndexEngine.get_loc
File "pandas/_libs/hashtable_class_helper.pxi", line 1675, in pandas._libs.hashtable.PyObjectHashTable.get_item
File "pandas/_libs/hashtable_class_helper.pxi", line 1683, in pandas._libs.hashtable.PyObjectHashTable.get_item
KeyError: 'path'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "cryosparc_master/cryosparc_compute/run.py", line 85, in cryosparc_compute.run.main
File "/home/xxxx/cryosparc/cryosparc_master/cryosparc_compute/jobs/imports/run.py", line 121, in run_import_particles
all_raw_paths = sorted(list(set(blobdf['path'])))
File "/home/xxxx/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.7/site-packages/pandas/core/frame.py", line 2906, in __getitem__
indexer = self.columns.get_loc(key)
File "/home/xxxx/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.7/site-packages/pandas/core/indexes/base.py", line 2897, in get_loc
raise KeyError(key) from err
KeyError: 'path'
I have already tried importing the micrographs that the particles were picked from separately, running CTF estimation on them again in cryosparc, and then providing the CTF estimation output as the micrograph/exposure input for the import job. I have also tried providing nothing in that field, as well as the output of just the import micrograph job. None of those things have made a difference. Any help would be appreciated.
I am hoping that the issue is as simple as adding something to the header or adding a column with some kind of information. The micrograph name does match the file name in the path I am giving cryosparc for the “particle data path”. I am providing this star file in the “particle meta path.” I am not sure what if anything I should be providing for the “micrographs” input above these (I have tried adding nothing, adding the separately imported micrographs that these particles came from, and adding CTF corrected exposures, also from which these particles were chosen).
cryosparc on this workstation is 3.3.1 I believe. Updating it is not an option for us at the moment. Could you perhaps direct me to an example of what the inputs for this particle import job should look like?
Have you tried to enable Ignore Raw Data and Ignore Pose Data on the import job along with supplying the microscope parameter info - similar to this issue with importing crYOLO particle picks?
Thanks so much for taking a look. It ended up working when I turned on those two options you mentioned and I cut 22 characters from the beginning of the micrograph input–I hadn’t realized that the CTF job was adding an additional string to the beginning of the micrograph name and that this was what the job was looking at to match the star file (I had assumed it was looking in the particle data path I gave it and at the star file).
Trying to now run 2D classification on these particles, It is saying that there are inputs missing. Are particles imported in this manner missing some kind of information that is required for 2D classification?
See error below:
[CPU: 80.3 MB] Project P21 Job J33 Started
[CPU: 80.5 MB] Master running v3.3.1, worker running v3.3.1
[CPU: 80.9 MB] Traceback (most recent call last):
File "cryosparc_worker/cryosparc_compute/run.py", line 52, in cryosparc_compute.run.main
File "/home/xxxx/cryosparc/cryosparc_worker/cryosparc_compute/jobs/runcommon.py", line 845, in check_default_inputs
assert False, 'Non-optional inputs from the following input groups and their slots are not connected: ' + missing_inputs + '. Please connect all required inputs.'
AssertionError: Non-optional inputs from the following input groups and their slots are not connected: particles.blob, particles.ctf. Please connect all required inputs.
Since you only imported the coordinates of the particles, you dont actually have particle images to classify. Therefore, you will need to extract them from the micrographs using Extract from Micrographs.