Hi! I’ve been trying to use a new program, vesicle-picker (GitHub - r-karimi/vesicle-picker: A program interfacing Segment Anything with cryoSPARC via cryosparc-tools to efficiently identify membrane proteins and complexes in cryo-EM micrographs.) that uses cryosparc tools to import curated micrographs from cryosparc before filtering and segmenting to generate picks.
However, there seems to be some problem with cryosparc tools being able to follow the symlinks generated by the import job to find the original micrograph image.
When I try to run the python script, I receive this error:
Connection succeeded to CryoSPARC command_vis at ptclfocuser-desktop:39003
Connection succeeded to CryoSPARC command_rtp at ptclfocuser-desktop:39005
0%| | 0/2891 [00:00<?, ?it/s]/home/ptclfocuser/software/home/software/anaconda3/envs/vesicle-picker/lib/python3.9/site-packages/cryosparc/tools.py:670: UserWarning: *** CommandClient: (ptclfocuser-desktop:39003/get_project_file) HTTP Error 422 UNPROCESSABLE ENTITY; please check cryosparcm log command_vis for additional information.
Response from server: b'Invalid file path J1/imported/017646539750839992624_filepath_DW.mrc; must be in project directory for P17'
with self.download(project_uid, path_rel) as response:
0%| | 0/2891 [00:00<?, ?it/s]
Traceback (most recent call last):
File "/home/ptclfocuser/software/home/software/anaconda3/envs/vesicle-picker/find_vesicles.py", line 49, in <module>
header, image_fullres = project.download_mrc(
File "/home/ptclfocuser/software/home/software/anaconda3/envs/vesicle-picker/lib/python3.9/site-packages/cryosparc/project.py", line 360, in download_mrc
return self.cs.download_mrc(self.uid, path_rel)
File "/home/ptclfocuser/software/home/software/anaconda3/envs/vesicle-picker/lib/python3.9/site-packages/cryosparc/tools.py", line 670, in download_mrc
with self.download(project_uid, path_rel) as response:
File "/home/ptclfocuser/software/home/software/anaconda3/envs/vesicle-picker/lib/python3.9/contextlib.py", line 119, in __enter__
return next(self.gen)
File "/home/ptclfocuser/software/home/software/anaconda3/envs/vesicle-picker/lib/python3.9/site-packages/cryosparc/command.py", line 225, in make_request
raise CommandError(error_reason, url=url, code=code, data=resdata)
cryosparc.errors.CommandError: *** (ptclfocuser-desktop:39003/get_project_file, code 422) HTTP Error 422 UNPROCESSABLE ENTITY; please check cryosparcm log command_vis for additional information.
Response from server: b'Invalid file path J1/imported/017646539750839992624_filepath_DW.mrc; must be in project directory for P17'
in cryosparc log command_vis, this is the log:
2024-08-16 14:47:29,594 request_handler INFO | Completed request for get_project_file in 0.00s
2024-08-16 14:48:00,434 request_handler INFO | Received request for get_project_file
2024-08-16 14:48:00,442 request_handler INFO | Completed request for get_project_file in 0.01s
2024-08-16 14:57:53,667 request_handler INFO | Received request for get_project_file
2024-08-16 14:57:53,674 request_handler INFO | Completed request for get_project_file in 0.01s
2024-08-16 14:57:53,682 request_handler INFO | Received request for get_project_file
2024-08-16 14:57:53,685 request_handler INFO | Completed request for get_project_file in 0.00s
2024-08-16 14:58:02,106 request_handler INFO | Received request for get_project_file
2024-08-16 14:58:02,115 get_project_file ERROR | Invalid file path J1/imported/017646539750839992624_blackk_file_path_mic_DW.mrc; must be in project directory for P17
2024-08-16 14:58:02,115 get_project_file ERROR | NoneType: None
2024-08-16 14:58:02,115 request_handler INFO | Completed request for get_project_file in 0.01s
Waiting for data... (interrupt to abort)
To test this further, I tried making a simple jupyter notebook to see if I can import a micrograph using cryosparc tools and mrcfile, and similarly also obtain this error where it suggests the micrograph is not present.
from cryosparc.tools import CryoSPARC
import mrcfile
import pandas as pd
cs = CryoSPARC(host="ptclfocuser-desktop", base_port=39000)
assert cs.test_connection()
project = cs.find_project("P17")
job = project.find_job("J1")
micrographs = job.load_output("imported_micrographs")
for mic in micrographs.rows():
micrograph_path = mic["micrograph_blob/path"]
with mrcfile.open(micrograph_path, permissive=True) as mrc:
data = mrc.data
print(f"Loaded micrograph with shape: {data.shape}")
FileNotFoundError Traceback (most recent call last)
Cell In[52], line 1
----> 1 with mrcfile.open(micrograph_path, permissive=True) as mrc:
2 data = mrc.data
3 print(f"Loaded micrograph with shape: {data.shape}")
File ~/software/home/software/anaconda3/envs/cryosparc-tools-example/lib/python3.8/site-packages/mrcfile/load_functions.py:145, in open(name, mode, permissive, header_only)
143 elif start[:2] == b'BZ':
144 NewMrc = Bzip2MrcFile
--> 145 return NewMrc(name, mode=mode, permissive=permissive,
146 header_only=header_only)
File ~/software/home/software/anaconda3/envs/cryosparc-tools-example/lib/python3.8/site-packages/mrcfile/mrcfile.py:109, in MrcFile.__init__(self, name, mode, overwrite, permissive, header_only, **kwargs)
106 self._mode = mode
107 self._read_only = (self._mode == 'r')
--> 109 self._open_file(name)
111 try:
112 if 'w' in mode:
File ~/software/home/software/anaconda3/envs/cryosparc-tools-example/lib/python3.8/site-packages/mrcfile/mrcfile.py:126, in MrcFile._open_file(self, name)
124 def _open_file(self, name):
125 """Open a file object to use as the I/O stream."""
--> 126 self._iostream = open(name, self._mode + 'b')
FileNotFoundError: [Errno 2] No such file or directory: 'J1/imported/015874051784107624481_filepath_mic_DW.mrc'
Do you know why CS tools is failing to find micrographs in this way? The files are definitely in the directories they are supposed to be in and are behaving normally within cryosparc itself. I’m using Cryosparc 4.4 and CS-tools 4.4.