Hi everyone
I got problem importing micrographs after upgrade from v4.2 to v4.4.1. When I run the ‘Import Movies’ job, I get this error:
Traceback (most recent call last):
File "cryosparc_master/cryosparc_compute/run.py", line 95, in cryosparc_master.cryosparc_compute.run.main
File "/home/cs/cryosparc/cryosparc_worker/cryosparc_compute/jobs/imports/run.py", line 1052, in run_import_movies_or_micrographs
gshapey, gshapex = read_gain_reference(gainref_abs_path, shape_only=True, data_format=data_format)
File "/home/cs/cryosparc/cryosparc_worker/cryosparc_compute/jobs/imports/run.py", line 1017, in read_gain_reference
assert hdr['nz'] == 1, "Gain reference must be a single image"
AssertionError: Gain reference must be a single image
guess there’s no problem with the gain reference file since I could import movies on the other workstation with v4.2 using the same files and gain reference.
Any idea how to fix this problem?
Welcome to the forum @U.J.Kim.
Please can you run these commands for both the (v4.2) successful and (v4.4.1) unsuccessful movie import job and post their outputs
project_id=P99 # replace with correct ID
job_id=J123 # replace with correct ID
cryosparcm cli "get_job('$project_id', '$job_id', 'params_spec')"
cryosparcm eventlog $project_id $job_id | tail -n 30
Thank you wtempel!
There could be some problem with upgrading process. (It was upgraded by another user)
and I am the first one who encountered this error.
Here’s the output of the successful one (v4.2)
[CPU RAM used: 255 MB] Reading headers of gain reference file /~/KE162_KE163_KE164_gain_reference.mrc
[CPU RAM used: 256 MB] Done importing.
[CPU RAM used: 256 MB] --------------------------------------------------------------
[CPU RAM used: 256 MB] ===========================================================
[CPU RAM used: 256 MB] Loaded 653 movies.
[CPU RAM used: 256 MB] Common fields:
[CPU RAM used: 256 MB] mscope_params/accel_kv : {130.0}
[CPU RAM used: 256 MB] mscope_params/cs_mm : {2.7}
[CPU RAM used: 256 MB] mscope_params/total_dose_e_per_A2 : {69.0}
[CPU RAM used: 256 MB] mscope_params/exp_group_id : {0}
[CPU RAM used: 256 MB] mscope_params/phase_plate : {0}
[CPU RAM used: 256 MB] mscope_params/neg_stain : {0}
[CPU RAM used: 256 MB] movie_blob/psize_A : {0.664}
[CPU RAM used: 256 MB] movie_blob/shape : [ 48 4092 5760]
[CPU RAM used: 256 MB] movie_blob/is_gain_corrected : {0}
[CPU RAM used: 256 MB] ===========================================================
[CPU RAM used: 256 MB] Making example plots. Exposures will be displayed without defect correction.
[CPU RAM used: 256 MB] Reading file…
Raw data J1/imported/016415094359687133597_FoilHole_14184361_Data_11911145_11911147_20231214_073102_fractions.tiff
[CPU RAM used: 785 MB] Reading file…
Raw data J1/imported/017697188750860636737_FoilHole_14184361_Data_11911148_11911150_20231214_073113_fractions.tiff
[CPU RAM used: 785 MB] Reading file…
Raw data J1/imported/017299415680234235935_FoilHole_14184365_Data_11911145_11911147_20231214_073122_fractions.tiff
[CPU RAM used: 794 MB] Done.
[CPU RAM used: 794 MB] --------------------------------------------------------------
[CPU RAM used: 794 MB] Compiling job outputs…
[CPU RAM used: 435 MB] Updating job size…
[CPU RAM used: 435 MB] Exporting job and creating csg files…
[CPU RAM used: 435 MB] ***************************************************************
[CPU RAM used: 435 MB] Job complete. Total time 13.08s
and the output of the unsuccessful one (v.4.4.1)
(http://stbio3:39002, code 400) Encountered ServerError from JSONRPC function “get_job” with params (‘P1’, ‘J1’, ‘params_spec’):
ServerError: P1 J1 does not exist.
Traceback (most recent call last):
File “/home/cs/cryosparc/cryosparc_master/cryosparc_command/commandcommon.py”, line 195, in wrapper
res = func(*args, **kwargs)
File “/home/cs/cryosparc/cryosparc_master/cryosparc_command/command_core/init.py”, line 5980, in get_job
raise ValueError(f"{project_uid} {job_uid} does not exist.")
ValueError: P1 J1 does not exist.
Thanks again
Please can you post specific error messages that indicate an upgrade problem.
Please confirm that the correct project and job identifiers were provided for the get_job() command. The output of the command is needed for further troubleshooting.
For the upgrade issue, I haven’t seen any issue messages, however, found that the original CryoSPARC directory was altered and renamed to “/cryosparc-old,” while a new directory was installed by another user. This make me to speculate that there might have been an issue during the upgrade process.
Here’s the output of the command from v4.4.1
{'_id': '65979dc746a001416bee7ce7', 'params_spec': {'accel_kv': {'value': 300}, 'blob_paths': {'value': '/home/~/*.tiff'}, 'cs_mm': {'value': 2.7}, 'gainref_path': {'value': '/run/media/cs/Seagate Hub/KE164_CHS_YU_20231211_mg_grid4_epu/gain/KE162_KE163_KE164_gain_reference.mrc'}, 'psize_A': {'value': 0.664}, 'total_dose_e_per_A2': {'value': 69}}, 'project_uid': 'P3', 'uid': 'J1'}
[cs@stbio3 P1]$ cryosparcm eventlog $project_id $job_id | tail -n 30
License is valid.
Launching job on lane default target stbio3 ...
Running job on master node hostname stbio3
[CPU RAM used: 178 MB] Job J1 Started
[CPU RAM used: 178 MB] Master running v4.4.1, worker running v4.4.1
[CPU RAM used: 178 MB] Working in directory: /home/cs/cryosparc-project/CS-a2-s/J1
[CPU RAM used: 178 MB] Running on lane default
[CPU RAM used: 178 MB] Resources allocated:
[CPU RAM used: 178 MB] Worker: stbio3
[CPU RAM used: 178 MB] CPU : [0]
[CPU RAM used: 178 MB] GPU : []
[CPU RAM used: 178 MB] RAM : [0, 1, 2, 3]
[CPU RAM used: 178 MB] SSD : False
[CPU RAM used: 178 MB] --------------------------------------------------------------
[CPU RAM used: 178 MB] Importing job module for job type import_movies...
[CPU RAM used: 252 MB] Job ready to run
[CPU RAM used: 252 MB] ***************************************************************
[CPU RAM used: 252 MB] Importing movies from /home/share/~/*.tiff
[CPU RAM used: 255 MB] Importing 10260 files
[CPU RAM used: 260 MB] Import paths were unique at level -1
[CPU RAM used: 260 MB] Importing 10261 files
[CPU RAM used: 260 MB] 'Skip Header Check' parameter enabled, checking first header only
[CPU RAM used: 261 MB] Reading headers of gain reference file /run/media/cs/Seagate Hub/~/KE162_KE163_KE164_gain_reference.mrc
[CPU RAM used: 261 MB] Traceback (most recent call last):
File "cryosparc_master/cryosparc_compute/run.py", line 95, in cryosparc_master.cryosparc_compute.run.main
File "/home/cs/cryosparc/cryosparc_worker/cryosparc_compute/jobs/imports/run.py", line 1052, in run_import_movies_or_micrographs
gshapey, gshapex = read_gain_reference(gainref_abs_path, shape_only=True, data_format=data_format)
File "/home/cs/cryosparc/cryosparc_worker/cryosparc_compute/jobs/imports/run.py", line 1017, in read_gain_reference
assert hdr['nz'] == 1, "Gain reference must be a single image"
AssertionError: Gain reference must be a single image
Are you sure that path to the gain ref is valid? It’s different from the 4.2 log.
1 Like
Paths with spaces in are usually a recipe for disaster*.
That gain reference path has a space in in (“/Seagate Hub/”)
*Unless appropriately quoted or escaped, which is not always guaranteed, and seems to be the case here? Thus, it is usually safer to avoid any sort of naming convention with spaces in.
edit: Also, as ~ is usually interpreted by bash/zsh/fish (not sure about csh etc) to mean “user home” (e.g.: /home/user) and I have no idea how Python will handle a path with a ~ in the middle. So that path has two potential pitfalls…
Hi rbs_sci,
Apologies for the confusion. My bad for noting the log from the external storage one and replacing the path directory with “~” to avoid showing private information.
I tested various paths that have gain reference file to confirm if the file itself are having any issues. The log I’ve posted is one of them. Indeed, as pointed out by @bogglid, the initial log was from a path without “~” and spaces.
Ah, OK. 
I normally make censored paths rather obvious… /path/to/CS-censoredProject was my last one. 
1 Like