Dear cryoSPARC team,
I experience an error querying the status of particular job types.
So far I got it consistently for “import_particles” and “import_micrographs” jobs.
The jobs are always created and completed, either executed by GUI, CLI or API.
However, when asking for the job status by either
- CLI, ICLI (
get_job("P80","J2")or - API (
job.wait_for_done()),
I got something like the one below.
However, get_job_status(...) works and returns “completed”. So there might be something amiss with get_job() for these imported jobs specifically. On the other hand, “import_volume” jobs can be accessed.
Error for get_job()
cryosparcm cli 'get_job("P80","J2")'
*** CommandClient: (http://xxx.xxx.com:39002/api) HTTP Error 500 INTERNAL SERVER ERROR; please check cryosparcm log command for additional information.
Response from server: b'<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">\n<title>500 Internal Server Error</title>\n<h1>Internal Server Error</h1>\n<p>The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application.</p>\n'
Traceback (most recent call last):
File "/opt/cryosparc/cryosparc_master/cryosparc_tools/cryosparc/command.py", line 104, in func
with make_json_request(self, "/api", data=data) as request:
File "/opt/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/contextlib.py", line 113, in __enter__
return next(self.gen)
File "/opt/cryosparc/cryosparc_master/cryosparc_tools/cryosparc/command.py", line 191, in make_request
raise CommandClient.Error(client, error_reason, url=url)
cryosparc_tools.cryosparc.command.Error: *** CommandClient: (http://xxx.xxx.39002/api) HTTP Error 500 INTERNAL SERVER ERROR; please check cryosparcm log command for additional information.
Response from server: b'<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">\n<title>500 Internal Server Error</title>\n<h1>Internal Server Error</h1>\n<p>The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application.</p>\n'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/opt/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/opt/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/opt/cryosparc/cryosparc_master/cryosparc_compute/client.py", line 57, in <module>
print(eval("cli."+command))
File "<string>", line 1, in <module>
File "/opt/cryosparc/cryosparc_master/cryosparc_tools/cryosparc/command.py", line 107, in func
raise CommandClient.Error(
cryosparc_tools.cryosparc.command.Error: *** CommandClient: (http:/xxx.xxx:39002) Did not receive a JSON response from method "get_job" with params ('P80', 'J2')
When building my own function ```wait_until_finished(project_uid, job_uid)``` using the CLI, I don't have any problems and the job status "completed" is red out as expected.
I restarting after the import and simply reading the job_status, everything works as expected as well.
my versions:
cryosparc version 4.2.0
cryosparc-tools version 4.2.0
### Here the code snippet.
```python
print("Importing STAR file...")
ptcls_import_job = workspace.create_job(
"import_particles",
params={"particle_meta_path": star_ptcls_path})
print(f"Created STAR import {ptcls_import_job.uid}")
ptcls_import_job.queue(lane)
ptcls_import_job.wait_for_done()
Here the output from cryosparcm log command_core
023-07-11 10:17:24,067 COMMAND.JOBS set_job_status INFO | Status changed for P80.J248 from launched to started
2023-07-11 10:17:24,073 COMMAND.CORE run INFO | Custom certificate mode - using /home/root/ca-bundle.pem for external request
2023-07-11 10:17:26,913 COMMAND.JOBS set_job_status INFO | Status changed for P80.J248 from started to running
2023-07-11 10:17:26,919 COMMAND.CORE run INFO | Custom certificate mode - using /home/root/ca-bundle.pem for external request
2023-07-11 10:17:26,979 COMMAND.DATA dump_project INFO | Exporting project P80
2023-07-11 10:17:26,983 COMMAND.DATA dump_project INFO | Exported project P80 to /data/cryo-em/manual_processing/cryosparc/fkrupp/CS-15069-03/project.json in 0.00s
[2023-07-11 10:17:30,224] ERROR in app: Exception on /api [POST]
Traceback (most recent call last):
File "/opt/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/site-packages/flask/app.py", line 2447, in wsgi_app
response = self.full_dispatch_request()
File "/opt/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/site-packages/flask/app.py", line 1952, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/opt/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/site-packages/flask/app.py", line 1821, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/opt/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/site-packages/flask/_compat.py", line 39, in reraise
raise value
File "/opt/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/site-packages/flask/app.py", line 1950, in full_dispatch_request
rv = self.dispatch_request()
File "/opt/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/site-packages/flask/app.py", line 1936, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/opt/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/site-packages/flask_jsonrpc/__init__.py", line 170, in wrapper
return jsonify_status_code(status_code, response_obj, is_batch=is_batch), status_code
File "/opt/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/site-packages/flask_jsonrpc/helpers.py", line 59, in jsonify_status_code
response = jsonify(*args, **kw)
File "/opt/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/site-packages/flask/json/__init__.py", line 370, in jsonify
dumps(data, indent=indent, separators=separators) + "\n",
File "/opt/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/site-packages/flask/json/__init__.py", line 211, in dumps
rv = _json.dumps(obj, **kwargs)
File "/opt/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/json/__init__.py", line 234, in dumps
return cls(
File "/opt/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/json/encoder.py", line 199, in encode
chunks = self.iterencode(o, _one_shot=True)
File "/opt/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/json/encoder.py", line 257, in iterencode
return _iterencode(o, 0)
File "/opt/cryosparc/cryosparc_master/cryosparc_command/commandcommon.py", line 36, in default
return super().default(obj)
File "/opt/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/site-packages/flask/json/__init__.py", line 100, in default
return _json.JSONEncoder.default(self, o)
File "/opt/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/json/encoder.py", line 179, in default
raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type bytes is not JSON serializable
2023-07-11 10:17:32,272 COMMAND.DATA dump_job_database INFO | Request to export P80 J248
2023-07-11 10:17:32,274 COMMAND.DATA dump_job_database INFO | Exporting job to /data/cryo-em/manual_processing/cryosparc/fkrupp/CS-15069-03/J248
2023-07-11 10:17:32,275 COMMAND.DATA dump_job_database INFO | Exporting all of job's images in the database to /data/cryo-em/manual_processing/cryosparc/fkrupp/CS-15069-03/J248/gridfs_data...
2023-07-11 10:17:32,311 COMMAND.DATA dump_job_database INFO | Writing 6 database images to /data/cryo-em/manual_processing/cryosparc/fkrupp/CS-15069-03/J248/gridfs_data/gridfsdata_0
2023-07-11 10:17:32,312 COMMAND.DATA dump_job_database INFO | Done. Exported 6 images in 0.04s
2023-07-11 10:17:32,312 COMMAND.DATA dump_job_database INFO | Exporting all job's streamlog events...
2023-07-11 10:17:32,316 COMMAND.DATA dump_job_database INFO | Done. Exported 1 files in 0.00s
2023-07-11 10:17:32,316 COMMAND.DATA dump_job_database INFO | Exporting job metafile...
2023-07-11 10:17:32,317 COMMAND.DATA dump_job_database INFO | Creating .csg file for imported_particles
2023-07-11 10:17:32,323 COMMAND.DATA dump_job_database INFO | Done. Exported in 0.01s
2023-07-11 10:17:32,323 COMMAND.DATA dump_job_database INFO | Updating job manifest...
2023-07-11 10:17:32,327 COMMAND.DATA dump_job_database INFO | Done. Updated in 0.00s
2023-07-11 10:17:32,327 COMMAND.DATA dump_job_database INFO | Exported P80 J248 in 0.05s
2023-07-11 10:17:32,339 COMMAND.JOBS set_job_status INFO | Status changed for P80.J248 from running to completed