I had a project on cryosparc instance 1. I detached the project from there and tried to attach it to another instance (instance 2). During reattaching in instance 2, the database filesystem got full and the reattachment did not complete and cryoSPARC was showing error in connecting. I proceeeded with cleaning up the filesystem to make space and tried to start cryoSPARC again. It started normally.
But when I opened by newly attached project, some of the last jobs in the last workspace were missing. Only half the jobs were showing up. I waited for sometime for it to load, but still didn’t help. I tried to detach and reattach the project. I am guessing that was probably a bad idea, as the project.json, job_manifest.json and workspace.json files got overwritten and do not show the missing jobs either now. However, those job directories and data do exist in the project directory. They are just not reflected in the json file and hence cryoSPARC can’t see it.
I was wondering if there is any way to recreate these json files with all the information about these missing jobs.
Prepare and set aside a copy of the project directory, in case the following steps result in unintended and unwanted changes to the project directory.
Post under this topic any questions you may have regarding the procedure below.
Save the following script to a file build_manifest.py:
project_dir = "/path/to/unattached/copy/of/projectdir/"
manifest_path = "/tmp/WpPwiAf5vi.job_manifest.json"
failed_job_docs_path = "/tmp/WpPwiAf5vi.failed.json"
import pathlib
import json
jobs = []
failed_job_dirs = {}
for doc in pathlib.Path(project_dir).glob('J*/job.json'):
job_dir = doc.parts[-2]
with open(doc) as handle:
try:
assert job_dir[1:].isdigit(), f"{doc} is not a valid job document path."
data = json.load(handle)
assert job_dir == data['uid'], f"Job uid does not match directory name {job_dir}."
jobs.append(data['uid'])
except Exception as e:
failed_job_dirs[job_dir] = str(e)
with open(manifest_path, 'w') as mhandle:
json.dump({'jobs': sorted(jobs)}, mhandle, indent=4)
if failed_job_dirs:
with open(failed_job_docs_path, 'w') as fhandle:
json.dump(failed_job_dirs, fhandle, indent=4)
In build_manifest.py, edit the project_dir definition on the first line to point to the unattached copy of the project directory.
Run the command python3 build_manifest.py
The command may take some time to complete. After completion, inspect (without modifying) the *.job_manifest.json and *.failed.json files created by the script inside the /tmp/ directory. *.job_manifest.json should include most job IDs in the project. *.failed.json should include records for a few job IDs along with errors that lead to those job’s exclusion from the job manifest.
Copy the *.job_manifest.json to the unattached project directory as a file named job_manifest.json.
Thank you for the detailed solution. I tried the steps as suggested. I was able to create job_manifest.json that included all the jobs in the directory. None of them shows any error. I also created a workspaces.json. I did not modify te project.json at all. But when i try to attach it, no workspaces or jobs show up on the project. However, all the directories still exist in the folder.
@Adwaith99 Thanks for sending the information. The command_core log, which you may browse using the command
cryosparcm log command_core | less
indicated a problem with the supplied workspaces.json file:
2025-09-03 10:34:30,833 import_project_run ERROR | workspaces_doc_data = load_workspaces_document(abs_path_export_project_dir)
[..]
2025-09-03 10:34:30,833 import_project_run ERROR | json.decoder.JSONDecodeError: Expecting property name enclosed in double quotes: line 13 column 5 (char 299)
The actual error is the trailing comma in the preceding line 12 of workspaces.json:
12 "deleted": false,
13 },
Removing the comma on line 12 (and, similarly, commas on lines 24, 36, 48, 60, 72) should make workspaces.json readable for the JSON decoder.
After a failed attachment attempt, job_manifest.json may be truncated. If truncation occurred (please check), the file may have to be replaced again by a “synthetic” job_manifest.json (see Corrupted project json files after faulty reattaching - #2 by wtempel) after detachment (if needed) and before a renewed import attempt.
I encountered a similar situation to this thread and followed the recovery procedure, but it did not succeed. I would greatly appreciate any further advice.
Background:
Original project was created and processed in CryoSPARC v4.6.0.
I attached it to a new instance running CryoSPARC v5.0.6.
After the first attach, only 371 jobs appeared in the UI (out of ~1810 total job directories on disk). Most workspaces were empty.
I waited several days with no further jobs appearing, then detached and re-attached the project – still no improvement.
At some point I created a new job and received the error “job dir is not empty”.
Now both project.json and workspaces.json are incorrect/corrupted.
There is no usable backup of the original intact project directory or of the original instance database.
What I have tried (following the procedure in this thread):
Detached the project.
Ran the build_manifest.py script – it successfully produced a job_manifest.json that lists all 1810 jobs.
Created a synthetic workspaces.json based on the workspace UIDs found in the job.json files (25 workspaces expected).
Placed both job_manifest.json and the synthetic workspaces.json into the project directory and tried to attach again.
After attach, no workspaces appear in the UI at all.
I ran cryosparcm snaplogs but did not find obviously useful error messages related to the attachment. I can provide the full log archive or specific excerpts if helpful.
Questions:
Is the recovery procedure from this thread still expected to work on CryoSPARC v5.0.6?
Is there any hope of recovering all ~1810 jobs and the 25 workspaces given that the project has already been incorrectly detached/attached multiple times and there is no remaining backup?
What additional information or files would be most useful for further diagnosis (e.g., contents of the synthetic workspaces.json, job_manifest.json, etc.)?
Thank you very much for any guidance. I understand this is a non-standard recovery situation and may not be fully recoverable.
Did you detach the project from the v4.6.0 instance?
Was this the first attempt to attach the project, or was there a previous attempt to attach the project to CryoSPARC instance with a version below v5.0.6?
Thank you for the follow-up questions. The original operator is no longer fully certain about the exact sequence of events.
Did you detach the project from the v4.6.0 instance?
The project may have been detached (and possibly re-attached) multiple times, on different computers, by different people. There is no clear record of a clean, single detachment from the original v4.6.0 instance.
Was this the first attempt to attach the project, or was there a previous attempt on a version below v5.0.6?
Earlier attach attempts cannot be ruled out. We do not have reliable logs to confirm this.
In short, the project’s transfer history appears messy . Would importing the jobs individually be a possible workaround?
Please let us know if this incomplete history changes the recovery outlook, or if any further diagnostic steps would still be useful.
I may iteratively request additional information that may help determine the best aproach.
Let’s start with
Do any jobs show up when you try display all jobs in the imported project, using a URL that ends with /browse/P99-W*-J* (when you replace P99 with the project UID of the newly imported project)?
Did the script produce a *.failed.json file? If so, did the file contain any job IDs?
Thanks @tosyl for trying that.
Please can you run these commands and email us the file created with these commands.
csprojectid=P999 # replace P999 with actual project ID
cd $(mktemp -d)
cryosparcm cli "api.projects.find_one('$csprojectid')" > ${csprojectid}_info.txt
cryosparcm cli "api.workspaces.find(project_uid='$csprojectid')" >> ${csprojectid}_info.txt
gzip *txt
realpath *.gz # display path of file to send us
I will send you a personal message with our email address.
. Would importing the jobs individually be a possible workaround?