Detaching and attaching a project

Data Management
1

Hi,
I wanted to transer a cryosparc project from an old cluster to a new cluster I recently purchased, and for this I detached the project in the old cluster with the idea to reattach it later on in the new cluster.
After detaching I copied all files from the old cluster to the new cluster (with rsync), but when I wanted to attach the project to the new cluster, I realized that the project.json and workspaces.json files were empty.
They were also empty on the original cluster.
First of all, this is super weird since I did not delete these files. Why did they get erased?
Second, since these files are empty, I cannot reattach now the project and it is a loss of many months of data processing!
Is there any way I can recover these files to reattach the project, or alternatively, recover only specific jobs from the old project?
Thanks!
Shifra

2

can you import anything to a few cryosparc project e.g. a particle stack from an extract job ? $pyem should work too if you want to output to use with RELION.

3

Welcome to the forum @shifral .

Please can you

  1. let us know the CryoSPARC version at the time when the project was created. You may find that version in the job.json file of an early job in the project directory with the command:
    grep '"version": "v' /path/to/project_dir/J1/job.json
  2. let us know the current CryoSPARC version of the CryoSPARC instance from which you detached the project.
  3. confirm that the CryoSPARC project directory with an empty project.json was in fact the project directory that was detached.
  4. inspect the command_core log for errors related to this project on the source CryoSPARC instance from which you detached the project.
    • to browse recent log entries, you may run the command on that CryoSPARC instance’s master host:
      cryosparcm log command_core | less
      (details on the less command)
    • to look for relevant errors that may have occurred longer ago, on the CryoSPARC master of the source instance, you may run
      grep -i error /path/to/cryosparc_master/run/command_core.log* | less
4

Hi,
Thanks for the input!
The cryosparc version is v4.6.2, and it was the same version when the project was created.
When checking for errors using your recommendations, it seems that the disk ran out of space, and therefore the files were deleted.
Is there a way to recover these files even if we cannot find a backup version of them? Could we perhaps generate a new project.json file using a template? (Are there a few essential rows we could put and fill in in the file so that I could reattach the project)? Thank you!

5

Hi,
I succeeded in recovering the project.json file!
I should note that I am trying to attach a project that had previously been detached from another cluster, so I modified the path in the project.json file.
However, after doing so, when I try to attach the project, I get an empty project.
The error is: “unable to import project from /path/to/directory: Ran out of input”.

I then tried to reattach the project in the old cluster. In the old cluster, after doing the attachment, I get an attached project with no error, but again an empty project.

Do you know why is this?
Thanks!

6

I should also mention that previously, to conserve space, I had deleted some of the job folders from my cryosparc project (mainly old extract jobs), directly from the cryosparc folder. Could the problem be related to this? Thanks!

7

@shifral Manual deletions/modifications of attached project directories may leave the CryoSPARC instance in an undefined state, as may a disk full event.
The project directory is likely damaged beyond just an empty project.json file. We unfortunately do not know how to recover the project.