Recover job information

Data Management
1

Hi,
Is there a way that I can recover (or reattach) job information in the cs interface?
Recently, 2 jobs showed up in the interface as building. It looks like they were accidentally cleared, although we do not know what happened. I successfully copied all contents back from an external backup. However, the jobs still show up as building in the cs interface. How can I rebuilt the cs database information for the 2 jobs in question?
This issue could be also a feature request: Reattach jobs with specific job number and keep linkage.
Thanks,
christian

2

Hi @cbiertue, if you go into the file system are the jobs still in the project ?

maybe Mongo DB deleted them, but if you

$ cd ~/project
$ ls -lah

does it output the job numbers there ? If not they could be gone

3

Hi @Mark-A-Nakasone , thanks for the suggestion. When I discovered the change in job status, I checked the directory contents. CS-project/Jxxx and CS-project/Jxxy were still there but there was not much inside (as in build stage). I copied Jxxx and Jxxy from an external backup to CS-project/Jxxx, and CS-project/Jxxy, respectively, so all files are there but the Mongo DB (CS interface) is not updated.
Is there a way to push Mongo DB to update its entries for the 2 compromised jobs based on their contents in the physical CS-project/Jxxx and CS-project/Jxxy directories?

4

Not 100% sure, but there were many “Import” type jobs that could work with that data and .cs file. Which type of data was it ?

5

PatchMotionCorr and PatchCTF - if I understand correctly, import will create a new job, however, we would like to keep the “old” connections.

6

That is a good point. One could export Path Motion Correction or Patch CTF, but the whole workflow won’t be there.

In some cases, I have detached a project, copied it, and then re-attached the copy of the project. This is inefficient and extreme, but for I wanted it worked and kept everything together.

7

Yes, that sounds like it would work. However, I am looking for a simpler solution e.g. modifying a text file to bring everything back as I have the actual data. The project has ~2k jobs with ~20k movies in several batches. Probably, we can still continue processing as the file locations are still there. The only drawback are the missing logs, graphs, info for those 2 jobs in the cs interface.

8

Hi Christian,

In case you encounter a similar situation in the future:

Is it possible that the non-final results were cleared from the project or workspace without first marking descendants of the affected jobs or the affected jobs themselves as final (which is distinct from the concept of job completion)?

Modifications of the project directory such as copying files into a project or job directory have the potential to make the project directory inconsistent with the CryoSPARC database, and to disrupt CryoSPARC function.
A potential and partial recovery procedure might entail:

  • restoring a backup of the project that still contains the intact job to a different path than the project already attached
  • attaching the restored project to the CryoSPARC instance as an additional project
  • exporting the relevant, intact job from the newly attached project
  • importing the exported job to the project where the job was accidentally cleared

This procedure would allow the job’s outputs to be used by newly created jobs, but would unfortunately not completely undo the accidental clearing of the jobs.