Project Only Displays a Subset of the Jobs

Data Management
1

Hello CryoSPARC community, I recently reinstalled my computer’s operating system. Although the project data and job directories were preserved, CryoSPARC itself was installed on the system drive and had to be reinstalled. I restored the database from a backup, but it’s an older version. So the current project only displays a subset of the jobs. I apologize for not having a more recent backup. Could you please advise on how to view all jobs in the card view?

2

@xyfzkd Restoring an outdated backup of the CryoSPARC database may lead to the corruption of CryoSPARC project directories (guide). You may want to shutdown the CryoSPARC instance to avoid additional corruption of CryoSPARC projects. When I have time, I will respond with some recovery suggestions.

3

If an outdated database backup is restored, project directories may be out of sync with the database, making some jobs inaccessible. For recovery, we suggest creating a fresh database and reimporting the project directories. The procedure below includes some checks that may detect inconsistencies between the database and project directories. If inconsistency is detected, stop interaction with the project and post the suggested information under this forum topic.

  1. shutdown CryoSPARC, to avoid additional corruption of CryoSPARC project directories
  2. create a new, empty database directory on a reliable filesystem with sufficient capacity. The database may grow to hundreds of gigabytes or even a few terabytes. Continuously monitor capacity of the volume and never allow it to fill up completely.
  3. inside cryosparc_master/config.sh, edit the definition export CRYOSPARC_DB_PATH= to point to the new directory.
  4. start CryoSPARC.
  5. create the first CryoSPARC user on the command line and, as needed, additional users on the command line or GUI.
  6. connect node- and/or cluster-type scheduler lane(s)
  7. attach projects. For each project you wish to attach
    1. navigate to the project directory.
    2. inspect the job_manifest.json file (with a command like less job_manifest.json) and confirm that it contains all jobs expected for the project. Compare the contents of the file to the output of the command
      ls -ld J*. If you find any jobs missing from the file, skip this project for now and post the outputs of these commands:
      ls -ld J* | tail -20
tail -n 20 job_manifest.json
    3. rename the cs.lock file
      mv cs.lock cs.lock.$(date +%s)
      This is a rare exception from the rule to not manipulate cs.lock files, justified by this recovery situation.
    4. try attaching the project
    5. wait for the attachment to complete and confirm that all jobs have been imported, are visible in the UI. If jobs are missing, inspect the command_core log for errors and post the errors. Do not interact with the project to avoid corruption of the project directory.
4

Many thanks~ I would have a try :smiling_face_with_three_hearts: