Unable to import project after V4.6 update

Troubleshooting
1

Hi,

I’m having trouble importing my projects after reinstalling CryoSPARC v4.6 on my server. When I try to import a project, I get the following error message:

Unable to import project from [my/directory/path]: ‘workspace_type’

The project window remains empty as if the project was not imported, but the project does appear in the Manage Project Data window. Additionally, a ‘cs.lock’ file has been created in the project folder.

For context, I initially attempted to update CryoSPARC from v4.5.3 to v4.6, but my VPN connection crashed, corrupting some files during the update process. I then detached all my projects from CryoSPARC v4.5.3, uninstalled it, and reinstalled CryoSPARC v4.6.

However, I’m now unable to reattach my projects to the new installation.

Any assistance would be greatly appreciated.

Thank you!

Pierre

2

Update:

I made a fresh installation of cryosparc V4.6 and rebooted the server, but the issue remains the same.

Before importing any old project, the web application works perfectly fine.

After importing my old project (created with V4.5.3) the web application is lagging and the project window remains blank. The imported project contains only an empty workspace (no thumbnails). I can still launch jobs in other projects, but the navigation in the web application is very slow (I need to refresh my navigator after clicking on any button).

After detaching the imported old project, cryosparc is no longer lagging and back to normal!

FYI, the issue is not specific to the new Cryosparc version V4.6 as I had the same issue after importing the same old project in cryosparc V4.5.3.

Please find below the log of the command cryosparcm log command_core

2024-09-30 17:16:45,903 import_project INFO | Importing project from /srv/CS_projects/CS-XXXX
2024-09-30 17:16:45,974 import_project INFO | Created project P5
2024-09-30 17:16:46,007 import_workspaces INFO | Created workspace W1 in P5
2024-09-30 17:16:46,010 import_project_run ERROR | Unable to import project from /srv/CS_projects/CS-XXXX
2024-09-30 17:16:46,010 import_project_run ERROR | Traceback (most recent call last):
2024-09-30 17:16:46,010 import_project_run ERROR | File “/srv/cryosparcuser/cryosparc/cryosparc_master/cryosparc_command/command_core/init.py”, line 4620, in import_project_run
2024-09-30 17:16:46,010 import_project_run ERROR | import_workspaces(workspaces_doc_data, abs_path_export_project_dir, new_project_uid, owner_user_id, notification_id)
2024-09-30 17:16:46,010 import_project_run ERROR | File “/srv/cryosparcuser/cryosparc/cryosparc_master/cryosparc_command/command_core/init.py”, line 4702, in import_workspaces
2024-09-30 17:16:46,010 import_project_run ERROR | if workspace[‘workspace_type’] == ‘live’:
2024-09-30 17:16:46,010 import_project_run ERROR | KeyError: ‘workspace_type’
2024-09-30 17:16:46,014 dump_project INFO | Exporting project P5
2024-09-30 17:16:46,019 dump_project INFO | Exported project P5 to /srv/CS_projects/CS-XXXX/project.json in 0.01s
2024-09-30 17:16:46,080 run ERROR | POST-RESPONSE-THREAD ERROR at import_project_run
2024-09-30 17:16:46,080 run ERROR | Traceback (most recent call last):
2024-09-30 17:16:46,080 run ERROR | File “/srv/cryosparcuser/cryosparc/cryosparc_master/cryosparc_command/commandcommon.py”, line 73, in run
2024-09-30 17:16:46,080 run ERROR | self.target(*self.args)
2024-09-30 17:16:46,080 run ERROR | File “/srv/cryosparcuser/cryosparc/cryosparc_master/cryosparc_command/command_core/init.py”, line 4620, in import_project_run
2024-09-30 17:16:46,080 run ERROR | import_workspaces(workspaces_doc_data, abs_path_export_project_dir, new_project_uid, owner_user_id, notification_id)
2024-09-30 17:16:46,080 run ERROR | File “/srv/cryosparcuser/cryosparc/cryosparc_master/cryosparc_command/command_core/init.py”, line 4702, in import_workspaces
2024-09-30 17:16:46,080 run ERROR | if workspace[‘workspace_type’] == ‘live’:
2024-09-30 17:16:46,080 run ERROR | KeyError: ‘workspace_type’
**custom thread exception hook caught something
**** handle exception rc
Traceback (most recent call last):
File “/srv/cryosparcuser/cryosparc/cryosparc_master/cryosparc_compute/jobs/runcommon.py”, line 2304, in run_with_except_hook
run_old(*args, **kw)
File “/srv/cryosparcuser/cryosparc/cryosparc_master/cryosparc_command/commandcommon.py”, line 73, in run
self.target(*self.args)
File “/srv/cryosparcuser/cryosparc/cryosparc_master/cryosparc_command/command_core/init.py”, line 4620, in import_project_run
import_workspaces(workspaces_doc_data, abs_path_export_project_dir, new_project_uid, owner_user_id, notification_id)
File “/srv/cryosparcuser/cryosparc/cryosparc_master/cryosparc_command/command_core/init.py”, line 4702, in import_workspaces
if workspace[‘workspace_type’] == ‘live’:
KeyError: ‘workspace_type’
Traceback (most recent call last):
File “/srv/cryosparcuser/cryosparc/cryosparc_master/cryosparc_compute/jobs/runcommon.py”, line 2304, in run_with_except_hook
run_old(*args, **kw)
File “/srv/cryosparcuser/cryosparc/cryosparc_master/cryosparc_command/commandcommon.py”, line 73, in run
self.target(*self.args)
File “/srv/cryosparcuser/cryosparc/cryosparc_master/cryosparc_command/command_core/init.py”, line 4620, in import_project_run
import_workspaces(workspaces_doc_data, abs_path_export_project_dir, new_project_uid, owner_user_id, notification_id)
File “/srv/cryosparcuser/cryosparc/cryosparc_master/cryosparc_command/command_core/init.py”, line 4702, in import_workspaces
if workspace[‘workspace_type’] == ‘live’:
KeyError: ‘workspace_type’

The error seems to be connected to this message:

if workspace[‘workspace_type’] == ‘live’:
KeyError: ‘workspace_type’

The imported project was not a live project.

Here is the content of the workspaces.json file of my project:

[
    {
        "id": {
            "$oid": "XXXXXXXXXX"
        },
        "updated_at": {
            "$date": "2024-09-23T14:14:05.190Z"
        },
        "created_at": {
            "$date": "2024-08-30T12:43:46.075Z"
        },
        "dumped_at": {
            "$date": "2024-09-23T14:14:05.190Z"
        },
        "autodump": false,
        "uid": "W1",
        "project_uid": "P5",
        "created_by_user_id": "XXXXXXXXXXX",
        "title": "027d_XXXX_Artica_New_processing",
        "last_accessed": {
            "name": "Pierre",
            "accessed_at": {
                "$date": "2024-09-23T14:14:02.192Z"
            }
        },
        "workspace_stats": {
            "updated_at": {
                "$date": "2024-09-20T19:14:43.419Z"
            },
            "job_count": 34,
            "job_sections": {
                "import": 1,
                "motion_correction": 2,
                "ctf_estimation": 1,
                "exposure_curation": 3,
                "particle_picking": 4,
                "extraction": 4,
                "particle_curation": 11,
                "reconstruction": 1,
                "refinement": 6,
                "local_refinement": 1
            },
            "job_status": {
                "completed": 34
            },
            "job_types": {
                "class_2D_new": 6,
                "nonuniform_refine_new": 5,
                "select_2D": 5,
                "extract_micrographs_multi": 4,
                "curate_exposures_v2": 2,
                "inspect_picks_v2": 2,
                "patch_ctf_estimation_multi": 1,
                "denoise_train": 1,
                "blob_picker_gpu": 1,
                "patch_motion_correction_multi": 1,
                "new_local_refine": 1,
                "local_motion_correction_multi": 1,
                "hetero_refine": 1,
                "homo_abinit": 1,
                "import_movies": 1,
                "template_picker_gpu": 1
            }
        },
        "imported": true,
        "uid_num": 1,
        "project_uid_num": 5,
        "last_exported": {
            "$date": "2024-09-23T11:39:37.189Z"
        }
    }

I am running Cryosparc on a standalone configuration.

Any help to import my old project would be greatly appreciated.

3

Did in the process you overwrite the pre-existing CryoSPARC database, or is that old database still available?

You may want to perform remote commands that are vulnerable to network disruption inside a screen or tmux session that you open on the remote server.

4

Hi,

I believe I overwrote the pre-existing Cryosparc database. I have a back-up of a database but only after the up-date crashed because of the VPN connexion. I do not think this back-up would be helpful.

Do you have any hint on what’s wrong with my project?

Thanks for your tip about ‘screen’ or ‘tmux’. I am actually using ‘screen’ now.

Thanks for your help,

Pierre

5

@PierreM. Am I understanding correctly that your are currently running a single CryoSPARC instance, and its version is 4.6.0?

When were these projects created? After the “fresh” installation of v4.6.0?
Please can you email us the file /tmp/projects_20241002.txt.gz created by these commands:

cryosparcm cli "list_projects()" > /tmp/projects_20241002.txt
gzip /tmp/projects_20241002.txt

so may confirm the feasibility of a recovery method that we are considering.

6

@PierreM. Am I understanding correctly that your are currently running a single CryoSPARC instance, and its version is 4.6.0?

@wtempel Correct, I am running a single cryosparc instance on a server with a standalone setup and its version is v4.6.0

When were these projects created? After the “fresh” installation of v4.6.0?

The project in which I can start jobs have been created after the “fresh” installation of v4.6.0 .
FYI, I can detach, delete from database and attach these freshly created projects without any trouble.

I am emailing you the requested file.

Thanks for your help,

Pierre