Deleting a project where the folder was already deleted fails if Live session was run

Data Management
1

We seem to have hit a minor issue when trying to remove projects that were already manually deleted (v3.1.0+210216).

When trying to delete the project from the web interface, a notification said it was unable to delete [path/to/project/]S1. Afterwards, the folder structure was apparently recreated by Cryosparc, but only the job folders. Trying to delete again still gave the same error, but manually making the S1 folder again solved the issue. With that folder in place, we could delete the project form the web interface.

Hope its a bug that can be fixed so such a project with the files missing is just removed, as it already happens for manually deleted projects without Live session folders.

2

Hey @boggild,

Thanks for reporting- this is definitely a bug, we’ll get it fixed for the next release.

3

Encountering this again today on another project, the folder structure didn’t re-appear on its own, but I still had to just manually create the S1-folder at the expected location to allow project deletion from the web interface.

edit: it seems pressing “clear intermediate results” is what can re-create the folder structure, minus the session folder(s).

4

Many thanks!
I deleted the empty project by creating the missed Sx folder~

5

The bug seems persist, not sure if there a patch already to fix this~ I am up-to-date

6

I’m also still seeing this on our up-to-date installation.

7

I’m still seeing this issue with the latest Cryosparc patch release (v4.2.1+230621).

We regularly need to delete old projects to free up storage space and need to remove these from the database as well to avoid filling up the whole partition where the database files are stored. When the project directory has already been deleted manually the project cannot be removed from the database, command_core.log shows an error like this:

2023-08-03 11:08:39,795 COMMAND.COMMON       wrapper              ERROR    | JSONRPC ERROR at request_delete_project
2023-08-03 11:08:39,795 COMMAND.COMMON       wrapper              ERROR    | Traceback (most recent call last):
2023-08-03 11:08:39,795 COMMAND.COMMON       wrapper              ERROR    |   File "/opt/cryosparc/cryosparc_master/cryosparc_command/commandcommon.py", line 200, in wrapper
2023-08-03 11:08:39,795 COMMAND.COMMON       wrapper              ERROR    |     res = func(*args, **kwargs)
2023-08-03 11:08:39,795 COMMAND.COMMON       wrapper              ERROR    |   File "/opt/cryosparc/cryosparc_master/cryosparc_command/commandcommon.py", line 251, in wrapper
2023-08-03 11:08:39,795 COMMAND.COMMON       wrapper              ERROR    |     assert os.path.isfile(
2023-08-03 11:08:39,795 COMMAND.COMMON       wrapper              ERROR    | AssertionError: validation error: lock file for P84 at /mnt/netstor/CS-00384/cs.lock absent or otherwise inaccessible.

The only way I found so far to delete the project using the UI is to create the missing directory and copy the file cs.lock from another project. Creating an empty cs.lock file does not work, because then Cryosparc complains that the file does not contain the instance ID.

Deletion through the UI seems to trigger an export of all jobs before things are deleted from the database, so I suppose that’s why the project directory needs to exist. Is there really no other way to purge the project from the database without jumping through these hoops? Of course one could do this directly in the database but that seems error-prone and hard to find all traces of a project manually. Ideally we would script this using the cli.

8

Welcome to the forum @stiobhan.

Deletion of a project directory for a CryoSPARC project that is still attached to the CryoSPARC instance leaves the CryoSPARC instance in an inconsistent state.
If you wish to delete a project directory, please delete the project first using the corresponding Delete Project GUI action.
A workaround for projects whose directory no longer exists is described in Job/project forced removal for a deleted project directory - #2 by nwong.