Cs.lock doesn't block other instance from attaching project

Troubleshooting
1

We have a small server dedicated to running Live at the scope, and a larger processing cluster for everything else. Both see the same storage server, but since the introduction of cs.lock files and the ability to detach/attach projects, we haven’t been too worried about the two cryosparc instances fighting over the same projects.

However, we don’t see the cs.lock file actually blocking another instance from attaching currently (both instances are on 4.3.0). Instead, if someone tries to attach a project while Live is still running, cs.lock is simply overwritting with the new instance’s info, leaving the project in a non-funcioning state.
We’ve been able to restore it to the original instance by copying back the correct info in the cs.lock file (from any other non-corrupted project).

I’m pretty sure the lock files were working as intended in earlier 4.x versions, but I can’t say when it might have stopped. Has anyone else experienced this issue, and perhaps found a fix?

2

Thanks @boggild for reporting this bug. We are working on a fix.

2 Likes
3

We still saw this issue on 4.4.1 - is it fixed in 4.5.x?

4

@boggild A fix was included in v4.4. Please can you describe any details that may help us to reproduce the issue in v4.4.1, such as

  • whether the presence of cs.lock inside the project directory was confirmed by a file listing prior to attachment to the new instance
  • the CryoSPARC version under which the relevant project was first created on the original instance
  • whether the project was attached on the new instance using the web app or command line
5

Hi @wtempel,

  1. The precence of the lock file was not checked beforehand, but I can make a test. How do I best delete a project from the original installation after it is “taken over” by the other, without deleting files from the project folder?
  2. The project was created in 4.3 and unfortunately the hardware on this machine doesn’t allow us to update. I assumed the check was on the installation attaching a project (v4.4.1), but perhaps the older installation on the machine it was created on is the cause of the issue here?
  3. Connected via the web app.
6

@boggild As you reported in September 2023, CryoSPARC v4.3 includes a bug that can cause the CryoSPARC instance to ignore, then overwrite a cs.lock file that is meant to lock the project directory to another instance. It is therefore possible to (erroneously) attach a project that is already locked to a CryoSPARC instance (v4.x) to another CryoSPARC instance (v4.3). In other words, the bug is fixed or not depending on the version of the destination CryoSPARC instance, not depending on the version of the CryoSPARC source instance that wrote the old cs.lock file.
You may want to consider blocking access by the CryoSPARC v4.3 master host to project directories that “belong” to other CryoSPARC instances.

To avoid any confusion in selecting the appropriate method, please can you

  1. confirm the version of the CryoSPARC instance from which you wish to delete the project
  2. confirm the version of the CryoSPARC instance that took over the project
  3. describe the action(s) (in GUI and/or CLI) performed on both the “source” and “destination” instances that resulted in the “takeover” of the project.
  4. outputs of the following commands
    cryosparcm cli "get_project('P1234', 'uid', 'project_dir', 'detached')"
cat $(cryosparcm cli "get_project_dir_abs('P1234')")/cs.lock
cryosparcm cli "get_instance_uid()"
    with P1234 replaced by the respective relevant project IDs
    • on the source instance
    • on the destination instance