Automatically archiving projects & how does a project know it has been archived

Data Management
1

We’re working on automating our archiving process as we get more and more data to archive. Currently, everything that happens outside of CS is automated, but we still have to go into CS and manually archive the project prior to being able to move it somewhere else in order for the UI to be happy and for us to be able to reattach it if needed later on. I have two related questions:

(1) How does a project “know” that it has been archived? I thought this would be an easy answer in the project.json, since there is an “archived”: false line, but this does not appear to be correlated to whether a project is archived (I archived a test project and it still says archived: false despite being archived in the UI, and I tried changing it to archived: true on a different test project and the project was not archived in the UI). If it is stored somewhere in the installation directory that would be helpful to know, as I don’t have access to that so I can’t poke around in there.

(2) If there is a way to archive a project outside of the UI (I looked in cryosparc tools, but I only see ways to tell if the project has been archived from inside the interface), how soon will the UI “know” that the project has been archived; i.e., would it have to be opened by a user or loaded by someone in order for it to update? Ideally we would have a situation where we automatically archive and then immediately move the CS project folder away from where it currently is located, even overnight (i.e. when no one is refreshing the interface), but I am sure we could get creative on this if needed.

I can see how a project that has background processes running would run into issues if archived automatically, so I suppose I am also curious if there are other considerations if we were to try to automate this somewhat – I am just lazy (and of the opinion that the more automated something is, the less likely it is that I will make mistakes).

Thanks!

2

In CryoSPARC v4.7, "archived": false is expected in the project.json file inside the project directory: After a project is archived, CryoSPARC ceases exporting database records, including the archived project attribute, to the project directory.

Editing the archived: attribute inside theproject.json inside an active (attached) project directory is not expected to archive the project, as project.json is just a (downstream) snapshot of CryoSPARC database records. The CryoSPARC database is considered the “authoritative source of truth” in this scenario.

In CryoSPARC v4.7, the archive project action may be initiated on the command line:

cryosparcm cli "archive_project('P99')"

and its success confirmed with the command

cryosparcm cli "get_project('P99', 'archived')"

A few cautions for future visitors of this topic:

  1. Commands shown in this post apply to CryoSPARC v4.7.1, and may not apply to newer/future CryoSPARC versions.
  2. The Archive CryoSPARC project action prepares the project directory for, but does not include, a move or copy of project data to archive storage. Moving or copying the project directory to archive storage needs to be performed separately, outside CryoSPARC, after the CryoSPARC archive or detach project action has been performed.
  3. Explicitly setting the archived property in the database is discouraged under normal circumstances, as vital other subtasks of the archive_project() function would be omitted, which may lead to CryoSPARC malfunction and corruption of the project directory.
3

Thanks for explaining, @wtempel. In our system user/my access to cryosparcm is (rightfully) gatekept, so we will continue to do it manually.