Cryosparc 4.7.0 - cleanup intermediate results deleted everything?

[biocit]

Hi

A user cleaned up one of his workspaces. But instead of cleaning up the intermediate results basically everything got deleted and one month of calculations are gone. The workspace is empty except for the Live Session. The data on the fileserver is also gone.

In the logs i can see this:

2025-05-15 11:57:36,023 clear_intermediate_results_run INFO | Clearing intermediate results for all jobs in P99

2025-05-15 12:23:09,824 dump_job_database INFO | Request to export P99 J2807
2025-05-15 12:23:09,824 dump_job_database INFO | Exporting job to cryosparc/Projects/CS-xxxxx/J2807
2025-05-15 12:23:09,825 dump_job_database INFO | Exporting all of job’s images in the database to cryosparc/Projects/CS-xxxxx/J2807/gridfs_d
ata…
2025-05-15 12:23:10,944 dump_job_database INFO | Writing 573 database images to cryosparc/Projects/CS-xxxxx/J2807/gridfs_data/gridfsdata_0
2025-05-15 12:23:10,944 dump_job_database INFO | Done. Exported 573 images in 1.12s
2025-05-15 12:23:10,945 dump_job_database INFO | Exporting all job’s streamlog events…
2025-05-15 12:23:11,222 dump_job_database INFO | Done. Exported 1 files in 0.28s
2025-05-15 12:23:11,222 dump_job_database INFO | Exporting job metafile…
2025-05-15 12:23:11,224 dump_job_database INFO | Creating .csg file for particles
2025-05-15 12:23:11,238 dump_job_database INFO | Creating .csg file for class_averages
2025-05-15 12:23:11,355 dump_job_database INFO | Done. Exported in 0.13s
2025-05-15 12:23:11,355 dump_job_database INFO | Updating job manifest…
2025-05-15 12:23:11,361 dump_job_database INFO | Done. Updated in 0.01s
2025-05-15 12:23:11,361 dump_job_database INFO | Exported P99 J2807 in 1.54s

and

2025-05-15 13:17:11,106 delete_job INFO | Deleting P99 J1380
2025-05-15 13:17:11,107 delete_job INFO | Now clearing job…

in job.json of the affected jobs i can see:

“deleted_at”: {
“$date”: “2025-05-15T11:17:17.506Z”

Is there a indication in the log files to clearly see which task was asked from the user? He told me that he selected “Clear intermediate results”. From the behavior i assume it was “Delete non-final jobs”.

Or Cryosparc would have a heavy bug. Of course i can’t test it again because i don’t want to kill another project.

[Haoran]

Did you mark your final job?

[biocit]

The user didn’t do that. I think no one in our instance does that really. But I would not expect that it deletes everything when a user select “Clear intermediate results”. Especially when “Delete non-final jobs” exists

[wtempel]

@biocit Please can you send us

• the tgz file created by the command
  cryosparcm snaplogs
• the corelogs.tar.bz2 file created by the command

  # ensure current working dir has plenty of space
  # replace /path/to with actual installation path
  tar jcvf corelogs.tar.bz2 /path/to/cryosparc_master/run/command_core.log*
  

If you can upload the files to a file sharing service, please send me a direct message with the link(s). Alternatively, we can make arrangements for you to upload the files.

[Mark-A-Nakasone]

correct, sometimes I go to the project level and just clear the intermediate results that could be TBs. That function of just clearing intermediate results should be safe. Some of our quotes and data polices are harsh.