When I try to import my project after an Ubuntu update, job J61 fails to import:
2024-03-26 10:36:23,920 wrapper ERROR | KeyError: ‘J61’
2024-03-26 10:40:54,717 import_project_run WARNING | Failed laying out tree in P4: ‘J61’
The tree view now doesn’t work and when I log onto cryosparc the project and its workspaces have lost their names. Can you please advise how to fix this? All other jobs import fine. I am using Cryosparc v4.4.
The job exists and looks fine:
ls -ltr J61
total 5916
-rw-rw-r-- 1 luca lp-group 515538 Dec 13 10:47 0_exposures_accepted.cs
-rw-rw-r-- 1 luca lp-group 15838 Dec 13 10:47 0_exposures_rejected.cs
-rw-rw-r-- 1 luca lp-group 5224842 Dec 13 10:47 J61_passthrough_exposures_accepted.cs
-rw-rw-r-- 1 luca lp-group 159462 Dec 13 10:47 J61_passthrough_exposures_rejected.cs
-rw-rw-r-- 1 luca lp-group 3392 Dec 13 10:47 J61_passthrough_exposures_manual_rejected.cs
drwxrwsr-x 2 luca lp-group 4096 Dec 13 10:47 gridfs_data
-rw-rw-r-- 1 luca lp-group 6257 Dec 13 10:47 job.log
-rw-rw-r-- 1 luca lp-group 22967 Mar 20 14:56 events.bson
-rw-rw-r-- 1 luca lp-group 1640 Mar 20 14:56 J61_exposures_accepted.csg
-rw-rw-r-- 1 luca lp-group 1616 Mar 20 14:56 J61_exposures_rejected.csg
-rw-rw-r-- 1 luca lp-group 88459 Mar 20 14:56 job.json
However, job J61 is not in mongoDB anymore:
meteor:PRIMARY> db.jobs.findOne({ project_uid: ‘P4’, uid: ‘J61’ }, { project_uid: 1, uid: 1, deleted: 1 })
null
Could you check your job_manifest.json file inside the project directory to see if J61 is included? If it’s not, try adding it to the file and retry the import.
If the above doesn’t work, please send us the entire log section from when the import runs.
Hi @nwong,
Thank you for getting back to me. I tried adding J61 to job_manifest.com but import fails again, now complaining about J97. The majority of jobs are imported but several are missing and the there’s no tree view.
How do I attach the log? I can only see a button to upload pictures. The relevant log section is too long to copy-paste here.
It looks like there may be an issue with your manifest not containing all the jobs correctly in your project folder. If J97 is failing with similar messages to J61, you may try to add J97 and any other jobs with the same problem to the manifest as well.
I will DM you an email address that you can send your logs to.
I managed to fix the problem by adding all missing jobs IDs in job_manifest.json. I realised that, when the project was detached, the job list was truncated after J603, so I added the remaining job IDs and the import was successful, I can now see all jobs and the tree view has been restored.
I’ve got an idea why the import failed, which might point to a bug. The job list in job_manifest.json stopped after J605, which was a failed job (Extract job that run out of space, like J604). Is it possible that the fact that the last job(s) to be executed failed caused the job_manifest.json file to be written out incorrectly? All the missing job IDs in job_manifest.json were the ones that would come after J605 according to the way Cryosparc indexes numerically the job IDs in the file (i.e. J1, J11, J111, J112 etc).
Glad you were able to resolve your issue, and thank you for your observations. We will look into this to see if it is a potential bug. To help with our investigation, could you send your logs to the email that I DM’ed to you earlier?