Upgrade to 2.11, database migration notifications stuck

Install, Configure and Update
1

I upgraded to 2.11.0, found some odd errors that based on other reports came from old 2.9.0 processes still being active. So, as suggested in that thread, I stopped cryosparc, killed all python processes, and restarted. It seems that this procedure killed active database migration processes, or maybe just the notification process.

Notifications for the following were stuck with no progress:
Exporting all jobs in P6
Database Migration: Exporting all projects in cryoSPARC

I cleared the notifications, and jobs are now able to run, but I am concerned whether this may have corrupted databases. Is there a mechanism to verify databases are consistent and/or correctly upgraded?

If it is just the notification system getting stuck for incomplete jobs, maybe it should have some sort of process verification to avoid stuck notifications.

2

Hi @Juno,

If you take a look at the Notification Manager, and check out the inactive notifications, you will probably see a “completed” notification for Database Migration: Exporting all projects in cryoSPARC. If this is there, you should be fine. The database migration does not modify the database, it just executes queries and writes out metadata to disk.

3

These still show as active ⟳ with progress bars in the “inactive” job list, with no change in progress since yesterday.

4

Hi @Juno

Please run cryosparcm restart to restart the database migration.

5

I already restarted cryosparc a few hours ago, and notifications remain unchanged. The notification system seems to think processes are still running, but they are not, and do not get restarted.

6

The notifications are permanently stuck as in-process. But clearing put them into the inactive list, and evrything still runs OK, so I think they are just lost notifications in a stale/zombie state

7

Hi -

FYI, we have the same issue (notifications seem stuck in “Exporting all jobs in P9
Database Migration: Exporting all projects in cryoSPARC” state), and no “completed” notification in the Notification manager.
Restarting cryosparc did not help either.

image
image1992×396 48 KB

Best,
Nicolas

8

Hi @ncoudray,

Did you check the “Inactive” notifications? Can you confirm there is no notification showing it is completed in that page? You can hit “clear” on the right side to clear the notification.

9

Thanks for the feedback

Yes, In the inactive notifications, I have:

image
image1032×186 18.8 KB

After I hit clear, like Juno, it moves to the “inactive” notifications, however, the progress bar associated is still low:

image

Please let me know if there’s anything else to check or do to make sure things will be working properly for that project.

10

Hi @ncoudray,

Notifications 3 and 4 indicate your migration completed successfully, you’re probably seeing those stale notifications due to cryoSPARC shutting down half way through a previous migration.

11

Ah, ok. Thanks a lot for the clarification