How to restore tree view after applying patch on AWS pcluster?

Install, Configure and Update
1

I installed the latest patch on an AWS pcluster installation so I could import an existing project, using the following:

cryosparcm patch --download
cryosparcm patch --install
cd ${CRYOSPARC_INSTALL_PATH}
mv cryosparc_master/cryosparc_worker_patch.tar.gz cryosparc_worker/
cryosparc_worker/bin/cryosparcw patch
cryosparcm restart

However, my project tree view is now completely broken with all the links wiped out. How do I restore the tree view? Restart and reboot failed to fix.

2

Hi @iphan,

Does this mean you see all the job cards, but they’re not connected to each other?
Can you try connecting an existing job to a new job and see what happens?
Also, are you able to send us screenshots of what you’re seeing?

3

Correct, I see all the jobs but the connections are gone. I tried connecting a new job and the link to the old one looks fine. I refreshed the tree-view page and it shows the single new link. All other jobs appear at the top in order of creation, unlinked.

I didn’t take a screenshot, sorry. Since I was completely stuck I continued to look for solutions. In the end I tore down the cluster and EC2 instance and recreated it with our AWS deploy script, re-imported from S3 via FSxL and the tree view was restored.

I am still unable to continue work on the restored project.

Failures depend on the jobs type: 2D class selection and sharpening work fine. Ab-initio stalls at “Loading a ParticleStack with x items…” I get no error from the GUI, can’t see anything in the job logs and nothing in the /var/log either, squeue and scontrol say the job is running. But the job just sits there.

To rule out RAM issues, I tested loading small subsets of particles. Still ab-initio gets stuck at "Loading a ParticleStack…”.

4

I’ve now installed 3.3.1 from scratch, no patching. Project now imports great. Still can’t continue processing. Ab-initio stalls at "Loading a ParticleStack…”.

I cloned an Ab initio job that previously completed and tried running it, using the same GPU lane. The cloned job stalls at "Loading a ParticleStack…”.

Grateful for any suggestions as I am again completely stuck.

5

@iphan Please can you provide the stdout for this job (see cryosparcm joblog ...), after obfuscating any confidential information.