Issue running cryosparcw v4.6.2

Hi

We have a cluster configuration, I upgraded the master to v4.6.2 successfully (from v4.4.1).

I installed the corresponding worker, it reports success and received expected output from cryosparcm cluster validate on the master, however there was no output file of the submitted job on the cluster (I think this may have happened previously although still worked).

Trying to submit jobs results in logged errors:

Traceback (most recent call last):
  File "/${csparcw_path}/cryosparc_worker/deps/anaconda/bin/conda", line 12, in <module>
    from conda.cli import main
ModuleNotFoundError: No module named 'conda'
Warning: Could not activate conda environment; this indicates that a cryoSPARC installation is either incomplete or in progress
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "cryosparc_master/cryosparc_compute/run.py", line 10, in init cryosparc_master.cryosparc_compute.run
ModuleNotFoundError: No module named 'Crypto'

${csparcw_path} is shown in full just truncated here for brevity

A similar error (first five lines) is displayed when trying to run any ./cryosparcw command on the command line.

I re-built the environment (rm -rf all conda directories, cryosparc_worker directory, checked environment vars were cleared and installed again).

Also note that in the output from the installation it reports:

To activate this environment, use:

    micromamba activate /${csparcw_path}/cryosparc_worker/deps/anaconda

Trying this results in:

micromamba: command not found

Any suggestions would be appreciated!

Kind regards

Stuart

Welcome to the forum @sjm000 .

Please can you again remove the cryosparc_worker/ directory, unpack cryosparc_worker.tar.gz and run installation

uname -a
cat /etc/*release
env | grep PATH
stat -f .
cd cryosparc_worker
# replace your-license-id with actual license id
./install --license "your-license-id" | tee /tmp/ahghaMiix9-cs-install-log.txt
find . -ls | tee -a /tmp/ahghaMiix9-cs-install-log.txt
gzip /tmp/ahghaMiix9-cs-install-log.txt

Please post the outputs of the first four commands and email us the file
/tmp/ahghaMiix9-cs-install-log.txt.gz.

Hi

I reinstalled a conda base environment that I had stripped out earlier (that puts the anaconda references in the PATHs), this changes the error when running cryosparcw commands (I previously had Cryosparc working for quite some time on the cluster using 4.4.1 and earlier versions).

The error I get when I run ./cryosparcw commands is now:

EnvironmentNameNotFound: Could not find conda environment: cryosparc_worker_env
You can list all discoverable environments with `conda info --envs`.


Warning: Could not activate conda environment; this indicates that a cryoSPARC installation is either incomplete or in progress

I still get the same error quoted in the original post if I try to launch jobs from the master.

The output from the commands requested is below:

$ uname -a
Linux erc-hpc-comp194 6.8.0-48-generic #48~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Mon Oct  7 11:24:13 UTC 2 x86_64 x86_64 x86_64 GNU/Linux

$ cat /etc/*release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=22.04
DISTRIB_CODENAME=jammy
DISTRIB_DESCRIPTION="Ubuntu 22.04.5 LTS"
PRETTY_NAME="Ubuntu 22.04.5 LTS"
NAME="Ubuntu"
VERSION_ID="22.04"
VERSION="22.04.5 LTS (Jammy Jellyfish)"
VERSION_CODENAME=jammy
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=jammy

$ env | grep PATH
MANPATH=:/opt/puppetlabs/puppet/share/man:/opt/slurm/share/man
PATH=/software/spackages_v0_21_prod/apps/linux-ubuntu22.04-icelake/gcc-13.2.0/anaconda3-2022.10-tjkkt6f5oslpe3qj7vrpvqrm7vru4k6e/bin:/software/spackages_v0_21_prod/apps/linux-ubuntu22.04-icelake/gcc-13.2.0/anaconda3-2022.10-tjkkt6f5oslpe3qj7vrpvqrm7vru4k6e/condabin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/puppetlabs/bin:/snap/bin:/opt/slurm/bin
MODULEPATH=/software/spackages_v0_21_prod/modules/linux-ubuntu22.04-icelake

$ stat -f .
  File: "."
    ID: fa5043700000004 Namelen: 255     Type: ceph
Block size: 4194304    Fundamental block size: 4194304
Blocks: Total: 467431835  Free: 465521408  Available: 465521408
Inodes: Total: 43599910   Free: -1

Kind regards

Stuart

Hi

I have tried the worker for versions 4.6.1 and 4.5.3 that give the same error as 4.6.2.

I have just reinstalled cryosparcw 4.4.1 and that does not show any errors when running ./cryosparcw commands .

I will roll back the master (apply any patches for the 4.4.1 branch to the master and workers) and let you know if it works.

Kind regards

Stuart

@sjm000 We, unfortunately, cannot explain the ModuleNotFoundErrors .
Some issues with broken dependencies may be resolved by a forced reinstallation of dependencies with the command

/path/to/cryosparc_worker/bin/cryosparcw forcedeps

Please note that commands related to CryoSPARC installation must be run with the cryosparc_worker/ (and, similarly, cryosparc_master/) directory mounted under same absolute path as the path under which CryoSPARC commands will run. In other words, moving cryosparc_worker/ and/or cryosparc_master/ to a different path after installation would result in a non-functional CryoSPARC installation.

Another suggestion is to keep the environment as “clean” as possible during CryoSPARC installation. For example, paths to binaries and libraries of non-system software, such as under /software in your case, are likely not required during installation, but may have unexpected side effects.

Hi

Curiously, I have just completed the rollback and patching of the master (and workers) to v4.4.1 and it looks like it is all working as expected.

Thank you for the responses, I will shortly be off until the new year but I will, no doubt at some point, revisit this and see if using the suggested forcedeps resolves the issue for the v4.6.2 worker.

Kind regards

Stuart

1 Like