Error running homogeneous refinement in v4.0.1

Hi, I am getting the following error when launching an homogeneous refinement in a fresh updated v4.0.1 instance. Previous jobs up to this point worked without problem (from importing raw frames).


[CPU: 1.21 GB]

====== Starting Refinement Iterations ======
[CPU: 1.21 GB]

----------------------------- Start Iteration 0
[CPU: 1.21 GB]

  Using Max Alignment Radius 18.057 (30.000A)
[CPU: 1.21 GB]

  Auto batchsize: 12300 in each split
[CPU: 1.60 GB]

-- THR 1 BATCH 500 NUM 6000 TOTAL 4.7805840 ELAPSED 17.061522 --
[CPU: 1.67 GB]

  Processed 24600.000 images in 17.592s.
[CPU: 1.87 GB]

  Computing FSCs... 
[CPU: 1.87 GB]

  Using full box size 256, downsampled box size 128, with low memory mode disabled.
[CPU: 1.87 GB]

  Computing FFTs on GPU.
[CPU: 977.2 MB]

Traceback (most recent call last):
  File "cryosparc_worker/cryosparc_compute/run.py", line 93, in cryosparc_compute.run.main
  File "cryosparc_worker/cryosparc_compute/jobs/refine/newrun.py", line 638, in cryosparc_compute.jobs.refine.newrun.run_homo_refine
  File "cryosparc_worker/cryosparc_compute/engine/gsigproc.py", line 463, in cryosparc_compute.engine.gsigproc.compute_all_fscs
  File "cryosparc_worker/cryosparc_compute/engine/gsigproc.py", line 506, in cryosparc_compute.engine.gsigproc.compute_all_fscs
  File "cryosparc_worker/cryosparc_compute/engine/gsigproc.py", line 316, in cryosparc_compute.engine.gsigproc.GPUFSC.edt
  File "cryosparc_worker/cryosparc_compute/engine/newcuda_kernels.py", line 6056, in cryosparc_compute.engine.newcuda_kernels.euclidian_distance_transform
  File "cryosparc_worker/cryosparc_compute/engine/cuda_core.py", line 416, in cryosparc_compute.engine.cuda_core.context_dependent_memoize.wrapper
  File "cryosparc_worker/cryosparc_compute/engine/newcuda_kernels.py", line 6036, in cryosparc_compute.engine.newcuda_kernels.get_edt_kernels
  File "/scicore/home/engel0006/GROUP/pool-engel/soft/cryosparc/cryosparc_worker/deps/anaconda/envs/cryosparc_worker_env/lib/python3.7/site-packages/pycuda/compiler.py", line 291, in __init__
    arch, code, cache_dir, include_dirs)
  File "/scicore/home/engel0006/GROUP/pool-engel/soft/cryosparc/cryosparc_worker/deps/anaconda/envs/cryosparc_worker_env/lib/python3.7/site-packages/pycuda/compiler.py", line 254, in compile
    return compile_plain(source, options, keep, nvcc, cache_dir, target)
  File "/scicore/home/engel0006/GROUP/pool-engel/soft/cryosparc/cryosparc_worker/deps/anaconda/envs/cryosparc_worker_env/lib/python3.7/site-packages/pycuda/compiler.py", line 137, in compile_plain
    stderr=stderr.decode("utf-8", "replace"))
pycuda.driver.CompileError: nvcc compilation of /scratch/tmp0x9q70ry/kernel.cu failed
[command: nvcc --cubin -arch sm_86 -I/scicore/home/engel0006/GROUP/pool-engel/soft/cryosparc/cryosparc_worker/deps/anaconda/envs/cryosparc_worker_env/lib/python3.7/site-packages/pycuda/cuda kernel.cu]
[stderr:
cicc: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.21' not found (required by /scicore/home/engel0006/GROUP/pool-engel/soft/cryosparc/cryosparc_worker/deps/anaconda/envs/cryosparc_worker_env/lib/./libLerc.so)
cicc: /lib64/libstdc++.so.6: version `CXXABI_1.3.9' not found (required by /scicore/home/engel0006/GROUP/pool-engel/soft/cryosparc/cryosparc_worker/deps/anaconda/envs/cryosparc_worker_env/lib/./libLerc.so)
]

Everything used to work fine before the update. Is there any fix for this?

Thank you!

Info about master node:

Linux worker08.cluster.bc2.ch 3.10.0-1160.66.1.el7.x86_64 #1 SMP Wed May 18 16:02:34 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
              total        used        free      shared  buff/cache   available
Mem:           2015          76          56           0        1882        1936
Swap:             7           0           7

Info about worker node (same as master):

diogori@worker08:cryosparc_worker$ eval $(bin/cryosparcw env)
diogori@worker08:cryosparc_worker$ echo $CRYOSPARC_CUDA_PATH
/scicore/soft/apps/CUDA/11.3.1
diogori@worker08:cryosparc_worker$ ${CRYOSPARC_CUDA_PATH}/bin/nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2021 NVIDIA Corporation
Built on Mon_May__3_19:15:13_PDT_2021
Cuda compilation tools, release 11.3, V11.3.109
Build cuda_11.3.r11.3/compiler.29920130_0
diogori@worker08:cryosparc_worker$ python -c "import pycuda.driver; print(pycuda.driver.get_version())"
(10, 1, 0)
diogori@worker08:cryosparc_worker$ uname -a && free -g && nvidia-smi
Linux worker08.cluster.bc2.ch 3.10.0-1160.66.1.el7.x86_64 #1 SMP Wed May 18 16:02:34 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
              total        used        free      shared  buff/cache   available
Mem:           2015          76          56           0        1882        1936
Swap:             7           0           7
Sat Oct  8 13:16:20 2022       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 515.43.04    Driver Version: 515.43.04    CUDA Version: 11.7     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA A40          On   | 00000000:23:00.0 Off |                    0 |
|  0%   34C    P0    35W / 300W |    743MiB / 46068MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
|   1  NVIDIA A40          On   | 00000000:41:00.0 Off |                    0 |
|  0%   34C    P0    32W / 300W |     22MiB / 46068MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
|   2  NVIDIA A40          On   | 00000000:A1:00.0 Off |                    0 |
|  0%   35C    P8    33W / 300W |     22MiB / 46068MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
|   3  NVIDIA A40          On   | 00000000:C1:00.0 Off |                    0 |
|  0%   33C    P8    31W / 300W |     22MiB / 46068MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|    0   N/A  N/A      4321      G   /usr/bin/X                         23MiB |
|    0   N/A  N/A     29801      G   .../napari/napari/bin/python        4MiB |
|    0   N/A  N/A     31419      G   ...himera-1.16/bin/python2.7       70MiB |
|    0   N/A  N/A     36338      G   ...himera-1.16/bin/python2.7      122MiB |
|    0   N/A  N/A     92685      G   ...himera-1.16/bin/python2.7      520MiB |
|    1   N/A  N/A      4321      G   /usr/bin/X                         22MiB |
|    2   N/A  N/A      4321      G   /usr/bin/X                         22MiB |
|    3   N/A  N/A      4321      G   /usr/bin/X                         22MiB |
+-----------------------------------------------------------------------------+

same problem for me with NU-refine

I think we have a related issue. Everything was working fine on v4.01 initially (jobs running etc), but after restarting the system cryosparc will not start the app (app: ERROR (spawn error)), and when I look in the app log I see the same missing libraries referenced above:

2022-10-08 14:56:12,849 INFO | Listening: http://localhost:39000
/home/exx/cryosparc/cryosparc_master/cryosparc_app/custom-server/nodejs/bin/node: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.21' not found (required by /home/exx/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/./libLerc.so)
/home/exx/cryosparc/cryosparc_master/cryosparc_app/custom-server/nodejs/bin/node: /lib64/libstdc++.so.6: version `CXXABI_1.3.9' not found (required by /home/exx/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/./libLerc.so)
/home/exx/cryosparc/cryosparc_master/cryosparc_app/custom-server/nodejs/bin/node: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.21' not found (required by /home/exx/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/./libLerc.so)
/home/exx/cryosparc/cryosparc_master/cryosparc_app/custom-server/nodejs/bin/node: /lib64/libstdc++.so.6: version `CXXABI_1.3.9' not found (required by /home/exx/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/./libLerc.so)
/home/exx/cryosparc/cryosparc_master/cryosparc_app/custom-server/nodejs/bin/node: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.21' not found (required by /home/exx/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/./libLerc.so)
/home/exx/cryosparc/cryosparc_master/cryosparc_app/custom-server/nodejs/bin/node: /lib64/libstdc++.so.6: version `CXXABI_1.3.9' not found (required by /home/exx/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/./libLerc.so)
/home/exx/cryosparc/cryosparc_master/cryosparc_app/custom-server/nodejs/bin/node: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.21' not found (required by /home/exx/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/./libLerc.so)
/home/exx/cryosparc/cryosparc_master/cryosparc_app/custom-server/nodejs/bin/node: /lib64/libstdc++.so.6: version `CXXABI_1.3.9' not found (required by /home/exx/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/./libLerc.so)
/home/exx/cryosparc/cryosparc_master/cryosparc_app/custom-server/nodejs/bin/node: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.21' not found (required by /home/exx/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/./libLerc.so)
/home/exx/cryosparc/cryosparc_master/cryosparc_app/custom-server/nodejs/bin/node: /lib64/libstdc++.so.6: version `CXXABI_1.3.9' not found (required by /home/exx/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/./libLerc.so)
/home/exx/cryosparc/cryosparc_master/cryosparc_app/custom-server/nodejs/bin/node: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.21' not found (required by /home/exx/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/./libLerc.so)
/home/exx/cryosparc/cryosparc_master/cryosparc_app/custom-server/nodejs/bin/node: /lib64/libstdc++.so.6: version `CXXABI_1.3.9' not found (required by /home/exx/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/./libLerc.so)
/home/exx/cryosparc/cryosparc_master/cryosparc_app/custom-server/nodejs/bin/node: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.21' not found (required by /home/exx/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/./libLerc.so)
/home/exx/cryosparc/cryosparc_master/cryosparc_app/custom-server/nodejs/bin/node: /lib64/libstdc++.so.6: version `CXXABI_1.3.9' not found (required by /home/exx/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/./libLerc.so)
/home/exx/cryosparc/cryosparc_master/cryosparc_app/custom-server/nodejs/bin/node: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.21' not found (required by /home/exx/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/./libLerc.so)
/home/exx/cryosparc/cryosparc_master/cryosparc_app/custom-server/nodejs/bin/node: /lib64/libstdc++.so.6: version `CXXABI_1.3.9' not found (required by /home/exx/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/./libLerc.so)
/home/exx/cryosparc/cryosparc_master/cryosparc_app/custom-server/nodejs/bin/node: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.21' not found (required by /home/exx/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/./libLerc.so)
/home/exx/cryosparc/cryosparc_master/cryosparc_app/custom-server/nodejs/bin/node: /lib64/libstdc++.so.6: version `CXXABI_1.3.9' not found (required by /home/exx/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/./libLerc.so)
/home/exx/cryosparc/cryosparc_master/cryosparc_app/custom-server/nodejs/bin/node: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.21' not found (required by /home/exx/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/./libLerc.so)
/home/exx/cryosparc/cryosparc_master/cryosparc_app/custom-server/nodejs/bin/node: /lib64/libstdc++.so.6: version `CXXABI_1.3.9' not found (required by /home/exx/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/./libLerc.so)
/home/exx/cryosparc/cryosparc_master/cryosparc_app/custom-server/nodejs/bin/node: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.21' not found (required by /home/exx/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/./libLerc.so)
/home/exx/cryosparc/cryosparc_master/cryosparc_app/custom-server/nodejs/bin/node: /lib64/libstdc++.so.6: version `CXXABI_1.3.9' not found (required by /home/exx/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/./libLerc.so)
/home/exx/cryosparc/cryosparc_master/cryosparc_app/custom-server/nodejs/bin/node: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.21' not found (required by /home/exx/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/./libLerc.so)
/home/exx/cryosparc/cryosparc_master/cryosparc_app/custom-server/nodejs/bin/node: /lib64/libstdc++.so.6: version `CXXABI_1.3.9' not found (required by /home/exx/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/./libLerc.so)
/home/exx/cryosparc/cryosparc_master/cryosparc_app/custom-server/nodejs/bin/node: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.21' not found (required by /home/exx/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/./libLerc.so)
/home/exx/cryosparc/cryosparc_master/cryosparc_app/custom-server/nodejs/bin/node: /lib64/libstdc++.so.6: version `CXXABI_1.3.9' not found (required by /home/exx/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/./libLerc.so)
/home/exx/cryosparc/cryosparc_master/cryosparc_app/custom-server/nodejs/bin/node: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.21' not found (required by /home/exx/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/./libLerc.so)
/home/exx/cryosparc/cryosparc_master/cryosparc_app/custom-server/nodejs/bin/node: /lib64/libstdc++.so.6: version `CXXABI_1.3.9' not found (required by /home/exx/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/./libLerc.so)
/home/exx/cryosparc/cryosparc_master/cryosparc_app/custom-server/nodejs/bin/node: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.21' not found (required by /home/exx/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/./libLerc.so)
/home/exx/cryosparc/cryosparc_master/cryosparc_app/custom-server/nodejs/bin/node: /lib64/libstdc++.so.6: version `CXXABI_1.3.9' not found (required by /home/exx/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/./libLerc.so)
/home/exx/cryosparc/cryosparc_master/cryosparc_app/custom-server/nodejs/bin/node: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.21' not found (required by /home/exx/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/./libLerc.so)
/home/exx/cryosparc/cryosparc_master/cryosparc_app/custom-server/nodejs/bin/node: /lib64/libstdc++.so.6: version `CXXABI_1.3.9' not found (required by /home/exx/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/./libLerc.so)

@kookjookeem fixed our issue by symlinking the newer version of libstdc++.so.6 from conda to /lib64/ so csparc can find what it is looking for - maybe the same issue?

+1 same issue but during cryosparc live session

@rdrighetto @olibclarke @CryoEM1 @prash thank you for reporting.
We are looking into the cause of this issue. To help us investigate, can you please provide:

  1. Your OS and version (output of uname -a )
  2. The version of CUDA with which you had installed cryoSPARC v4.0.0 (i.e. before updating to v4.0.1) and the version of CUDA with which you installed v4.0.1 (if different)

While we’re working on solving this please downgrade to v4.0.0, or use the workaround posted above by @olibclarke if it’s not possible to downgrade.
Thanks!

Hi @apunjani,

  1. OS version was already in original post:
Linux worker08.cluster.bc2.ch 3.10.0-1160.66.1.el7.x86_64 #1 SMP Wed May 18 16:02:34 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

It is CentOS 7.9.

  1. Also the CUDA version (11.3):
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2021 NVIDIA Corporation
Built on Mon_May__3_19:15:13_PDT_2021
Cuda compilation tools, release 11.3, V11.3.109
Build cuda_11.3.r11.3/compiler.29920130_0

The CUDA version did not change between v4.0.0 and v4.0.1. I updated from v3.4.0 to v4.0.1 directly.

Thank you!

Hi @apunjani,

For our system:

  1. Linux c112384 3.10.0-1160.76.1.el7.x86_64 #1 SMP Wed Aug 10 16:21:17 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

  2. CUDA 11.2, driver version 460.67, not changed between 4.0 and 4.0.1.

Cheers
Oli

Please correct me if I am wrong but linking libstdc++.so.6 to the system /lib64 MIGHT make the system quite unstable. I think adding the libstdc++.so.6 from anaconda env into PRELOAD variable should provide a more viable solution:
In cryosparcw
[…]
Set shell variables. Clear out LD_PRELOAD so that if cryosparm or cryosparcw

are called with [mutual] recursion, they don’t interfere with each other

export PATH="$CRYOSPARC_PATH:$PATH"
export LD_PRELOAD=""
for lib in libstdc++.so libtiff.so libpython3.7m.so; do
lib_path="$CRYOSPARC_ROOT_DIR/deps/anaconda/envs/$CRYOSPARC_CONDA_ENV/lib/$lib"
[…]

JC

4 Likes

OS: Linux ultron.csb.vanderbilt.edu 3.10.0-1160.36.2.el7.x86_64 #1 SMP Wed Jul 21 11:57:15 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

CUDA: NVIDIA-SMI 470.57.02 Driver Version: 470.57.02 CUDA Version: 11.4

Thank you for helping

Prash

Thanks, JC! This fix works.

Can confirm that the fix proposed by @jcducom works! Thank you!

Hi @olibclarke, @prash, @CryoEM1, @rdrighetto,

How was the CUDA being used here installed? Was it installed on the same Linux version as the current worker? If not, which version was it?
Also, can you edit cryosparc_master/bin/cryosparcm and cryosparc_worker/bin/cryosparcw to not add libtiff to LD_PRELOAD:

export LD_PRELOAD=""
for lib in libpython3.7m.so ; do

Then restart CryoSPARC (cryosparcm restart) and try running a job again.

This likely started occurring due to the fix that was implemented for the problems I experienced here. It appears that LD_LIBRARY_PATH is now set to a blank value as of 4.0.1:

$ cryosparcm env
<snip>
export "LD_LIBRARY_PATH="
<snip>

Since the library path for the Anaconda install is no longer in LD_LIBRARY_PATH, only the system library paths are being searched now.

1 Like

Hi, I got the same errors for Local Refinement from v4.0.1. Had to downgrade to v3.4.0.
Here is my info:

  1. Linux sh02-15n13.int 3.10.0-1160.76.1.el7.x86_64 #1 SMP Wed Aug 10 16:21:17 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
  2. Cuda compilation tools, release 9.2, V9.2.148

When I downgraded to v4.0.0 the job completes.

Looking at the output of $ cryosparcw env between v4.0.0 and v4.0.1 the LD_LIBRARY_PATH is quite different.

v4.0.0:

$ cryosparcw env | grep LD_LIBRARY
export "LD_LIBRARY_PATH=/usr/local/cryosparc2/cryosparc2_worker/cryosparc_compute/blobio:/usr/local/cryosparc2/cryosparc2_worker/deps/anaconda/envs/cryosparc_worker_env/lib:/usr/local/cryosparc2/cryosparc2_worker/deps/external/cudnn/lib:/usr/local/cuda-11.2/lib64"

v4.0.1:

$ cryosparcw env | grep LD_LIBRARY
export "LD_LIBRARY_PATH=/usr/local/cryosparc2/cryosparc2_worker/deps/external/cudnn/lib:/lsi/local/packages/cuda/11.2/lib64"

Perhaps the changes to the cryosparcw script caused this?

$ diff cryosparcw_v4.0.0 cryosparcw_v4.0.1 
58,63c58
<         if conda activate $CRYOSPARC_CONDA_ENV ; then
<             # Use newest conda libraries instead of system-provided (e.g., libstdc++.so for Node.js)
<             export LD_LIBRARY_PATH="$CRYOSPARC_ROOT_DIR/deps/anaconda/envs/$CRYOSPARC_CONDA_ENV/lib${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}"
<         else
<             echo "Warning: Could not activate conda environment; this indicates that a cryoSPARC installation is either incomplete or in progress" 1>&2
<         fi
---
>         conda activate $CRYOSPARC_CONDA_ENV || echo "Warning: Could not activate conda environment; this indicates that a cryoSPARC installation is either incomplete or in progress" 1>&2
69d63
< 
113,118d106
< # Use to selectively load the libraries required for cryoSPARC's compute modules
< # which would otherwise interfere with unrelated processes
< activate_compute_libraries() {
<     export LD_LIBRARY_PATH="$CRYOSPARC_ROOT_DIR/cryosparc_compute/blobio:${LD_LIBRARY_PATH}"
< }
< 
143d130
<     activate_compute_libraries
162d148
<     activate_compute_libraries
169d154
<     activate_compute_libraries
293d277
<     activate_compute_libraries
310d293
<     activate_compute_libraries
409d391
<     activate_compute_libraries

I haven’t tested it yet but the v4.0.0’s LD_LIBRARY_PATH does have the libstdc++.so.6 shared library file in it.

$ ls /usr/local/cryosparc2/cryosparc2_worker/deps/anaconda/envs/cryosparc_worker_env/lib | grep libstdc++
libstdc++.so
libstdc++.so.6
libstdc++.so.6.0.30

Hope that helps someone.

Today I just updated the cryosparc from v3.4.0 to v4.0.1 again (previously had downgraded from v4.0.1 to v3.4.0 because of errors) and my Local Refinement job worked without any errors! Not sure if anything has been updated on the cryosparc side.

@rdrighetto @CryoEM1 @olibclarke @prash @jcducom @DanielAsarnow @bmdennis @donghuachen @clil16
The CryoSPARC v4.0.1+221017 patch includes a fix for this problem. Before installing the fix, please undo the links that you configured as a workaround.

2 Likes