Hi,
we are using RTX 2070 super cards (8 GB RAM) for cryoEM reconstructions in Cryosparc.
When we run the DeepEMhancer outside of Cryosparc we get this type of error:
(1) Unknown: Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above.
[[{{node conv3d_1/convolution}}]]
0 successful operations.
0 derived errors ignored.
which can be dealt with (as recommended on github: GitHub - rsanchezgarc/deepEMhancer: Deep learning for cryo-EM maps post-processing), by setting this value TF_FORCE_GPU_ALLOW_GROWTH=‘true’
prior to invoking deepemhancer command with all its options:
TF_FORCE_GPU_ALLOW_GROWTH=‘true’ deepemhancer -p highRes
In Cryosparc DeepEMhancer wrapper however, at this moment we cannot see a way to use this option, so instead, we modified the run.py script in the cryosparc_worker installation.
In our case this script is found in the following path:
…/cryosparcinstall/cryosparc2_worker/cryosparc_compute/jobs/deepemhancer/run.py
we added the following line
os.environ[‘TF_FORCE_GPU_ALLOW_GROWTH’] = ‘true’
in the following context:
import subprocess
os.environ[‘TF_FORCE_GPU_ALLOW_GROWTH’] = ‘true’
def run_deepemhancer_wrapper(job):
The GPU error does not appear anymore. It may be wiped be a new CS update though.
I hope this could be useful to someone.
All the best
Lukasz K