Topaz in CryoSPARC and Nvidia 5090

Particle Picking
1

Dear CryoSPARC staffs,

When I use Topaz train in CryoSPARC in recent days, many warnings and troubles were shown. Could you please help to solve the problem?

\[CPU:  266.1 MB  Avail: 478.79 GB\]

return self.\_conv_forward(input, self.weight, self.bias)

\[CPU:  266.1 MB  Avail: 478.82 GB\]

File "/home/kemove/miniconda3/envs/topaz/lib/python3.6/site-packages/torch/nn/modules/conv.py", line 443, in \_conv_forward

\[CPU:  266.1 MB  Avail: 478.85 GB\]

self.padding, self.dilation, self.groups)

\[CPU:  266.1 MB  Avail: 478.88 GB\]

RuntimeError: Unable to find a valid cuDNN algorithm to run convolution

\[CPU:  266.1 MB  Avail: 485.09 GB\]

**Traceback (most recent call last):   File "cryosparc_master/cryosparc_compute/run.py", line 129, in cryosparc_master.cryosparc_compute.run.main   File "/home/kemove/software/cryosparc_worker/cryosparc_compute/jobs/topaz/run_topaz.py", line 384, in run_topaz_wrapper_train     utils.run_process(train_command)   File "/home/kemove/software/cryosparc_worker/cryosparc_compute/jobs/topaz/topaz_utils.py", line 99, in run_process     assert process.returncode == 0, f"Subprocess exited with status {process.returncode} ({str_command})" AssertionError: Subprocess exited with status 1 (/home/kemove/miniconda3/envs/topaz/bin/topaz train --train-images /mnt/ab68010c-5ccf-4cf5-8d83-dcd1d6c18465/liuxb/K160_3_results/CS-k160-3/J69/image_list_train.txt --train-targets /mnt/ab68010c-5ccf-4cf5-8d83-dcd1d6c18465/liuxb/K160_3_results/CS-k160-3/J69…)**

this is the first problem.

\[CPU:  264.7 MB  Avail: 483.36 GB\]

TypeError: concat() takes 1 positional argument but 2 were given

\[CPU:  264.7 MB  Avail: 483.43 GB\]

**Traceback (most recent call last):   File "cryosparc_master/cryosparc_compute/run.py", line 129, in cryosparc_master.cryosparc_compute.run.main   File "/home/kemove/software/cryosparc_worker/cryosparc_compute/jobs/topaz/run_topaz.py", line 332, in run_topaz_wrapper_train     utils.run_process(split_command)   File "/home/kemove/software/cryosparc_worker/cryosparc_compute/jobs/topaz/topaz_utils.py", line 99, in run_process     assert process.returncode == 0, f"Subprocess exited with status {process.returncode} ({str_command})" AssertionError: Subprocess exited with status 1 (/home/kemove/miniconda3/bin/topaz train_test_split --number 1608 --seed 1679832333 --image-dir /mnt/ab68010c-5ccf-4cf5-8d83-dcd1d6c18465/liuxb/K160_3_results/CS-k160-3/J70/preprocessed /mnt/ab68010c-5ccf-4cf5-8d83-dcd1d6c18465/liuxb/K160_3_results/CS-k160-…)**

this is the second.

\[CPU:  260.1 MB  Avail: 485.94 GB\]

# WARNING: no coordinates are observed with x_coord > 100 or y_coord > 103 .                     Did you scale the micrographs and particle coordinates correctly?

this is the third.

# WARNING: \~0 particles are out of bounds for image /mnt/ab68010c-5ccf-4cf5-8d83-dcd1d6c18465/liuxb/K160_3_results/CS-k160-3/J72/preprocessed/006742460034114535093_FoilHole_28788073_Data_28782394_28782396_20250829_094330_EER_patch_aligned_doseweighted.mrc. Did you scale the micrographs and particle coordinates correctly?

this is the fourth.

Could you please tell me how to run the Topaz by using Nvidia 5090?

2

Welcome to the forum @Xiaobo. Due to a complex network of version constraints between dependencies of CryoSPARC versions ≤ 4.7.1 and topaz, the CryoSPARC wrapper for topaz does not currently work with the rtx5090 GPU.

3

This work for me with the old wrapper.
conda install -y -c tbepler -c conda-forge -c pytorch
topaz=0.2.5
numpy=1.26.4
scipy=1.11.4
pandas=1.5.3
scikit-learn=1.3.2
h5py=3.11.0
pillow=10.4.0
tqdm=4.66.5
future=0.18.3
joblib=1.3.2
python-dateutil=2.8.2
pytz=2024.1

pip install “torch==2.7.1” “torchvision==0.22.1” “torchaudio==2.7.1”
–index-url https://download.pytorch.org/whl/cu128

4

update on last reply
Wrapper no changed

For topaz extract

anaconda3/envs/topaz/lib/python3.11/site-packages/topaz/model/classifier.py

I add this at the END

— Patch for PyTorch >= 2.6 safe serialization —

try:
from torch.serialization import add_safe_globals
except (ImportError, AttributeError):
add_safe_globals = None

if add_safe_globals is not None:

Allow safe deserialization of LinearClassifier and common torch.nn modules

from torch.nn.modules.container import Sequential
from torch.nn.modules.conv import Conv2d, ConvTranspose2d
from torch.nn.modules.batchnorm import BatchNorm2d
from torch.nn.modules.pooling import MaxPool2d, AvgPool2d
from torch.nn.modules.dropout import Dropout
from torch.nn.modules.activation import ReLU

add_safe_globals([
    LinearClassifier,
    Sequential,
    Conv2d,
    ConvTranspose2d,
    BatchNorm2d,
    MaxPool2d,
    AvgPool2d,
    Dropout,
    ReLU,
])

— End patch —

anaconda3/envs/topaz/lib/python3.11/site-packages/topaz/model/features/resnet.py

I add this at the end

— Patch for PyTorch >= 2.6 safe serialization —

try:
from torch.serialization import add_safe_globals
except (ImportError, AttributeError):
add_safe_globals = None

if add_safe_globals is not None:

Allow safe deserialization of ResNet variants and related building blocks

globals_to_allow =
for cls_name in [
“ResNet”, “ResNet6”, “ResNet8”, “ResNet16”,
“MaxPool”, “BasicConv2d”, “ResidA”
]:
cls = globals().get(cls_name)
if cls is not None:
globals_to_allow.append(cls)
if globals_to_allow:
add_safe_globals(globals_to_allow)

— End patch —