Hi All,
I encountered several issues with Topaz after upgrading cryoSPARC to v5 (currently on v5.0.2). The first issue is that Topaz Cross Validation job requires an input for the Expected number of particles to run, even though this is exactly the parameter I am optimizing for the Topaz Cross Validation job and have provided an value for Initial value to begin with and Value to increment parameter by. Even if I started the job after providing an Expected number of particles, the job failed after finishing the eight corss-validation sub-jobs with the error message below:
Traceback (most recent call last):
File “/home/cryosparcuser/Applications/cryosparc/cryosparc_worker/.pixi/envs/worker/lib/python3.12/site-packages/pandas/core/groupby/groupby.py”, line 1943, in _agg_py_fallback
res_values = self._grouper.agg_series(ser, alt, preserve_dtype=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/home/cryosparcuser/Applications/cryosparc/cryosparc_worker/.pixi/envs/worker/lib/python3.12/site-packages/pandas/core/groupby/ops.py”, line 864, in agg_series
result = self._aggregate_series_pure_python(obj, func)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/home/cryosparcuser/Applications/cryosparc/cryosparc_worker/.pixi/envs/worker/lib/python3.12/site-packages/pandas/core/groupby/ops.py”, line 885, in _aggregate_series_pure_python
res = func(group)
^^^^^^^^^^^
File “/home/cryosparcuser/Applications/cryosparc/cryosparc_worker/.pixi/envs/worker/lib/python3.12/site-packages/pandas/core/groupby/groupby.py”, line 2460, in
alt=lambda x: Series(x, copy=False).mean(numeric_only=numeric_only),
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/home/cryosparcuser/Applications/cryosparc/cryosparc_worker/.pixi/envs/worker/lib/python3.12/site-packages/pandas/core/series.py”, line 6560, in mean
return NDFrame.mean(self, axis, skipna, numeric_only, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/home/cryosparcuser/Applications/cryosparc/cryosparc_worker/.pixi/envs/worker/lib/python3.12/site-packages/pandas/core/generic.py”, line 12439, in mean
return self._stat_function(
^^^^^^^^^^^^^^^^^^^^
File “/home/cryosparcuser/Applications/cryosparc/cryosparc_worker/.pixi/envs/worker/lib/python3.12/site-packages/pandas/core/generic.py”, line 12396, in _stat_function
return self._reduce(
^^^^^^^^^^^^^
File “/home/cryosparcuser/Applications/cryosparc/cryosparc_worker/.pixi/envs/worker/lib/python3.12/site-packages/pandas/core/series.py”, line 6468, in _reduce
return op(delegate, skipna=skipna, **kwds)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/home/cryosparcuser/Applications/cryosparc/cryosparc_worker/.pixi/envs/worker/lib/python3.12/site-packages/pandas/core/nanops.py”, line 147, in f
result = alt(values, axis=axis, skipna=skipna, **kwds)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/home/cryosparcuser/Applications/cryosparc/cryosparc_worker/.pixi/envs/worker/lib/python3.12/site-packages/pandas/core/nanops.py”, line 404, in new_func
result = func(values, axis=axis, skipna=skipna, mask=mask, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/home/cryosparcuser/Applications/cryosparc/cryosparc_worker/.pixi/envs/worker/lib/python3.12/site-packages/pandas/core/nanops.py”, line 720, in nanmean
the_sum = _ensure_numeric(the_sum)
^^^^^^^^^^^^^^^^^^^^^^^^
File “/home/cryosparcuser/Applications/cryosparc/cryosparc_worker/.pixi/envs/worker/lib/python3.12/site-packages/pandas/core/nanops.py”, line 1701, in _ensure_numeric
raise TypeError(f"Could not convert string ‘{x}’ to numeric")
TypeError: Could not convert string ‘–’ to numeric
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File “cli/run.py”, line 106, in cli.run.run_job
File “cli/run.py”, line 211, in cli.run.run_job_function
File “/home/cryosparcuser/Applications/cryosparc/cryosparc_worker/compute/jobs/topaz/run_topaz.py”, line 1152, in run_topaz_wrapper_cross_validation
cv_results = cv_results.drop(“split”, axis=1).groupby([“N”, “epoch”]).mean().reset_index().drop(“fold”, axis=1)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/home/cryosparcuser/Applications/cryosparc/cryosparc_worker/.pixi/envs/worker/lib/python3.12/site-packages/pandas/core/groupby/groupby.py”, line 2458, in mean
result = self._cython_agg_general(
^^^^^^^^^^^^^^^^^^^^^^^^^
File “/home/cryosparcuser/Applications/cryosparc/cryosparc_worker/.pixi/envs/worker/lib/python3.12/site-packages/pandas/core/groupby/groupby.py”, line 2004, in _cython_agg_general
new_mgr = data.grouped_reduce(array_func)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/home/cryosparcuser/Applications/cryosparc/cryosparc_worker/.pixi/envs/worker/lib/python3.12/site-packages/pandas/core/internals/managers.py”, line 1469, in grouped_reduce
applied = sb.apply(func)
^^^^^^^^^^^^^^
File “/home/cryosparcuser/Applications/cryosparc/cryosparc_worker/.pixi/envs/worker/lib/python3.12/site-packages/pandas/core/internals/blocks.py”, line 395, in apply
result = func(self.values, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/home/cryosparcuser/Applications/cryosparc/cryosparc_worker/.pixi/envs/worker/lib/python3.12/site-packages/pandas/core/groupby/groupby.py”, line 2001, in array_func
result = self._agg_py_fallback(how, values, ndim=data.ndim, alt=alt)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/home/cryosparcuser/Applications/cryosparc/cryosparc_worker/.pixi/envs/worker/lib/python3.12/site-packages/pandas/core/groupby/groupby.py”, line 1947, in _agg_py_fallback
raise type(err)(msg) from err
TypeError: agg function failed [how->mean,dtype->object]
Because of the issues with Topaz Cross Validation, I had to use a simple Topaz Train job instead, which was finished successfully. But I encountered another issue with Topaz Extract. The job always fails unless the Absolute path of directory containing preprocessed directory contains the preprocessed images for all the Input Microgrpahs. On v4.7.1, the Topaz Extract job would first preprocess micrographs not found in the indicated directory, which should be the expected behavior.
I am not sure if these issues are due to incompatibility of Topaz v0.2.5a with cryoSPARC v5.0.2. If that is indeed the case, could you please provide some guidance to install Topaz v0.3.0 (or later version) for cryoSPARC v5? The current instructions for installing Topaz v0.2.5a results in errors when trying to install Topaz v0.3.0 and later.
Thanks!