Just updated to the latest version 4.1.2. Both topaz and deepemchaner do not work. See below for an example of an error. Let me know if you need any additional information.
#######################
Unable to queue P78 J268: ServerError: Failed to enqueue P78 J268 (topaz_train): Job has builder errors. Path to Topaz executable: Invalid path specified: /home/cryosparc/bin/topaz.sh directory allowed: False; file allowed: True; glob allowed: False
Pease can you provide additional details:
ps xuww | grep supervisor
stat /home/cryosparc/bin/topaz.sh
@wtempel Sorry. I just realized it is not due to cryosparc version issue. But rather due to this project was imported from another machine and it seems to have been set to “/home/cryosparc/bin/topaz.sh” for topaz script, which is different from the current machine. Despite I changed the path in the job, it still go for this project-level parameter. I tried to remove this setting from the project-level parameters tab. But it does not help.
Were you able to disable the project-level setting?
What is the error you are seeing now?
Were you able to set a job-level path to the topaz script?
What is the output of
ps xuww | grep supervisor
stat /path/to/topaz-script # job-level definition
?
Hi, we also have this issue with DeepEMhancer in v4.1.2. Trying to queue the job led to the similar error message (I am curious why the error message complains about the topaz.sh script while I was trying to run a DeepEMhancer job):
Unable to queue P7 J182: ServerError: Failed to enqueue P7 J182 (deepemhancer): Job has builder errors. Path to deepEMhancer executable: Invalid path specified: /home/yangyang/Applications/cryoem_tools/topaz.sh directory allowed: False; file allowed: True; glob allowed: False
However, DeepEMhancer ran successfully in another Project that does not have project-level setting for Topaz and DeepEMhancer. In addition, resetting the project-level setting fort the problematic project did not solve the issue.
Here is the output of ps xuww | grep supervisor:
yangyang 718771 0.0 0.0 50096 10112 ? Ss Feb21 0:25 python /home/yangyang/Applications/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/bin/supervisord -c /home/yangyang/Applications/cryosparc/cryosparc_master/supervisord.conf
yangyang 1796278 0.0 0.0 17868 2424 pts/1 S+ 04:57 0:00 grep --color=auto supervisor
Output of stat /home/yangyang/Applications/cryoEM_tools/deepemhancer.sh:
File: /home/yangyang/Applications/cryoEM_tools/deepemhancer.sh
Size: 521 Blocks: 8 IO Block: 4096 regular file
Device: 10305h/66309d Inode: 44818688 Links: 1
Access: (0775/-rwxrwxr-x) Uid: ( 1000/yangyang) Gid: ( 1000/yangyang)
Access: 2023-02-22 00:57:48.104234973 -0600
Modify: 2023-02-18 00:17:43.762822316 -0600
Change: 2023-02-18 00:28:17.471878237 -0600
Birth: 2023-02-18 00:16:05.996473718 -0600
@YYang Updating your instance to CryoSPARC v4.2 should resolve this issue. For information on v4.2 and the subsequent 230302 patch, please see the release notes.
Please can you post the
Please can you post the output of these commands
cryosparcm cli "get_job('P6', 'J34', 'params_spec')"
stat /programs/x86_64-linux/system/sbgrid_bin/topaz
/full/path/to/cryosparc_worker/bin/cryosparcw call /programs/x86_64-linux/system/sbgrid_bin/topaz --versioncryosparcm cli “get_job(‘P6’, ‘J34’, ‘params_spec’)”
_id : 655f6eaesf534434’, ‘project_uid’ : ‘P6’, ‘uid’: 'J34,}
/full/path/to/cryosparc_worker/bin/cryosparcw call /programs/x86_64-linux/system/sbgrid_bin/topaz --versionCryoSPARC System master node installed at
/home/cryosparc/cryosparc/cryosparc_master
Current cryoSPARC version: v4.4.0+231114
I am connected using ssh on a machine on same network for remote execution of commands.
@vijayverma Unfortunately, I have not tested this kind of setup and would need to see the output of the remaining commands
to provide additional suggestions.
Alternatively, you might:
topaz.sh.@wtempel , I did these command you mentioned in linux that is on master node.
cryosparcm cli "get_job('P6', 'J34', 'params_spec')"
stat /programs/x86_64-linux/system/sbgrid_bin/topaz
/full/path/to/cryosparc_worker/bin/cryosparcw call /programs/x86_64-linux/system/sbgrid_bin/topaz --version
for the command/full/path/to/cryosparc_worker/bin/cryosparcw call /programs/x86_64-linux/system/sbgrid_bin/topaz --version
I do not get worker path- [cryosparc@hermes ~]$ which cryosparcw
/usr/bin/which: no cryosparcw in (/programs/x86_64-linux/system/sbgrid_bin:/usr/local/cuda/bin:/home/cryosparc/cryosparc/cryosparc_master/bin:/home/cryosparc/.local/bin:/home/cryosparc/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/programs/share/bin:/programs/share/sbgrid/bin:/programs/x86_64-linux/sbgrid_installer/latest)
cryosparcm cli “get_job(‘P6’, ‘J34’, ‘params_spec’)”
_id : 655f6eaesf534434’, ‘project_uid’ : ‘P6’, ‘uid’: 'J34,}/full/path/to/cryosparc_worker/bin/cryosparcw call /programs/x86_64-linux/system/sbgrid_bin/topaz --version
I do not get worker path-
[cryosparc@hermes ~]$ which cryosparcw
/usr/bin/which: no cryosparcw in (/programs/x86_64-linux/system/sbgrid_bin:/usr/local/cuda/bin:/home/cryosparc/cryosparc/cryosparc_master/bin:/home/cryosparc/.local/bin:/home/cryosparc/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/programs/share/bin:/programs/share/sbgrid/bin:/programs/x86_64-linux/sbgrid_installer/latest)
Hope this is more clear now.
Could it be
/home/cryosparc/cryosparc/cryosparc_worker/bin/cryosparcw call /programs/x86_64-linux/system/sbgrid_bin/topaz --version
?
This is just a guess based on the path to your cryosparc_master/ directory.
@wtempel , Thank You , I will try this
/home/cryosparc/cryosparc/cryosparc_worker/bin/cryosparcw call /programs/x86_64-linux/system/sbgrid_bin/topaz –version
should I write the version of cryosparc v4.4.0+231114
and this command from master node or remote node.
I assumed that your CryoSPARC installation is a single-node, combined master/worker instance, but I failed to confirm this assumption.
If you know that this CryoSPARC installation is a single-node, combined master/worker instance, please run the command
/home/cryosparc/cryosparc/cryosparc_worker/bin/cryosparcw call /programs/x86_64-linux/system/sbgrid_bin/topaz --version
on the CryoSPARC master node without specifying a version, and post here the output of that command.
If you are not sure that this CryoSPARC installation is a single-node, combined master/worker instance, please run the commands, also on the master node
cryosparcm status | grep HOSTNAME
cryosparcm cli "get_scheduler_targets()"
**@wtempel **
/home/cryosparc/cryosparc/cryosparc_worker//cryosparcw call /programs/x86_64-linux/system/sbgrid_bin/topaz --version
/home/cryosparc/cryosparc/cryosparc_worker/bincryosparcw: line 272
Usage:Topaz {-h} {version} …
Topaz: error: the following arguments are required:
cryosparcm status | grep HOSTNAME
/home/cryosparc/cryosparc/cryosparc_master
export CRYOSPARC_DB_PATH="/home/cryosparc/cryosparc/cryosparc_database
cryosparcm cli “get_scheduler_targets()”
