I’ve been running CS Live v2.15.0 for a while (haven’t gotten a chance to update yet) but all of a sudden the GPUs on the worker node have become unavailable. The RTP_worker jobs start but nothing is logged in the job log or running on the worker node.
I also checked command_core log and not information about it there.
The gpus show up correctly in nvidia-smi and can be used in other programs without issue.
Ive reloaded the nvidia drivers and unloaded and reloaded the worker nodes to the master node.