Heartbeat issue-can't send heartbeat

Troubleshooting
1

Hello,

I’m currently using version v4.7.0.
Recently, I’ve been experiencing an issue where jobs fail after a certain period of time, regardless of the job type.
I tried increasing the heartbeat timeout to 4 hours, but the jobs still fail exactly at the 4-hour mark.
I checked the logs and noticed that no heartbeat signals are being sent at all.
What should I do in this case?

image
image1134×801 17 KB

Sincerely,
Junhong Kim

2

@JHKIM Please can you post:

  1. for a job that has not been restarted after failing in this way, the project and job UIDs P* and J*, respectively
  2. the output of the command
    `cryosparcm cli “get_scheduler_targets()”.

Please post the information as text.