Hey
I was trying to processing some data and didn't realize the disk is running out of space. and of course the job gives error message.
After the error message, i found out that i cannot connect to the local cryosparc interface (localhost:39000). (it gives message like error connection).
Just want to ask if anyone ran into similar problems as this one before and if so, how to deal with it.. Also, just wanna ask what will happen if, in the middle of running, the space runs out, will the GPU automatically stop? can I continue to run from where I stopped if I got more space?
This is normal when the disk fills up.
You need to make some space on the disk, stop cryosparc on the command line (cryosparcm stop), check for any zombie cryosparc processes that may be running and kill them manually.
At this point, when there are no CS processes running, you should check in the /tmp directory for mongodb .lock files - there should not be any there if the database is not running, if there is one there, delete it, as otherwise you can run into database errors on restart.
After all that, you can then restart cryosparc (cryosparcm start). Good luck!
Thanks!
when I entered the cryosparcm stop commend line. it outputs the following
unix:///tmp/cryosparc-supervisor-b7b467717d546431de58b1020117c4c0.sock refused connection
and I checked the list of the /tmp directory,
cryosparc-supervisor-b7b467717d546431de58b1020117c4c0.sock
d71799fe-8bed-4420-a90e-d37871e21df2.tmp
file.tiff
hsperfdata_root
lua_jQ0PI7
mongodb-39001.sock
qtsingleapp-Aspera-3b57-3e8
qtsingleapp-Aspera-3b57-3e8-lockfile
qtsingleapp-Aspera-459b-3e8
qtsingleapp-Aspera-459b-3e8-lockfile
systemd-private-c585698c5b7c4781968d08e879b1ae26-colord.service-ij5fc1
systemd-private-c585698c5b7c4781968d08e879b1ae26-fwupd.service-UTe5uw
systemd-private-c585698c5b7c4781968d08e879b1ae26-geoclue.service-01yUfY
systemd-private-c585698c5b7c4781968d08e879b1ae26-ModemManager.service-DAOMG6
systemd-private-c585698c5b7c4781968d08e879b1ae26-rtkit-daemon.service-hXoNXI
ToDesk_Service_HUIfj32hjjgq.lock
I am not sure whether any of those are zombie processes I should delete…
and after cryosparcm start commend, it still cannot connect to the interface…
is there any suggestions?
I would check if any cryosparc processes are still running, using ps -x | grep cryosparc. If there are, kill them. If, after killing them, you still have the cryosparc & mongodb sock files in your /tmp directory, delete those then run cryosparcm start.
(but before doing any of this you need to make some space on the disk! Otherwise it definitely won’t start)
Thanks for your quick reply!
the output of the ps -x | grep cryosparc is the following
12483 pts/0 S+ 0:00 grep --color=auto cryosparc
i am not sure what it means. also, could you tell me the commend line for killing the cryosparc process? thanks very much!
This indicates that no cryosparc processes are running - it is just picking up the grep command you used to identify them. I would say you can safely delete the two sock files in /tmp/, then start cryosparc.
thanks!
i successfully accessed to the interface. you are a genius.
1 Like
No worries glad you got it sorted!
It would be great to have some kind of automatic buffer built in to avoid filling up the disk, we have run into this in the past ourselves and it is always a bit of a pain (and if you try restarting CS without killing rogue processes or deleting the sock files you end up with scary-looking database errors…)