Update went well without any error but no projects or workspaces are showing up. Also seems after i clicked on projects tab once everything else is also not loading anymore. app log showing something about ram
Get sessions error: Executor error during find command :: caused by :: errmsg: "Sort operation used more than the maximum 33554432 bytes of RAM. Add an index, or specify a smaller limit." T5 [MongoServerError]: Executor error during find command :: caused by :: errmsg: "Sort operation used more than the maximum 33554432 bytes of RAM.
Add an index, or specify a smaller limit."
cryosparc test showing an error, though i think not related
cryosparcuser@kermit103:/home/ansible$ cryosparcm test i
Running installation tests...
✓ Running as cryoSPARC owner
✓ Running on master node
✓ CryoSPARC is running
✓ Connected to command_core at http://kermit103:39002
✓ CRYOSPARC_LICENSE_ID environment variable is set
✓ License has correct format
✓ Insecure mode is disabled
✓ License server set to "https://get.cryosparc.com"
✓ Connection to license server succeeded
✓ License server returned success status code 200
✓ License server returned valid JSON response
✓ License exists and is valid
✓ CryoSPARC is running v4.0.0
✓ Running the latest version of cryoSPARC
Could not get latest patch (status code 404)
✓ Patch update not required
✓ Admin user has been created
Traceback (most recent call last):
File "/home/cryosparcuser/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/home/cryosparcuser/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/home/cryosparcuser/cryosparc_master/cryosparc_command/instance_tests/installation_test.py", line 81, in <module>
run_cryosparc_installation_tests()
File "/home/cryosparcuser/cryosparc_master/cryosparc_command/instance_tests/installation_test.py", line 71, in run_cryosparc_installation_tests
if len(target['resource_slots']['GPU']) > 0:
KeyError: 'resource_slots'
We’re looking into the error using the report logs sent via email. I just wanted to ask you how many projects, workspaces/sessions, and users you currently have in the instance? This appears to be a request failing do to size constraints.
Just around 10-15 users not totally sure and 69 projects.
As i update i just noticed as i wanted to look project numbers up, i started cryosparc and still had the projects overview html in browser and it did load correctly, though switching to another field like sessions didnt load and then back to projects it didnt load anymore also.
So it seems only the first asked page is laoded and then it hangs.
I got the same Keyerror: ‘resource_slots’ after upgrading to v4.0. Also, I am not able to see anything in the central window, which actually appears as a blank screen, and the Job builder shows up as a blank tab as well.
This happened when I made the regular cluster installation, which worked fine in v3.4
FYI not sure if related, there is no functionality of the GUI with old Safari browser (13). I had to update mac beyond Catalina now to either Big Sur or Monterey.
First i just noticed after doing the commands that P1 was deleted before and doesnt exist:D
Doesnt now either but the rest is showing up now, though it is still not normal, but i found some more detail.
Its somehow linked to the home page tab.
When i load on the project view directly i can switch between it and session view and manager view and instance view and others without problem, BUT as soon as i load home once it gets displayed also correctly, but then switching to any other page, it is not loading correctly. also normal browser refresh is not working, i have to click refresh then cancel the refresh and then refresh again then its working again until i have been in home page once.
Also the get sessions error from above still appears in log but also when switching between just sessions and project view, which seems to work. So not sure its related actually.
I still see the error in the log, though we can use cryosparc normally if we just dont use the home tab ever:D i recorded my screen and will send it via email to you, since people here may want to keep projects private.
Basically what i see
going to home tab → any other tab after will not load properly → fixable by refresh-cancel refresh- refresh again
if you do the refresh thing on the home tab itsself → other tabs will just be blank or management not even open
the sort error appears anyway when switching to projects/session tab but does not seem to really have an impact
Sorry, i was really busy the last weeks and didnt manage to came back to this.
The actual situation ist that we are running v4.0.2 and everything works except:
Going to home tab breaks loading of following tabs, we still then need to refresh, cancel refresh, refresh again. This is what i already shown you in the videos
Tutorial & Guides and everything doesnt load on home tab.
But as long as we just avoid the home tab, everything works perfect.
We connect vie ssh tunnel. The server has http, https proxy defined in /etc/environment, in capital and non capital. The server has no other way of reaching the outside. Before in version 3+ tutorials and stuff where still loading.
Since you cant reproduce it, could it be that the problems are actually related. That if the home tab cant reach the online sources and gets this errors somehow breaks and then when switching to another view its not loading correctly?
I mean could you just cancel outside connection on your test system so that the tutorials and stuff also is not loading and then try if going to another view then home as shown in the videos is breaking the loading of them?
i just updated and i sadly got to inform you that its still the same.
first load of home tab is totally normal → clicking on projects will be just white instead of any project shown
can be fixed with refresh - cancel refresh - refresh again just like before, then everything works aslong as i avoid home tab
going to home tab then will display home tab normally
going back to projects will display projects itsself but no content like job images in it, cant open resource manager etc
going back to home tab will now not load processing history, job by status etc
doing refresh - cancel - refresh now on home tab will disaply it again correctly but then going back to projects its totally white again, just like when at point 1
Edit:
i just had the white projects page open while writing this and it loaded normally suddenly after a few minutes without me doing anything:D
Edit2:
Ok it gets interesting now:D I just timed it several times now:
After openening home tab first time or doing refresh-cancel-refresh on it then going to project tab the tab is white
content load in everytime roghly after 120s(128 mostly), so theres a timeout in your code i would suspect, if that helps you
other cryosparc tabs are also unresponsive in that 120s, for example if i just open a new tab with normal localhost:39000, its stuck loading until that 120s from the first tab are over, and then instantly loads exact at the same time as the project view in the other browser tab
Edit3:
I tried it with 2 different machines, the freezing is local to one machine and one browser in the 120s. I normally use chrome but if i open in the 120s localhost:39000 tab in safari it laods normally. But if i do it generally in safari its the same, so its not a bug just on chrome. (I found also actually that on safari it just needs a normal refresh and not refresh-cancel-refresh as in chrome btw :D)
But in general its reproducible on the other machine.
Could you double check you’re running on v4.0.3? We’re pretty confident that the changes introduced in that version resolve the only possible way certain requests from the browser to server result in a timeout (usually after 1-2 minutes as you described).
In order to debug I’ll need more technical information as well - please email us with your logs (cryosparcm snaplogs) and browser diagnostics.
cryosparcuser@kermit103:/sbdata/software/apps/kilian$ cryosparcm update --check
CryoSPARC current version v4.0.3
update starting on Fri 04 Nov 2022 02:41:04 PM UTC
current version v4.0.3
new version v4.0.3
Already up to date.
I will put the logs together and send it to you via mail:)