Avoiding the combination of (non-secure) http: and remote hosts proved to resolve some performance problems, as in Web UI won't load - #3 by ccgauvin94. Your own check will help clarify whether the delay you reported is related.
If you find that access via a loopback (127.*.*.*) address resolves the issue, an https reverse proxy, set up once for each master installation, would be an alternative to the repeated establishment of an ssh tunnel.