Change Cylc Hub URL or base path

Cylc Cylc Support
1

Is there a way to change the default base path from /hub to something else? We’re trying to make the hub accessible from the outside via a proxy-auth server we have via “https://proxy/cylc/” but the Hub keeps re-writing the URL back to “https://proxy/hub/” but we need it to go to “https://proxy/cylc/hub/”.

The farthest I was able to get was to set bind_url in my jupyter_config.py to:
c.JupyterHub.bind_url = 'http://:43600/cylc'

This seems to work for the application/api but then none of the static assets/scripts load because they’re all pointing to /hub/ still.

Thanks!

2

I think c.JupyterHub.base_url might do the trick, but I haven’t tried it myself.

Worth taking a look at these helpful notes JupyterHub put together for proxying.

3

Ok, I tried base_url and it just overwrote my bind_url value. I’ll work with our web dept using those notes. I believe we use Apache.

4

Alright, we got it loaded but apparently _xsrf isn’t making it back to the server and the web guys swear they’re not filtering it out. We unfortunately don’t have web debug tools available to us so I’ll have to wait to inspect once I get to a personal computer.

The error we’re seeing:

[W 2024-11-06 20:48:01.238 CylcHubApp web:1873] 403 POST /cylc/user/{...}/cylc/graphql ({...}): '_xsrf' argument missing from POST

Edit: I am temporarily using the DummyAuthenticator… I see a post commenting they had to implement the following method in their authentication module to get it to work:

def check_xsrf_cookie(self):
        return
5

I have tested locally (without a proxy) using:

c.JupyterHub.base_url = '/foo'
c.JupyterHub.authenticator_class = 'jupyterhub.auth.DummyAuthenticator'

And accessing via http://localhost:8000/foo. It seems to work fine (so long as I give the dummy authenticator a valid user name).

Ideas for debugging:

  • Try running the most similar configuration possible locally (without a proxy).
  • Make sure to wipe cookies/cache.
  • Make sure all Cylc / Jupyter components are up to date (there was a recent spate of xsrf token changes in the Jupyter ecosystem).
  • [1] If you have JupyterLab installed, try accessing it rather than the Cylc UI Server (e.g. http://localhost:8000/foo/user/<username>/lab). If Jupyter Lab works, but Cylc UI Server doesn’t, then it’s an issue we can resolve at our end.

[1] If Jupyter Lab works, but the Cylc UI Server doesn’t.

And if you are able to patch the Cylc source code, try applying this patch:

diff --git a/cylc/uiserver/handlers.py b/cylc/uiserver/handlers.py
index c4e50fc..d8f0ec6 100644
--- a/cylc/uiserver/handlers.py
+++ b/cylc/uiserver/handlers.py
@@ -192,6 +192,7 @@ class CylcStaticHandler(CylcAppHandler, web.StaticFileHandler):
     def get(self, path):
         # authenticate the static handler
         # this provides us with login redirection and token caching
+        print(f'CylcStaticHandler: path={path}')
         if not path:
             # Request for /index.html
             # Accessing xsrf_token ensures xsrf cookie is set

And report back the CylcStaticHandler lines.

[1] If Jupyter Lab does not work either.

It is probably a setup issue.

It might be worth running the setup past the Jupyter folks (they also have a Discourse forum).

6

Alright, thanks for the guidance! It does work when I access the Hub directly. We’re currently still on 8.2.4 but I’m working to get that upgraded to 8.3.5.

1 Like
7

Ah I think that would be it. You’ll need cylc-uiserver 1.5 for compatibility with JupyterHub 4.1 XSRF changes (see changelog), which needs cylc-flow 8.3

8

That appears to be specific to static assets which appear to be working since I’m able to get to the login page. It’s just after logging in and getting the dashboard POST requests to /cylc/graphql don’t have the _xsrf field defined (or one of the two header options apparently).

I’m having trouble understanding how GraphQL/Tornado requests are defined or where the field/headers would be set or which one is being used. So it might be the proxy forwarder we’re using has some default that scrubs headers for our IA compliance requirements.

9

The fix for JupyterHub 4.1+ includes a change that makes the server response to GET / (i.e. index.html) include the XSRF cookie, which is then used by the browser on subsequent requests to /cylc/graphql etc. Without the fix, those subsequent requests won’t have the XSRF cookie so will fail.

See these lines of diff: Don't protect static assets from XSRF by minrk · Pull Request #592 · cylc/cylc-uiserver · GitHub

10

Ah, I see. I didn’t realize it had to be passed along like that; I assumed it was tracked by the host session. This is the deepest I’ve had to look at XSRF. Usually it “just works”.