Catch-all host regex not working as expected in 7.8.10 global.rc

Cylc Cylc Support
1

I don’t know for how long this goes back so I’m just speaking to the latest 7.8.10:

If you only specify a .* pattern in the top-level hosts directive, Cylc will still generate a psuedo localhost section that it will default to (which also has defaults like “run directory = $HOME/cylc-run”) which results in “cylc register” failing unless $HOME/cylc-run exists, which we don’t want it to because we keep all that in a centralized location out of $HOME.

So for example (highly simplified to only the relevant lines):

$HOME/.cylc/7.8.10/global.rc:

[hosts]
    # The only host definition:
    [[.*]]
        run directory = /some/path/cylc-run

Output from “cylc get-global-config”:

[hosts]
    [[localhost]]
        run directory = /home/<user>/cylc-run
    [[.*]]
        run directory = /some/path/cylc-run

Our workaround has been to specify only a localhost section under [hosts] even though localhost isn’t one of our suite hosts, but it still gets interpreted correctly in that case. Almost like using localhost works like “.*” probably should… Or am I misunderstanding something?

We regularly kick off suites with “cylc run” and monitor from a non-suite host and the suites start up correctly on the dedicated suite hosts and they write their output to the correct cylc-run path, so somehow Cylc is figuring it out when you only provide a localhost subsection in the hosts section…

2

Hello,

$HOME/.cylc/7.8.10/global.rc:

Using the Cylc version number in the configuration file path is a Cylc 8 feature so I wouldn’t expect this file to be loaded?

The way Cylc 7 handles the [hosts] section is somewhat unusual. The localhost item is evaluated relative to the host on which the config is loaded. So if your interactive box and suite server share the same filesystem then the localhost configuration will be applied to both. Kinda logical but very confusing, consequently localhost works kinda like you might expect .* to.

Explicitly named hosts take precedence over globs which may or may not explain why .* isn’t working the way you expect it to, I’m not sure. Sorry, not the most helpful answer.

The good news is in Cylc 8 we have overhauled this with the new platforms configuration. Platforms can still be named using globs, however, these are now matched in a much more logical way.

It is no longer possible to override the run directory in the Cylc 7 way, however, Cylc 8 now has symlink-dirs which can be used to move workflow run directories (or specific parts of run dirs e.g. log/) to another filesystem leaving behind a symlink for Cylc to follow (this is closely related to the old Rose root dir feature).

Some additional Cylc 8 reading:

Cylc 8.0rc1 should be out very soon.

Oliver

3

Using the Cylc version number in the configuration file path is a Cylc 8 feature so I wouldn’t expect this file to be loaded?

It actually is loading so maybe that feature got backported? Been working that way for all the 7.8.x series as far as I know. If I don’t have the version-specific config in place, we get full defaults. If I do have it in place, it’ll get picked up.

Your explanation around the localhost behavior matches up exactly with what we’re seeing so I’ll just continue to leave it that way (we do have a shared network storage connected).

Our situation is a little complex so we’re currently restricted to the 7.8.x series for the time being, but I am looking forward to Cylc 8 eventually!

Thanks for the explanation!