Migration: unbound variable

Cylc Cylc 8 Migration
1

Hi, migrating to cylc 8, I have a missing environment variable.

Cylc is invoked something like this:

export OPT_NAME=test_quick
cylc vip --verbose

The variable is used in rose-suite.conf:
PLOT_DIR="$DATADIR/res_cylc_8/$OPT_NAME"

When run, I can see the validation step correctly uses the variable:

DEBUG - Setting Jinja2 template variables:
    ...
    + PLOT_DIR=/data/users/bblay/res_cylc_8/test_quick

However, cylc then crashes with the following error:

DEBUG:cylc:running command:
$ ssh -oBatchMode=yes -oConnectTimeout=8 \ 
    -oStrictHostKeyChecking=no -n exvcylc09 env \ 
    CYLC_VERSION=8.4.1 CYLC_ENV_NAME=cylc-8.4.1-1 bash --login \ 
    -c 'exec "$0" "$@"' cylc play --verbose res_cylc_8/run5 \ 
    --host=localhost --color=always

Welcome to the Met Office

WARNING: exvcylc09 is provided for the processing of OFFICIAL
information. Unauthorised access may constitute a criminal offence.
By continuing to use this facility you agree to abide by the Cyber
Security in Practice policy. All activity on the system is liable
to monitoring in accordance with the Monitoring policy.

DEBUG - Loading site/user config files
DEBUG - Reading file /home/h03/fcm/metomi-
    site/etc/cylc/flow/8/global.cylc
DEBUG - Processing with Jinja2
DEBUG - Setting Jinja2 template variables:
    + CYLC_TEMPLATE_VARS={'CYLC_VERSION': '8.4.1', 'CYLC_TEMPLATE_VARS': {...}}
    + CYLC_VERSION=8.4.1
DEBUG - /home/h02/bblay/cylc-
    run/res_cylc_8/run5/log/scheduler: directory created
DEBUG - /home/h02/bblay/cylc-run/res_cylc_8/run5/log/job:
    directory created
INFO - Extracting job.sh to /net/home/h02/bblay/cylc-
    run/res_cylc_8/run5/.service/etc/job.sh
2025-03-07T11:50:14Z DEBUG - Starting
2025-03-07T11:50:14Z DEBUG - Configure curve: *[/home/h02/bblay/cylc-run/res_cylc_8/run5/.service/client_public_keys]
2025-03-07T11:50:14Z INFO - Workflow: res_cylc_8/run5
2025-03-07T11:50:14Z DEBUG - Reading file /home/h02/bblay/cylc-run/res_cylc_8/run5/flow.cylc
2025-03-07T11:50:14Z ERROR - Workflow shutting down - PluginError: Error in plugin cylc.pre_configure.rose
    ConfigProcessError: env=PLOT_DIR: OPT_NAME: unbound variable
ERROR: ERROR: command terminated by signal 1: ssh -oBatchMode=yes -oConnectTimeout=8 -oStrictHostKeyChecking=no -n exvcylc09 env CYLC_VERSION=8.4.1 CYLC_ENV_NAME=cylc-8.4.1-1 bash --login -c ''"'"'exec "$0" "$@"'"'"'' cylc play --verbose res_cylc_8/run5 --host=localhost --color=always

I read this as the environment variable not being available on the remote node. However, this was working in Cylc 7. Is this a known difference with Cylc 8? If so, what is the recommended approach?

Perhaps I just broke something while migrating. I tried adding this, which I thought I’d used in the past to get variables onto remote nodes, but it doesn’t help:
export CYLC_TASK_ENV_OPT_NAME=$OPT_NAME

Any advice would be greatly appreciated!

2

For completeness, as it looks like it’s processing flow.cylc at the time of the error, I’ll add that OPT_NAME is not used in there. It has environment sections like this: PLOT_DIR = {{PLOT_DIR}}/charts.

3

Hi :wave:,

I read this as the environment variable not being available on the remote node.

Yes, I expect this is the case.

For completeness, as it looks like it’s processing flow.cylc at the time of the error, I’ll add that OPT_NAME is not used in there

The rose-suite.conf file needs to be read as part of parsing the flow.cylc file, so this is expected.

Is this a known difference with Cylc 8

Rose 2019 used to write all of the template variables defined in the rose-suite.conf file to the top of the suite.rc file (in the cylc-run dir) when you ran rose-suite.run. The suite.rc file was then loaded on the Cylc server. I guess Rose must have expanded environment variables at this stage if this worked before?

Any advice would be greatly appreciated!

I would recommend looking into Rose optional configs to implement this OPT_NAME switch.

Optional configs allow you to define fragments of a Rose config (rose-suite.conf in this case) to be applied when the option is activated.

E.g, you might have a file like this:

# opt/rose-suite-test_quick

[template variables]
PLOT_DIR="$DATADIR/res_cylc_8/test_quick"

Which you could activate like this:

cylc vip -O test_quick

At Cylc 8, Rose optional configs (and all other Rose installation options) are remembered for future restarts or reinstallations of the workflow so only need to be specified once (when you first install the workflow).

Optional configs can also be specified via a Rose environment variable (this will be detected at install time and turned into a config option so will work on the Cylc server).