Marking Keys as Required in flow.cylc for Site-Specific Settings

Cylc Cylc Support
1

Hi,

Is it possible to mark a key in flow.cylc as “required” so that if it is not provided with a value, an error is raised during cylc validate/install?

The reason I need this is that while writing portable workflows, I want to mark some site-specific settings (like platform or certain environment variables) as “required.” This way, they must be overwritten by the site include file.

Thank you in advance for your help and suggestions!

2

Summary

You could either

  • Write checks into the flow.cylc file using Jinja2
  • Use Rose.

Jinja2 Checks

This may be neater, especially if you don’t want to add the Rose dependency or the checks are simple and you only have a few variables, but could quickly become a pain if you have lots of variables.

#site.cylc
{% set platform_name = "actual_name" %}

#!jinja2
# flow.cylc
{% set platform_name = "default" %}
%%include site.cylc

{{ assert(platform_name != "default", "Platform Name should be overridden") }}

# ...
[runtime]
    [[task]]
        platform = {{platform_name}}

> cylc validate 
Jinja2Error: Jinja2 Assertion Error: Platform Name should be overridden
...

Use Rose

Rose allows you to create a file alongside your workflow containing configuration items:

#!jinja2
# flow.cylc
...

[runtime]
    [[task]]
        platform = {{PLATFORM}}
        [[[environment]]]
            MYVAR={{MYVAR}}

# rose-suite.conf
[template variables]
PLATFORM="mycomputer11"
MYVAR=42

If you don’t put a value in the rose-suite.conf files you’ll get an error:

> cylc validate
Jinja2Error: 'platform_name' is undefined
...

Jinja2 in Cylc reference material

Rose Optional Configs

You could, using the latter store site specific settings in a file in opt: opt/rose-suite-sitename.conf and have an empty rose-suite.conf.

In this case the validation would fail if one did not validate & install the workflow with:

cylc validate --opt-conf-key sitename         # -O for short

Optional configs tutorial

Rose Metadata Validation

Alternatively you could use Rose metadata checking to carry out more complex checks on the values. See the tutorial on Rose metadata

4 Likes