There is a change coming with Cylc8/Rose2 to the way template variables are evaluated. Very few configurations should be affected by this change.
Rose2019.01.4 will log warnings when it encounters any template variables that are incompatible with Cylc8/Rose2 in the output of rose suite-run.
What Are Template Variables?
Rose can be used to configure “template variables” which are passed to the “template engine” which is used to process the suite.rc file.
For example if you define the variable answer in the rose-suite.conf file:
[jinja2:suite.rc]
answer=42
Then you could use it in the suite.rc file:
#!Jinja2
[runtime]
[[root]]
[[[environment]]]
ANSWER = {{ answer }}
What Types Does Rose2019 Support?
Rose2019 uses the Jinja2 template engine to parse values specified in the [jinja2:suite.rc] section. This was intended to allow defining variables in different types. Jinja2 is very close to Python so this gives us the ability to specify inputs in the standard Python formats:
# All the intended use cases
[jinja2:suite.rc]
string="mystring"
int=42
float=12.34
bool=True
list=[1, 2, 3]
tuple=(1, 2, 3)
dict={'a': 1, 'b': 2, 'c': 3}
# python3
set={1, 2, 3}
However, an unintended consequence is that it also let through Jinja2 expressions:
# Examples of unintended usage
[jinja2:suite.rc]
yuck=[0] + range(5)
mess=[1, 2, 3] + range(3) | len + 5
nooo=(5 / 3) != 2
This was never the intended purpose of the [jinja2:suite.rc] section. There are a number of reasons that this is problematic, not least that a large proportion of these uses will break with the upcoming Python3 Rose. For example, in Python3 range() returns a generator not a list so [0] + range(5) will break and the rules for integer division have changed so (5 / 3) != 2 might not function as intended.
What Types Will Rose2/Cylc8 Support?
With the upcoming Python3.7+ Cylc8/Rose2 releases template variables will be restricted to Python literals (i.e. all of the intended use cases in the first example).
You will not be able to provide expressions, functions or conditionals:
# All things which won't be possible in Cylc8/Rose2
[jinja2:suite.rc]
expression=40 + 2
function=range(5)
conditional=42 == 84/2
How Do I Upgrade Jinja2 Expressions?
Before
rose-suite.conf
[env]
VAR=40
[jinja2:suite.rc]
list_expression=[0] + [1]
range1=[0] + range(1, 5) | list
range2=range(0, 10, 2)
maths=$VAR + 2
After
rose-suite.conf
[env]
[jinja2:suite.rc]
list_expression=[0, 1]
_range1=(0, 5)
_range2=(0, 10, 2)
VAR=40
suite.rc
#!Jinja2
# NOTE: the list() here protects against Python3 changes
{% set range1=list(range(*_range1)) %}
{% set range2=list(range(*_range2)) %}
{% set maths = VAR + 2 %}
Sidenote: Cylc Template Variable Type Support
Cylc7 allows Jinja2 variables to be specified to cylc run using the -s command line option, however, these variables are only ever interpreted as strings. This can cause some problems e.g:
suite.rc
[cylc]
[[parameters]]
foo = 0 .. {{ INT + 2 }}
$ cylc run <suite> -s INT=40 # this
Traceback ...
Type Error: "40" + 2
With Cylc8 template variables will be evaluated the same way in Cylc as in Rose meaning that they will be safely inter-operable:
-s INT=42
-s LIST='[1, 2, 3]' # note these quotes are stripped by the shell
-s 'STRING="string"' # note the outer quotes are needed to prevent
# the inner ones from being stripped by the shell
-s BOOL=True
-s 'DICT={"a": 1, "b": 2}'
# and so on ...