# Cylc submit and jinja2 template variables

**URL:** <https://cylc.discourse.group/t/cylc-submit-and-jinja2-template-variables/355>\
**Category:** Cylc Support\
**Created:** [July 17, 2021, 2:27pm UTC](https://cylc.discourse.group/t/cylc-submit-and-jinja2-template-variables/355 "2021-07-17T14:27:50Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![schaferk](https://yyz2.discourse-cdn.com/free1/user_avatar/cylc.discourse.group/schaferk/32/92_2.png) [@schaferk](https://cylc.discourse.group/u/schaferk)\
**Post date:** [July 17, 2021, 2:27pm UTC](https://cylc.discourse.group/t/cylc-submit-and-jinja2-template-variables/355/1 "2021-07-17T14:27:50Z")

</div>

version 7.8.8  
cylc submit -s myVar1=bob -s myVar2=fred --dry --debug mySuite myTask.1

```auto
...snip
2021-07-17T14:21:39Z DEBUG - Processing with Jinja2

2021-07-17T14:21:39Z DEBUG - Setting Jinja2 template variables:

+ myVar1=bob

+ myVar2=fred

2021-07-17T14:21:39Z DEBUG - Expanding [runtime] namespace lists and parameters
...snip

```

i was trying to replace values in a task script.  
suite.rc

```auto
#!jinja2
[scheduling]
    cycling mode = integer
    initial cycle point = 1
    final cycle point = 1

    [[dependencies]]
        graph = "myTask"

[runtime]
    [[root]]

    [[myTask]]
        script = echo myVar1 = {% set myVar1 = myVar1 %} myVar2 = {% set myVar2 = myVar2 %}

```

script validates.  
cylc submit …produces:  
[myTask.1] Job ID: 49915

however, cylc get-config --sparse mySuite  
shows

```auto
[scheduling]
    cycling mode = integer
    initial cycle point = 1
    final cycle point = 1
    [[dependencies]]
        graph = myTask
[runtime]
    [[root]]
    [[myTask]]
        script = echo myVar1 = myVar2 =

```

am i going down the wrong path here?

show a use case where the  
"-s " switch in ‘cylc submit’ is used successfully?

---

<div class="post-metadata">

**Author:** ![kinow](https://yyz2.discourse-cdn.com/free1/user_avatar/cylc.discourse.group/kinow/32/145_2.png) [@kinow](https://cylc.discourse.group/u/kinow)\
**Post date:** [July 18, 2021, 6:51am UTC](https://cylc.discourse.group/t/cylc-submit-and-jinja2-template-variables/355/2 "2021-07-18T06:51:27Z")

</div>

I made a small change to your `suite.rc` to use the Jinja variable, instead of declaring it.

```auto
#!jinja2

[scheduling]
    cycling mode = integer
    initial cycle point = 1
    final cycle point = 1

    [[dependencies]]
        graph = "myTask"

[runtime]
    [[root]]

    [[myTask]]
        script = echo myVar1 = {{ myVar1 }} myVar2 = {{ myVar2 }}

```

Now using `{{ myVar1 }}` syntax and also specifying the Jinja variable values in the command line (`-s`). After that, I think your example should work.

```bash
kinow@ranma:~/cylc-run/cylc7test$ /opt/cylc/usr/bin/cylc get-config -s myVar1=bob -s myVar2=fred --sparse cylc7test
[scheduling]
    cycling mode = integer
    initial cycle point = 1
    final cycle point = 1
    [[dependencies]]
        graph = myTask
[runtime]
    [[root]]
    [[myTask]]
        script = echo myVar1 = bob myVar2 = fred

```

Not sure why validation didn’t fail, although I think your suite was valid, but with empty variables.

Cheers  
Bruno

---

<div class="post-metadata">

**Author:** ![schaferk](https://yyz2.discourse-cdn.com/free1/user_avatar/cylc.discourse.group/schaferk/32/92_2.png) [@schaferk](https://cylc.discourse.group/u/schaferk)\
**Post date:** [July 18, 2021, 11:33am UTC](https://cylc.discourse.group/t/cylc-submit-and-jinja2-template-variables/355/3 "2021-07-18T11:33:48Z")

</div>

thank you bruno.

i did try your instance initially without 'get-config’ing it, e.g., i always, register, validate, …, get-config, etc…  
the validation fails for me…

```auto
Jinja2Error:
  File "<template>", line 15, in top-level template code
UndefinedError: 'myVar1' is undefined
Context lines:

    [[myTask]]
        script = """
            echo myVar1 = echo myVar1 = {{myVar1}} myVar2 = {{myVar2}}	<-- Jinja2Error

```

```auto
[mach]$ cylc --version
7.8.4-10-gde7d

```

FWIW; using --debug in cylc submit, ie.

```auto
cylc submit -s myVar1=bob -s myVar2=fred --dry --debug mySuite myTask

```

does not differentiate the cases of using {{myVar1}} vs. {%set myVar1=myVar1 %}.

hence, i was not able to diagnose using --debug.

---

<div class="post-metadata">

**Author:** ![kinow](https://yyz2.discourse-cdn.com/free1/user_avatar/cylc.discourse.group/kinow/32/145_2.png) [@kinow](https://cylc.discourse.group/u/kinow)\
**Post date:** [July 18, 2021, 8:29pm UTC](https://cylc.discourse.group/t/cylc-submit-and-jinja2-template-variables/355/4 "2021-07-18T20:29:01Z")

</div>

Hi!

> the validation fails for me…

Ah, yes. I tested the `validate` command before changing the suite and it worked. But now it fails due to Jinja not finding the variables defined in the `suite.rc` file (the error is similar to referencing a variable before it is declared in Python).

This should work:

```bash
kinow@ranma:~/cylc-run/cylc7test$ /opt/cylc/usr/bin/cylc validate -s myVar1=bob -s myVar2=fred cylc7test
Valid for cylc-7.9.3

```

So you will always have to define the variables in your `cylc` sub-commands that need to parse the `suite.rc` file, like `submit`, `validate`, `get-config`, etc.

I have not used much Rose, besides reading the documentation and doing a few code reviews for the new code, but I think one of the features of Rose is to allow you to save your variables so you don’t have to keep repeating them in your command line.

But others can confirm if that’s the case, or if there are other workarounds.

Note too that functionalities of Rose are being incorporated in Cylc 8, either directly in the code or as plug-ins.

> does not differentiate the cases of using {{myVar1}} vs. {%set myVar1=myVar1 %}.

I **think** your previous example is a valid `suite.rc`. The generated `echo` command is valid, but does not print anything.

And the Jinja statements you used were valid. They were declaring variables, but not outputting anything.

Cheers  
Bruno

---

<div class="post-metadata">

**Author:** ![hilary.j.oliver](https://yyz2.discourse-cdn.com/free1/user_avatar/cylc.discourse.group/hilary.j.oliver/32/4_2.png) [@hilary.j.oliver](https://cylc.discourse.group/u/hilary.j.oliver)\
**Post date:** [July 18, 2021, 10:58pm UTC](https://cylc.discourse.group/t/cylc-submit-and-jinja2-template-variables/355/5 "2021-07-18T22:58:48Z")

</div>

Hi @schaferk ,

@kinow pretty much nailed it, but I can add a couple of comments.

It helps to think of Jinja2 as a preprocessor that generates the final `suite.rc` file that the Cylc scheduler loads. The Jinja2 `{% ... %}` syntax is for statements that set and manipulate Jinja2 variables but don’t print anything to your `suite.rc` file. The `{{ ... }}` syntax is the print statement, which writes text out to your `suite.rc` file (which @kinow is calling the template “output”).

If your `suite.rc` contains Jinja2 variables that have to take their values from command line inputs, then you will have to supply those inputs on every command line that parses the `suite.rc` file (including `cylc validate`) otherwise the command will fail due to referencing undefined variables. However, you can set defaults to be used if you don’t supply values on the command line:

```auto
{{ VAR | default("blah") }}

```

> [@kinow](#):
>
> but I think one of the features of Rose is to allow you to save your variables so you don’t have to keep repeating them in your command line.
> 
> But others can confirm if that’s the case, or if there are other workarounds.

Yes, `rose suite-run` takes Jinja2 variables defined in the `rose-suite.conf` file and writes the variable definitions to the `suite.rc` at install time. However, that functionality has been migrated into Cylc 8, so don’t bother with it now unless you are already using `rose suite-run` with Cylc 7.

Hilary