Run a task only once within jinja2 loop

[gturek]

Hi, I have the following (simplified) situation summarized by this workflow:

#!Jinja2

{% set casts=['a','b','c'] %}

[scheduler]
    cycle point format = %Y%m%d
[scheduling]
    initial cycle point = now
    final cycle point = '20250731'

    [[graph]]

{% for cast in casts %}
        R/2024-W01-3/P7D = """
             {{ cast }}_task3[-P7D] => {{ cast }}_task1 => {{ cast }}_task2 => {{ cast }}_task3
        """
{% endfor %}

[runtime]

{% for cast in casts %}
   [[ {{ cast }}_task1 ]]
        script = """
            sleep 15
        """
   [[ {{ cast }}_task2 ]]
        script = """
            sleep 15
        """
   [[ {{ cast }}_task3 ]]
        script = """
            sleep 15
        """
{% endfor %}

I modified flow.cylc by taking task1 out of the loop and running it only once:

   [[graph]]
        R1 = """
            task1
        """
{% for cast in casts %}
        R/2024-W01-3/P7D = """
             {{ cast }}_task3[-P7D] => task1 => {{ cast }}_task2 => {{ cast }}_task3
        """ 
{% endfor %}

[runtime]

   [[ task1 ]]
        script = """
            sleep 15
        """
{% for cast in casts %}
..........

I still need to tell it to wait on task1 at least the first time around, and it’s here where I get stuck.
The above modification still makes task1 run at every scheduled cycle. Is the only way to solve this to set run mode = skip by broadcast after the first time around?
Thanx!
Gaby

[gturek]

As a follow up to this question we actually tried broadcasting to set the run mode to skip in a post-script to the task. This caused the task to be marked as “failed” and the workflow to stop. I think here we ran into the polling issue that was discussed in @sparonuz’s post https://cylc.discourse.group/t/cylc-broadcast-error-workflowstopped/. So this approach is definitively not an option

[hilary.j.oliver]

Hi @gturek !

I don’t have time for a close look right now, but generally speaking if you only want task1 to run in the first cycle then it should only appear in the first cycle R1 expression. Every task in the P7D expression will run in every cycle - that what it says to do.

However you can have the first of the “every cycle” tasks wait on task1 but putting that dependency in the R1.

You should definitely not need to dynamically set the task to skip mode to do this.

A very simplified example:

[scheduling]
    initial cycle point = 2025
    [[graph]]
        R1 = "task1 => every1"
        P7D = """
            every1 => every2
            every1[-P7D] => every1
        """
[runtime]
    [[task1, every1, every2]]

Result:

(But apologies if I’ve over-simplified your requirements by being in hurry! … I’ll come back tomorrow if no one else responds first).

H

[update] - note parameterizing tasks with Jinja2 can make a workflow look more complicated, but it doesn’t change what I’ve said above - the parameters get expanded to generate more task names, but the same principles apply as for any task names.

[gturek]

Thanx Hilary! This graph works perfectly:

    [[graph]]

{% for cast in casts %}
       R1 = """
             task1 => {{ cast }}_task2
        """
{% endfor %}


{% for cast in casts %}
        R/2024-W01-3/P7D = """
             {{ cast }}_task3[-P7D] => {{ cast }}_task2 => {{ cast }}_task3
        """
{% endfor %}