Can multiple inter-cycle triggers be simplified?

Cylc Cylc Support
1

I have a workflow that generates hourly output, then a task that compiles results every 24 hours with a graph like the one below. Is there syntax that might make this graph more simple? Like specifying a range of previous cycles that a task depends.

[[[ PT1H ]]] 
 graph = """
    make_data
"""
[[[ T23!^ ]]]
graph = """
    make_data => join_all

    # But only do `join_all` if all the previous hours have completed
    make_data[-PT1H] => join_all
    make_data[-PT2H] => join_all
    make_data[-PT3H] => join_all
    make_data[-PT4H] => join_all
    make_data[-PT5H] => join_all
    make_data[-PT6H] => join_all
    make_data[-PT7H] => join_all
    make_data[-PT8H] => join_all
    make_data[-PT9H] => join_all
    make_data[-PT10H] => join_all
    make_data[-PT11H] => join_all
    make_data[-PT12H] => join_all
    make_data[-PT13H] => join_all
    make_data[-PT14H] => join_all
    make_data[-PT15H] => join_all
    make_data[-PT16H] => join_all
    make_data[-PT17H] => join_all
    make_data[-PT18H] => join_all
    make_data[-PT19H] => join_all
    make_data[-PT20H] => join_all
    make_data[-PT21H] => join_all
    make_data[-PT22H] => join_all
    make_data[-PT23H] => join_all
"""

Thanks for your ideas and pointing out anything I have missed in the docs.

2

Hello,

This sort of dependency is a little tricky to express in the raw config at present. The best solution I can offer is to use Jinja2 to automate the dependency writing:

#!Jinja2
                                                                                                         
[scheduling]    
    initial cycle point = previous(T00)    
    [[dependencies]]    
        [[[PT1H]]]    
            graph = """    
                make_data    
            """    
        [[[T23!^]]]    
            graph = """    
{% for offset in range(24) %}    
                make_data[-PT{{ offset }}H] => join_all    
{% endfor %}    
            """

Which expands to this:

[scheduling]
    initial cycle point = previous(T00)
    [[dependencies]]
        [[[PT1H]]]
            graph = make_data
        [[[T23!^]]]
            graph = """
                make_data[-PT0H] => join_all
                make_data[-PT1H] => join_all
                make_data[-PT2H] => join_all
                make_data[-PT3H] => join_all
                make_data[-PT4H] => join_all
                make_data[-PT5H] => join_all
                make_data[-PT6H] => join_all
                make_data[-PT7H] => join_all
                make_data[-PT8H] => join_all
                make_data[-PT9H] => join_all
                make_data[-PT10H] => join_all
                make_data[-PT11H] => join_all
                make_data[-PT12H] => join_all
                make_data[-PT13H] => join_all
                make_data[-PT14H] => join_all
                make_data[-PT15H] => join_all
                make_data[-PT16H] => join_all
                make_data[-PT17H] => join_all
                make_data[-PT18H] => join_all
                make_data[-PT19H] => join_all
                make_data[-PT20H] => join_all
                make_data[-PT21H] => join_all
                make_data[-PT22H] => join_all
                make_data[-PT23H] => join_all
            """

To make it prettier you can put the Jinja2 logic into a macro or a custom filter.

2 Likes
3

Thanks for the tip! Yes, Jinja is what I needed.