Parameterized clock triggers?

This is slightly related to the “subcycle” problem I asked about a little while ago.

I have some data processing tasks that I want to run at an hourly interval for a particular cycle point, i.e. at the 00Z cycle point I run tasks every hour from 18Z to 12Z wallclock - then from 06Z I also run tasks from 04Z to 18Z wallclock etc. Essentially, “run every hour from 2 hours before the cycle point to 12 hours after the cycle point”.

Is there a straightforward way to parameterize this? One way to be something like a parameterized xtrigger where the string would be available for the offset, e.g.

[cylc]
    [[parameters]]
        offset = -2..12
[scheduling]
    initial cycle point = next(00)
    [[xtriggers]]
        delay<offset> = wall_clock("PT%(offset)dH")
    [[dependencies]]
        [[[PT6H]]]
            graph = "@delay<offset> => process<offset>"

Jinja2 sorcery is another option (with a loop creating either an xtrigger or clock-trigger entry for every task in the list), but I’m wondering if there’s a simpler way to do this that I’m missing. It’s important that I be able to maintain the 6 hour cycling (or at least a connection between the 0, 6, 12, and 18Z cycle points!)

Hi Tim,

IMO doing the “parameterization” yourself with a Jinja2 loop is fine for this. With a Python API (as an alternative to suite.rc + Jinja2) planned for the not-too-distant future, we’re not desperate to extend built-in parameterization to replace absolutely everthing you could do with Jinja2 loops. On the other hand, I suppose parameterized xtriggers are in the same ball-park as parameterized tasks (even easier in fact, because they don’t appear in the runtime section). Maybe we’ll consider it…

Hilary

Added a reference to this conversation to an appropriate Issue page on GitHub: https://github.com/cylc/cylc-flow/issues/2453#issuecomment-518420184 [Now taken on by https://github.com/cylc/cylc-flow/issues/3454]

Thanks! Jinja2 it is for now.

I had just coded my suite to use xtriggers and parameters in the xtriggers but see this is not possible. That is a shame. It will make my workflow very messy now.

@hilary.j.oliver is there any chance this will come in at some point or does cylc8 do some much cooler?

[FAIL] ERROR, undefined xtrigger label: amm15data_084_132_25hourm

This error made me very sad so back to the drawing board.

Hi @kaday,

Sorry for your pain … but at the moment we are still rather swamped with higher priorities, and there is a decent workaround for this: do your xtrigger parameterization with a Jinja2 loop (and you can avoid duplication if you need the same parameters in xtriggers and tasks, by defining your task parameter range with the same Jinja2 variables).

Of course this is an Open Source project; as a former Cylc developer maybe you’d like to take this on! :grinning: https://github.com/cylc/cylc-flow/issues/3454

Hilary