Bug in graphing xtrigger dependencies?

Cylc Cylc Support
1

Hi y’all. I have been testing external triggers. The relevant portion of my flow.cylc [scheduling] section below:

    [[ xtriggers ]]
        nowcast = workflow_state("test_nowcast//%(point)s/block_NEMO_cycle1", offset="-P1D", is_trigger=True):PT2S

    [[ graph ]]

        R1 = """ clean """

        {{ recur[cast['exemode']] }} = """
            clean[^] => init => block_DATA => block_NEMO<cycle=0>
            {% if cast['ig_modesystem'] == 0 %}
                block_NEMO<cycle=0> => ola2wola<cycle=0> & concat_bias_hts
                block_NEMO<cycle=0> => com2newcom => MROA
                ola2wola<cycle=0> & concat_bias_hts => create_lightout
                create_lightout => block_BIASTS
                block_NEMO<cycle=1> => ola2wola<cycle=1>
                MROA => cpmxpackcmz2cpmx => cpmx2cdf => cdf2cmx
                cdf2cmx => cmx2splitcmx => canredcmx2cmx_forsplit
                canredcmx2cmx_forsplit => cpmx2cdf_forsplit
                block_BIASTS & cdf2cmx => block_NEMO<cycle=1>
            {% endif %}
    """

    {{ RECUR_THU }} = @nowcast => block_NEMO<cycle=0>
    P1D ! {{ RECUR_THU }} = block_NEMO<cycle=0>[-P1D] => block_NEMO<cycle=0>

This works exacltly as I wanted it to. On all days except thursday my daily_forecast waits on the preceeding day’s daily_forecast. On thursdays, it has to wait on the preceeding day’s nowcast. Verified by looking at the progress in tui (the tasks do a touch and sleep).

I then plotted the graph:

df-ncst-deps
df-ncst-deps1904×1119 219 KB

And the trigger dependance has the wrong date in it. Shouldn’t it say 20250319?

2

What are you using for RECUR_THU?

3

Hi @gturek -

No, xtriggers are graphed (somewhat redundantly) with the cycle point of the tasks that depend on them.

To the scheduler, an xtrigger is just some arbitrary Python function that it calls periodically with a given list of arguments. The arguments can include the cycle point of the dependent task, via string template, but the function can interpret and use its arguments however it likes. That even goes for the built-in workflow_state trigger. Its offset argument happens to be used (inside the function) to compute the cycle point of the target task in the target workflow, but the calling scheduler doesn’t know that.

2 Likes
4

The @nowcast xtrigger is in Thursday the 20th’s Cycle, and I think that’s correct. The general reasoning makes sense - that an xtrigger can be more or less anything - at the extreme I could write a python function which polls a web connected coffee machine and triggers when 5 people have brought a cappuchino.

Whether it might make sense for this particular trigger to have special behaviour… (probably not, we don’t necessarily know much about the upstream workflow, so using the cycle point from the trigger could mislead)

1 Like
5

p.s. is your workflow_state xtrigger targeting your own workflow, not another workflow?? If so, you can do that, but it is better and clearer to use the plain graph notation for all internal dependencies if possible.

1 Like
6

Hi Hilary, no, we’re experimenting with each “cast” being its own workflow, so I was testing the xtrigger.

1 Like
7

Yes, that’s kind of my point. It’s in the 20th cycle which is correct (it’s a thursday), but the date on the trigger itself could be misleading. But I think it has to do with my choice of graph options:

df-ncst-deps1
df-ncst-deps11260×920 63.2 KB

Without the -c option, the date on the trigger entry clearly tells me the trigger happens on the 20th.
With the -c option, I was expecting the date on which the trigger acts for some reason

8

RECUR_THU=R/+P3D/P7D
I have come up with a way to come up with the correct recurrence string regardless of the date the workflow is started on

9

So in other words, when you’re graphing with the -c option, the date below each task does not contribute any additional information

10

The -c option is for formatting cycling graphs in a more readable way, without making other changes. I suppose we could in principle remove the cycle points from task labels in that case, given that it’s printed at the top of the cycle point box (but then, task IDs are cycle/name not just name…)

1 Like