Different clock triggers for different cycles?

Cylc Cylc Support
1

(cylc 7.8.x)

I have a task that runs every 6 hours. At times 00Z and 12Z, I want it to have a 2-hour clock offset. At times 06Z and 18Z, I want it to have a 1.5-hour clock offset. Other than that, the task is identical (i.e. same task script, same resources required, etc.). All offsets are to make the task run prior to the cycle point wallclock, but that won’t strictly matter for this example. When I set up a suite that looks like:

[cylc]
    UTC mode = True

[scheduling]
    initial cycle point = now

    [[xtriggers]]
        clock_synoptic = wall_clock(-PT2H)
        clock_offtime = wall_clock(-PT1H30M)

    [[dependencies]]
        [[[T00,T12]]]
            graph = """ @clock_synoptic => foo """
        [[[T06,T18]]]
            graph = """ @clock_offtime => foo """

I get an error (when turning on debug options) along the lines of:

Traceback (most recent call last):
  File "/users/whitcomb/working_copies/cylc-working/bin/cylc-validate", line 153, in <module>
    main()
  File "/users/whitcomb/working_copies/cylc-working/bin/cylc-validate", line 90, in main
    output_fname=options.output, mem_log_func=profiler.log_memory)
  File "/users/whitcomb/working_copies/cylc-working/lib/cylc/config.py", line 558, in __init__
    self.load_graph()
  File "/users/whitcomb/working_copies/cylc-working/lib/cylc/config.py", line 2127, in load_graph
    old_xtrig.func_kwargs['offset'])
KeyError: 'offset'

While looking through the code to track this down, several questions arose:

  1. (minor) Is it valid to not use keyword arguments for the wall_clock builtin xtrigger? The user manual says that it’s fine, but the checks in config.py (in load_graph) are explicitly looking only for an offset keyword argument and never checking for positional arguments (and in fact set an offset of zero when comparing with another clock trigger if keyword arguments are not used)

  2. (major) From looking at the code, it seems like the error is coming from the second portion of the graph being parsed and the task foo checking for other clock triggers, and resetting the clock trigger to be the one with the larger offset. The error had to do with the absence of keyword args, but it looks like the intended behavior (even if that’s fixed) is not what I want here. Am I correct in that task foo in the above example will have its clock trigger over-ridden when the second part of the graph is parsed and that I can’t have different clock xtriggers for the same task in different (non-overlapping) cycles?

2

Hi Tim,

Thanks, you have definitely found a bug (damn it!).

  1. (minor) - agreed, you should be able to use a positional or keyword arg for the clock offset. This should be easy to fix, I’ll check Monday.

  2. (major) - the override is intended and correct for single tasks with multiple clock triggers (we might as well only keep the one with the largest date-time value). However, the problem is that xtriggers, unlike task triggers, are not being tied to the cycling sequence, so your two foo instances are not being recognized as different with respect to xtriggers (oops, my fault I think). From a quick look this bug affects all xtriggers, not just clock triggers (in the general case, both foo instances would end up with both xtriggers).

I will get endeavor to get a fix for this up quickly … https://github.com/cylc/cylc-flow/issues/3283

… in the meantime the workaround is to use a different logical task that inherits the original foo content:

...
    [[dependencies]]
        [[[T00,T12]]]
            graph = "@clock_synoptic => foo-a"
        [[[T06,T18]]]
            graph = "@clock_offtime => foo-b"
[runtime]
    [[foo]]
        # (original foo job config)
        ...
    [[foo-a, foo-b]]
        inherit = foo
...

Hilary

3

Thanks! I see the PRs in progress and that looks good. This also opens up another snag that means that I might need to rename the tasks anyway: with the syntax as it currently stands, I don’t think I can have cycle-specific expiration times

With the new PRs in place, it looks like I can support cycle-specific wallclock offsets (great!) but I realized today that I’m still constrained with expirations to have the same expire offset across all cycles.

4

Yes, clock-expiration is currently considered a property of the task, whereas clock xtriggers are entities in the workflow that tasks can trigger off of (and tasks can thus - after the fix! - trigger off of different xtriggers in different cycle points).

We should probably re-implement clock-expiration as an xtrigger. I’ll put an issue up for consideration (however, this one might not happen quickly).

5

The xtrigger fix has passed review and been merged to master - it will be in the upcoming cylc-7.8.4 release. https://github.com/cylc/cylc-flow/pull/3285