Interdependency for specific days of month

Cylc Cylc Support
1

I want to run several interdependent monthly tasks, which may take a long time, without holding up my 4/day cycling. For example, if I get 1 day per hour wallclock, and I have a runahead limit of <=2 days, then I need the first task to start at the same time as 5 or more cycle days earlier.
The final task dependency is at 01T00.
I want to run one task 5+ days earlier. This task will complete within a short while, because it orphans a process (to stage data on tape). It could take 30 min to a couple hours to complete. I don’t care too much if it doesn’t quite finish before my next task starts.
The next task (pull data from tape) should run about 2 cycle days (a couple hours?) later. It may finish in 30 mins, or a few hours (and can fail and retry). Once succeeded, the final task at 01T00 can start.

My thought was to try to tie each task to a specific calendar day, so the delay between the first two tasks running will be a result of how long the suite (it’s cylc7) takes to cycle through, basically using the runahead limit as a brake.
The alternative is to do A[-P5D] => B, but I can’t see how to do this for 3 linked task, i.e.
A[-P5D] => B[-P3D] => C

What is the best way to tackle this?

2

The alternative is to do A[-P5D] => B, but I can’t see how to do this for 3 linked task, i.e.
A[-P5D] => B[-P3D] => C

This inter-cycle dependence can be written:

    a[-P2D] => b
    b[-P3D] => c

The next thing is to work out what recurrence you want this inter-cycle dependency to repeat on.

  • P1D - daily
  • P5D - every 5 days

E.G.

P5D = """
    # apply these inter-cycle dependencies, only every 5 days
    a[-P2D] => b
    b[-P3D] => c
"""

Interdependency for specific days of month

You can also use:

  • P1M - monthly on the first of the month
  • 05T00 - monthly on the fifth of the month
  • 28T00 - monthly on the 28th of the month

Note, we can’t currently write a recurrence on the 29th, 30th or 31st of the month in the Gregorian calendar

As a result of this issue: cycling: last day of the month - 31T00 · Issue #2382 · cylc/cylc-flow · GitHub

3

Thanks for your response.
My past experience with cylc is that

a[-P2D] => b
b[-P3D] => c

Would end up defining two b tasks at different times. Although the cycling spec was usually something like T06H rather than P5D.

I want a frequency of 1 month for these tasks, with the last task running on the first of the month, and the earlier tasks running days before. The rest of the suite runs at PT6H.

So would (for cylc7 since that is what I am using) this work? My quick test suggests no.

[[[P1M]]] 
graph="""
a[-P2D] => b
b[-P3D] => c
"""

I’m happy to avoid dates 29,30,31, but of course the space between the 25th and the 1st will vary each month. So I can’t both specify that b runs on the 25th, and that it runs N days before c.

I suppose I could be make it work if I made c run on 28T18, the last common cycle each month, as an artificial dependency, since it will be guaranteed to run before 01T00.

4

I want a frequency of 1 month for these tasks, with the last task running on the first of the month, and the earlier tasks running days before

You’ll need to use two recurrences to do this, something along these lines:

[scheduling]
    initial cycle point = 2000
    [[graph]]
        T00 = """
            # every day
            a
        """
        01T00 = """
            # every month on the first of the month
            a[-P2D] => b
        """
        04T00 = """
            # every month on the fourth of the month
            b[-P3D] => c
        """

$ cylc graph . -t 20000225 20000305

tmpwfrwjbul
tmpwfrwjbul1034×946 31.2 KB