Different task configuration at different cycle points?

Cylc Cylc Support
1

I was wondering if it is possible to have a different task configuration at different cycle points? For example, say it is known that when you run your 06Z cycle, the infrastructure is very busy, so you want to use less nodes, so at 06Z you want a different configuration to be used.

    [[[task / T06]]]
         [[[job]]]
             execution time limit = PT90M
         [[[directives]]]
            -l select=1:ncpus=1:mem=5G

    [[[task ! T06]]]
         [[[job]]]
             execution time limit = PT30M
         [[[directives]]]
            -l select=1:ncpus=5:mem=25G

Or, perhaps you want to run multiple cycles at once, but the earlier cycles don’t run as long or need as many resources

    [[[task ! $]]]
         [[[job]]]
             execution time limit = PT90M
         [[[directives]]]
            -l select=1:ncpus=1:mem=5G

    [[[task/$]]]
         [[[job]]]
             execution time limit = PT30M
         [[[directives]]]
            -l select=1:ncpus=5:mem=25G

From my experimentation, I don’t believe this is doable, but would it be something the Cylc team would consider adding? Or it could be in the subsections (e.g. [[[job ! $]]], [[[directives / $]]])?

The second use case is the one I’m wanting to accomplish at the moment.

2

Hello,

Cylc doesn’t allow dynamic task configurations, this is because Cylc phillosopy is to keep the scheduling and configuration components of a workflow separated (into [scheduling] and [runtime] respectively).

The problem can be solved in the scheduling section:

[scheduling]
    initial cycle point = 1
    final cycle point = 3
    cycling mode = integer
    [[dependencies]]
        [[[P1]]]
            graph = """
                d[-P1] => a => b
                c => d
            """
        [[[P1!$]]]
            graph = """
                b => task => c
            """
        [[[R1/$]]]
            graph = """
                b => task_long => c
            """

There is a little duplication but the solution is flexible and explicit.

3

Yes, that is the approach I’ve had to adopt. It just adds complexity to the graph section, which, when dealing with large and already complex NWP graphs, can make it harder to follow. I felt that allowing more flexibility in the runtime area would lead to a cleaner result.

4

IMO that’s arguable - it seems a clean and explicit solution to me: two abstract tasks are used to represent the two different behaviours at different cycles, and under [runtime] all the common settings are inherited, so there doesn’t need to be any duplication at all. And the abstract task names can be chosen to make it clear what is happening by merely inspecting the graph (i.e. that the two tasks are actually variants of the same task - e.g. model-5G and model-25G or whatever).

Hilary

5