Workflow with suicide triggers

Cylc Cylc 8 Migration
1

I have an existing cylc7 workflow for a monitoring suite which boils down to the following:

graph = """
        @clock => a => b => c
        housekeeping & a:fail => !a
        housekeeping & (a:fail | b:fail) => !b
        housekeeping & (a:fail | b:fail | c:fail) => !c
        a:finish => housekeeping
"""

The premise is that at each cycle tasks a, b and c will run in sequence. If any of them fail then I need the whole chain to fail nicely and disappear so that it does not cause issues with the runahead limit. It is okay for the tasks to fail if that is what happens, but the suite needs to be able to continue. The housekeeping task is only triggered off the a task as that is the only task that is guaranteed to have run.

I am trying to work out how to best migrate this to cylc8 as the above does not validate:

GraphParseError: Opposite outputs b:failed and b:succeeded must both be optional if both are used

Blindly putting “?” after all the a, b, and c tasks (except the a:finish) validates but with the warning

WARNING - 4 suicide triggers detected. These are rarely needed in Cylc 8 - have you upgraded from Cylc 7 syntax?

Is there any way of implementing this workflow with optional outputs and no suicide triggers?

2

With Cylc 8 (excluding compatibility mode) we no longer need suicide triggers to remove failed tasks from the workflow.

To convert your example, put ?'s after any tasks which are allowed to fail then remove the suicide triggers. I think this should do the job:

# get as far as possible along the a => b => c pipe
@clock => a? => b? => c?
a:finish => housekeeping
3 Likes
3

(As an aside: behold the awesome power of Cylc 8 :grin: - that’s quite a simplification!)

4

I had been trying a lot of variations, but hadn’t managed to fall on that one! Excellent to know that this does indeed work and it is a great simplification.

I still have one concern though. I have set up a workflow with this graph and have forced task b to fail. When I open cylc gui all the cycles still remain with task c in a waiting state. Is this expected behaviour?

workflow1.PNG
workflow1.PNG1920Ă—1201 186 KB

Workflow:

[scheduler]

    UTC mode = True

[scheduling]

    initial cycle point = previous(T00)
    initial cycle point constraints = T00,T06,T12,T18

    [[graph]]

        PT1H = """
               @wall_clock => a? => b? => c?
               a:finish => housekeeping
               """

[runtime]

    [[root]]
        platform = localhost

    [[a]]
        script = passfail.sh 0

    [[b]]
        script = passfail.sh 1

    [[c]]
        script = passfail.sh 0

    [[housekeeping]]
        script = passfail.sh 0

where bin/passfail.sh is simply

#!/usr/bin/bash
echo $1
exit $1
5

Yes, the Cylc 8 UIs show (by default) active tasks, plus what comes next in the graph. Your c tasks are shown as waiting because they are next in the graph, but they are not ready to run yet (because they will only trigger if b succeeds).

6

Once you grok how optional outputs work, it should be much easier to understand than the old way.

@clock => a? => b? => c?

This just means:

  • trigger a when the clock time is right
  • trigger b if a succeeds
  • trigger c if b succeeds
  • a, b, and c (if they run at all) may or may not succeed (i.e. they are not required to succeed, by the workflow)

And that’s it!

7

Apologies @paulearnshaw - that was not a well-considered response! The failed tasks (1 per cycle) should indeed not be piling in up the GUI like that, given that success of b is optional according to your graph.

Investigating…

… yep, seems to be a bug in the datastore that feeds the UI. In which case, thanks for the bug report!

1 Like
8

FYI, there’s a fix in the works, for the upcoming 8.0 release: Data-store prune of optional discontinued branch by dwsutherland · Pull Request #4976 · cylc/cylc-flow · GitHub

9

Brilliant, thanks for looking into this and coming up with a solution.