I was wondering if it were possible to trigger off of a task succeeding or suiciding? To the best of my knowledge it isn’t, so a toy example like below might be done.
graph = """
a => b & c & d
a:message => !b & !c & !d
a:message | (b & c & d) => end
"""
But if you get more complex, it gets harder to make triggers that are easy to follow. For example:
graph = """
a => c => b & d
a:message => !b & !c & !d
c:message => !b & !d
a:message | c:message | (b & d) => end
"""
Any advice on handling this sort of thing (other than don’t design it this way)? I think task:finish counting “suicides” as finished might make some sense here, because the task is finished as far as this cycle is concerned.
It is not possible to trigger off of a suicide event. Some thoughts:
If the logic is getting complex there you can use family groupings:
a => c => B_D
a:message => !B_C_D
c:message => B_D
a:message | c:message | B_D:succeed-any => end
You kinda can trigger off of suicide events indirectly by adding a dummy task with the same triggering:
a:message => suicide_1
suicide_1 => !b & !c & !d
Which would effectively allow you to trigger off of a suicide event:
suicide_1 | suicide_2 | (b & d) => end
With the caveat that you would need to suicide your dummy tasks.
Suicide triggering is kinda horrible and should be avoided wherever possible.
To explain why using your example:
a:message => !b & !c & !d
c:message => !b & !d
This looks correct but if we re-arrange these dependencies like so:
a:message => !c
a:message & c:message => !d # oh no
a:message & c:message => !b # oh no
Going at the problem programmatically might help.
Not something I’ve ever tried, but if you really need suicide triggers and have complex logic surrounding them you could try writing the suicide depenencies automatically.
Here’s you example written in Python.
# list all of your suicide triggers
suicides = {
'a:message': set(['b', 'c', 'd']),
'c:message': set(['c', 'd'])
}
# reverse the dictionary to see what suicides
# depend on which triggers
rev_suicides = {}
for trigger, tasks in suicides.items():
for task in tasks:
if task in rev_suicides:
rev_suicides[task].append(trigger)
else:
rev_suicides[task] = [trigger]
# output the suicide triggers
for task, triggers in rev_suicides.items():
print('%s => %s' % (' | '.join(triggers), task))
Which outputs:
c:message | a:message => c
a:message => b
c:message | a:message => d
This kind of logic could easily be done in Jinja2 or written as Python functions imported in Jinja2.
Suicide triggers are somewhat nasty as @oliver.sanders says. From a pure workflow logic perspective we shouldn’t need them at all, but a detail of Cylc’s internal implementation sometimes makes them necessary in practice: objects in the server program that represent waiting tasks in current active cycle points have to be removed from the “task pool” if they won’t be needed due to upstream events. Our development roadmap will actually make suicide triggers redundant in the future, but we haven’t pinned a date on that milestone yet (watch this space!).
I am doing the FAMILY approach already, I just wondered if there was a better way.
I’m not sure how to avoid suiciding paths when you only want to run certain paths if certain conditions are met, and if they aren’t meant, they should never run and the suite should be able to cycle. The simplest option would be to make a task succeeded instead of remove it from the graph really, but I don’t think Cylc supports that by default. Of course, if it is a message trigger, then I could change the state of the tasks to succeeded instead inside the script.
Oh, and @oliver.sanders good point in 3. I had written it incorrectly, highlighting that this case does indeed get hard to manage very quickly. It should have been
a:message | c:message => !b & !d
a:message => !c
a => c => b & d
a:message | c:message | (b & d) => end
If your tasks use little resource or aren’t submitted to a batch system then a pretty simple way to avoid suicide triggers is to build the suicide logic into the script:
script = """
if [[ <condition> ]]; then
true # skip
else
run-my-script
fi
"""
Obviously non-ideal as dependency management is meant to be Cylc’s job but for some edge cases an acceptable way to avoid suicide triggers.
This is a solution to something that had me in circles for a while today:
graph = """
a => !b
a:fail => b
a | b:finish => hk
"""
It looks so simple now, but it kept hanging on the hk task. “b” is a task that checks to see if it was a failure that I was expecting and if so succeeds and if not fails so I can go back and look at the error and see if I need to stop the suite. The suite has to run even if I have failures which is not ideal but at least can now see at a glance if it was an expected failure as “b” passes if so.
Please not th original problem was much bigger as it used lots of cool parameters! So did not always hang.