I need a task visible but only to be used manually when required, the only current way i can see to do this is “task:Impossible_message_trigger => manual_trigger_task” and then removed by a suicide trigger later before the workflow stalls.
Is there a way to do this safely while avoiding using suicide triggers, as i know we are generally not advised to use them in cylc 8?
Ah, that’s an interesting requirement!
task:impossible_message_trigger => manual_trigger_task
That’s not really a proper dependency, is it? Because manual_trigger_task does not actually ever trigger off of task.
Can you explain the use case more clearly?
A task which is only to be triggered manually to recover from specific situations. It can not run automatically as it is an organisational decision to make based on current weather situations or external vendors. Run from cylc so all interactions are logged, and because of our access arrangements which limit manual cli interaction.
We generally prefer our tasks always visible instead of operator’s having to do more complex actions than edit run it.
Hi,
Tasks can always be triggered irrespective of whether they are visible in the GUI or not. E.G. from the GUI, select the “trigger” command, then click the pencil icon to the right and fill in the task ID you want to trigger. Obviously typing this information manually isn’t as convenient as having the task visible in the GUI for the operator to click on, but it still a perfectly viable approach.
One feature we have considered for similar use cases is to allow “custom” commands in the GUI. The idea is that key operations (e.g. triggering a recovery task) could be defined by the workflow designer, and made available to the operator as a command in the GUI. These custom commands could have inputs for the operator to fill in and could call one or more Cylc commands with these inputs (e.g. trigger, broadcast, stop, etc).
Correct. But, it did work in Cylc7. It may be a bit of an abuse of the graph, but it worked.
e.g.
task:impossible => special_task
task => !special_task
Worked as expected. Interestingly, in Cylc8, this does not work as the special_task is never removed from the graph by the suicide trigger (at least Cylc8.2.3, I really should get around to installing 8.2.4).
start => a => end
start:impossible => recover
end => !recover
Leads to recover never being removed, despite end completing. I am unsure if that is expected, or a bug. making it start:impossible? does allow recover to be removed, but then it is not always visible.
2024-02-29T00:50:18Z INFO - [20200101T1800Z/recover waiting(runahead) job:00 flows:1]
2024-02-29T00:50:18Z INFO - [20200102T0000Z/end submitted job:01 flows:1] => running
2024-02-29T00:50:18Z INFO - [20200102T0000Z/end running job:01 flows:1] health: execu
2024-02-29T00:50:21Z INFO - [20200101T0000Z/end running job:01 flows:1] => succeeded
2024-02-29T00:50:21Z INFO - [20200101T0000Z/recover waiting(runahead) job:00 flows:1]
2024-02-29T00:50:21Z INFO - [20200101T0600Z/end running job:01 flows:1] => succeeded
2024-02-29T00:50:21Z INFO - [20200101T0600Z/recover waiting(runahead) job:00 flows:1]
2024-02-29T00:50:21Z INFO - [20200101T1200Z/end running job:01 flows:1] => succeeded
2024-02-29T00:50:21Z INFO - [20200101T1200Z/recover waiting(runahead) job:00 flows:1]
2024-02-29T00:50:21Z INFO - [20200102T0000Z/end running job:01 flows:1] => succeeded
2024-02-29T00:50:21Z INFO - [20200102T0000Z/recover waiting(runahead) job:00 flows:1]
2024-02-29T00:50:22Z ERROR - Incomplete tasks:
* 20200101T0000Z/start did not complete required outputs: ['impossible']
* 20200101T0600Z/start did not complete required outputs: ['impossible']
* 20200101T1200Z/start did not complete required outputs: ['impossible']
* 20200101T1800Z/start did not complete required outputs: ['impossible']
* 20200102T0000Z/start did not complete required outputs: ['impossible']
In the case Diquan is describing, I do not believe the commands are Cylc commands, but an actual script to generate some cold start data, although as you say, a Cylc command could be to trigger a task that does not sit in the graph but is defined in the runtime section. I like having tasks visible, but, to go down the idea you mention, would the options be a dropdown list so people don’t need to look up and type them in, they can just select the appropriate run, fill in or modify something, and it then goes and does stuff? I think they may still need things like pre-script, script, environment, really, its perhaps a list of special tasks which can be triggered by selecting them from a list?
Remember that whether a future or past task is visible in the Cylc 8 GUI is merely a choice!
By default you see the current “active tasks” (which includes finished-but-incomplete tasks) plus n=1 graph edges beyond them, but you can change the window extent to see more of the graph, and you can trigger tasks anywhere in the graph.
Cylc 7 was very restrictive by comparison - you could only see the content of the scheduler’s current “active task pool” (which was however unnecessarily bloated with extra tasks by Cylc 8 standards which seemed to give users the impression they were seeing “all the tasks” or something).
visible in this context I think is more about having a helper task that is for manual intervention visible. Not visibility of general tasks.
I think that might also be an abuse of the term “as expected” 
Interestingly, in Cylc8, this does not work as the
special_taskis never removed from the graph by the suicide trigger (at least Cylc8.2.3,
Actually, it does get removed. Cylc 8.2.x only logs removal by suicide trigger in DEBUG mode (it’s back to INFO on master for 8.3). The reason for the stall, as your log says, is the incomplete start task. The graph path from start:impossible is not optional, so start gets retained as incomplete.
R1 = """
start => a => end
start:impossible => recover # :impossible is a required output
"""
When start finishes it will be retained as incomplete, so it will remain visible in the GUI, and recover will be visible as a future task at n=1. But you’ll have to remove the incomplete task to avoid a stall.
R1 = """
start => a => end
start:impossible? => recover # :impossible is an optional output
"""
Here, if start succeeds it can recede into the past as complete, and you’ll need to extend the GUI window to see recover. But if it fails it will be retained as incomplete, and visible.
Replicate the klunky Cylc 7 way:
R1 = """
start => a => end
start:impossible? => recover
# Artificially keep "recover" at n=0 in the GUI window:
start:submitted => recover
end => !recover
"""
The start:submit trigger spawns recover into the active pool (where the scheduler tracks completion of its other prerequisites). It will stay there waiting on start:impossible until removed by the suicide trigger. (Replace start:submitted with any other trigger, to spawn recover when you like).
@Diquan & @TomC - I’ve edited my two responses above, please re-read them - I have a tendency to give too much explanatory detail, which can create a misleading impression of complexity!
Can you explain the requirement that recover is only ever triggered by manual intervention?
It seems strange to me that it can’t be determined automatically (most obviously, but not necessarily, by failure of the upstream task), and yet a non-expert production operator can make the decision?
This thankfully ended up not actually being required for my use case as it was decided its fine to run automatically.
But i believe it might still be required for recovery tasks by some of our systems @TomC ?