All dependencies being met to trigger a family of tasks when on hold

Cylc Cylc Support
1

I’m using Cylc 8.2.3 and noticed the following when the workflow is on hold. If you want to trigger an entire family of tasks by using “Trigger”, it doesn’t actually trigger the tasks unless…

  • you “Trigger” one task by one across the family, OR
  • you “Set Outputs” as succeeded of all prerequisites of that family of tasks. Then a simple “Trigger” on the family level, it triggers all the tasks under that family.

Is this behaviour expected?

Thank you

2

Hi,

Triggering individual tasks should always work as expected. Triggering a group of tasks, e.g. a family or cycle is somewhat trickier. Currently, if you try to trigger a group, Cylc will only trigger the subset of tasks within the group that the scheduler is actively managing. This is sufficient for many use cases, e.g. manually re-triggering failed tasks (cylc trigger '<workflow>//<family>:failed), but there are some uses this doesn’t satisfy.

The reason for this limitation is technical, note that Cylc 8 differs from Cylc 7 in that, not all tasks defined in the workflow will necessarily be run by it (see “graph branching” in the docs for an explanation), which makes this trickier.

There is an issue on GitHub you can follow for progress on this matter: Command task matching requirements · Issue #5827 · cylc/cylc-flow · GitHub

Cheers,
Oliver

3

To add to @oliver.sanders 's response a little:

Cylc will only trigger the subset of tasks within the group that the scheduler is actively managing.

That is what we call the “active window” of the scheduler. It includes:

  • literally active tasks (submitted and running);
  • tasks that have partially satisfied prerequisites (actively waiting on the rest of their prerequisites);
  • tasks that are actively waiting on external triggers (including clock triggers)
  • tasks that are ready to run but held back by a queue or by the runahead limit

As Oliver indicates, manually triggering multiple tasks by name or family pattern outside of the active window is potentially problematic because of optional branching (or, e.g. you could easily trigger two branches at once where only one or the other is supposed to run).

Triggering individual tasks outside of the active window technically has the same problem though (you might trigger a task that would not have run naturally) but a least that has a more limited and obvious effect, so it is safer.

However, I expect in a future release we will extend that to multiple tasks because it would make some use cases easier to handle.

4

Thank you both for the quick responses.