Is it not possible to trigger a task while paused?

From what I can see it’s not possible to trigger a task while the workflow is paused from either command line or the web GUI, but I can see in the documentation that pause is listed as replacing hold but doesn’t seem to carry over that functionality. I can also see that hold functionality does still exist in the options with “–hold-after” but not the generic “–hold”.

This especially causes an issue when it comes to wanting to start the workflow in a held/paused position and run tasks individually, since pause doesn’t allow for this and the only work around i have come up with is setting “–hold-after” to 6 hours prior to the first run but is quite a hacky approach to get this functionality.

So essentially, I am wondering if I have missed something with pause or is this just its correct functionality?

In Cylc 8 you can:

  • “pause” the workflow
    • suspend all job submission; resume with “play”
  • “hold” individual tasks
    • don’t submit jobs even if ready to run; resume with “release”
    • all tasks (not just waiting ones) can be held, which means don’t submit the next job until released, e.g. for automatic retry on failure

You’re right that manually triggering a task has no immediate effect if the workflow is paused.

You can manually trigger a held task though. So for your use case I think cylc play --hold-after=P is the right thing to do (where P is earlier than the initial cycle point).

This does make me think we need a --hold-all option though, to start with all initial tasks held, without having to specify an early cycle point for “hold after”. I’ll flag this with the dev team…

Start paused or held, then manually trigger individual tasks · Issue #5912 · cylc/cylc-flow · GitHub

3 Likes

cylc pause does not “replace” cylc hold, this command still exists and isn’t going anywhere.

If you want the Cylc 7 style behaviour where you can hold back all tasks in the workflow and manually trigger the ones you want to run, you can still do it as before

# hold all tasks
cylc hold '<workflow>//*'

# trigger a task
cylc trigger '<workflow>//<cycle>/<task>'

# release all tasks at a later date
cylc release '<workflow>//*'

Note, you can hold/release all tasks from the GUI by clicking on any task and pressing the pencil icon to the right of the hold/release command. This brings up the command editor from which you can change the task from the one selected to *.

2 Likes

@Diquan, there’s one caveat with what @oliver.sanders suggested, which users have been known to find confusing.

Wildcards (such as in cylc hold <workflow>//* only match tasks in the scheduler’s active window, not future tasks beyond the window (in cycling workflows there could be an infinite number of future matches).

So if the workflow is already running (i.e., not paused), cylc hold <worfklow>//* might appear to do exactly nothing. A held running task won’t submit retries but the already-running job can still satisfy the prerequisites of downstream tasks that have not been held, in which case those tasks will run.

For your use case, using the hold-after option might be better, because every task will be held based entirely on cycle point.

Another option:

  • start the workflow paused: cylc play --pause <wf>
    • this results in all initial tasks, in the active window, staying in the waiting state
  • then do cylc hold <wf>//* to hold those waiting tasks
  • then un-pause the workflow: cylc play <wf>
  • now you can release held tasks, individually
    • (but note again that future tasks have not been held by the wildcard command - you may need to hold those individually if you don’t want them to run automatically downstream of released tasks).
1 Like
 # Hold all tasks after cycle point 1234 in my_workflow:
  $ cylc hold my_workflow// --after=1234

Could the documentation around this be clarified. Is it inclusive or only after? I could read this two ways, that cycle 1234-> is held, or that cycle 1235-> is held. I’m guessing the former, but it would be clearer to say in the documentation that it is 1234 onwards held.

No, it’s as per the literal meaning of “after” - i.e., not inclusive.

(And I just did a quick test run to confirm this)

But I suppose it wouldn’t hurt to say that in the docs anyway. (Are you volunteering?)