Is it possible to trigger tasks from a previously completed job cycle?

Cylc Cylc Support
1

I’m a new Cylc user, so thanks for taking some time to answer a basic usage question.

I primarily rely on the GUI to view my tasks as they are running. By right clicking a job, I can trigger it to “run now”. I needed to make an edit to fix a bug in a particular task, and now I want to rerun (trigger) that job’s task for all the cycles that have already completed, without starting the suite over from the beginning. But in the GUI, when a cycle completes and is inactive, it is removed from the list and I can’t “trigger” the job again.

I tried using the command line to trigger the specific job,

cylc trigger <suite_name> <task_name>/<task_job_name>

but that didn’t start running the tasks I tried to trigger.

Is what I’m trying to do possible? Is it possible to trigger jobs from previously completed cycles, or do I need to stop and start the suite from the beginning?

2

Hi @blaylockbk,

Actually your question is not quite as basic as you might hope. Cycling workflows are potentially infinite in extent so Cylc can’t be “aware of” the whole workflow at once. Instead it maintains a pool of “task proxies” as a moving window on the workflow, and finished tasks that are no longer needed (to satisfy the prerequisites of upcoming tasks) are forgotten as the workflow moves on. If you want to re-run one of those, you have to manually “insert” the corresponding task proxy back into the pool before you can trigger it (and it will require manual triggering if its upstream tasks have also been forgotten).

By default an inserted task will spawn its own successor (same task, next cycle point) as normal - which is what you want (to run multiple instances of that same task at different cycle points) - but spawning only happens at job submit time, so you will have to trigger them one by one.

From your description you only want to run the inserted task at each cycle, not the tasks downstream of it in the graph as well - right? I suggest you try it in a simple test workflow like this:

# register as "toast"
[scheduling]
   cycling mode = integer
   initial cycle point = 1
   [[dependencies]]
       [[[P1]]]
           graph = "foo[-P1] => foo => bar & baz"
[runtime]
   [[root]]
       script = sleep 20

Let this run on to cycle point 8 or so, then hold the workflow and insert tasks baz.2:

cylc insert toast baz.2  # or via the GUI

Then trigger baz.2 (CLI or GUI) to run it and spawn its next instance, then trigger that, and so on until baz catches up to the main window again.

Alternatively, note that this does not really require a workflow engine as you are only running a series of single tasks, disregarding dependencies etc. So you could just write a short script to run each one in succession with the cylc submit command - which runs tasks separately, outside of the the workflow engine. The job logs will still end up in the same place.

Hope that helps!

2 Likes
3

(p.s. I should add, somewhat apologetically, that users should not really have to know about the task pool and insertion - we have had to expose an aspect of internal implementation here, because infinite cycling workflows are difficult to manage … But, we are working on ways to make this transparent to users in the future)

1 Like