Best practice to force succeeded state in Cylc 8?

Cylc Cylc Support
1

In Cylc 7 you could Reset State → succeeded an individual job if you had a job you knew would never succeed and thus would allow current cycle to finish and advance to the next one.

In Cylc 8, I assumed the comparable method would be to use set-output to succeeded. This does trigger any children jobs to run as expected, but the job’s failed state persists in the graph thus keeping the cycle from completing and advancing.

What is the recommended method to move things along in Cylc 8 if I just want to move the workflow forward?

Thanks,
Steve

2

Hi @jaworsks

Yes, the persistent failed task needs some explanation.

In Cylc 7, to continue as if a task had succeeded when in fact it had failed, you had to force reset its state to succeeded, to make it look (to the scheduler) like it really had succeeded.

For Cylc 8’s event-driven scheduler you just tell it to continue as if the task succeeded by spawning the downstream tasks that depend on the “succeeded” output. And yes, that’s what cylc set-outputs does. (By default, it spawns downstream of the “succeeded” output).

Now to the failed task. In Cylc 8 “incomplete” tasks - which finished without completing “required” outputs (i.e., outputs not marked with ? as optional) - are retained in the scheduler’s active window to stall the workflow unless dealt with by the operator. They indicate something went wrong at runtime that the graph does not handle automatically (in this case, there is no optional graph branch to handle task failure). See Scheduling Configuration — Cylc 8.2.2 documentation (required and optional outputs).

As of 8.2.3 you have to deal with incomplete tasks like this:

  • fix the problem and retrigger to complete them naturally
    OR
  • remove them from the active window with cylc remove so they won’t stall the workflow (in which case you are taking responsibility for the effect on the workflow, of removing a task that did not complete its required outputs).

Note however, this will change at the upcoming 8.3.0 release: cylc set-outputs will be replaced by cylc set which can “set” prerequisites as well as outputs, and manually setting required outputs will automatically complete an incomplete task. So for your use case:

  • Cylc 8.2.3: use cylc set-outputs to continue as if the task had succeeded, and cylc remove to remove the incomplete task
  • Cylc 8.3.0 (coming soon): just use cylc set to continue AND complete the incomplete task
3

Oh, and in Cylc 8, if your task is expected to fail sometimes, and the failure can be handled automatically (which can mean doing nothing extra, if that’s appropriate), you can use optional outputs so that no manual intervention is required at all!

4

@hilary.j.oliver,

Thanks for the response. We had just recently discovered the new “?” annotation and have used in a few places where we can accept the failure with no intervention needed. It’s a great addition to the workflow.

In the case for these jobs, if it fails we want to hopefully be able to correct the issue and retrigger, but there are times where we just have to accept the failure and move one. We will continue with the set-output followed by remove for now and will look forward to Cylc 8.3

Do you have a rough estimate/goal date for Cylc 8.3 other than “soon” (understanding that an estimate isn’t a promise)?

5

At this point, I hope by the end of December.