How to know all tasks of a cycle point finished

Cylc Cylc Support
1

Hi,

There are many tasks to run dependently or independently at a cycle point in my suite.
They run in different speed, some finishes early, some slower.
I 'd like to write a semaphore file after all the tasks are finished.

I had a solution was to do Done task creating semaphore file after all jobs succeeded, like
FAMILY1:succeed-all => Done
FAMILY2:succeed-all => Done

But this would cause a hefty graph, which is slow to load/debug and looks messy.

Is there a way in cylc to know after all the tasks of a cycle point are finished?
and after that a task/event handler can be triggered to write semaphore file?

Thanks
Hailin

2

Hi Hailin,

Cycle point is task-specific, not global, in Cylc so it’s not so easy to know “when a cycle is finished”, as you’ve discovered.

There are several ways you could do it. E.g. (off the top of my head):

  • define a family that contains all tasks in the cycle sequence of interest, and trigger a “cycle_done” task off them all finishing (essentially what you’ve just described)
    • the downside of this is a mess of extra dependencies, as you’ve noted
  • constrain your suite to max active cycle points = 1 which forces each cycle to finish before the next one starts
    • the downside of this is: no multi-cycle fast catch-up - but maybe that doesn’t matter to you
  • put a task in the workflow that repeatedly checks (perhaps via automatic retries) to see if all tasks in the cycle have finished yet
    • you can use cylc list -p POINT,POINT to list all the tasks that can appear in the current cycle point, and then cylc suite-state -p POINT to query the suite database for all task states recorded for POINT … and finally return success (and write your semaphore file) when they all show up finished.

(Note for POINT use $CYLC_TASK_CYCLE_POINT in task job scripts).

The last option might be a good choice?

Hilary

3

Hi Hilary,
Yes, last option sounds good for me.
I already have a task Done.
I will add automatic retries to check the status of all tasks excluding Done itself.

Thanks
Hailin