There are many tasks to run dependently or independently at a cycle point in my suite.
They run in different speed, some finishes early, some slower.
I 'd like to write a semaphore file after all the tasks are finished.
I had a solution was to do Done task creating semaphore file after all jobs succeeded, like
FAMILY1:succeed-all => Done
FAMILY2:succeed-all => Done
But this would cause a hefty graph, which is slow to load/debug and looks messy.
Is there a way in cylc to know after all the tasks of a cycle point are finished?
and after that a task/event handler can be triggered to write semaphore file?
Cycle point is task-specific, not global, in Cylc so it’s not so easy to know “when a cycle is finished”, as you’ve discovered.
There are several ways you could do it. E.g. (off the top of my head):
- define a family that contains all tasks in the cycle sequence of interest, and trigger a “cycle_done” task off them all finishing (essentially what you’ve just described)
- the downside of this is a mess of extra dependencies, as you’ve noted
- constrain your suite to
max active cycle points = 1 which forces each cycle to finish before the next one starts
- the downside of this is: no multi-cycle fast catch-up - but maybe that doesn’t matter to you
- put a task in the workflow that repeatedly checks (perhaps via automatic retries) to see if all tasks in the cycle have finished yet
- you can use
cylc list -p POINT,POINT to list all the tasks that can appear in the current cycle point, and then
cylc suite-state -p POINT to query the suite database for all task states recorded for POINT … and finally return success (and write your semaphore file) when they all show up finished.
(Note for POINT use
$CYLC_TASK_CYCLE_POINT in task job scripts).
The last option might be a good choice?
Yes, last option sounds good for me.
I already have a task Done.
I will add automatic retries to check the status of all tasks excluding Done itself.