As we migrate more and more of our suites to Cylc, it’s becoming more and more difficult to keep an eye on the suites to ensure they start when they should. This means the only way we can tell something is wrong is if the cycle points are behind the wall clock (depending on the suite). When we have dozens of suites with 10+ cycle points loaded in each, the gscan interface gets choppy and long (i.e., difficult to read).
A potential solution was to implement an email on late events but what we’re finding is the “late offset” setting doesn’t seem to be getting respected.
I have it set to PT12H, since most of our cycle points are every 12 hours, so we have T00/T12 graphs, but tasks will naturally run sometime in that 12 hour window depending on what triggers them. With a “late offset” of PT12H, I would expect tasks to not trigger the late event until the next 12 hour cycle, or am I misunderstanding the mechanic?
Right now, I have tasks in 20210121T1200Z kicking off late events but I wouldn’t expect them to do that unless they hadn’t run yet by 20210122T0000Z…
Does that make sense?