When developing to-become-operational suites, what do people do regarding any cycle duration wallclock limit caps on queues for non-operational work?
I’ve been happily developing a suite intending to target a 6 hour cycle - I’m now getting to stage where I’d like to match the target operational cycle length in some of my tests. I’ve just realised local caps mean this value isn’t catered for in suite dev/testing - I can only go up to 3 hours.
I think I could have options including:
- Aim for a shorter operational cycle duration, which fits within acceptable dev durations (e.g. 3h)
- Can do, but would prefer 6H for archiving retrieval - this is a nowcast model, not forecast, so want to minimise archive retrievals needed to reconstruct events
- Try to parametrise run duration, such that I have Jinja magic letting me switch durations depending on context (3h for dev, changing to suck-it-and-see/pray 6h works when I throw it over to the operational team)
What do people with 6h cycling operational suites (e.g. UM, no?) do?
If it’s parametrised run duration, does anyone have a handy link to a suite implementing the required Jinja incantations - I’m particularly keen to see what people do about normalising (correct term?) start times wrt a presumably ~random start time - going from
initial cycle point = now behaviour to
initial cycle point = next([...]) behaviour.
But I’m less clear if/how I can link this cadence up to the corresponding next() incantation like Matt’s example here.
Specifically, I can’t see if there’s a way to automagically generate:
next(T00, T06, T12,T18)from
next(T00, T03, ..., T21)from
- … other next variants from other durations, with something to ensure this folds into 24H with no remainder
Not to worry if not possible - just trying to avoid my other non-DRY solution: defining / maintaining consistency between two implicitly linked Jinja vars, a la
CYCLE_NEXT="next(T00, T03, ..., T21)"
MO people: I’m linking to this q on our internal channels in case there’s a specific internal answer to this!
Extra brownie points for examples which do duration arithmetic - if my cycle nominally lasts PT6H, I’d like to be able to define padded execution time limits, e.g. PADDED = (110% * NOMINAL) + PAD_C, with padding “slope” (110%) and offset being fine-tuneable once I understand how my model performs at different durations.