# Parametrised cycle durations: consistency cf next(\[...\])
**URL:** https://cylc.discourse.group/t/parametrised-cycle-durations-consistency-cf-next/377
**Category:** Cylc Support
**Created:** [September 13, 2021, 1:11pm UTC](https://cylc.discourse.group/t/parametrised-cycle-durations-consistency-cf-next/377 "2021-09-13T13:11:31Z")
**Posts on this page:** 8
**Page:** 1
### Author:  [@edmundh](https://cylc.discourse.group/u/edmundh)
#### Post date: [September 13, 2021, 1:11pm UTC](https://cylc.discourse.group/t/parametrised-cycle-durations-consistency-cf-next/377/1 "2021-09-13T13:11:31Z")
When developing to-become-operational suites, what do people do regarding any cycle duration wallclock limit caps on queues for non-operational work?
I’ve been happily developing a suite intending to target a 6 hour cycle - I’m now getting to stage where I’d like to match the target operational cycle length in some of my tests. I’ve just realised local caps mean this value isn’t catered for in suite dev/testing - I can only go up to 3 hours.
I think I could have options including:
- Aim for a shorter operational cycle duration, which fits within acceptable dev durations (e.g. 3h)
- Can do, but would prefer 6H for archiving retrieval - this is a nowcast model, not forecast, so want to minimise archive retrievals needed to reconstruct events
- Try to parametrise run duration, such that I have Jinja magic letting me switch durations depending on context (3h for dev, changing to suck-it-and-see/pray 6h works when I throw it over to the operational team)
What do people with 6h cycling operational suites (e.g. UM, no?) do?
If it’s parametrised run duration, does anyone have a handy link to a suite implementing the required Jinja incantations - I’m particularly keen to see what people do about normalising (correct term?) start times wrt a presumably ~random start time - going from `initial cycle point = now` behaviour to `initial cycle point = next([...])` behaviour.
For the actual cycle duration, I can see I can ~happily use Jinja to achieve sth like @funkapus’ [parametrised run duration/cadence](https://cylc.discourse.group/t/using-a-jinja2-variable-in-an-include-statement/363).
But I’m less clear if/how I can link this cadence up to the corresponding next() incantation like [Matt’s example here](https://cylc.discourse.group/t/wall-clock-synchronization-real-time-scheduling/58/2).
Specifically, I can’t see if there’s a way to automagically generate:
- `next(T00, T06, T12,T18)` from `CYCLE_DURATION="PT6H"`
- `next(T00, T03, ..., T21)` from `CYCLE_DURATION="PT3H"`
- … other next variants from other durations, with something to ensure this folds into 24H with no remainder
Is this technically achievable? E.g. via a cunning call in Jinja of [`cylc cyclepoint` as discussed here](https://cylc.discourse.group/t/arithmetic-with-cylc-task-cycle-point-in-standard-date-time-format/81/5) by @hilary.j.oliver & @jonnyhtw ? Has anyone got a shareable example of doing this?
Not to worry if not possible - just trying to avoid my other non-DRY solution: defining / maintaining consistency between two implicitly linked Jinja vars, a la
- `CYCLE_DURATION="PT3H"`
- `CYCLE_NEXT="next(T00, T03, ..., T21)"`
MO people: I’m linking to this q on our internal channels in case there’s a specific internal answer to this!
_Extra brownie points for examples which do duration arithmetic - if my cycle nominally lasts PT6H, I’d like to be able to define padded execution time limits, e.g. PADDED = (110% \* NOMINAL) + PAD\_C, with padding “slope” (110%) and offset being fine-tuneable once I understand how my model performs at different durations._
Thanks,
Edmund
---
### Author:  [@oliver.sanders](https://cylc.discourse.group/u/oliver.sanders)
#### Post date: [September 13, 2021, 3:20pm UTC](https://cylc.discourse.group/t/parametrised-cycle-durations-consistency-cf-next/377/2 "2021-09-13T15:20:23Z")
> Specifically, I can’t see if there’s a way to automagically generate:
Here’s a Jinja2 filter that can convert a duration (e.g. `PT6H`) into times of the day (e.g. `T00`, `T06`, `T12`, `T18`).
```python
from isodatetime.data import Duration, TimePoint
from isodatetime.parsers import TimeRecurrenceParser, TimePointParser, DurationParser
def duration_to_time(duration, offset='00'):
"""Generates times of the day from a duration.
Works with durations that are shorter than one day.
Arguments:
duration (str):
The duration as a string e.g. "PT3H".
offset (str):
The offset from which to count,
(i.e. the earliest time to consider).
e.g. "00" (midnight, the default)
Returns:
str
Examples:
# every six hours starting at T00
>>> list(duration_to_time('PT6H'))
['T0000', 'T0600', 'T1200', 'T1800']
# every six hours starting at T0030
>>> list(duration_to_time('PT6H', '0030'))
['T0030', 'T0630', 'T1230', 'T1830']
"""
tp1 = TimePointParser().parse('20000101T' + offset + 'Z')
tp2 = TimePointParser().parse('20000102T00Z')
dur = DurationParser().parse(duration)
while tp1 < tp2:
yield 'T%02d%02d' % (tp1.hour_of_day, tp1.minute_of_hour)
tp1 = tp1 + dur
```
---
### Author:  [@edmundh](https://cylc.discourse.group/u/edmundh)
#### Post date: [September 13, 2021, 3:55pm UTC](https://cylc.discourse.group/t/parametrised-cycle-durations-consistency-cf-next/377/3 "2021-09-13T15:55:32Z")
Fantastic, thanks Oliver!
I’ve not come across Jinja2 filters before - trying to mug up on them now as it’s looking like this isodatetime library can likely solve my secondary question about duration arithmetic for setting padded execution time limits.
Can I check:
- Will this work with Cylc7 (I need this in an operational suite context, PS45)?
- Does cylc give me access to the isodatetime library at runtime?
- I see I can’t simply do an `import isodatetime` successfully from within the Met Office’s standard “scientific software stack” (at least the default version - assume same for operational flavours?)
- I see I can [install isodatetime from conda](https://anaconda.org/conda-forge/metomi-isodatetime) if I want to extend my own environment for Jinja filter development purposes - sadly though I can’t use my own env in operations
- Guessing/hoping cylc/rose must rely on this library, so I can safely rely on this being available to my custom Jinja2 filters at runtime?
- If I’ve written one of these Jinja2 filters for my suite, and want to put this in a file (rather than inlining in suite.rc), where should I put this?
- From the [Cylc8 docs on custom Jinja2 filters](https://cylc.github.io/cylc-doc/8.0b0/html/user-guide/writing-suites/jinja2.html#custom-jinja2-filters-tests-and-globals) I can infer the intended filename, but not the path\*
- Is the path `${my-suite}/bin`, `${my-suite}/lib/python` - or somewhere else?
\* I think there may be something which has gone missing from the Cylc8 docs at that point:
> Cylc also supports custom Jinja2 globals, filters and tests. A custom global, filter or test is a single Python function in a source file with the same name as the function (plus `.py` extension) and stored in one of the following locations:
>
> In the argument list[…]
I.e. I think there’s a set of missing locations which were intended to be listed between
> […] locations:
and
> In the argument list[…]
---
### Author:  [@edmundh](https://cylc.discourse.group/u/edmundh)
#### Post date: [September 13, 2021, 4:17pm UTC](https://cylc.discourse.group/t/parametrised-cycle-durations-consistency-cf-next/377/4 "2021-09-13T16:17:32Z")
Aha - re path, think found the relevant sections of Cylc7 docs which suggests it’s `${my-suite}/lib/python/` _(or poss `${my-suite}/lib/Jinja2Filters/` - only former [mentioned in Cylc8 equivalent to 2.](https://cylc.github.io/cylc-doc/8.0b0/html/user-guide/writing-suites/configuration.html#suite-configuration-directories) though, so assume preferred)_:
1. [Cylc7: Custom jinja2 filters](https://cylc.github.io/cylc-doc/7.9.3/singlehtml/index.html#customjinja2filters)
2. [Cylc7: Suite configuration dirs](https://cylc.github.io/cylc-doc/7.9.3/singlehtml/index.html#suite-configuration-directories)
---
### Author:  [@hilary.j.oliver](https://cylc.discourse.group/u/hilary.j.oliver)
#### Post date: [September 13, 2021, 11:56pm UTC](https://cylc.discourse.group/t/parametrised-cycle-durations-consistency-cf-next/377/5 "2021-09-13T23:56:17Z")
> Can I check:
>
> - Will this work with Cylc7 (I need this in an operational suite context, PS45)?
Yes
> - Does cylc give me access to the isodatetime library at runtime?
Yes
> …
>
> - Guessing/hoping cylc/rose must rely on this library, so I can safely rely on this being available to my custom Jinja2 filters at runtime?
👍
> - If I’ve written one of these Jinja2 filters for my suite, and want to put this in a file (rather than inlining in suite.rc), where should I put this?..
> I.e. I think there’s a set of missing locations which were intended to be listed between …
You’re right about the Cylc 8 docs, which are still undergoing heavy revision, but the Cylc 7 docs have it (as I think you’ve noted in your follow-up post):
> > Cylc also supports custom Jinja2 globals, filters and tests. A custom global, filter or test is a single Python function in a source file with the same name as the function (plus “.py” extension) and stored in one of the following locations:
> >
> > - `/lib/Jinja2[namespace]/`
> > - `[suite configuration directory]/Jinja2[namespace]/`
> > - `$HOME/.cylc/Jinja2[namespace]/`
> >
> > where `[namespace]/` is one of `Globals/` , `Filters/` or `Tests/` .
---
### Author:  [@oliver.sanders](https://cylc.discourse.group/u/oliver.sanders)
#### Post date: [September 14, 2021, 8:49am UTC](https://cylc.discourse.group/t/parametrised-cycle-durations-consistency-cf-next/377/6 "2021-09-14T08:49:43Z")
Here’s an example integrated into a toy suite:
> **[cylc-examples/jinja2/duration\_to\_time at master · oliver-sanders/cylc-examples](https://github.com/oliver-sanders/cylc-examples/tree/master/jinja2/duration_to_time)**
>
> master/jinja2/duration\_to\_time
---
### Author:  [@edmundh](https://cylc.discourse.group/u/edmundh)
#### Post date: [September 14, 2021, 12:17pm UTC](https://cylc.discourse.group/t/parametrised-cycle-durations-consistency-cf-next/377/7 "2021-09-14T12:17:23Z")
Thanks _very_ much Oliver for creating this - FWIW, as a user, toy suites like this / ones @wxtim has produced are _super_-useful for the noobier/dimmer amongst us for making link between description in documentation and how one can implement things.
And nicely runnable to boot - _documplementation?_
---
### Author:  [@wxtim](https://cylc.discourse.group/u/wxtim)
#### Post date: [September 20, 2021, 7:55am UTC](https://cylc.discourse.group/t/parametrised-cycle-durations-consistency-cf-next/377/8 "2021-09-20T07:55:31Z")
I doubt anyone here should be described as “dimmer”, and I dislike the term “noob” because everyone has to start somewhere.
But
Some of those giving customer support still need to write toy examples to be sure that they give sensible advice. 🙂