I want to cycle through the year 1999 starting on the first Monday and running weekly on Monday.
I have the following in scheduling, the get_data script is run once in each cycle and the run and st_archive are ensembles of 10 members each. I’m confused by the dates cylc appears to cycle on: 19990103T1800-0619990109T1800-0619990116T1800-0619990123T1800-0619990130T1800-06 - why?
Firstly, the T1800-06 is because your UTC (Z) initial and final cycle points are getting translated to local time (UTC-06) because you haven’t set UTC mode = True in the suite (or else used local time throughout).
Secondly, to make an ongoing weekly cycle starting at the initial cycle point, the full and abbreviated expressions are:
R/^/P1W # full
R//P1W
P1W
The following expressions, however, make an ongoing weekly cycle up to the final cycle point (i.e. ending exactly on the final cycle point):
R/P1W/$ # full
P1W/$
R/P1W
So, I presume you meant to write R//P1W not R/P1W (a subtle difference I guess … this is powerful notation, but it has to be wielded carefully!). The two forms will generate the exact same sequence so long as the final cycle point is on-sequence - which in your case it isn’t, because 26 Dec 1999 was a Sunday, not a Monday.
Thank you. I understand this is powerful notation, but I have a hard time grasping it.
I have had a request to run for every Monday in August from 1999 to 2019 - can this be expressed in the suite.rc?
Fair enough, I have to look up the documentation plenty myself. But on the upside it can generate almost any conceivable regular date-time sequence, and is also “standards based” (ISO 8601 recurrences). In my opinion, it is harder to make mistakes if you stick to the non-abbreviated recurrence forms (in case that helps).
That’s an interesting one. Cylc’s graph recurrence notation can define any regular sequence, with optional exclusions that can also be regular sequences, but I can’t see how to express a multi-year weekly cycle that excludes all months but August. (Maybe someone else can).
If you have to, you could resort to a simple weekly Monday cycle R/W-1/P1W and tasks that share a few lines of scripting to simply succeed without doing anything if the cycle point month is not August.
A more elegant but more involved way is a yearly-cycling suite that runs a Mondays-in-August sub-suite for each year:
[cylc]
cycle point format = %Y
[scheduling]
initial cycle point = 1999
final cycle point = 2019
[[dependencies]]
[[[R1]]]
graph = prep => august
[[[P1Y]]]
graph = august[-P1Y] => august
[runtime]
[[prep]]
script = """rm -r $HOME/cylc-run/august
cylc register august $HOME/suites/august"""
[[august]]
script = cylc run --no-detach --set=YEAR=$CYLC_TASK_CYCLE_POINT august
And the sub-suite, in $HOME/suites/august/suite.rc:
#!Jinja2
[cylc]
cycle point format = %Y-%m-%d
[scheduling]
initial cycle point = {{YEAR}}-08-01
final cycle point = {{YEAR}}-08-31
[[dependencies]]
[[[R/W-1/P1W]]]
graph = foo
Note there is no overlap between successive runs of the sub-suite, so I just re-used the same sub-suite run directory for each year. Result, in the sub-suite run directory, on running the main suite:
Also, I always try to get users to experiment with trivial example workflows to see how these things work. Here’s the one I used for your first question:
[cylc]
cycle point format = %Y-%m-%d
[scheduling]
initial cycle point = 1999-01-04 # Monday
#final cycle point = 1999-12-26 # Sunday!!!
final cycle point = 1999-12-27 # Monday
[[dependencies]]
# from initial point:
#[[[R/^/P1W]]]
#[[[R//P1W]]]
#[[[P1W]]]
# to final point:
[[[R/P1W]]]
graph = foo
Then you can use cylc graph [START-POINT] [STOP-POINT] to see the generated cycle points, or cylc list --points=START,STOP, e.g.:
[drugs-and-money]$ cylc list --points=1999-01,2000-01 jim
foo.1999-01-04
foo.1999-01-11
foo.1999-01-18
...
foo.1999-12-13
foo.1999-12-20
foo.1999-12-27
Now Cylc sets the variable CYLC_TASK_PARAM_m=cat in task foo_cat; and it sets CYLC_TASK_PARAM_m=dog in task foo_dog; and so on. I.e. parameterized tasks can retrieve their own specific parameter value (or values, if multiple parameters are used) from their job environment.
In your example, what value were you expecting $CYLC_TASK_PARAM_mname have in the runtime job environment of task prep, which is not parameterized by mname?
Looking back to my example, the only month you asked for was august, so “august” was not a parameter value. To do the same for multiple months you’d presumably want to loop over the list of months and register the sub-suite for each one, inside the prep task. Only parameterized tasks get access to the values of the parameters they are constructed from, as I just described (and then, only to the specific parameter value(s) for the task, not the whole range(s)). I.e. we don’t automatically pass all parameter lists to all tasks, as that’s almost never needed. To do that, you need to initialize your parameters with a Jinja2 variable, and then you can do whatever you want with the variable throughout the suite.