Good afternoon,
I have a workflow where I would like to run a task twice. It could be simplified as:
[[ graph ]]
#
# Run just once at the initial cycle point
#
R1 = """
build_model => init_vars => copy_data_init
"""
#
# Run at each cycle point
#
T00 = """
copy_data_init => init_vars => run_model
"""
The task init_vars initializes some variables that I need both for copying the initialization data, and for running the model, and these variables depend on the cycle point, so they need to be recalculated each day at midnight.
Of course I get an error
WorkflowConfigError: circular edges detected
Is there a way of telling cylc that’s not a circular dependency? (I feel like this problem has a very easy solution, but somehow I can’t see it …)
Thanks a lot for your help in advance,
Stella
Hi.
R1 means, “run once at the initial cycle point”.
T00 means, “run every day at midnight from the initial cycle point onwards”.
In your case, I suspect that the initial cycle point is at midnight. As a result the midnight cycle contains both the R1 and T00 graphs, i.e:
# R1
build_model => init_vars => copy_data_init
# T00
copy_data_init => init_vars => run_model
Which is a cyclic graph because init_vars must run both before and after copy_data_init.
I think you probably want to exclude the initial cycle point from the T00 graph like so:
[scheduling]
[[graph]]
# Run just once at the initial cycle point
R1 = """
build_model => init_vars => copy_data_init
"""
# Run every day at midnight (except in the first cycle)
T00 ! R1 = """
copy_data_init => init_vars => run_model
"""
Otherwise, if there is a requirement to run the same task twice within the same cycle, you can use an alias to copy the task like so:
[scheduling]
[[graph]]
# Run just once at the initial cycle point
R1 = """
build_model => init_vars_startup => copy_data_init
"""
# Run every day at midnight
T00 = """
copy_data_init => init_vars => run_model
"""
[runtime]
# define the task once
[[INIT_VARS]]
script = ...
# but make two copies
[[init_vars]]
inherit = INIT_VARS
[[init_vars_startup]]
inherit = INIT_VARS
Hi Olivier,
If I understand correctly with the first solution, the “run_model” will be executed starting from the second cycle point, and skipped at the initial one? Is this equivalent to put ! ^ ?
This is not what I want, so the second solution is what I need.
Thanks a lot for your help!
Stella
I was anyway trying to implement the first approach, but for some reason I can’t make it work …
In reality my workflow does not start at midnight, I have a slightly more complicated recurrence
[scheduling]
initial cycle point = '20241111'
final cycle point = '20241124'
[[graph]]
R1 = """
init_vars => copy_data_init
"""
R/2024-W01-1/P7D ! R1 = """
copy_data_init => init_vars => run_model
"""
And with this I still get the circular graph error. I don’t understand, as soon as I change R/2024-W01-1/P7D → T00, it works.
Is there any problem with the way I specify my recurrence expression?
Hi @sparonuz
I think the fact that your recurrence expression starts before the initial cycle point of the workflow is causing some kind of a problem with recurrence processing.
We’ll need to investigate that, but excluding the initial point specifically seems to work fine:
R/2024-W01-1/P7D ! ^
It also works if you start the P7D recurrence at the initial point, and exclude R1:
R/^/P7D ! R1 # or just "P7D ! R1"
(Then you have to make sure the initial cycle point falls on the right week day, but you’re already doing that for the R1 recurrence).
Or figure out which week corresponds to your initial cycle point and start the recurrence there:
R/2024-W46-1/P7D ! R1
Or start it the following week so you don’t need the exclusion:
R/2024-W47-1/P7D
1 Like
Thanks @hilary.j.oliver !
You are right, as soon as I change my start point to
[scheduling]
initial cycle point = '20240101'
final cycle point = '20240114'
It validates the worflow without errors.
The problem is that we where searching a way of specifying a workflow running each Monday (or another specific day of the week), and this was the suggestion given in a previous discussion. And we would like to have a recurrence expression independent from the initial cycle point (so, not change the week to 46 if the initial cycle point is the 11 November).
Implementing your first workaround avoids the cycling graph error, but still the workflow doesn’t seems to run correctly. My worflow is now:
[scheduling]
initial cycle point = '20241111'
final cycle point = '20241118'
[[graph]]
R1 = """
init_vars => copy_data_init
"""
R/2024-W01-1/P7D ! ^ = """
FAMBUILD[^]:succeed-all => init_vars => run_model
"""
[runtime]
[[FAMBUILD]]
[[FAMRUN]]
[[ init_vars ]]
platform = belenos_login
script = """
echo "task init vars"
"""
[[copy_data_init]]
platform = belenos_login
script = """
echo "task copy_data_init"
"""
inherit = FAMBUILD
[[run_model]]
platform = belenos_login
script = """
echo "task run_model"
"""
inherit = FAMRUN
And the graph shows the workflow as intended
But when I play the workflow, first in the tui it shows a point that is not in the graph (20240101)
And then the flow gets stuck at the FAMBUILD, it never triggers the 2024/11/18 run.
Am I using Families in the wrong way?Or is still the same problem as before (i.e. the recurrence expression starts before the 1st cycle point)?
Many thanks in advance for your help!
Best,
Stella
Hi Stella,
A quick test shows the problem occurs even if you don’t use the family name in the graph, so it’s not that.
The initial cycle point defines the start of the graph, so it wouldn’t be unreasonable for dependence on (non-existent) tasks prior to that to cause a stall. But Cylc is supposed to ignore pre-initial-point tasks as a convenience, to make writing the start of the graph easier, so it should work.
Your example runs fine so long as the recurrence start point is >= initial point. So until we figure out what’s happening here, I think you’ll have to do that.
And we would like to have a recurrence expression independent from the initial cycle point (so, not change the week to 46 if the initial cycle point is the 11 November).
Is there some reason why you can’t just do this:
R/^/P7D ! ^
This requires making sure that the initial cycle point is the first day of a week, but you have to do that for the R1 expression anyway.
1 Like
Hi @hilary.j.oliver
Yes, you’re right, I will need to specify that anyway, so I can simplify my recurrence expression!
Thanks a lot,
Stella
