Can I use the suite_state xtrigger without a message? I have been trying the example in 12.2 in the documentation. As far as I can tell, the cylc message and the output, x, need to be the same as the message in suite_state for it to work. I tried setting the message to None but it didn’t succeed in this case. How can I get it to trigger just on the successful completion of the task?
By removing the message variable I managed to make this work but when I had ‘message=None’ or ‘message=none’ in the suite_state function it didn’t work…
Another question about suite_state usage. I am trying to pass some environment variables into it. i.e
suite_state(suite = {SUITE_NAME}, task = {TASK_NAME}… etc
But these do not seem to be being evaluated - i.e. the downstream task waits indefinitely and when I look at the prerequisites the environment variables have not been evaluated and are there as ${SUITE_NAME} etc. What is the best way to past environment variables (I could do Jinja too…) to the function?
Hi Emily,
Sorry for missing these questions last week (I think many of us were adjusting to the new “locked down” reality).
Your questions are good ones, I can see we need to document this functionality more clearly (sorry again!). I’ll try to explain. Feel free to ask more if it’s still not clear.
The “suite state function signature” shown in the user guide is the actual Python function definition signature. Arguments like offset=None default to None (not defined) if you don’t pass anything in for them. However, what’s probably not clear is the difference between passing arguments to the function from within a Python program, and from the Cylc suite configuration file. In the latter case, the Cylc file parser extracts the argument strings, without any knowledge of Python, from the function signature in your suite.rc, and passes them verbatim to the underlying Python function. The “no knowledge of Python” bit means that if you put message=None in your functional call signature it will be interpreted literally as the string “None” not the Python value None. We will document this better!
Relevant to other parts of your questions:
- You can’t use shell environment variables for task name etc. in these function arguments because the shell never sees any of this. The suite.rc file is a configuration file parsed by the Python Cylc scheduler program, which calls the
suite_statePython function with the arguments you listed. (Environment variables like$CYLC_TASK_NAMEare written to Cylc-defined to task job scripts, but that’s because jobs are executed as shell scripts). - Jinja2 can be used anywhere throughout the suite config file, but it is only a preprocessor, used at start-up before the suite starts running, so you can’t pass dynamic information like a particular task name or cycle point in that way. Jinja2 is really a way to programmatically construct the static suite configuration used to initialized the scheduler.
Long story short,
- to trigger off of upstream task success only do not use the
messageargument at all (that is for custom output messages), just usestatus=succeeded. - for the upstream suite and task name use a literal string (the actual names)
- if the upstream cycle point is related to the cycle point of the dependent task in your suite (presumably it is) use Python-like string templating as described (but not well enough!) in the user guide:
%(point)sgets replaced with the cycle point of the dependent task. Andoffset=PT6H(say) if the upstream cycle point is offset from the local dependent task’s cycle point.
All of the available string templates are described a little lower down in the user guide chapter, under “custom xtrigger functions”: https://cylc.github.io/doc/built-sphinx-single/index.html#custom-trigger-functions (in the current user guide). However, note that they all refer to values in the suite that calls the the xtrigger function. So in the suite_state (inter-suite triggering) case, they are the downstream (dependent) suite values, and the only relevant template argument for that xtrigger function is %(point)s.
Hilary
(Or Jinja2 variables that contain those static names, if you like).
Another question about suite_state function usage:
One of the suites that I want to use the xtrigger to poll off has a different format for the cycle_point_format than the downstream suite (and indeed other suites that I am polling). Can I redefine this format (or make a new string template like %(point)s ) to feed in the cycle point in the correct format? How would I do this?
Hi @EmilyLane,
The cycle point should be converted automatically to the format used by the target suite (the target suite database, which the xtrigger function looks at, knows the format). So it should just work - have you tried it and found a problem?
Hilary
What seems to be the problem is if I specify the cycle point format under [cylc]. If I do that then %(point)s seems to be hardwired to that one. When I comment it out it doesn’t matter that there is a difference in the format.
The upstream suite has the format %y%m%dT%h%MZ for the cycle point
The following hangs indefinitely:
[cylc]
cycle point format = %Y%m%dT%H%M
[scheduling]
initial cycle point = 20200505T0300
final cycle point = 20200507T0300
[[xtriggers]]
upstream = suite_state(suite=u-ak198, task=fcst_done, point=%(point)s, cylc_run_dir=/home/mooresa/cylc-run):PT10S
[[dependencies]]
[[[PT12H]]]
graph = “”"
@upstream => blam
“”"
[runtime]
[[root]]
script = sleep 5
[[blam]]
script = “”" echo Triggered “”"
Whereas if you delete the cycle point format line or change it to be consistent with the upstream format it works
Not sure if this is what it is supposed to do or not…
Actually I think I might have found the problem. The upstream suite didn’t specify a cycle point format explicitly. In this case it seems to revert to the format of the downstream suite.
Hi Emily,
You’re right, I’ve just confirmed that the cycle point format conversion is only done if the upstream suite sets its explicitly.
Thanks for the excellent detective work and bug report!
Do you have control over the upstream suite (so that you can change it to set the format)?
Hilary
New issue posted to track the fix: https://github.com/cylc/cylc-flow/issues/3600
No but Stuart Moore, who owns that suite, changed it for me; so all good.
(Fix merged for next release).