Using Cylc 8.4.2
PBS batch
I’m having a little bit tough time understanding the new graph branching for Cylc 8 that not require suicide triggers (without turn on backward compatibility) due to English is not my primary language.
Short recap:
I want “task1:messRUN” to run if cylc message said run. However, I want to tell cylc to forget about “run_suite” if cylc message said done and no need to run task1:messRUN.
Hence:
task1:messRUN => run_suite
task1:messDONE => !run_suite
How do I translate these graph to CYLC 8?
task1:messRUN => run_suite
task1:messDONE? => run_suite
or
task1:messRUN | task1:messDONE? => run_suite
The problem is that first part won’t run run_suite even though the message sent run. The latter, I don’t think that is what I’m looking for?
How do I tell Cylc to forget about run_suite and no need to have it run?
In case you asked:
[[task1]]
script = ....
....
[[[directives]]]
....
[[[outputs]]]
messRUN = "run"
messDONE = "done"
cylc message triggered from the shell script such as cylc message -- ${CYLC_WORK_ID}//${CYLC_TASK_NAME} "run"
Something like that. With your help will be greatly appreciated.
Hi @hollabigj
How do I tell Cylc to forget about run_suite and no need to have it run?
With Cylc 8, if run_suite’s prerequisites get satisfied, we run it; if not, we don’t run it - there’s no need to “tell Cylc to forget about” it.
So, if I’ve understood your requirements there’s no need for the “messDONE” output anymore - all you need is this.
task1:messRUN? => run_suite
This says:
- if the “messRUN” output is completed (by the associated message), trigger
run_suite to run
- the “messRUN” output is optional (by the
? character), so task1 is allowed to finish without completing it
Sorry for confusion since it was written in hurry.
I mean it works for Cylc 7 but suicide trigger was removed in non-backward compatibility.
messRUN starts the run_suite and keep running. It wakes up, check things that need to digest the data as in two options:
messRUN:
- check things that need to digest data
a. YES: run run_suite. After digested data done, tell messDONE I’m done, and sleep.
b. NO, sleep and tell messDONE not to run run_suite.
It wakes up every 10 minutes and check files that need to process. If nothing to process, it tells messDONE no need to run “run_suite” aka suicide trigger.
messDONE:
- waiting for status from messRUN
a. NO, don’t trigger suicide
b. YES, trigger suicide
I was thinking maybe this part?
task1:messRUN => run_suite
task1:messDONE => run_suite:finished
messRUN to tell messDONE no need to process run_suite → finish as suicide trigger?
I saw one was “cylc remove” but the documentation cautious that it will cause workflow instability or atrophy (the reason suicide trigger was not required and removed later release).
Or your approach is the solution if you think this is correct way to do so? We definitely needs to kill the task process aka suicide trigger.
I notice people were stumped about suicide trigger was removed and asked in this help forum. It’s starting to be more FAQ.
I’m having trouble understanding your description, sorry.
For my initial response, above, I assumed the following:
task1 completes output messRUN if task run_suite should run- OR
task1 completes output messDONE if run_suite should NOT run
If that’s correct, then:
- you DO need a suicide trigger in Cylc 7
- (
run_suite gets spawned ahead, before we know if it is needed or not, so a suicide trigger must remove it if it turns out not to be needed)
- you DO NOT need a suicide trigger in Cylc 8
- (
run_suite will only be spawned if it needs to run)
Have I misunderstood the scenario?
Is it that you always run run_suite but sometimes need to kill it after it starts running??
I notice people were stumped about suicide trigger was removed
Suicide triggers have not been removed, they are still supported, but they are rarely needed in Cylc 8. In particular, they are not needed for optional branching scenarios - which was by far their main use case in Cylc 7.
Ok, I think I get it. Sorry for confusion earlier. Right, they don’t want backward compatibility for Cylc 8 (without rely on Cylc 7).
Perhaps this might work?
task1:messRUN? => run_suite
that would be similar to:
task1:messRUN => run_suite
task1:messDONE >= !run_suite
If there is nothing to digest data, it will not send messRUN message (I think, I need to look at code again). Perhaps, looks like I need to re-write the external script such as shell script, python, etc. (it was inherit from someone else, not mine) to rely on messRUN since messDONE sounds like not necessary to have it. I can re-factor that and remove messDONE to conform Cylc 8.
Like you said no need to tell it to forget about it. I’ll talk to the person tomorrow to suggest the change.
With your suggestion or correction would be greatly appreciated. I appreciate all of your help and created a great product, keep up with a great work 
1 Like
Yes, I think we’re on the same page now. If I have correctly understood your scenario, all you need is this:
task1:messRUN? => run_suite
(Which works as I described earlier).
You probably don’t even need to refactor the external task script. The messDONE message can still be sent by the script even if it is not used anymore by the workflow. The scheduler will just log it.
Yup that you mentioned earlier which is super helpful.
Thanks, I appreciate it. If there’s any issue pops up or regard to this, I’ll circle back later.
I have one question (off-subject): can cylc do anything without orchestrate all climate, weather, and weather-driven forecasting workflows.
Like for example, let’s say I wanted to set up a workflow for network sweeper, detection, packet sniffer, etc. Would Cylc performs these task other than weather?
The reason I asked because I found Cylc workflow can be very useful for other things.
Yes, Cylc is actually not specialized to weather and climate at all!
The fact that Cylc can do date-time cycling in a uniquely efficient way (no barrier between cycles) makes it very useful for those workflows, but it can do non-cycling and integer-cycling flows just as well.
1 Like