Can someone please suggest on these two sub topics:
Suppose I have workflow like it installed in workflow/runname
A =>B =>C =>D1 =>D2 =>D3 , where D1-3 are the 3 instances of task D.
In order to play : cylc play workflow/runname , which runs all tasks.
Suppose task D2 failed. I fixed it, and I want to start from broken point..
What will be the syntax of “cylc play” say starting from task D2?
“cylc install” assumes that previous workflow/runname should not exist. I have to clean this directory, before I issue “cylc install”. Is there any way to force cylc install to overwrite previous workflow directory? If not, do we have “cylc clean” type of command to clean the previous workflow directory instead of “rm -rf”.
cylc play naturally restarts from wherever the workflow got to previously, but if there’s a failed task present you have tell the scheduler what to do about that. Either:
manually retrigger the failed task after fixing it (cylc trigger)
or carry on as if it succeeded, by manually setting outputs (cylc set).
The procedure is the same even if the workflow was stopped or killed: it will revive in the same state, with the failed task still present.
To be explicit, if D2 fails due to a fixable bug in it:
fix the bug (and install the fix with cylc reinstall, if it is in a workflow source file)
retrigger the failed task - it should now succeed so the flow can continue
(Note if the bug is in the flow.cylc rather than a script, and you’re intervening in a running workflow rather than restarting a stopped one, you also need to do cylc reload to tell the running scheduler to update it’s configuration from the altered flow.cylc.)
Note also we have shortcut commands for these processes:
cylc vip - validate, install, play
(run from scratch)
cylc vr - validate, reinstall, reload or play
(update a running workflow after changing the source directory)
It sounds like you are using an explicit run name (e.g. cylc install --run-name=blah). If so, to rerun your workflow from scratch you must either:
install to a new run name, for the new run
or clean out the old run directory, to reuse the old run name
It’s a bit easier to use the default numbered runs instead of run names. Then you can install new runs as you like and it will automatically increment the run number.
Either way (run names or numbers) previous runs still exist (and you can even continue running them) until you clean them up.
If not, do we have “cylc clean” type of command to clean the previous workflow directory instead of “rm -rf”.
That’s exactly what cylc clean does - it removes run directories. But it’s better than rm -rf because:
it checks that the target workflows are not running at the time
it automatically cleans up symlinked run sub-directories (i.e., if follows the symlinks)
it automatically cleans up remote run directories too, for distributed workflows
We don’t support this because it’s a dangerous practice - if you accidentally rerun a workflow from scratch in an existing run directory then you’ve wiped out the correct workflow state and contaminated its outputs.
If you really want to force it, you can manually remove the workflow .service directory, so that Cylc will not recognise it as an existing run directory anymore - not recommended!