With Cylc, workflows can be installed into a hierarchical structure. This can help you organise your workflows into groups.
Here’s a screenshot of the Cylc GUI showing workflows which have been installed hierarchically:
These groups can be expanded / collapsed to make the workflow list more manageable.
You still search and filter workflows when they are grouped in this way.
Workflow Indicators
The indicators help you to track the status of each workflow, hover over them for details:
When workflows are installed hierarchically, these states “bubble” up the tree giving you a combined status for the group:
This can assist with monitoring large numbers of workflows.
Workflow Management
Once your workflows are grouped, you can manage them as a “block”.
# stop all "testing" workflows:
cylc stop 'testing/*'
# re run all "failed" tasks in "improver" workflows:
cylc trigger 'improver/*:failed'
# change an environment variable for all tasks in all "az675" workflows:
cylc broadcast 'az675/*' -s '[environment]FOO=42'
# delete all stopped "testing" workflow installations:
cylc clean 'testing/*'
For more information on the “glob” format (the * symbol), run cylc help id.
You can also perform these “group” actions in the GUI.
When you bring up the command list, press the pencil icon next to the command you want to run:
Then edit the “workflows” list as required:
You can also edit the list of tasks in the same way. The glob format used to select workflows in the GUI is the same as the format used on the CLI.
How To Install Hierarchically
To change the ID that the workflow is installed under, use the --workflow-name (-n for short) when you install the workflow:
In this example the workflow will be installed with the ID testing/u/aa012/n96/run1:
$ cylc install -n testing/u/aa012/n96
INSTALLED testing/u/aa012/n96/run1 from ...
The workflow “run name” is also a part of this hierarchy. By default, cylc install adds run1, run2, run3, … onto the end of each workflow installation. But you can customise this using the --run-name option (-r for short), or remove it using --no-run-name:
$ cylc install -n testing/u/aa012 -r n96
INSTALLED testing/u/aa012/n96 from ...
$ cylc install -n testing/u/aa012/n96 --no-run-name
INSTALLED testing/u/aa012/n96/run1 from ...
How To Configure A Hierarchy By Default
When you install a workflow, Cylc will reflect any hierarchy from the source directory structure into the installed run directory structure.
When you install a workflow, Cylc will look for its source directory in your configured source dirs. By default this is ~/cylc-src. E.G, this command will install look for a workflow called foo in ~/cylc-src and install it with the ID foo/run1:
$ cylc install foo
INSTALLED foo/run1 from ~/cylc-src/foo
If you arrange your workflow source directories hierarchically, then the workflows will install hierarchically by default.
For example, if you have a workflow stored in ~/cylc-src/foo/bar:
$ tree ~/cylc-src/
~/cylc-src/
`-- foo/
`-- bar/
`-- flow.cylc
Then, by default, it will install as foo/bar/run1
$ cylc install foo/bar
INSTALLED foo/bar/run1 from ~/cylc-src/foo/bar
The default directory is ~/cylc-src, however, you can add more by configuring your source dirs (e.g. you might want to add ~/roses to the list if you are using Rosie).