Tip: Using data to define your workflow

We often want to read in a dataset for use in defining our workflow.

With Jinja2, we can read in data files and use them to generate our workflow configuration file, e.g. define tasks, environment, directives and more.

There is now a worked example of this design pattern in the Cylc documentation:

https://cylc.github.io/cylc-doc/nightly_8.4/html/user-guide/examples/external-data-files/index.html

2 Likes