# Using arbitrary Python modules via Jinja2

**URL:** https://cylc.discourse.group/t/using-arbitrary-python-modules-via-jinja2/667
**Category:** Tips
**Created:** [June 5, 2023, 10:39pm UTC](https://cylc.discourse.group/t/using-arbitrary-python-modules-via-jinja2/667 "2023-06-05T22:39:46Z")
**Posts on this page:** 2
**Page:** 1

<div class="post-metadata">

### Author: ![hilary.j.oliver](https://yyz2.discourse-cdn.com/free1/user_avatar/cylc.discourse.group/hilary.j.oliver/32/4_2.png) [@hilary.j.oliver](https://cylc.discourse.group/u/hilary.j.oliver)
#### Post date: [June 5, 2023, 10:39pm UTC](https://cylc.discourse.group/t/using-arbitrary-python-modules-via-jinja2/667/1 "2023-06-05T22:39:46Z")

</div>

Did you know you Cylc extends Jinja2’s import mechanism to allow direct use of arbitrary Python modules?

Here’s an example that creates tasks at start-up, to process each file in a directory:

```console
$ ls /path/to/data/
cat dog fish

```

```ini
#!Jinja2
{# import Python's os module: #}
{% from "os" import listdir %}

{% set DATADIR = "/path/to/data" %}

[task parameters] {# use os.listdir() #}
   id = {{ listdir(DATADIR) | join(", ") }}
[scheduling]
   [[graph]]
      R1 = "prep => proc<id> => results"
[runtime]
   [[prep]]
      # ...
   [[proc<id>]]
      script = """
         do-something.py {{DATADIR}}/${CYLC_TASK_PARAM_id}
      """
   [[results]]
      # ...

```

Result after Jinja2 processing (use `cylc view -j`):

```ini

[task parameters]
   id = fish, cat, dog
[scheduling]
   [[graph]]
      R1 = "prep => proc<id> => results"
[runtime]
   [[prep]]
      # ...
   [[proc<id>]]
      script = """
         do-something.py /path/to/data/${CYLC_TASK_PARAM_id}
      """
   [[results]]
      # ...

```

And (use `cylc graph`):

 ![image](https://global.discourse-cdn.com/free1/uploads/cylc1/original/1X/c8986e6a52ac5d1a7d3bb3be4185d65c1ddb7bda.png)

Documented here in the User Guide: [Jinja2 — Cylc 8.1.4 documentation](https://cylc.github.io/cylc-doc/stable/html/user-guide/writing-workflows/jinja2.html#importing-python-modules)

---

<div class="post-metadata">

### Author: ![oliver.sanders](https://yyz2.discourse-cdn.com/free1/user_avatar/cylc.discourse.group/oliver.sanders/32/110_2.png) [@oliver.sanders](https://cylc.discourse.group/u/oliver.sanders)
#### Post date: [June 12, 2023, 10:00am UTC](https://cylc.discourse.group/t/using-arbitrary-python-modules-via-jinja2/667/2 "2023-06-12T10:00:13Z")

</div>

If you have a workflow which would need to perform a lot of logic in Jinja2, you can use this import mechanism to implement the logic in Python. Python modules in a workflow’s `lib/python` directory are added to the path automatically.

> Note that Jinja2 and any modules you import with it will run under the same Python environment that Cylc is installed under.