# How can i find origin of validation warnings?

**URL:** <https://cylc.discourse.group/t/how-can-i-find-origin-of-validation-warnings/767>\
**Category:** Cylc Support\
**Created:** [September 28, 2023, 3:18am UTC](https://cylc.discourse.group/t/how-can-i-find-origin-of-validation-warnings/767 "2023-09-28T03:18:59Z")\
**Posts on this page:** 16\
**Page:** 1

<div class="post-metadata">

**Author:** ![jonnyhtw](https://yyz2.discourse-cdn.com/free1/user_avatar/cylc.discourse.group/jonnyhtw/32/232_2.png) [@jonnyhtw](https://cylc.discourse.group/u/jonnyhtw)\
**Post date:** [September 28, 2023, 3:18am UTC](https://cylc.discourse.group/t/how-can-i-find-origin-of-validation-warnings/767/1 "2023-09-28T03:18:59Z")

</div>

hi there,

is there a way to find the location of errors like this in a workflow?..

```auto
> cylc val .
WARNING - Support for integers with leading zeros was dropped in Jinja2 v3. Rose will extend support until
    a future version.
    Please amend your Rose configuration files e.g:
     * 01 => 1
     * 00 => 0
     * 09 => 9

```

i have lots of `rose` configuration files and so lookng through all of them would be very time consuming. i know it doesn’t matter for now because i can still run with this warning but it would be great to be able to home in on them a bit better. i’ve tried `cylc val -v` but that doesn’t seem to help.

thanks,

jonny

---

<div class="post-metadata">

**Author:** ![hilary.j.oliver](https://yyz2.discourse-cdn.com/free1/user_avatar/cylc.discourse.group/hilary.j.oliver/32/4_2.png) [@hilary.j.oliver](https://cylc.discourse.group/u/hilary.j.oliver)\
**Post date:** [September 28, 2023, 3:48am UTC](https://cylc.discourse.group/t/how-can-i-find-origin-of-validation-warnings/767/2 "2023-09-28T03:48:51Z")

</div>

Technically, I’d say that’s the `cylc-rose` plugin’s fault, not Cylc itself.

I would just regex grep for integers starting with `0` …

---

<div class="post-metadata">

**Author:** ![oliver.sanders](https://yyz2.discourse-cdn.com/free1/user_avatar/cylc.discourse.group/oliver.sanders/32/110_2.png) [@oliver.sanders](https://cylc.discourse.group/u/oliver.sanders)\
**Post date:** [September 28, 2023, 8:58am UTC](https://cylc.discourse.group/t/how-can-i-find-origin-of-validation-warnings/767/3 "2023-09-28T08:58:04Z")

</div>

Rose doesn’t preserve line numbers when it reads in configs so we can’t give you a position.

FYI, Jinja2 completely dropped support for zero-prefixed integers. We thought this was too disruptive so we hacked Jinja2 to maintain support for the short term. This hack is tricky, we won’t be able to keep it going with the next Jinja2 upgrade, so definitely act on this warning, next time it will be an error.

---

<div class="post-metadata">

**Author:** ![jonnyhtw](https://yyz2.discourse-cdn.com/free1/user_avatar/cylc.discourse.group/jonnyhtw/32/232_2.png) [@jonnyhtw](https://cylc.discourse.group/u/jonnyhtw)\
**Post date:** [September 29, 2023, 1:38am UTC](https://cylc.discourse.group/t/how-can-i-find-origin-of-validation-warnings/767/4 "2023-09-29T01:38:41Z")

</div>

what is they say?.. something like _if you use regex now you have two problems_ 😆

i’ll try and work out how to do this!

thanks

---

<div class="post-metadata">

**Author:** ![jonnyhtw](https://yyz2.discourse-cdn.com/free1/user_avatar/cylc.discourse.group/jonnyhtw/32/232_2.png) [@jonnyhtw](https://cylc.discourse.group/u/jonnyhtw)\
**Post date:** [September 29, 2023, 2:08am UTC](https://cylc.discourse.group/t/how-can-i-find-origin-of-validation-warnings/767/5 "2023-09-29T02:08:23Z")

</div>

thanks @oliver.sanders

---

<div class="post-metadata">

**Author:** ![jonnyhtw](https://yyz2.discourse-cdn.com/free1/user_avatar/cylc.discourse.group/jonnyhtw/32/232_2.png) [@jonnyhtw](https://cylc.discourse.group/u/jonnyhtw)\
**Post date:** [September 29, 2023, 2:09am UTC](https://cylc.discourse.group/t/how-can-i-find-origin-of-validation-warnings/767/6 "2023-09-29T02:09:01Z")

</div>

@hilary.j.oliver from the error message i guess this error is only relevant to `rose*.conf` files?

---

<div class="post-metadata">

**Author:** ![jonnyhtw](https://yyz2.discourse-cdn.com/free1/user_avatar/cylc.discourse.group/jonnyhtw/32/232_2.png) [@jonnyhtw](https://cylc.discourse.group/u/jonnyhtw)\
**Post date:** [September 29, 2023, 2:22am UTC](https://cylc.discourse.group/t/how-can-i-find-origin-of-validation-warnings/767/7 "2023-09-29T02:22:41Z")

</div>

the reason i ask this is that this only show one result which is in the `flow.cylc` file and seems unrelated…

```auto
> grep -rE '=0[0-9]' *
flow.cylc: -W umask=0022

```

 ![image](https://global.discourse-cdn.com/free1/uploads/cylc1/original/1X/2666cc33fa10c2e6f88a7677d885e1675b72e6a5.png)

---

<div class="post-metadata">

**Author:** ![hilary.j.oliver](https://yyz2.discourse-cdn.com/free1/user_avatar/cylc.discourse.group/hilary.j.oliver/32/4_2.png) [@hilary.j.oliver](https://cylc.discourse.group/u/hilary.j.oliver)\
**Post date:** [September 29, 2023, 2:26am UTC](https://cylc.discourse.group/t/how-can-i-find-origin-of-validation-warnings/767/8 "2023-09-29T02:26:00Z")

</div>

Yes, a zero-prefixed integer in your `flow.cylc` Jinja2 code wlll result in a syntax error, with the bad line highlighted:

```auto
{% set foo = 01 %}

```

result:

```auto
$ cylc validate .

Jinja2Error: expected token 'end of statement block', got 'integer'
File /home/oliverh/cylc-src/bug/flow.cylc
   {% set foo = 01 %} <-- TemplateSyntaxError

```

(The error message doesn’t seem terribly appropriate unfortunately, but that’s down to Jinja2, not Cylc)

… whereas, these are OK:

```auto
{% set foo = 1 %}
{% set bar = "01" %}

```

---

<div class="post-metadata">

**Author:** ![jonnyhtw](https://yyz2.discourse-cdn.com/free1/user_avatar/cylc.discourse.group/jonnyhtw/32/232_2.png) [@jonnyhtw](https://cylc.discourse.group/u/jonnyhtw)\
**Post date:** [September 29, 2023, 2:42am UTC](https://cylc.discourse.group/t/how-can-i-find-origin-of-validation-warnings/767/9 "2023-09-29T02:42:27Z")

</div>

roger that, thanks @hilary.j.oliver

---

<div class="post-metadata">

**Author:** ![jonnyhtw](https://yyz2.discourse-cdn.com/free1/user_avatar/cylc.discourse.group/jonnyhtw/32/232_2.png) [@jonnyhtw](https://cylc.discourse.group/u/jonnyhtw)\
**Post date:** [September 29, 2023, 2:56am UTC](https://cylc.discourse.group/t/how-can-i-find-origin-of-validation-warnings/767/10 "2023-09-29T02:56:05Z")

</div>

ok got it!

i gave up using regex to be honest (too hard for me at the best of times let alone a friday afternoon) and deleted each directory inside the workflow definition until i was left with only a `rose-suite.conf` and a `flow.cylc` file left.

once i’d got there i realised that Jinja2 was complaining about this in my `rose-suite.conf`…

`EXPT_BASIS=1978,09,01,00,00`

which needed to be changed to…

`EXPT_BASIS=1978,9,1,0,0`

quite how on earth one would construct a regex to find _that_ i don’t know haha

cheers,

jonny

---

<div class="post-metadata">

**Author:** ![wxtim](https://yyz2.discourse-cdn.com/free1/user_avatar/cylc.discourse.group/wxtim/32/151_2.png) [@wxtim](https://cylc.discourse.group/u/wxtim)\
**Post date:** [September 29, 2023, 8:46am UTC](https://cylc.discourse.group/t/how-can-i-find-origin-of-validation-warnings/767/11 "2023-09-29T08:46:02Z")

</div>

I think that the following works

```bash
> sed -E 's@^\.0([0-9]+)@\1@g' ${FILE}

```

```bash
cat > testfile << __HERE__
# Simple example
> FOO=0043
> # Datetime
> EXPT_BASIS=1978,09,01,00,00
> # Decimal
> BAR=0.004
> __HERE__

> sed -E 's@^\.0([0-9]+)@\1@g' testfile
# Simple example
FOO=0043
# Datetime
EXPT_BASIS=1978,09,01,00,00
# Decimal
BAR=0.004

```

YMMV - not extensively tested: Given this I’d tend to do

```bash
> sed -E 's@^\.0([0-9]+)@\1@g' testfile > testfile.new
> difftool testfile > testfile.new
# Manually inspect.
> mv testfile.new testfile

```

---

<div class="post-metadata">

**Author:** ![hilary.j.oliver](https://yyz2.discourse-cdn.com/free1/user_avatar/cylc.discourse.group/hilary.j.oliver/32/4_2.png) [@hilary.j.oliver](https://cylc.discourse.group/u/hilary.j.oliver)\
**Post date:** [October 1, 2023, 1:08am UTC](https://cylc.discourse.group/t/how-can-i-find-origin-of-validation-warnings/767/12 "2023-10-01T01:08:10Z")

</div>

> [@jonnyhtw](#):
>
> ```auto
> > grep -rE '=0[0-9]' *
> flow.cylc: -W umask=0022
> 
> ```

Yes, that is unrelated, the `0022` there is just plain old text, not a Jinja2 value.

---

<div class="post-metadata">

**Author:** ![oliver.sanders](https://yyz2.discourse-cdn.com/free1/user_avatar/cylc.discourse.group/oliver.sanders/32/110_2.png) [@oliver.sanders](https://cylc.discourse.group/u/oliver.sanders)\
**Post date:** [October 2, 2023, 10:10am UTC](https://cylc.discourse.group/t/how-can-i-find-origin-of-validation-warnings/767/13 "2023-10-02T10:10:00Z")

</div>

Even simpler:

```auto
\b0\d

```

- `\b` - word break incl commas etc.
- `0` - the number 0
- `\d` - any number

> **[regex101: build, test, and debug regex](https://regex101.com/r/VrwNR4/1)**
>
> Regular expression tester with syntax highlighting, explanation, cheat sheet for PHP/PCRE, Python, GO, JavaScript, Java, C#/.NET, Rust.

---

<div class="post-metadata">

**Author:** ![jonnyhtw](https://yyz2.discourse-cdn.com/free1/user_avatar/cylc.discourse.group/jonnyhtw/32/232_2.png) [@jonnyhtw](https://cylc.discourse.group/u/jonnyhtw)\
**Post date:** [October 3, 2023, 11:02pm UTC](https://cylc.discourse.group/t/how-can-i-find-origin-of-validation-warnings/767/14 "2023-10-03T23:02:02Z")

</div>

hi @oliver.sanders this doesn’t work for me i’m afraid…

```auto
> cat rose-suite.conf |grep EXPT_BASIS
EXPT_BASIS=1978,09,01,00,00

> grep -rE '\b0\d' *
[returns zero matches]

```

double quotes don’t work either.

fyi `grep` version is…

```auto
> grep --version
grep (GNU grep) 2.20

```

cheers,

---

<div class="post-metadata">

**Author:** ![oliver.sanders](https://yyz2.discourse-cdn.com/free1/user_avatar/cylc.discourse.group/oliver.sanders/32/110_2.png) [@oliver.sanders](https://cylc.discourse.group/u/oliver.sanders)\
**Post date:** [October 4, 2023, 8:30am UTC](https://cylc.discourse.group/t/how-can-i-find-origin-of-validation-warnings/767/15 "2023-10-04T08:30:42Z")

</div>

The regex works, but `grep` don’t support it, here’s the same thing adapted for `sed`:

```auto
sed -rn '/\b0[0-9]/p' <file>

```

(note set doesn’t support `\d`)

---

<div class="post-metadata">

**Author:** ![jonnyhtw](https://yyz2.discourse-cdn.com/free1/user_avatar/cylc.discourse.group/jonnyhtw/32/232_2.png) [@jonnyhtw](https://cylc.discourse.group/u/jonnyhtw)\
**Post date:** [October 5, 2023, 3:45am UTC](https://cylc.discourse.group/t/how-can-i-find-origin-of-validation-warnings/767/16 "2023-10-05T03:45:39Z")

</div>

> [@oliver.sanders](#):
>
> `sed -rn '/\b0[0-9]/p'`

ok thanks that works for `sed`…

```auto
> sed -rn '/\b0[0-9]/p' rose-suite.conf
EXPT_BASIS=1978,09,01,00,00

```

i’ve put in a PR in the `cylc-rose` repo to amend the warning text.

cheers,