# Symlink resolution when restarting a workflow

**URL:** <https://cylc.discourse.group/t/symlink-resolution-when-restarting-a-workflow/915>\
**Category:** Cylc Support\
**Created:** [March 6, 2024, 9:24am UTC](https://cylc.discourse.group/t/symlink-resolution-when-restarting-a-workflow/915 "2024-03-06T09:24:03Z")\
**Posts on this page:** 7\
**Page:** 2

<div class="post-metadata">

**Author:** ![oliver.sanders](https://yyz2.discourse-cdn.com/free1/user_avatar/cylc.discourse.group/oliver.sanders/32/110_2.png) [@oliver.sanders](https://cylc.discourse.group/u/oliver.sanders)\
**Post date:** [March 8, 2024, 4:18pm UTC](https://cylc.discourse.group/t/symlink-resolution-when-restarting-a-workflow/915/21 "2024-03-08T16:18:35Z")

</div>

Try turning your relative symlink into an absolute symlink and it will work:

```auto
.
|-- link-me
`-- workflow
    |-- bin
    | |-- bar -> /var/tmp/tmp.T8JCH73dFl/workflow/bin/../../link-me
    | `-- foo -> ../../link-me
    `-- flow.cylc

```

```auto
$ cat ./workflow/flow.cylc
[scheduling]
    [[graph]]
        R1 = foo & bar

[runtime]
    [[foo]]
        script = foo
    [[bar]]
        script = bar

```

When I run this example `foo` fails (relative symlink does not resolve) and `bar` succeeds (absolute symlink does resolve).

In your example the script and symlink have the same name, in my example I’m using different names so the `foo` script never works.

---

<div class="post-metadata">

**Author:** ![abarral](https://avatars.discourse-cdn.com/v4/letter/a/3ab097/32.png) [@abarral](https://cylc.discourse.group/u/abarral)\
**Post date:** [March 8, 2024, 4:23pm UTC](https://cylc.discourse.group/t/symlink-resolution-when-restarting-a-workflow/915/22 "2024-03-08T16:23:32Z")

</div>

For the sake of clarity, here’s the script to replicate my case:

```auto
mkdir example && cd example
mkdir -p flow/bin
echo "echo Hello" > script_real.sh
chmod +x script_real.sh
cd flow/bin
ln -s ../../script_real.sh script_real_SYMLK.sh
cd ..
cat > flow.cylc <<eod
[scheduling]
    cycling mode = integer
    [[graph]]
        R1 = do_stuff
[runtime]
    [[do_stuff]]
        script=". script_real_SYMLK.sh; error"
eod

```

then

```auto
cd example/flow
cylc vip
# wait
cylc log flow//1/do_stuff

```

---

<div class="post-metadata">

**Author:** ![abarral](https://avatars.discourse-cdn.com/v4/letter/a/3ab097/32.png) [@abarral](https://cylc.discourse.group/u/abarral)\
**Post date:** [March 8, 2024, 4:24pm UTC](https://cylc.discourse.group/t/symlink-resolution-when-restarting-a-workflow/915/23 "2024-03-08T16:24:23Z")

</div>

> [@oliver.sanders](#):
>
> Yes, I think that the `cylc vip` and `cylc play` commands are using different working directories. Or preserving `$PATH` differently somehow?

Well that could indeed explain it then ! But that’s very surprising for the user ^^

---

<div class="post-metadata">

**Author:** ![oliver.sanders](https://yyz2.discourse-cdn.com/free1/user_avatar/cylc.discourse.group/oliver.sanders/32/110_2.png) [@oliver.sanders](https://cylc.discourse.group/u/oliver.sanders)\
**Post date:** [March 8, 2024, 4:31pm UTC](https://cylc.discourse.group/t/symlink-resolution-when-restarting-a-workflow/915/24 "2024-03-08T16:31:37Z")

</div>

I tried running your example and it failed for me:

```auto
[example] $ cd $(mktemp -d)
[tmp.uZZvZTW10O] $ mkdir example && cd example
[example] $ mkdir -p flow/bin
[example] $ echo "echo Hello" > script_real.sh
[example] $ chmod +x script_real.sh
[example] $ cd flow/bin
[bin] $ ln -s ../../script_real.sh script_real_SYMLK.sh
[bin] $ cd ..
[flow] $ cat > flow.cylc <<eod
> [scheduling]
> cycling mode = integer
> [[graph]]
> R1 = do_stuff
> [runtime]
> [[do_stuff]]
> script=". script_real_SYMLK.sh; error"
> eod
[flow] $ cylc vip -N
$ cylc validate /var/tmp/tmp.uZZvZTW10O/example/flow
Valid for cylc-8.2.4
$ cylc install /var/tmp/tmp.uZZvZTW10O/example/flow
INSTALLED flow/run1 from /var/tmp/tmp.uZZvZTW10O/example/flow
$ cylc play -N flow/run1

 ▪ ■ Cylc Workflow Engine 8.2.4
 ██ Copyright (C) 2008-2024 NIWA
▝▘ & British Crown (Met Office) & Contributors

...

2024-03-08T16:24:39Z INFO - [1/do_stuff running job:01 flows:1] => failed
2024-03-08T16:24:39Z WARNING - [1/do_stuff failed job:01 flows:1] did not complete required outputs: ['succeeded']
2024-03-08T16:24:39Z ERROR - Incomplete tasks:
      * 1/do_stuff did not complete required outputs: ['succeeded']
2024-03-08T16:24:39Z CRITICAL - Workflow stalled
2024-03-08T16:24:39Z WARNING - P3D stall timer starts NOW

```

```auto
$ cylc cat flow//1/do_stuff -f e
/.../cylc-run/flow/run1/log/job/1/do_stuff/01/job: line 42: script_real_SYMLK.sh: No such file or directory
2024-03-08T16:24:38Z CRITICAL - failed/ERR

```

Which makes sense because the symlink does not resolve:

```auto
$ cat ~/cylc-run/flow/run1/bin/script_real_SYMLK.sh 
cat: /.../cylc-run/flow/run1/bin/script_real_SYMLK.sh: No such file or directory

```

I don’t understand how this works for you. I suspect you have something in your `.bashrc` or `.bash_profile` that is putting extra things into the `$PATH` maybe? Inspecting the environment may yield insights.

> Well that could indeed explain it then ! But that’s very surprising for the user ^^

After investigation it looks like they are using the same working directory, so it isn’t that. So I’m not sure why this works for `vip` but not `play`, however, I don’t actually understand how this is working for you in the first place.

---

<div class="post-metadata">

**Author:** ![abarral](https://avatars.discourse-cdn.com/v4/letter/a/3ab097/32.png) [@abarral](https://cylc.discourse.group/u/abarral)\
**Post date:** [March 8, 2024, 4:37pm UTC](https://cylc.discourse.group/t/symlink-resolution-when-restarting-a-workflow/915/25 "2024-03-08T16:37:48Z")

</div>

I just tried running it on another machine (my remote cluster), and I get the exact same result (the symlinked script runs).

To reduce incertitude to a minimum, I’m simply running a [single script](https://pastebin.com/4sUsfg3a) which is just a concatenation of the previous steps.

---

<div class="post-metadata">

**Author:** ![abarral](https://avatars.discourse-cdn.com/v4/letter/a/3ab097/32.png) [@abarral](https://cylc.discourse.group/u/abarral)\
**Post date:** [March 8, 2024, 4:48pm UTC](https://cylc.discourse.group/t/symlink-resolution-when-restarting-a-workflow/915/26 "2024-03-08T16:48:55Z")

</div>

I have some interesting results !

```auto
> cylc vip . # Works fine
# versus
> cylc install .
> cylc play flow # ...SYMLK.sh: No such file or directory

```

Which explains why it didn’t work in your case

---

<div class="post-metadata">

**Author:** ![oliver.sanders](https://yyz2.discourse-cdn.com/free1/user_avatar/cylc.discourse.group/oliver.sanders/32/110_2.png) [@oliver.sanders](https://cylc.discourse.group/u/oliver.sanders)\
**Post date:** [March 11, 2024, 9:54am UTC](https://cylc.discourse.group/t/symlink-resolution-when-restarting-a-workflow/915/27 "2024-03-11T09:54:41Z")

</div>

> Which explains why it didn’t work in your case

Nope, this does not work for me [either way around](https://cylc.discourse.group/t/symlink-resolution-when-restarting-a-workflow/915/24). This approach should not logically work as the symlink does not resolve [as you confirmed](https://cylc.discourse.group/t/symlink-resolution-when-restarting-a-workflow/915/14).

To explain what `cylc install` is doing. We write and edit workflow definitions in the “source directory” (`~/cylc-src` by default), when we run `cylc install`, Cylc copies the workflow files into the “run directory” (`~/cylc-run`). This keeps your running workflows separate from their source allowing you to continue working on your workflow source code without affecting any running installations of that workflow.

So if your `~/cylc-src` directory looks like this:

```auto
 ~cylc-src/
├── flow
│ ├── bin
│ │ └── script_real_SYMLK.sh ⇒ ../../script_real.sh
│ └── flow.cylc
└── script_real.sh

```

The relative symlink is pointing at `~/cylc-src/script_real.sh`.

When you install the workflow your `~/cylc-run` directory will look like this:

```auto
 ~cylc-run/
└── flow/
     └── run1/ # "cylc install" copied this from "~/cylc-src/flow"
          ├── flow.cylc
          └── bin/
               └── script_real_SYMLK.sh ⇒ ../../script_real.sh

```

The relative symlink is now pointing at `~/cylc-run/flow/script_real.sh`. Since this file does not exist, the symlink is broken.

How to work around this? If the script is only required by the one workflow, then it can be moved into the workflow’s bin directory where it will be copied across by `cylc install`. If the script is common to multiple workflows (and you don’t want to make multiple copies to make maintenance easier), you could put the script in a central location e.g. `~/bin` and ensure that directory is in your `$PATH`, e.g. by adding `export PATH="$HOME/bin:$PATH"` in your `~/.bashrc` file.

At our site, when multiple workflows share common resources, we typically move these resources to a common version-controlled location so multiple users can access them. Workflows then install these resources (at a specified revision) into the workflow using an install task. These often use Rose [file installation](https://metomi.github.io/rose/doc/html/api/configuration/file-creation.html) for convenience.

[Previous page](https://cylc.discourse.group/t/symlink-resolution-when-restarting-a-workflow/915.md?page=1)