Cylc8 custom share directory location issue

Cylc Cylc Support
1

Hi,

I’m running a rose stem via Cylc8 as:

rose stem -n lfric_cylc8 rose-stem --opt-conf-key=$platform --group=developer --source=lfric=$lfric
cylc play lfric_cylc8

with a ~/.cylc/flow/global.cylc file as:

[install]
    [[symlink dirs]]
        [[[gadi]]]
            log = /scratch/dp9/dl9118
            share = /scratch/dp9/dl9118
            work = /scratch/dp9/dl9118

(where gadi is the remote host). On the remote host the log and work directories are softlinked to ~/cylc-run/lfric_cylc8/run1 from /scratch/dp9/dl9118/cylc-run/lfric_cylc8/run1 as expected, however the share directory is not being soft linked properly:

lrwxrwxrwx  1 dl9118 dp9   49 Sep 14 13:19 log -> /scratch/dp9/dl9118/cylc-run/lfric_cylc8/run1/log
drwxr-sr-x  3 dl9118 dp9 4.0K Sep 14 15:10 .service
drwxr-xr-x  6 dl9118 dp9 4.0K Sep 14 13:20 share
lrwxrwxrwx  1 dl9118 dp9   50 Sep 14 13:19 work -> /scratch/dp9/dl9118/cylc-run/lfric_cylc8/run1/work

Running cylc config I see:

[dl9118@ood-vn5 gungho]$ cylc config
[scheduler]
    [[host self-identification]]
        method = hardwired
        host = ood-vn5
[install]
    source dirs = ~/cylc-src, ~/roses
    [[symlink dirs]]
        [[[ood]]]
        [[[gadi]]]
            log = /scratch/dp9/dl9118
            share = /scratch/dp9/dl9118
            work = /scratch/dp9/dl9118
...

so I don’t know why this isn’t being picked up and a soft link being created for share on the remote host.

Any help would be gratefully appreciated.

Cheers, Dave.

2

Seems to be coming from the task script rather than Cylc, it’s running

++ rose host-select gadi.nci.org.au
+ HOST=gadi.nci.org.au
++ echo /home/562/saw562/cylc-run/lfric_cylc8/run1/share/source
++ sed 's|/home/562/saw562/||'
+ RELATIVE_SOURCE_DIRECTORY=cylc-run/lfric_cylc8/run1/share/source
+ ssh gadi.nci.org.au mkdir -p cylc-run/lfric_cylc8/run1/share/source
+ rsync -avz /home/562/saw562/cylc-run/lfric_cylc8/run1/share/source/ gadi.nci.org.au:cylc-run/lfric_cylc8/run1/share/source/
3

Thanks Scott! I’ll see if I can find a way to install the entire suite under /scratch instead of ~/.

4

To link the entire workflow run directory use run.

https://cylc.github.io/cylc-doc/stable/html/reference/config/global.html#global.cylc[install][symlink%20dirs][<install%20target>]run

Note this will move all files including those that Cylc manages such as the workflow database.

1 Like
5

Thank you for making me aware of this Oliver. I tried setting the whole run dir to /scratch, but as Scott says, the initial task on the localhost goes off and makes the share dir first, so this may require some modification of the suite itself.

6

Ah right, yes creating the share dir manually will break the symlink dirs feature.

A quick workaround is to run a blank task on the remote platform before this local task runs.

install_remote => install_local

[runtime]
    [[install_remote]]
        # a blank task which triggers Cylc's remote installation
        # (run this before local tasks which attempt to install into the remote cylc-run directory)
        platform = my_platform
    [[install_local]]
         # a local task which installs things into the remote cylc-run directory
         script = rsync /something my_platform:cylc-run/${CYLC_WORKFLOW_ID}/share/something
        platform = localhost
7

Thank you Oliver, this looks like a nice work-around. Long term we will no-doubt have to do some restructuring of the LFRic rose stems, so we can address this more holistically at that point. Thanks again, Dave.