Cylc8 custom share directory location issue


I’m running a rose stem via Cylc8 as:

rose stem -n lfric_cylc8 rose-stem --opt-conf-key=$platform --group=developer --source=lfric=$lfric
cylc play lfric_cylc8

with a ~/.cylc/flow/global.cylc file as:

    [[symlink dirs]]
            log = /scratch/dp9/dl9118
            share = /scratch/dp9/dl9118
            work = /scratch/dp9/dl9118

(where gadi is the remote host). On the remote host the log and work directories are softlinked to ~/cylc-run/lfric_cylc8/run1 from /scratch/dp9/dl9118/cylc-run/lfric_cylc8/run1 as expected, however the share directory is not being soft linked properly:

lrwxrwxrwx  1 dl9118 dp9   49 Sep 14 13:19 log -> /scratch/dp9/dl9118/cylc-run/lfric_cylc8/run1/log
drwxr-sr-x  3 dl9118 dp9 4.0K Sep 14 15:10 .service
drwxr-xr-x  6 dl9118 dp9 4.0K Sep 14 13:20 share
lrwxrwxrwx  1 dl9118 dp9   50 Sep 14 13:19 work -> /scratch/dp9/dl9118/cylc-run/lfric_cylc8/run1/work

Running cylc config I see:

[dl9118@ood-vn5 gungho]$ cylc config
    [[host self-identification]]
        method = hardwired
        host = ood-vn5
    source dirs = ~/cylc-src, ~/roses
    [[symlink dirs]]
            log = /scratch/dp9/dl9118
            share = /scratch/dp9/dl9118
            work = /scratch/dp9/dl9118

so I don’t know why this isn’t being picked up and a soft link being created for share on the remote host.

Any help would be gratefully appreciated.

Cheers, Dave.

Seems to be coming from the task script rather than Cylc, it’s running

++ rose host-select
++ echo /home/562/saw562/cylc-run/lfric_cylc8/run1/share/source
++ sed 's|/home/562/saw562/||'
+ RELATIVE_SOURCE_DIRECTORY=cylc-run/lfric_cylc8/run1/share/source
+ ssh mkdir -p cylc-run/lfric_cylc8/run1/share/source
+ rsync -avz /home/562/saw562/cylc-run/lfric_cylc8/run1/share/source/

Thanks Scott! I’ll see if I can find a way to install the entire suite under /scratch instead of ~/.

To link the entire workflow run directory use run.[install][symlink%20dirs][<install%20target>]run

Note this will move all files including those that Cylc manages such as the workflow database.

1 Like

Thank you for making me aware of this Oliver. I tried setting the whole run dir to /scratch, but as Scott says, the initial task on the localhost goes off and makes the share dir first, so this may require some modification of the suite itself.

Ah right, yes creating the share dir manually will break the symlink dirs feature.

A quick workaround is to run a blank task on the remote platform before this local task runs.

install_remote => install_local

        # a blank task which triggers Cylc's remote installation
        # (run this before local tasks which attempt to install into the remote cylc-run directory)
        platform = my_platform
         # a local task which installs things into the remote cylc-run directory
         script = rsync /something my_platform:cylc-run/${CYLC_WORKFLOW_ID}/share/something
        platform = localhost

Thank you Oliver, this looks like a nice work-around. Long term we will no-doubt have to do some restructuring of the LFRic rose stems, so we can address this more holistically at that point. Thanks again, Dave.