Access task dir from a different task

Cylc Cylc Support
1

Good afternoon,

I guess my question has a simple solution, yet I could not find it on the forum or in the documentation.

I am writing a cycling workflow using cylc 8.3.3. Someting that could be simplifyed like this

[ scheduling ]

    initial cycle point = {{ EXP_START_DATE }}
    final cycle point = {{ EXP_END_DATE }}

    [[ graph ]]
        # Run just once at the initial cycle point, needs to be completed before the run model
        R1 = """ create_statics => run_model """

        P7D = """

                # PREPARE INPUTS FOR RUN
                recup_data & fill_namelist

                # RUN
                FAMPREP:succeed-all =>  run_model
                run_model[-P7D] => run_model
                [...]
        """

The task in the “Prep” family, as the name suggest, are retrieving data need by my model to run.

As you see, we have some “static” data files, i.e. they don’t depend on the cycle point, and some “dynamic” ones, i.e. they depend on the date, and will no longer be needed once the model runs that specific date, and archives the outputs.

So ideally I would like the prep tasks to put data diretctly in the run directory.

Now the question: the run_model tasks, runs in his own $CYLC_TASK_WORK_DIR. Is there a way to access this variable in other tasks?

For the moment what I am doing is constructing the path manually like

model_rundir = ${CYLC_WORKFLOW_WORK_DIR}/$CYLC_TASK_CYCLE_POINT/run_model

and using this variable in the various tasks that needs it, but this looks error prone to me.

Is there a more elegant solution I am not seeying?

Thanks a lot in advance,

Best,
Stella

2

Hi Stella,

Task work sub-directory locations are configurable, so you can make several tasks share the same work directory if they need to read and write from $PWD at runtime.

cylc-doc: task work directory

However, I think you are describing exactly what the workflow share directory is for: a shared IO space for multiple tasks as the workflow runs. The share directory is created automatically and is available as $CYLC_WORKFLOW_SHARE_DIR in task environments.

You can then create sub-directories under share as needed, and refer to those locations in the task environment too, e.g.:

[runtime]
    [[root]]
        [[[environment]]]
            STATIC_DATA = ${CYLC_WORKFLOW_SHARE_DIR}/static

Your Prep task could create the static/ directory and put the static data there, and other tasks could use it as needed (both via the $STATIC_DATA variable).

cylc-doc: workflow share directory

1 Like
3

Not directly.

I would expect work from tasks needed by other tasks to be placed in ${CYLC_WORKFLOW_SHARE_DIR}/ if its from the R1 tasks or ${CYLC_WORKFLOW_SHARE_DIR}/${CYLC_TASK_CYCLE_POINT} if it’s from the prep family (which should allow you to write a housekeeping task).

For a complete list of task variables see the Job Script environment variables reference.

1 Like
4

Hello Hilary and Tim, thanks for the quick reply!

I completely missed on the doc the possibility of modifying the task work dir, that’s what I was looking for I think.
I will put the static folder into share folder, it makes sense.

Many thanks again,
Stella

1 Like
5

No worries - Cylc can do a lot, and we try to ensure that we document it all: The only downside to this that I have seen is that there is a lot of documentation, probably more than most people will read from end-to-end.

6

Hi Tim,
I agree, it is somethimes difficult to find the exact page you’re searching for, and google is not doing a great job either.

Also, I did not understand this was what I needed, since it looked a bit criptic to me:

Path: flow.cylc[runtime][]work sub-directory

(and even after your explanation, I tryed to pass an absolute path to sub-dir ended up with the entire work folder structure duplicated inside /work :woman_facepalming:). Maybe a simple example would have done the trick.

I guess that like everything, I will just get use to the way the doc it written!

Thanks again for the promt help,
Stella

7

Working on the basis that you are looking at [this page](https://touch ~/.extended_vdi_assignment):

flow.cylc[runtime][<namespace>]work sub-directory is shorthand for

# a flow.cylc file
[runtime]
    [[task or FAMILY name]]
        work sub-directory = location

documentation of the shorthand

I’m not sure why you are trying to change the directory where the job runs rather than just having the job put the output somewhere else?

8

Well, after thinking about it, I think I shouldn’t.

I will do as you suggest, and just tell the various tasks where to put the output.