Best way to set up conditional compiling of binaries

Cylc Cylc Support
1

Kia ora! I have what perhaps is more of a style question. In our suite, we’d like to compile our various components once at the beginning of the suite, but not all the time (the compilation takes for ever). I can think of a number of ways of approaching this.

  1. compile at first install only
    • how do I detect first install?
  2. compile dependent on a T/F flag using Jinja2
    • check of flag in graph section or in runtime section?

In all these scenarios the binaries would be installed in a specified location outside of cylc-run so as to be visible to all runs, and not accidentally “cleaned”

… or perhaps I really need to get on with Rose?
Cheers!

2

Hi,

We usually schedule these sorts of tasks to run on workflow startup. Here’s an example where the compile task is run on startup:

[scheduling]
    initial cycle point = 2000
    [[graph]]
        R1 = """
            compile
        """
        P1D = """
            # chain of tasks to run every day
            pre => run => post

            # wait for the compile before running the executable
            compile[^] => run
        """

The compile task will only run once, the pre, run and post tasks will run every day.

We would store the executable inside the workflow (i.e. inside ~/cylc-run/<workflow>). This is important as the implementation of the compile task may change the outcome of the executable (options, compiler flags, etc).

If you want to work with resources (e.g. executables) stored outside of the workflow, I would recommend that you use a separate process to manage these resources, e.g. install them into environments.

A common practice is to copy or link executables stored outside of the workflow into it using Rose fileinstallation. This won’t build your executable for you, but it will provide an install-time solution for linking them into the workflow.

2 Likes