Spack install cylc-flow

Cylc Cylc Support
1

Hello,

I would like to add cylc-flow to my Spack environment.
Are you working on a Spack package recipe?

Maybe I can help.

Best,
Teemsis

2

Not that I’m aware of. Please go ahead and let us know how it goes!

1 Like
3

We haven’t looked into Spack to date, the main reasons for which being:

  • Cylc is a universal Python package (no build step required).
  • As a system tool, Cylc is typically installed into its own environment with dependencies of workflow tasks being installed into other environments (sometimes Spack environments). This allows Cylc to be upgraded independently of other dependencies and for parallel installations of Cylc on the same system via the provided wrapper script.

I think Spack might help to broaden Cylc’s hardware support by making it easier for people to compile their own Python, zmq, etc, but I’m not sure what the advantages are beyond that. Would be interested in finding out more.

You might find our conda-forge feedstock useful for getting started on your Spack recipe. It works by pulling in the platform-independent source code files from pypi, then installing them with pip install --no-deps (to ensure all dependencies are installed from conda-forge rather than pypi).

Notes:

  • The pip install step is necessary (because it registers Cylc CLI commands and extensions).
  • setuptools is both an installation and runtime dependency of Cylc (because it’s used to load CLI commands and extensions).

Would be interested in finding out how you get on.

1 Like
4

Awesome, thank you for the hints.
That’s exactly what I was looking for :slight_smile:

For some reasons (sorry, I cannot be more specific) I really need to include Cylc into my Spack environment, mainly because I am not allowed to use conda in my system.

Just one thing, I’m not sure to understand the difference between cylc-flow and cylc-base. Should I create both cylc-base and cycl-flow recipes in Spack?

5

cylc-flow-base is described here: Conda Environments — Cylc 8.1.0 documentation

6

There are some optional dependencies of Cylc, you can find the full list in the setup.cfg file here:

For Conda we currently offer two outputs:

  • cylc-flow-base - which contains only the bare essentials
  • cylc-flow - which includes the following optional dependency groups: empy,report-timings,tutorials,graph

The cylc-flow-base output is useful for:

  • Projects/environments which have Cylc as a dependency but want to leave it up to the end user to decide what optional extras they want.
  • Installation on remote platforms where Cylc jobs are submitted to but where the optional extras are not necessary.
1 Like
7

BTW: It might be worth looking into conda-pack which allows you to transfer Conda environments to other platforms with compatible architecture without the need for Conda to be installed on the platform itself.

4 Likes
8

Hi,

It’s been a long time, but my colleague and I have achieved the Spack recipes for Cylc, Cylc Rose, and Cylc UI Server.

spack install py-cylc-flow

Or:

spack install py-cylc-flow py-cylc-rose py-cylc-uiserver

Here is the “spack info py-cylc-flow”:

PythonPackage:   py-cylc-flow

Description:
    A workflow engine for cycling systems.

Homepage: https://cylc.org

Preferred version:  
    8.2.0    https://files.pythonhosted.org/packages/source/c/cylc-flow/cylc-flow-8.2.0.tar.gz

Safe versions:  
    8.2.0    https://files.pythonhosted.org/packages/source/c/cylc-flow/cylc-flow-8.2.0.tar.gz
    8.1.4    https://files.pythonhosted.org/packages/source/c/cylc-flow/cylc-flow-8.1.4.tar.gz

Deprecated versions:  
    None

Variants:
    Name [Default]               When    Allowed values    Description
    =========================    ====    ==============    ======================================

    build_system [python_pip]    --      python_pip        Build systems supported by the package

Build Dependencies:
    py-aiofiles       py-colorama            py-jinja2              py-promise   py-pyzmq       py-tomli  python
    py-ansimarkup     py-graphene            py-metomi-isodatetime  py-protobuf  py-rx          py-urwid
    py-async-timeout  py-importlib-metadata  py-pip                 py-psutil    py-setuptools  py-wheel

Link Dependencies:
    None

Run Dependencies:
    py-aiofiles    py-async-timeout  py-graphene            py-jinja2              py-promise   py-psutil  py-rx          py-tomli  python
    py-ansimarkup  py-colorama       py-importlib-metadata  py-metomi-isodatetime  py-protobuf  py-pyzmq   py-setuptools  py-urwid

Licenses: 
    None

Thank you for your help.

Best regards,

2 Likes