`conda: command not found` inside workflow task

Cylc Cylc Support
1

I’m not making this an issue on GitHub because this is probably me being stupid…

flow.cylc :

#!Jinja2

# Create task families for conda environments.
%include 'conda.cylc'

[scheduling]
    cycling mode = integer
    initial cycle point = 1
    [[graph]]
        P1 = raw_to_mzml
[runtime]
    [[raw_to_mzml]]
        inherit = None, CONDA_TRFP
        script = thermorawfileparser --help

conda.cylc :

#!Jinja2

{% set conda_envs = {
    'CONDA_TRFP': 'wf-trfp',
    'CONDA_BINNER': 'wf-binner',
    'CONDA_DATAMUNGING': 'wf-datamunging',
    'CONDA_INFLUX': 'wf-influx'
    } %}

[runtime]
{% for env, conda_env_name in conda_envs.items() %}
    [[{{env}}]]
        pre-script = """
            set +eu
            conda activate {{ conda_env_name }}
            set -eu
        """
{% endfor %}

job.err

/home/fontain/cylc-run/test-conda/run1/log/job/1/raw_to_mzml/01/job: line 44: conda : command not found
/home/fontain/cylc-run/test-conda/run1/log/job/5/raw_to_mzml/01/job: line 50: thermorawfileparser : command not found
2024-06-05T10:14:34+02:00 CRITICAL - failed/ERR

As a sanity check, I added a echo $PATH and echo $USER to the pre-script. It’s indeed my user, but condabin isn’t added to the path.

This is a global conda install (root-owned, with group ‘conda’ which my username is in). But the issue started before that, with a local $HOME/miniforge3. So I tried reinstalling conda to solve it.

_
My .bashrc :

Summary 
# >>> conda initialize >>>
# !! Contents within this block are managed by 'conda init' !!
__conda_setup="$('/var/local/miniforge3/bin/conda' 'shell.bash' 'hook' 2> /dev/null)"
if [ $? -eq 0 ]; then
    eval "$__conda_setup"
else
    if [ -f "/var/local/miniforge3/etc/profile.d/conda.sh" ]; then
        . "/var/local/miniforge3/etc/profile.d/conda.sh"
    else
        export PATH="/var/local/miniforge3/bin:$PATH"
    fi
fi
unset __conda_setup
# <<< conda initialize <<<

# Toggle specific Cylc Flow version (select corresponding conda environment)
unset CYLC_VERSION
export CYLC_VERSION=8.2.4

_

My cylc wrapper:

Summary 
##############################!!! EDIT ME !!!##################################
# Centrally installed Cylc releases:
CYLC_HOME_ROOT="${CYLC_HOME_ROOT:-/opt}"

# Users can set CYLC_HOME_ROOT_ALT as well (see above), e.g.:
CYLC_HOME_ROOT_ALT=/var/local/miniforge3/envs

# Global config locations for Cylc 8 & Rose 2 (defaults: /etc/cylc & /etc/rose)
# export CYLC_SITE_CONF_PATH="${CYLC_SITE_CONF_PATH:-/etc/cylc}"
# export ROSE_SITE_CONF_PATH="${ROSE_SITE_CONF_PATH:-/etc/rose}"
###############################################################################

_

Versions:

Summary 
fontain@server:~$ cylc version --long
8.2.4 (/var/local/miniforge3/envs/cylc-8.2.4)

Plugins:
  cylc-rose       1.3.3   /var/local/miniforge3/envs/cylc-8.2.4/lib/python3.9/site-packages
  cylc-uiserver   1.4.4   /var/local/miniforge3/envs/cylc-8.2.4/lib/python3.9/site-packages

Entry Points:
  cylc.command:
    gui = cylc.uiserver.scripts.gui:main
    hub = cylc.uiserver.scripts.hub:main [hub]
    hubapp = cylc.uiserver.scripts.hubapp:main [hub]
  cylc.post_install:
    rose_opts = cylc.rose.entry_points:post_install
2

Hi,

The script, pre-script, etc that you configure is run in a Bash “login shell” environment.

This means that your ~/.bash_profile file should be sourced (automatically by Bash), which is usually configured to source your ~/.bashrc file which should in turn source the conda.sh script which provides Conda.

So this should work, I have workflows configured this way, but worth checking that the chain of assumptions holds.

1 Like
3

My .bashrc had the ‘interactive’ guard at the top:

# If not running interactively, don't do anything
case $- in
    *i*) ;;
      *) return;;
esac

:man_facepalming:

Thank you Oliver. I’m glad I didn’t make it a GitHub issue.

1 Like