We’ve just got access to our new supercomputer and now trying to set up to submit jobs using Rose/Cylc and having problems. The problem with cylc referencing its directories via
~ rear its head again.
In a similar setup to our current HPC, ARCHER, the compute nodes on the new HPC also cannot see the
/home disk. With ARCHER, you may remember, we were able to get around the issue of cylc referencing its directories via
~ by setting
init-script = "export HOME=$DATADIR" in the
suite.rc file. This worked, I believe, because the job script commands prior to the
aprun were executed on the job launcher (MOM) nodes which could see
/home. On ARCHER2 (SLURM) the job script and all commands prior to the
srun are run on pe0 (as there are no job launcher nodes) so cannot see
/home at all.
Any ideas on how we can get around this are much appreciated?
Hacking around with the cylc generated job script and manually submitting on the HPC, things like the path to the stdout and error don’t work…
#SBATCH --output=cylc-run/u-al624/log/job/1/archer/08/job.out #SBATCH --error=cylc-run/u-al624/log/job/1/archer/08/job.err
It needs the full path to the /work disk, etc
Hope that makes sense and happy to provide any more information.