Hello! I’m having trouble figuring out how to get rose apps to run with preopts the same as they did in cylc 7. For the latter, we use the environment section of our task definitions and simply specify ROSE_LAUNCHER_PREOPTS. This appears to have the effect that after cylc has submitted the task to our cluster, rose itself actually further uses srun with the command line options specified in preopts to actually run our task.
However, in cylc 8 this isn’t happening. The target of the app is just called directly.
Am I doing something wrong? Thanks.
Rose apps don’t call srun directly. The variable you refer to applies to the rose mpi-launch command.
https://metomi.github.io/rose/doc/html/api/environment-variables.html#envvar-ROSE_LAUNCHER_PREOPTS
Typically the Rose app will call a script and then rose mpi-launch is called within that script.
It looks like you haven’t specified a launcher. Did you have this configured centrally with Rose 2019 (as used with Cylc 7)? Try running the command rose config rose-mpi-launch in your Cylc 7 environment. You’ll probably find you have launcher-list=srun (or something like that). Then try the same command with Rose 2 (as used with Cylc 8) and make sure you have the same settings.
1 Like
Thank you so much for the quick reply – we’ll definitely check that out!
That was indeed our problem. Thank you so much!