Feature request: optional argument for cylc stop to automatically clean run afterwards?

Cylc Cylc Support
1

Good morning,

I was wondering if there was an easier way to keep “on top of” run folders in the workflow directory.

There have been previous conversations about this: Automatic cylc clean? . But the stated solution seems rather dangerous, especially if you would like to investigate the run after stopping.

For context, I have been developing and testing models using cylc8. It’s a tiny bit of a time sink to put: 1) “cylc stop X”, 2) Wait, 3) “cylc clean X”. My question is could this be an optional argument in “cylc stop”, like “cylc stop --clean X”?

For very large cleans (such as rose stem tests) it might be worth also parsing somehow a timeout option.

2

Depending on what you aim to achieve, this might be better handled with a little bit of shell.

If you deem your run is of no further use immediately, you can kill it and clean in one go:

cylc stop --kill my-workflow/run1 && sleep 5 && cylc clean my-workflow/run1

This will kill any active tasks before cleaning to ensure the workflow is not running and doesn’t produce more files. But if you want to wait for the tasks to naturally finish before cleaning then then you’d need to do something more complicated, likely with cylc ping.

EDIT: command updated to add a sleep to let the stop happen.

1 Like
3

Just to note that the above suggestion may not be 100% reliable, as it relies on the workflow stopping before the clean happens. While this will often work, the stop command doesn’t wait for the workflow to actually stop before returning. You should find the following more reliable.

stop-n-clean.sh:

#!/usr/bin/bash env

flow="$1"
cylc stop --kill "${flow}"
while cylc ping "${flow}" >/dev/null; do
  echo "Waiting for workflow to stop."
  sleep 1
done
cylc clean -y "${flow}"

Then use it like ./stop-n-clean.sh my-workflow/run1.

4

The stop command actually has a built-in option to wait for shutdown, which makes your original suggestion work @jfrost-mo. E.g. to wait for up to 60 seconds:

$ cylc stop --kill --max-polls=12 --interval=5 <workflow-ID>

If the worklfow has not shut down by max-polls, the stop command will exit with error status:

$ cylc stop --max-polls=10 --interval=1 <workflow-ID>
..........
ERROR - failed after 10 polls
CylcError: Workflow did not shut down after 10 polls
2 Likes