I have recently started properly running some of my tasks and apps in Cylc 8, for Limited-Area-Models for some deterministic and ensemble NWP suites, based on the UM. I am finding it challenging to understand the dependencies between the various tasks, especially when working across many tasks on different suites. For example in the image below, the *_RECONF_DIFF runs first, followed by *_DIFF_DONE and then *_ENGL_DIFF and the ordering can be confusing.
More importantly, in the past with cylc gui <suite> we had the ability to select any task and check the dependencies. This is very important for fast debugging and troubleshooting. It would be greatly appreciated if this is considered as a future feature, or there’s some workaround that could help in a similar way.
Have you tried the graph view? At the top right of the GUI is a button to add views. Graph view should make this a little more obvious. It may be slow to load if your graph is very large.
I can’t find the issue, but I think that we are planning to put the dependency data back into the GUI.
The Cylc 8 GUI puts all tasks and families in alphabetical order.
The “show” functionality which listed a task’s prerequisites and outputs is sill available on the command line (cylc show <id>) and will be arriving in the Cylc 8 GUI in due course.
Once you get used to Cylc 8 it will be less confusing than Cylc 7.
In all the UI views, you see current “active tasks” plus future (waiting) and past (finished) tasks out to n (default 1) edges in the dependency graph. (c.f. Cylc 7, where exactly which past and future tasks you saw depended on scheduling algorithm internals, and that was not configurable).
In general it’s not possible to list tasks in their “correct run order” in a linear tree view because the dependencies form a non-linear graph. So an alphanumeric sort order, along with quick task state and name filtering, is really the only sensible way to do it.
So, as @wxtim and @oliver.sanders note above, use the graph view to see dependencies; or cylc show on the command line - and that will make its way into the web UI in a future release.
Hi @wxtim I have tried out the graph view and it’s the closest thing I have seen to help us understand dependencies. I will play around with it a bit more to see if parts of the graph can be expanded/condensed to work a bit better.
Note, there’s also the cylc graph command which can be run on a stopped workflow or working copy.
This command plots the first three cycles (by default) of the entire workflow which is good for getting an idea of the overall structure of the workflow. The --grouping option can be used to expand/collapse families which is useful.
Similar to cylc show, the functionality of the cylc graph command will arrive in the GUI in due course.
I’ve just noticed something a little strange and is possibly a bug or simply me not knowing the details of the new tools for Cylc 8.
I have the following graph on the Cylc GUI and these tasks are in a cycle I haven’t run yet.
However, when I use cylc show I’m finding that the outputs don’t quite align with the status on the Cylc GUI. I’ve used the commands below and this T1800Z cycle hasn’t run yet and the housekeep for the previous cycle is still held/hasn’t run.
>>>cylc show --list access_ce4_bn_TestCylc//20230501T1800Z/enbn_get_city_anl
20230501T1800Z/enbn_start succeeded
20230501T1800Z/enbn_start_get_city_anl succeeded
>>>cylc show access_ce4_bn_TestCylc//20230501T1800Z/enbn_get_city_anl
title: Get external data
importance: CAT-1, high
description: (not given)
URL: https://hpc-gitlab.bom.gov.au/ACCESS/APS3/City/cityensemblesuite/-/blob/dev/Suite/doc/get_city_anl.md
state: waiting
prerequisites: ('-': not satisfied)
- 20230501T1800Z/enbn_start succeeded
- 20230501T1800Z/enbn_start_get_city_anl succeeded
outputs: ('-': not completed)
- 20230501T1800Z/enbn_get_city_anl expired
- 20230501T1800Z/enbn_get_city_anl submitted
- 20230501T1800Z/enbn_get_city_anl submit-failed
- 20230501T1800Z/enbn_get_city_anl started
- 20230501T1800Z/enbn_get_city_anl succeeded
- 20230501T1800Z/enbn_get_city_anl failed
Is there a reasonable explanation for this or something obvious I’m missing?
I’m finding that the outputs don’t quite align with the status on the Cylc GUI
I think what you’ve shown above looks consistent to me. What task states were you expecting to see?
The cylc show command lists the prerequisites and outputs of a command. Prerequisites or outputs which have been completed are show in green with a + symbol, and those which have not been completed are shown in red with a - symbol. The --list-prereqs option shows all of the tasks prerequisites irrespective of whether they have been completed or not.
My confusion was that the enbn_start and enbn_start_get_city_anl for T1800Z cycle haven’t run yet, but the cylc show --list says that they’ve succeeded.
