Cylc graph: Print out a reference format for the dependencies in SUITE

what does this invocation of cylc graph do?

i initiated the invocation thinking, well maybe we don/t need gtk for this:

   3/ cylc [prep] graph [OPTIONS] --reference SUITE [START[STOP]]
 Print out a reference format for the dependencies in SUITE.

% cylc graph --reference example 2015-03-15T06

ERROR: no X environment? No module named gtk

some utilities for non-gtk would be useful (IMHO).

what does this invocation of cylc graph do?

For a given suite scheduling (or if the --namespace option if also provided, the runtime namespace inheritance) configuration, it will, instead of opening an interactive GUI window to display the visual graph of nodes & edges, print to STDOUT a textual representation of those graphs.

The returned text is the equivalent to the Graphviz code used to render the visual graph on the GUI (for the same command minus the--reference option), but converted to a more human-readable format, namely (I believe) in the Graphviz ‘plain’ format.

For example (taking one from our test battery), for the following simple suite:

[cylc]
    cycle point time zone = +12

[scheduling]
    initial cycle point = 20140808T00
    [[dependencies]]
       [[[PT6H]]]
            graph = foo
       [[[T00]]]
            graph = foo => bar

[visualization]
number of cycle points = 3

it will return:

$ cylc graph --reference
edge "foo.20140808T0000+12" "bar.20140808T0000+12"
graph
node "bar.20140808T0000+12" "bar\n20140808T0000+12"
node "foo.20140808T0000+12" "foo\n20140808T0000+12"
node "foo.20140808T0600+12" "foo\n20140808T0600+12"
node "foo.20140808T1200+12" "foo\n20140808T1200+12"
stop

Uses for it include as a means to systematically compare the graph of two suites, via [cylc graph-diff](https://cylc.github.io/doc/built-sphinx-single/index.html#graph-diff) which under the hood simply takes the output of those commands from the two suites & does a diff command to highlight lines that vary.

i initiated the invocation thinking, well maybe we don/t need gtk for this:

To the crux of your question, yes, given the above by functionality the --reference option does not on its own require GTK (unless I have overlooked some subtlety).

However, I’ve taken a look at the logic of the relevant files from the codebase & the code that runs for that option does go through a (try / except) block where it checks for the gtk module & exits if it is not there, with that error you report, despite not needing that module (or some of the others it checks for there) for the --reference option utility.

So, there is a flaw in the code logic I believe; thanks for noticing that! A small refactor could allow cylc graph --reference to run in an environment without gtk & other unrequired modules. I will write this up in a small Issue on GitHub.

In honest, though, this deficiency is unlikely to get addressed (at least soon), because we are developing Cylc 8 & its new UI, which will work in the browser (with no local Cylc installation required), in earnest, & there the graphing of suites will be managed by completely new code. We are focusing efforts on the new Cylc-8 system rather than making enhancements to Cylc-7 code specific to its tech stack which we are replacing, including GTK which we are leaving behind.

some utilities for non-gtk would be useful (IMHO).

We absolutely agree on this, however, & there are plans for enhanced graphing features in Cylc 8. If you are interested, we have some Issues on GitHub that discuss some of these, see e.g this one on cylc graph & this one on graph comparisons.

Oops, forgot to add earlier that I put the GitHub Issue up here, FYI.

Hi @schaferk,

You’ll see we’ve closed that Issue as “won’t fix”. It’s not a priority for the reasons @sadie.bartholomew gave (Cylc 8 is coming, with no more GTK), and because the graph reference format is not really useful to users anyway (it’s used for graph comparison in our automated tests, during development). We’re trying to get Cylc 8 done as quickly as possible, then none of us will have to fight with pygtk anymore :slight_smile:

Hilary

could someone post the plot of the graph represented by the graph reference format (above)?

IMHO, the graph reference format is more easily parsed than the graph strings in the suite.
it/s usefulness, then, is in the eye of the beholder.

could someone post the plot of the graph represented by the graph reference format (above)?

It’s not a very exciting graph, but here you go:
mrs

IMHO, the graph reference format is more easily parsed than the graph strings in the suite.
it/s usefulness, then, is in the eye of the beholder.

If you’re talking about machine parsing that’s true, because Cylc’s graph configurations are patterns for generating infinite graphs composed of repeating nodes, in a format of our own invention. A reference graph (and any specific visualization in fact) is necessarily just some finite subset of the full workflow that does not see the full picture - so for “human parsing” there’s not really any substitute for understanding what the graph config strings mean.

The problem with providing what you want is this: the text format “reference graphs” should not require pygtk in principle, but they are still generated by the underlying graph library that does.

However, it turns out you’re in luck! I had forgotten that we had already re-implemented reference graphs for testing in Cylc 8 (no GTK) on the repository master branch, and it only took me 5 minutes to back-port that to Cylc 7. I’ll post an update on this shortly:

The new command cylc ref-graph will be in the upcoming 7.8.5 release. In the meantime you could manually copy the new file bin/cylc-ref-graph, and the small changes in bin/cylc and bin/cylc-help into your 7.8.4 release version. (It will probably work with 7.8.3 too, but I haven’t checked that).

1 Like

h.

i cloned just now:
(p27) bash-3.2$ cylc ref-graph trivial.01
snip…
graph
node “foo.1999-01-06T00:00:00” “foo\n1999-01-06T00:00:00”
node “foo.1999-01-13T00:00:00” “foo\n1999-01-13T00:00:00”
node “foo.1999-01-20T00:00:00” “foo\n1999-01-20T00:00:00”
stop

thanks for taking the time.