OK, for topical reasons, I’m wondering if I should start another thread . . .
So, I’ve been checking out the subsuite approach. I would like to make it work, because it does seem less crufty than using parametrized tasks. When using simple stubs for the tasks in the subscript, everything works wonderfully. But the effort to create the tasks for the subsuite is a bear: the subsuite approach is much more elegant at the suite definition level, but it’s proving really challenging at the task script level.
The problem is that the (Python-based) task scripts for the parent suite need to use a great many functions which are common among several different scripts; so they’re contained in utility modules and imported from those modules by the various task scripts. The task scripts for the tasks in the subsuite will need these functions too. So I either have to have two copies of them – one in the utility modules for the parent suite, and a second copy for the subsuite’s tasks – OR, I need the subsuite task scripts to import from the parent suite’s utility modules. The former is a bad idea because these functions get tweaked a lot, and having functions duplicated just makes it likely that some change won’t get propagated between them. OK, I can do the latter: I can have the subsuite’s tasks import functions from the parent suite’s utility modules. But the problem is that the utility modules, in turn, depend heavily upon the environment. When imports are done from these modules by the subsuite task, many of the environment variables set as constants in the utility modules won’t be defined. There’s ways around that, too; but this starts to become epicycles after a while. So it’s not clear anymore what the cleanest approach would be.
The only way I know of to pass portions of the environment from a parent task to a subsuite it kicks off is through the “–set” switch in the “cylc run” command – either specifying the environment variables individually, or by constructing a JSON string – and then using the Jinja2 variables so created to set new environment variables in the subsuite’s definition file. Is there any other way for a subsuite to inherit environment from the parent?