Holding families that do not exist in the task pool

Hi,

I have a workflow, made up of many families

HHH_000_gl
HHH_012_gl
HHH_024_gl
...
HHH_240_gl

Where there is a graph like

wait<block-1>  => wait<block>
wait<block> => a<block> => b<block> => c<block>

Early on, whilst only a few families exist, say HHH_000_glHHH_024_gl and I execute cylc hold workflow '//*/HHH_0[3-9]*_gl'

I can see the next couple of families held. I manually trigger the first (and only visible) task in HHH_036_gl, which causes new tasks to appear, including the next HHH wait task. I see the pause state does not apply to this new task, and it runs straight away even though my intention was to pause future tasks and manually work through each task one by one, whilst allowing the earlier blocks to continue processing (i.e. I do not want to pause HHH_000 … HHH_024), nor am I wanting to pause all other cycles, nor other tasks not in these HHH families.

Can someone please advise how to do what I’m wanting to do?

I guess a nicer way of saying this might be:

Tasks, when added to the task pool, do not inherit the pause status via inheritance. I think they should. If you hold a family, it should go through all tasks which inherit from it, and hold them.

The cylc hold command will hold future tasks - which is what you want - if targeted specifically by cycle-point/task-name. (Under the hood, the scheduler has to maintain a list of future tasks to check against as the workflow evolves).

It also works with pattern globbing and “selectors” (such as task state) for current active tasks (i.e. the “task pool”.)

Unfortunately, holding a family falls into the second category. I think we have a plan to extend this functionality, but I vaguely recall that’s it’s trickier than it might seem (maybe to do with figuring out when the pattern or family name has been “used up” and no more matching tasks will appear in the future).

(Note Cylc 7 only does task pool globbing, although you might not have noticed due to the many waiting tasks spawned ahead in Cylc 7).

So, for now, not ideal, but you’d have to hold each individual family member.

(Note Cylc 7 only does task pool globbing, although you might not have noticed due to the many waiting tasks spawned ahead in Cylc 7).

Now that you mention it, yes, I’ve seen this in Cylc 7, especially for trying to reset many cyclepoints succeeded.

(maybe to do with figuring out when the pattern or family name has been “used up” and no more matching tasks will appear in the future).

I won’t pretend to understand all the complexity that flow and such have introduced, but, one idea may be to have a number of edges/depth type option with a default of 0 (current behaviour) and a maximum of some other number (1? 3? 6? 12?). So instead of looking all the way down a potential tree, it can be limited to N levels down.