Hi. Cylc 7 question.
Because of PBS issues after an “upgrade” on a platform I use (but do not control), tasks are intermittently and unpredictably failing to submit, and Cylc correctly flags them with a submit failed status.
I would like these tasks to automatically re-submit. I am aware of “execution retry delays” for retrying tasks that have failed; is there anything equivalent for tasks that have failed submission? I don’t see anything in the Cylc 7 docs. The docs do cover addressing this via a trigger in the scheduling graph, e.g. " C:submit-fail => D", which I suppose could be done as “C:submit-fail => C”; but this would cause infinite retries, and would dramatically multiply the size of our scheduling graph, since the particular tasks and invocations that are causing submit failures are unpredictable. The docs do also refer to the submit-retrying state, which leads me to think that there has to be some way to get a task into that state.
Thanks much.