Index out of range error

Cylc Cylc Support
1

I saw this error in a log this morning.

2024-11-27T03:56:15Z INFO - [20241127T0200Z/finish_getobs:succeeded] already finished and completed (flows=1))
2024-11-27T03:56:17Z INFO - [20241126T1200Z/bn_um_recon_ls_cov_6/01:submitted] (polled)started
2024-11-27T03:56:17Z INFO - [20241126T1200Z/bn_um_recon_ls_cov_6/01:submitted] setting implied output: submitted
2024-11-27T03:56:17Z INFO - [20241126T1200Z/bn_um_recon_ls_cov_6/01:submitted] submitted to platform:None[None]
2024-11-27T03:56:17Z WARNING - Unhandled jobs-poll output: 2024-11-27T03:56:16Z|20241126T1200Z/bn_um_recon_ls_cov_6/01|{"job_runner_name": "pbs", "job_id": "4185188", "job_runner_exit_polled": 0, "time_submit_exit": "2024-11-27T03:53:42Z", "time_run": "2024-11-27T03:55:48Z"}
2024-11-27T03:56:17Z WARNING - list index out of range
2024-11-27T03:56:37Z INFO - [20241126T1400Z/bn_um_fcst_14/01:submitted] (polled)started
2024-11-27T03:56:37Z INFO - [20241126T1400Z/bn_um_fcst_14/01:submitted] setting implied output: submitted
2024-11-27T03:56:37Z INFO - [20241126T1400Z/bn_um_fcst_14/01:submitted] submitted to another-platform:None[None]
2024-11-27T03:56:37Z WARNING - Unhandled jobs-poll output: 2024-11-27T03:56:37Z|20241126T1400Z/bn_um_fcst_14/01|{"job_runner_name": "pbs", "job_id": "4185508", "job_runner_exit_polled": 0, "time_submit_exit": "2024-11-27T03:55:34Z", "time_run": "2024-11-27T03:55:43Z"}
2024-11-27T03:56:37Z WARNING - list index out of range
2024-11-27T03:57:20Z INFO - platform: my-platfrm - remote file install (on remote-host)
2024-11-27T03:57:24Z INFO - platform: my-platfrm - remote file install complete
2024-11-27T03:57:32Z INFO - [20241126T1400Z/bn_backup_var_anl/01:submitted] setting implied output: submitted
2024-11-27T03:57:32Z INFO - [20241126T1400Z/bn_backup_var_anl/01:submitted] submitted to my-platfrm:None[None]
2024-11-27T03:57:32Z CRITICAL - An uncaught error caused Cylc to shut down.
    If you think this was an issue in Cylc, please report the following traceback to the developers.
    https://github.com/cylc/cylc-flow/issues/new?assignees=&labels=bug&template=bug.md&title=;
2024-11-27T03:57:32Z ERROR - list index out of range
    Traceback (most recent call last):
      File "/homeminiconda3/cylc-8.3.5/lib/python3.11/site-packages/cylc/flow/scheduler_cli.py", line 689, in cylc_play
        asyncio.get_running_loop()
    RuntimeError: no running event loop
    During handling of the above exception, another exception occurred:
    Traceback (most recent call last):
      File "/homeminiconda3/cylc-8.3.5/lib/python3.11/site-packages/cylc/flow/scheduler.py", line 652, in run_scheduler
        await self._main_loop()
      File "/homeminiconda3/cylc-8.3.5/lib/python3.11/site-packages/cylc/flow/scheduler.py", line 1537, in _main_loop
        self.process_queued_task_messages()
      File "/homeminiconda3/cylc-8.3.5/lib/python3.11/site-packages/cylc/flow/scheduler.py", line 907, in process_queued_task_messages
        if self.task_events_mgr.process_message(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/homeminiconda3/cylc-8.3.5/lib/python3.11/site-packages/cylc/flow/task_events_mgr.py", line 716, in process_message
        self.process_message(
      File "/homeminiconda3/cylc-8.3.5/lib/python3.11/site-packages/cylc/flow/task_events_mgr.py", line 770, in process_message
        self._process_message_submitted(itask, event_time, forced)
      File "/homeminiconda3/cylc-8.3.5/lib/python3.11/site-packages/cylc/flow/task_events_mgr.py", line 1497, in _process_message_submitted
        self._insert_task_job(
      File "/homeminiconda3/cylc-8.3.5/lib/python3.11/site-packages/cylc/flow/task_events_mgr.py", line 1541, in _insert_task_job
        job_conf = itask.jobs[-1]
                   ~~~~~~~~~~^^^^
    IndexError: list index out of range
2024-11-27T03:57:32Z CRITICAL - Workflow shutting down - list index out of range
2024-11-27T03:57:32Z WARNING - Orphaned tasks:
    * 20241126T1200Z/bn_backup_um_fcst (running)
    * 20241126T1200Z/bn_um_recon_ls_cov_6 (submitted)
    * 20241126T1400Z/bn_um_fcst_14 (submitted)
    * 20241126T1400Z/bn_set_varbc (submitted)
    * 20241126T1400Z/bn_backup_var_anl (submitted)

Is this something we did or a bug somewhere in Cylc?

2

It may be have been triggered by something unusual you did, but it’s a bug regardless - can you post an issue on the cylc-flow repo please, along with steps to reproduce if possible.

3

Ok, posted an Issue at Workflow disappeared - `list index out of range` error · Issue #6500 · cylc/cylc-flow · GitHub - best guess from me is a fail in the callback functions.

1 Like
4

I think this issue has already been fixed in Cylc 8.3.6.

1 Like