How to find out job run time?

Cylc Cylc Support
1

Hi,

I’m playing around with the number of processors to find the best balance of quick submission time and quick run time for various total run lengths. I need to know the time I submitted each job, the time it ended, and ideally how long it took between submission and starting. Is there a simple way to do this?

Cheers!

1 Like
2

as discussed in person :slight_smile: i know you can get this from the beginning and end of the job.out file but it would really good to know if there is an easier/automated way of doing this!

cheers

3

You can extract that timing info from the workflow run database - an sqlite DB in the workflow log directory):

$ sqlite3 -header ~/cylc-run/demo/runN/log/db "select * from task_jobs"
cycle|name|submit_num|flow_nums|is_manual_submit|try_num|time_submit|time_submit_exit|submit_status|time_run|time_run_exit|run_signal|run_status|platform_name|job_runner_name|job_id
1|data|1|[1]|0|1|2023-06-21T16:57:37+12:00|2023-06-21T16:57:38+12:00|0|2023-06-21T16:57:38+12:00|2023-06-21T16:57:49+12:00||0|localhost|background|10513
1|build-model|1|[1]|0|1|2023-06-21T16:57:37+12:00|2023-06-21T16:57:38+12:00|0|2023-06-21T16:57:38+12:00|2023-06-21T16:57:48+12:00||0|localhost|background|10512
2|data|1|[1]|0|1|2023-06-21T16:57:37+12:00|2023-06-21T16:57:38+12:00|0|2023-06-21T16:57:38+12:00|2023-06-21T16:57:57+12:00||0|localhost|background|10515
3|data|1|[1]|0|1|2023-06-21T16:57:50+12:00|2023-06-21T16:57:51+12:00|0|2023-06-21T16:57:51+12:00|2023-06-21T16:58:03+12:00||0|localhost|background|10625
4|data|1|[1]|0|1|2023-06-21T16:57:51+12:00|2023-06-21T16:57:51+12:00|0|2023-06-21T16:57:52+12:00|2023-06-21T16:58:07+12:00||0|localhost|background|10660
5|data|1|[1]|0|1|2023-06-21T16:57:59+12:00|2023-06-21T16:57:59+12:00|0|2023-06-21T16:58:00+12:00|2023-06-21T16:58:15+12:00||0|localhost|background|10701
1|model_m2|1|[1]|0|1|2023-06-21T16:58:05+12:00|2023-06-21T16:58:05+12:00|0|2023-06-21T16:58:06+12:00|2023-06-21T16:58:22+12:00||0|localhost|background|10742
1|model_m1|1|[1]|0|1|2023-06-21T16:58:09+12:00|2023-06-21T16:58:09+12:00|0|2023-06-21T16:58:10+12:00|2023-06-21T16:58:26+12:00||0|localhost|background|10783
1|model_m3|1|[1]|0|1|2023-06-21T16:58:17+12:00|2023-06-21T16:58:17+12:00|0|2023-06-21T16:58:18+12:00|2023-06-21T16:58:33+12:00||0|localhost|background|10839
1|post1_m2|1|[1]|0|1|2023-06-21T16:58:23+12:00|2023-06-21T16:58:23+12:00|0|2023-06-21T16:58:24+12:00|2023-06-21T16:58:40+12:00||0|localhost|background|10890
1|post1_m1|1|[1]|0|1|2023-06-21T16:58:27+12:00|2023-06-21T16:58:27+12:00|0|2023-06-21T16:58:28+12:00|2023-06-21T16:58:39+12:00||0|localhost|background|10931
1|post2_m2|1|[1]|0|1|2023-06-21T16:58:35+12:00|2023-06-21T16:58:35+12:00|0|2023-06-21T16:58:36+12:00|2023-06-21T16:58:48+12:00||0|localhost|background|10977
1|post2_m1|1|[1]|0|1|2023-06-21T16:58:40+12:00|2023-06-21T16:58:40+12:00|0|2023-06-21T16:58:41+12:00||||localhost|background|11024
2|model_m2|1|[1]|0|1|2023-06-21T16:58:40+12:00|2023-06-21T16:58:41+12:00|0|2023-06-21T16:58:41+12:00||||localhost|background|11059
1|post1_m3|1|[1]|0|1|2023-06-21T16:58:49+12:00|2023-06-21T16:58:50+12:00|0|||||localhost|background|11105
...
1 Like
4

You can also look in the job.status file which is easier to parse or source. e.g.

#!/bin/bash
. job.status
echo "Start time: $CYLC_JOB_INIT_TIME"
echo "End time: $CYLC_JOB_EXIT_TIME"
echo "Elapsed: $(( $(date -d "$CYLC_JOB_EXIT_TIME" +%s) - $(date -d "$CYLC_JOB_INIT_TIME" +%s) )) s"

Example output:

Start time: 2023-05-09T10:00:26Z
End time: 2023-05-09T10:02:56Z
Elapsed: 150 s
1 Like
5

Also see:

  • The cylc report-timings command.
  • The analysis view in the Cylc GUI.
1 Like