# Run host ranking algorithm extensions?

**URL:** <https://cylc.discourse.group/t/run-host-ranking-algorithm-extensions/1107>\
**Category:** Cylc Support\
**Created:** [January 20, 2025, 2:48am UTC](https://cylc.discourse.group/t/run-host-ranking-algorithm-extensions/1107 "2025-01-20T02:48:47Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![TomC](https://yyz2.discourse-cdn.com/free1/user_avatar/cylc.discourse.group/tomc/32/115_2.png) [@TomC](https://cylc.discourse.group/u/TomC)\
**Post date:** [January 20, 2025, 2:48am UTC](https://cylc.discourse.group/t/run-host-ranking-algorithm-extensions/1107/1 "2025-01-20T02:48:47Z")

</div>

Hi, looking at [Global Configuration — Cylc 8.4.0 documentation](https://cylc.github.io/cylc-doc/stable/html/reference/config/global.html#global.cylc%5Bscheduler%5D%5Brun%20hosts%5Dranking) - I was wondering if there was any way to extend the ranking algorthim to be a bit smarter. From how I read the documentation, the load limiting is based on current memory, load etc. But, if you launch a workflow at a temporary quiet period on that VM, the results could be a bit misleading.

I wondered if there was some way to give weights to workflows for example, or other things outside of the psutil attributes. For example,

```auto
VM1 = [a, b,c] # all large workflows
VM2 = [z, y, x] # all small workflows

```

Run `cylc vip`, and its quiet, so it launches on VM1, but, then, the workflows there start running and the VM is flooded, causing slowness due to resource contention. But, if there was a way to say `a` and `x` has a weight of 9, `b` of 6, `c` of 8, whereas z,y are weight 1 and 3. So, the new workflow should ideally end up on `VM2` as it will on average have lower load.

More smarts would be expected/planned resource load profiles and Cylc could compare against expected peak loads and launch workflows in such a way to smooth out expected/planned load.

---

<div class="post-metadata">

**Author:** ![oliver.sanders](https://yyz2.discourse-cdn.com/free1/user_avatar/cylc.discourse.group/oliver.sanders/32/110_2.png) [@oliver.sanders](https://cylc.discourse.group/u/oliver.sanders)\
**Post date:** [January 20, 2025, 10:52am UTC](https://cylc.discourse.group/t/run-host-ranking-algorithm-extensions/1107/2 "2025-01-20T10:52:26Z")

</div>

Hi,

We can use any of the functionalities of the [psutil module](https://psutil.readthedocs.io/en/latest/), do basic maths with any numbers returned and hardcode thresholds, etc. The expressions you provide are parsed by Python in a restricted environment which only allows certain operations (e.g. you cannot import modules). This isn’t presently pluggable.

On our site, we use the following:

```auto
ranking = """
    getloadavg()[2] < 20
    virtual_memory().available > 2000000000
    -1 * virtual_memory().available
"""

```

This ranks hosts by available memory, but excludes hosts with high server load. We find ranking by available memory works pretty well, our servers are about as evenly loaded as could be hoped for. Here’s a screenshot of today’s memory loading (0-100%), the servers are all within 5% of each other:

 ![Screenshot from 2025-01-20 10-39-18](https://global.discourse-cdn.com/free1/uploads/cylc1/original/1X/6c85c65c9cd583a87582bf545b3506d694e63dee.png)

If you’re worried about erratic spikes in workflow activity hitting your servers hard, two things you can do:

- Look at using time averaged metrics from `psutils`, e.g. load average comes in 5, 10 and 15 min bins.
- Scale more vertically, the more workflows you run on a server, the more stable the load on will be on the server.

---

<div class="post-metadata">

**Author:** ![TomC](https://yyz2.discourse-cdn.com/free1/user_avatar/cylc.discourse.group/tomc/32/115_2.png) [@TomC](https://cylc.discourse.group/u/TomC)\
**Post date:** [January 20, 2025, 11:19am UTC](https://cylc.discourse.group/t/run-host-ranking-algorithm-extensions/1107/3 "2025-01-20T11:19:43Z")

</div>

Thanks. One last question on this, is there a default ranking approach used in cylc, is the default an empty string?

---

<div class="post-metadata">

**Author:** ![oliver.sanders](https://yyz2.discourse-cdn.com/free1/user_avatar/cylc.discourse.group/oliver.sanders/32/110_2.png) [@oliver.sanders](https://cylc.discourse.group/u/oliver.sanders)\
**Post date:** [January 20, 2025, 11:24am UTC](https://cylc.discourse.group/t/run-host-ranking-algorithm-extensions/1107/4 "2025-01-20T11:24:19Z")

</div>

The default (`ranking = `) is random selection (will clarify that in the docs).