Moving Cylc processes from one machine to another

I am new to Cylc, but already a fan. I’m currently running tests of our software using Cylc and I am pleased with how it performs.

In order to sustain operations, we sometimes switch our processing to a different login node. When we do this, I want to have our Cylc environment ready on the new machine so that we can resume processing where we left off.

One idea I have for this is to simply rsync the Cylc database files from the source to the destination login node. I have not tested this yet, but I think that it should work. Is there a better way to do this? Does Cylc have any built in functions for doing this kind of thing?

I’m new to Cylc, so please bear with me if this is a dumb question.

Thanks!

1 Like

Hi @Keith

To do a restart of a suite daemon on another host, you can copy the suite database over. The only catch is, you shouldn’t copy an sqlite DB file while it is being updated so you need to stop the suite on the original host first. In the worst case, with no database you can do a “warm start” from the beginning of the most recent cycle point - at the cost of potentially re-running some jobs.

(With Cylc 8, by the way, you can start the scheduler at any point in the dependency graph without needing a corresponding check point state in the DB, so check points and warm starts will not be needed anymore … soon).

However, we recommend using a small pool of Cylc VMs on a shared filesystem. Then you can use Cylc’s built-in self-migration capability, as described here: 13. Running Suites — The Cylc Suite Engine 7.9.3 documentation