Hetzner Failover IP OCF script part deux: local DNS resolving

Two weeks ago we published a script that allows one to update the failover address provided by Hetzner using an OCF script. This makes it possible to provide redundant services between two systems within the Hetzner network. Even though this script by itself seems to function properly, it does have one shortcoming.

Consider a setup where both systems provide a set of services that use the same data store (e.g. a MySQL database). Even though these database services are replicated, queries must always be processed by the master node. Naively, one could solve this by simply letting all these services use the failover address provided by Hetzner. This will however not work, for the reason that even though traffic from the outside will always be routed to exactly one of the two systems, both systems have the address defined locally. The only way to perform connections between both systems, is by using the per-system (non-failover) addresses.

We solved this by creating a second OCF script, which can be used alongside the original one. This script maintains an entry in the /etc/hosts file, referring to the per-system address of the master node. The script uses three configuration parameters, namely the name of the DNS entry, the IP address of the system and the IP address of its peer. Unfortunately, this implies that the configuration on both systems is not identical. Therefore, the OCF script uses an additional configuration file to store the IP addresses. An example of this configuration file is shown below, which we shall call /etc/ha.d/update-dns-mysql:

ME=1.2.3.4
OTHER=2.3.4.5

Now we can add it to the CRM configuration as follows:

primitive DNS_mysql ocf:kumina:update-dns \
        op start interval="0" \
        params config="/etc/ha.d/update-dns-mysql" hostname="MYSQL-MASTER"

Now if everything is configured correctly, the OCF script will maintain an entry in /etc/hosts on both systems, pointing to the correct host:

$ getent hosts MYSQL-MASTER
1.2.3.4  MYSQL-MASTER

By using this hostname for all services that use the database, they will always connect to the MySQL instance running on the master node. In fact, the reason why it is implemented as a separate script, is because it can even be used without failover IP addresses from Hetzner — it can be used for any kind of service of which only a single instance suffices.

As mentioned in the previous post, be sure to let us know whether or not it works for you!

The files:

Tags: , , , , ,


Comments are closed.

Kumina designs, builds, operates and supports Kubernetes solutions that help companies thrive online. As Certified Kubernetes Service Partner, we know how to build real solutions.