In one of our datacenters, our upstream-provider needed to change the VLAN-ID of our uplink to be in line with their policies. We wanted to do this with minimal downtime.
The upstream interface on our gateway machines are in a linux bridge, that also contains the virtual interfaces for the virtual machines hosted on those machines. Here’s some (semi-censored) output of what I mean:
root@host:~# cat /etc/network/interfaces
auto bond0
iface bond0 inet manual
slaves eth0 eth1
bond-mode active-backup
bond-miimon 100
bond-downdelay 200
bond-updelay 200
auto ubr1
iface ubr1 inet static
address 192.0.2.2
netmask 255.255.255.0
gateway 192.0.2.1
bridge_ports bond0.910
bridge_stp off
bridge_fd 0
root@host:~# brctl show ubr1
bridge name bridge id STP enabled interfaces
ubr1 8000.90b11c4a8b39 no bond0.910
vm314_eth0
vm316_eth0
We had our upstream provider configure their switches beforehand so we could switch the VLAN when we wanted. The IP addressing remained the same on both sides, so the interface with the old VLAN had to be removed before the new one could be added so there wouldn’t be any IP conflicts. To do this all in one go, we made a small script to do this:
#!/bin/sh
IF=bond0
BR=ubr0
VLAN_OLD=910
VLAN_NEW=512
# Change the config so these changes persist after a reboot
sed -i “s/$IF\.$VLAN_OLD/$IF.$VLAN_NEW/” /etc/network/interfaces
brctl delif $BR $IF.$VLAN_OLD
ifconfig $IF.$VLAN_OLD down
vconfig rem $IF.$VLAN_OLD
vconfig add $IF.$VLAN_NEW
ifconfig $IF.$VLAN_NEW up
brctl addif $BR $IF.$VLAN_NEW
We ran this script on the gateway machines and the VLAN-IDs were changed almost instantly, total unavailability was less than a second.