Almost all serious project require redundancy, reallyenglish is no exception. carp(4), developed by OpenBSD project, is a free implementation of VRRP, which is patented by Cisco. carp(4) has load balancing based on L2 and L3, but here, I'm going to focus on its fail over feature.

            | 192.168.100.254/24
     .------+------.
 vr0 |         vr0 |
 .---+---. vr3 .---+---.
 | core1 +-----+ core2 |
 `---+---'     `---+---'
 vr1 |          vr1|
     `------+------'
            | 192.168.200.254/24

Firstly, enable IPv4 forwarding. Also, enable preemption to failover all carp interfaces altogether when one interface fails.

# sysctl net.inet.ip.forwarding=1
# echo net.inet.ip.forwarding=1 >> /etc/sysctl.conf
# sysctl net.inet.carp.preempt=1
# echo net.inet.carp.preempt=1 >> /etc/sysctl.conf

Using carp(4) is fairly simple. Here is what hostname.if(5) looks like.

/etc/hostname.vr0:     up
/etc/hostname.vr1:     up
/etc/hostname.vr3:     inet 172.16.0.1/30
/etc/hostname.carp0:   192.168.100.254 vhid 1 carpdev vr0 pass mypass
/etc/hostname.carp1:   192.168.200.254 vhid 2 carpdev vr1 pass mypass
/etc/hostname.pfsync0: syncdev vr3

Bring up the interfaces. You can also use ifconfig(8), but I always prefer startup script to make sure that the configuration will not be lost.

# sh /etc/netstart vr0
# sh /etc/netstart vr1
# sh /etc/netstart vr3
# sh /etc/netstart carp0
# sh /etc/netstart carp1
# sh /etc/netstart pfync0

If you insists on using ifconfig(8):

# ifconfig vr0 up
# ifconfig vr1 up
# ifconfig vr3 inet 172.16.0.1/30
# ifconfig carp0 create
# ifconfig carp1 create
# ifconfig carp0 192.168.100.254 vhid 1 carpdev vr0 pass mypass
# ifconfig carp1 192.168.200.254 vhid 2 carpdev vr0 pass mypass
# ifconfig pfsync0 syncdev vr3

Repeat almost same configuration on core2. The difference is each carp interface has advskew keyword and vr3 for pfsync device has different IP address. A router with higher advskew value becomes slave.

/etc/hostname.vr3:     inet 172.16.0.2/30
/etc/hostname.carp0: 192.168.100.254 vhid 1 carpdev vr0 pass mypass advskew 100
/etc/hostname.carp1: 192.168.200.254 vhid 2 carpdev vr1 pass mypass advskew 100

Now you have redundant firewalls. Easy!

As explained in the earlier entry, we use OSPF for internal routing. How do I establish OSPF neighbor? Because ospfd(8) cannot be enabled actively on carp'ed interface, you need real interface that connects OSPF cloud and core routers. Additionally, real interface cannot preempt carp interface. That means when the real interface fails, you cannot failover. To workaround this problem, you need two OSPF-enabled routers and two links on each core. Things get messy from now on.

                                     carp0
           udav1   vr1    sis0      /
-----.      .------.     .-------. / 
     +------+  r1  +-----+ core1 |<
     |      `------'     `-------' \
     |     udav0|vr2\   / sis1  \   \
     |          |    \ /         \   carp1
OSPF |          |     X        pfsync0
     |          |    / \         /   carp0
     |     udav0|vr1/   \ sis0  /   /
     |      .------.     .-------. / 
     +------|  r2  +-----+ core2 |<
-----'      `------'     `-------' \
           udav1   vr2     sis1     \
                                     carp1

In this setup, when one of the link from core to OSPF cloud fails, traffic will be routed via the link between r1 and r2. When carp'ed interface fails, preemption kicks in and ospfd(8) on core1 will remove the routes and r1 will not route traffic to core1. claudio@, one of "the networking guy" in OpenBSD admitted it's not optimal. Hopefully, it will be fixed in the future release.

ALIX 2D3 board, which is used as r1 and r2, has only three ports. And one port is used for diskless boot in this testing. So, i added two USB ethernet adopters, deteced as udav(4). Make sure the right patch goes to the right port. Each link between the routers has a /30 subnet. If confused, draw a diagram on paper.

Let's enable OSPF on all interface on every router, but not on real interfaces used for carp. ospfd.conf(5) looks like:

# core1 and core2
auth-type crypt
auth-md 1 "mypass"
auth-md-keyid 1

area 0.0.0.0 {
  # link to r1/r2
  interface sis0 { metric 10 }
  interface sis1 { metric 20 }
  # you can omit "passive" on carp interface. it's always passive anyway
  interface carp1 { passive }
  interface carp0 { passive }
}

# r1 and r2
auth-type crypt
auth-md 1 "mypass"
auth-md-keyid 1

area 0.0.0.0 {
  # link to core
  interface vr1 { metric 10 }
  interface vr2 { metric 20 }
  interface udav0 { }
  interface udav1 { }
}

Run ospfd(8) on every router. The output of ospfctl(8) on core1 looks like this.

# ospfctl sh int
Interface   Address            State  HelloTimer Linkstate  Uptime    nc  ac
carp0       192.168.100.254/24 DOWN   -          master     00:00:00   0   0
carp1       192.168.200.254/24 DOWN   -          master     00:00:00   0   0
sis1        192.168.0.5/30     BCKUP  00:00:00   active     1d06h19m   1   1
sis0        192.168.0.1/30     BCKUP  00:00:00   active     1d06h19m   1   1

# ospfctl sh ne
ID              Pri State        DeadTime Address         Iface     Uptime
172.16.254.202  10  FULL/DR      00:00:33 192.168.0.6     sis1      1d06h20m
172.16.254.201  1   FULL/DR      00:00:32 192.168.0.2     sis0      1d06h20m

Now the fun (and boring) part. While pinging from a host in OSPF cloud to the other end of core, pull out the cable randomly. You can shutdown one of the router, too. Nothing should happen, other than normal ping output. If it doesn't work, tcpdump(8) is your friend. You can manually fail over the master by:

# ifconfig -g carp carpdemote 10

This setup requires no additional package and works out of the box. If you need more redundancy, i.e. redundancy at application level, than this setup, OpenBSD also provides relayd(8). If you need redundant DHCP servers, modified version of dhcpd(8) has synchronization support. OpenBSD makes redundant networking fairly easy.

Happy redundancy!