About the Site

This weblog is edited and run by members of reallyenglish, a company offering a total English learning solution based in London, Beijing, Shanghai and Tokyo. Visit our corporate site to know more about what we do.

Notes are posted by members from various cultural and geographical backgrounds, and the topics range from education, business and international communication to software development, the internet culture, and more.

Staff

Masatomo Nakano http://twitter.com/masatomon /m/mt-static/support/assets_c/userpics/userpic-2-100x100.png simonl davida jeremyw Go Kameda gavin b No name tomoyukis

 

Recent Comments

  • Tomoyuki Sakurai: we've been using vether(4) successfully since 4.7. http://old.nabble.com/kernel-6318:-arp-cache-problem-bridge%284%29-with-vether%284%29-td27635451.html (the bug read more
  • RofR: Hi everyone Where about to setup serveral OBSD 4.9 or read more
  • Tomoyuki Sakurai: the above configuration has been working fine since 4.6. read more
  • Jean Aumont: Hi Tomoyuki, If you capture the packets with tcpdump and read more
  • Tomoyuki Sakurai: you will not see OSPF packets on physical interface. > read more
  • Jean Aumont: Hi everyone, I have been trying to pass OSPF in read more
  • Anonymous: without any other information, all i can say is doing read more
  • Jean Aumont: I have a similar set-up, but my 2 end points read more
  • Tomoyuki Sakurai: as for spoke, it's a simple matter of static routing. read more
  • vpnman: Excellent article! I am wondering how, in both a mesh read more

OSPF over GRE over IPSec with OpenBSD

| 13 Comments
reallyenglish.com is geographically diverse. The HQ in UK, a few branches in Asia, maybe more in the future. Except for few exceptions, the employees expect universal services, such as source code repository, project management system and wiki. All of them must be protected by strong encryption. IPsec is a standard protocol designed for encrypting packets from one place to another. We extensively use IPsec between our offices.

As a sysadmin, whenever someone says "the Internet is down", I say "no, it isn't". The problem would be caused by networking issue, misconfiguration by someone, maybe his own fault or whatever. The Internet is fine. But, partially, he is right. The Internet is the network of networks. Different network is managed by different entity. As such, links between networks
cut off for various reason every day somewhere on the Internet. You just don't know it is happening. The Internet as a whole works somehow thanks to "dynamic routing".

What dynamic routing does is, in short, "if there are multiple paths to the destination, use the best path" and "tell neighbors about your directly connected networks so that others can understand the whole network map". Without dynamic routing, you have to teach every possible route to every single router. For our intranet, we're using OSPF.

Okay, for robust intranetworking, all you need is multiple meshed IPsec tunnels with OSPF. IPSec is a standard and mature protocol, and so is OSPF. Sounds easy? Not quite. What I don't really like about IPsec is, it wasn't designed with routing in mind. As Security Association is statically configured, you cannot specify multiple path to a destination. Moreover, it doesn't handle multicast packet. OSPF is a routing protocol and uses multicast for its communication.

One more thing to make this problem complicated is proprietary products, Netscreen security devices to be specific, we're still using. If you stick with a specific vendor, you will not have interoperability problem, but you're successfully vendor locked-in. What you can do is what that particular vendor implemented. If they don't, you cannot do anything with it. Anyway, I strongly prefer open standards and open platforms. The choice in this case is OpenBSD. However, there wasn't documentation about how to implement meshed, dynamically routed IPsec with OpenBSD and ScreenOS.

IPsec doesn't like multicast pakcet. Okay, use GRE, Generic Routing Encapsulation. By encapsulating multicast packet into GRE unicast packet, IPsec happily forward the packet to its peer. When the link to the peer goes down, ospfd(8) marks the link as unavailable and use alternative path to the destination. Sounds simple, isn't it? Again, not quite. You have to understand all the network layers involved. There would be 6 endpoints, or IP addresses, involved in this setup. IPsec endpoints, GRE tunnel endpoints and a /30 network for OSPF communication.

Let's configure OpenBSD first. Make sure both packet forwarding and GRE are enabled. Also, you better to disable pf(4) during the testing. You can enable it again after everything works. OpenBSD 4.6 and later enable pf by default.

# sysctl net.inet.gre.allow=1
# sysctl net.inet.ip.forwarding=1
# pfctl -d

Next, create GRE tunnel.

# ifconfig gre0 create
# ifconfig gre0 tunnel myglobal.ip.addre.ss peerglobal.ip.addre.ss (outer header)
# ifconfig gre0 inet mygre.ip.addre.ss peergre.ip.addre.ss netmask 0xffffffff link0 up (inner header)

The packets between mygre.ip.addre.ss and peergre.ip.addre.ss will be encapsulated by GRE unicast packets with the global IP addresses of the both endpoints as source and destination header. GRE packets are not encrypted, yet.

Then, configure IPsec. OpenBSD's newer ipsec.conf(5) makes IPsec configuration much easier than ever. The security policy would be "encrypt all packets between the endpoints with their global address". Due to performance limitation of the hardware, we use SHA1 and 3DES.

# vi /etc/ipsec.conf
mypeer = "peerglobal.ip.addre.ss"
me = "myglobal.ip.addre.ss"
mypassword = "password"
ike esp from $me to $mypeer peer $mypeer \ main auth hmac-sha1 enc 3des group modp1024 \ srcid $me/32 dstid $mypeer/32 \ psk $mypassword

Start isakmpd and tell it to read security policy from /etc/ipsec.conf.

# isakmpd -K
# ipsecctl -f /etc/ipsec.conf


NS 5GT's config is not so straight forward, but almost same you did above on OpenBSD.

set interface tunnel.1 zone untrust
set interface tunnel.1 ip mygre.ip.addre.ss/30 (inner header)
set interface tunnel.1 tunnel encap gre
set interface tunnel.1 tunnel local-if untrust dst-ip global.ip.addre.ss (outer GRE header, GRE endpoints)
set interface tunnel.1 tunnel keep-alive (optional, but useful to see GRE packet on the wire)
set ike gateway mypeer address peerglobal.ip.addre.ss main outgoing-interface "untrust" preshare password sec-level compatible
set vpn mypeer gateway jp proposal "nopfs-esp-3des-sha" 
set vpn mypeer bind interface tunnel.1
set vpn mypeer proxy-id local-ip global.ip.addre.ss/32 remote-ip peerglobal.ip.adre.ss/32 ANY

The gotcha was, I thought it wasn't possible for OpenBSD and 5GT to talk to each other in a setup like this because OpenBSD's GRE tunnel is point-to-point, 5GT's is a normal /30 subnet. I even didn't give it a try during the initial testing. In fact, it is. Now, you can ping the peer using GRE /30 IP address each other.


# ping peergre.ip.addre.ss
PING peergre.ip.addre.ss (peergre.ip.addre.ss): 56 data bytes
64 bytes from peergre.ip.addre.ss: icmp_seq=0 ttl=64 time=277.665 ms
...


You'll see ICMP packets on the GRE interface.
# tcpdump -tnei gre0 -c 10 icmp 
tcpdump: listening on gre0, link-type NULL
mygre.ip.addre.ss > peergre.ip.addre.ss: icmp: echo reply
peergre.ip.addre.ss > mygre.ip.addre.ss: icmp: echo request
...


Make sure the traffic is encrypted. Replace vr0 with your egress interface's name.
# ipsecctl -s all
FLOWS:
flow esp in from myglobal.ip.addre.ss to peerglobal.ip.addre.ss peer peerglobal.ip.addre.ss srcid myglobal.ip.addre.ss/32 dstid peerglobal.ip.addre.ss/32 type use
flow esp out from peerglobal.ip.addre.ss to myglobal.ip.addre.ss peer peerglobal.ip.addre.ss srcid peerglobal.ip.addre.ss/32 dstid myglobal.ip.addre.ss/32 type require

SAD:
esp tunnel from myglobal.ip.addre.ss to peerglobal.ip.addre.ss spi 0x1275cf8c auth hmac-sha1 enc aes
esp tunnel from peerglobal.ip.addre.ss to myglobal.ip.addre.ss spi 0x20eec763 auth hmac-sha1 enc aes

# tcpdump -tni vr0 -c 10
tcpdump: listening on vr0, link-type EN10MB
esp myglobal.ip.addre.ss > peerglobal.ip.addre.ss spi 0x61a862f2 seq 45586 len 372
esp peerglobal.ip.addre.ss > myglobal.ip.addre.ss spi 0x84cd621d seq 45973 len 132
Now that both IPsec and GRE tunnel worked, the last thing you need is enabling OSPF. ospfd.conf is very simple. Replace vr1 with your internal network interface.

# vi /etc/ospfd.conf
password="mypassword"
auth-md 1 $password
auth-type crypt
auth-md-keyid 1
area 0.0.0.0 {
 interface gre0 { }
 interface vr1 { passive }
}

Start ospfd(8) in debug and verbose mode.

# ospfd -dv
... lots of debug information ...

Do the same thing on 5GT.

set vr trust-vr protocol ospf enable
set interface tunnel.1 protocol ospf area 0.0.0.0
set interface tunnel.1 protocol ospf authentication md5 mypassword key-id 1
set interface tunnel.1 protocol ospf authentication active-md5-key-id 1
set interface tunnel.1 protocol ospf enable
set interface trust protocol ospf area 0.0.0.0
set interface trust protocol ospf passive
set interface trust protocol ospf enable

Now you'll see OSPF neighbor relationship is established.

# ospfctl show neighbor
ID                      Pri State      DeadTime Address                Iface     Uptime
peergre.ip.addre.ss     1   FULL/P2P   00:00:30 peergre.ip.addre.ss    vr1       00:00:23

run "ospfctl show rib" to see RIB and "ospfctl show fib" to see FIB. It should show you the network at the peer's end. If it doesn't work for you, make sure IPsec flow is established,  OSPF packet is flowing between both hosts. To make a meshed IPsec, simply repeat this procedure. Tedious? Well, for that matter, you have to replace all the proprietary boxes with OpenBSD and use some kind of automated configuration management system.

13 Comments

Excellent article. I am curious as to why you set the link0 flag in your ifconfig statement. I was under the impression that the link0 flag disables multicast packets. Your stated reason for using GRE tunnels was to allow multicast encapsulation. Quite possibly, I've missed something.

ifconfig(8) states "Setting this flag stops all IP multicast packets from being forwarded by the bridge" in BRIDGE section. gre(4) says link0 "Set the operation mode to the specified IP protocol value", IPPROTO_GRE. This is the default value when you create gre(4). I prefer explicitly setting values.

Ah, thank you, clearly I did miss something. I did read gre(4). Also, you offer some good advice to explicitly set default values. I do much the same in my pf.conf. Have a great New Year in 2011.

Excellent article! I am wondering how, in both a mesh and hub and spoke topology, you can force all traffic to travel over the vpn, not just traffic destined for the LAN. Thanks.

as for spoke, it's a simple matter of static routing. for mesh, net.inet.ip.multipath would be your friend (here be dragons).

I have a similar set-up, but my 2 end points are OpenBSD 4.9 server.

The ospf is going through the GRE tunnel and the IPSEC encryption and the OSPF neighbors become established.

After the OSPF dead timer, the OSPF neighbors are lost, and re-initialisation of the OSPF is re-startded.

They become neighbor again, and this bad cycle re-start again.
Any ideas why ?

Thanks for the great article.

without any other information, all i can say is doing tcpdump and diagnosing OSPF messages. also, ospfd -dv would be useful.

Hi everyone,

I have been trying to pass OSPF in an IPSEC tunnel with no success.

The ospf is going through the GRE/GIF tunnel and the IPSEC encryption and the OSPF neighbors become established.

After the OSPF dead timer, the OSPF neighbors are lost, and re-initialisation of the OSPF is re-startded.

I have done 2 tests. The first one between 2 OpenBSD 4.9 servers and seconf one between 2 FreeBSD 8.2 servers. Both test were unsuccessfull.

With OpenBSD4.9, I have capture file that show that multicast messages are sent to the gre0 interface but are not pass to the physical interface em0 (Packets are lost)

With FreeBSD8.2, a similar situation, I have capture file that show that multicast messages are sent to the gif0 interface but are not pass to the physical interface em0 (again, Packets are lost)

I would like know, which version of OpenBSD were using when you managed to get OSPF going properly through the IPSEC tunnel ?

Thanks,
Jean Aumont
JeanAumont@gmail.com

you will not see OSPF packets on physical interface.

> sudo tcpdump -ni gre0 -c 2 proto ospf

06:27:04.199345 10.100.255.2 > 224.0.0.5: OSPFv2-hello 48: rtrid 10.101.255.1 backbone [tos 0xc0] [ttl 1]
06:27:05.818788 10.100.255.1 > 224.0.0.5: OSPFv2-hello 48: rtrid 10.100.48.254 backbone [tos 0xc0] [ttl 1]

Hi Tomoyuki,

If you capture the packets with tcpdump and then use Wireshark to analyse them, you can see the packet esasily.

Packets are lost/dropped on OpenBSD and FreeBSD.

So this is why I would like to know which version of OpenBSD you were using when you managed to get OSPF going properly through the IPSEC tunnel ?

Thanks,

Jean Aumont
JeanAumont@gmail.com

the above configuration has been working fine since 4.6.

Hi everyone

Where about to setup serveral OBSD 4.9 or 5.0 boxes with OSPF between them and will be using some kind of ipsec+gre or similar to make routing happen over untrusted networks.

Have anyone had or look, or tested 'vether(4)'. It was introduced in 4.8.
http://www.openbsd.org/cgi-bin/man.cgi?query=vether&sektion=4

Thanks for a great article

we've been using vether(4) successfully since 4.7.

http://old.nabble.com/kernel-6318:-arp-cache-problem-bridge%284%29-with-vether%284%29-td27635451.html

(the bug was fixed long time ago, btw)

Leave a comment