Building DMVPN with mGRE, NHRP and IPSec VPN

I – OVERVIEW

This lab will treat the design and deployment of dynamic multipoint VPN architectures by moving step by step into the configuration and explaining how mGRE (multipoint Generic Router Encapsulation), NHRP (Next-Hop Resolution Protocol) and IPsec VPN are mixed to build a dynamic secure topology over the Internet for large enterprises with hundreds of sites.

Figure1: physical Topology

A hypothetical enterprise (figure1) with a central office (HUB) and several branch office sites (spokes) connecting over the Internet is facing a rapid business grow and more and more direct connections between branch offices are needed.

Spokes are spread over different places where it is not always possible to afford a static public addresses therefore company needs more scalable method than a simple Hub and spoke with point-to-point tunneling or a full mesh topology where the administration tasks of the IPSec traffic become extremely burdensome.

II – DEPLOYMENT

Figure2: Address scheme

The DMVPN model provides a scalable configuration for a dynamic-mesh VPN with the only static relationships configured are those between spokes and the HUB.

DMVPN concepts include several components like mGRE, NHRP and IPSec Proxy instantiation that will be explained during this lab.

II-1 Address scheme:

Each site has its own private address space 10.0.1.0/24, 10.0.2.0/24, 10.0.3.0/24 and 10.0.4.0/24 for central site, site1, site2 and site3 respectively.

Each spoke router obtains its public IP address dynamically from the ISP; only the HUB has a static permanent public IP.

Table1: address scheme

Network	Addresses
OSPF Area 10 – central site	10.0.1.0/24
OSPF Area 11 – site 1	10.0.2.0/24
OSPF Area 12 – site 2	10.0.3.0/24
OSPF Area 13 – site 3	10.0.4.0/24
Multipoint GRE network	172.16.0.128/26
Area 0 link subnet between HUB and Internet	192.168.0.16/30
Area 0 link between SPOKEA and Internet	192.168.0.12/30
Area 0 link between SPOKEB and Internet	192.168.0.8/30
Area 0 link between SPOKEC and Internet	192.168.0.4/30

II-2 Multipoint Generic Router Encapsulation – mGRE:

Table2: Tunnel configuration guideline

Router HUB
Interface tunnel 0	Ip /mask	172.16.0.129/28
	Source ip	S1/0
	Dest ip	–
	Tunnel type	GRE multipoint
Router SPOKEA
Interface tunnel 0	Ip /mask	172.16.0.130/28
	Source ip	S1/0
	Dest ip	–
	Tunnel type	GRE multipoint
Router SPOKEB
Interface tunnel 0	Ip /mask	172.16.0.131/28
	Source ip	S1/0
	Dest ip	–
	Tunnel type	GRE multipoint
Router SPOKEC
Interface tunnel 0	Ip /mask	172.16.0.132/28
	Source ip	S1/0
	Dest ip	–
	Tunnel type	GRE multipoint

A Hub n Spoke point-to-point GRE would be a heavy burden for the administration of the topology because all spokes IP addresses must be known in advance.

Moreover such traditional solutions consume lot of CPU and memory resources because each tunnel will create its own IDB (Interface Descriptor Block).

The alternative for point-to-point GRE will be multipoint GRE where a single interface will terminate all spokes GRE tunnels and will consume a single IDB and conserves interface memory structures and interface process management on the HUB.

With point-to-multipoint GRE the tunnel source should be public routable, in our case we set mGRE packet to take serial 1/0 as an output interface to the SP network, the tunnel destination address is dynamically assigned.

Tunnel source and destination form the outer IP header that will carry the encapsulated traffic throughout the physical topology.

All tunnels participating to mGRE network are identified by a tunnel key ID and belongs to the same subnet 172.16.0.129/26, from the private address space and routable only inside the company’s network.

HUB:

interface Tunnel0

ip address 172.16.0.129 255.255.255.192

!!The mGRE packet will be encapsulated out of the physical interface

tunnel source Serial1/0

!!Enable mGRE

tunnel mode gre multipoint

!! Tunnel identification key, match the NHRP network-id

tunnel key 1

SPOKEA :

interface Tunnel0

ip address 172.16.0.130 255.255.255.192

tunnel source Serial1/0

tunnel mode gre multipoint

tunnel key 1

SPOKEB :

interface Tunnel0

ip address 172.16.0.131 255.255.255.192

tunnel source Serial1/0

tunnel mode gre multipoint

tunnel key 1

SPOKEC :

interface Tunnel0

ip address 172.16.0.132 255.255.255.192

tunnel source Serial1/0

tunnel mode gre multipoint

tunnel key 1

So far, all tunnel protocol state still down, because tunnels destination addresses are unknown, so unreachable.

The HUB router knows that traffic destined to 172.16.0.128/26 will be forwarded to the serial interface (tunnel source) to be encapsulated into the needed mGRE tunnel and all other traffic will be reachable through the next hop physical interface 192.168.0.18, therefore no particular information about spokes on the HUB except that they belong to 172.16.0.128/26.

II-3 Static routing:

HUB:

ip route 0.0.0.0 0.0.0.0 192.168.0.18

ip route 172.16.0.0 255.255.255.192 Serial1/0

As opposed to the HUB, all spokes know the HUB physical IP address 192.168.0.17 and how to reach it statically (through s1/0). All other traffic will be forwarded to the next hop physical interface IP address.

On SPOKEA:

!! Forward all traffic to the ISP next-hop destination

ip route 0.0.0.0 0.0.0.0 192.168.0.14

!! The HUB physical interface is reachable through serial1/0

ip route 192.168.0.17 255.255.255.255 Serial1/0

On SPOKEB:

ip route 0.0.0.0 0.0.0.0 192.168.0.10

ip route 192.168.0.17 255.255.255.255 Serial1/0

On SPOKEC:

ip route 0.0.0.0 0.0.0.0 192.168.0.5

ip route 192.168.0.17 255.255.255.255 Serial1/0

II-4 Dynamic routing:

Figure3: routing topology

Each spoke protects a set of private subnet, not known by others; all other routers must be informed of these subnets.

With distance vector protocols like RIP or EIGRP the specification of “no-split horizon” is required for the updates to be sent back to the mGRE interface to other spokes.

With Link State protocols like OSPF the appropriate next-hop is automatically reflected within the subnet.

Typically OSPF is configured in point-to-multipoint mode on mGRE interfaces, this cause spokes to install routes with the HUB as next-hop, which negates DMVPN network topology concept, for that reason DMVPN cloud must be treated as broadcast.

NBMA – Non Broadcast Multiple Access are networks where multiple hosts/devices are connected, but data is sent directly to the destination over a virtual circuit or switching fabric like ATM, FR or X.25.

Broadcast networks – All hosts connected to the network listen to the media but only the host/device to which the communication is intended will receive the frame.

Cisco prefer Distance Vector protocols and recommend using EIGRP for large scale deployments.

Spokes OSPF priorities are set to 0 to force them in DROTHER mode and never become DR/BDR, whereas the HUB priority is set to a higher value “10” for instance.

All routers must be configured to announce the mGRE subnet 172.16.0/128 as area 0 and 10.0.1.0/24, 10.0.2.0/24, 10.0.3.0/24 and 10.0.4.0/24 as OSPF non-backbone area10, area11, area12 and area13 respectively.

Note that the routing protocol is dealing only with mGRE logical topology.

HUB:

interface Tunnel0

ip ospf network broadcast

ip ospf priority 10

!

router ospf 10

router-id 1.1.0.1

!! Announce local networks.

network 10.0.1.0 0.0.0.255 area 10

!! Announce mGRE network as OSPF backbone

network 172.16.0.128 0.0.0.63 area 0

SPOKEA:

interface Tunnel0

ip ospf network broadcast

ip ospf priority 0

!

router ospf 10

router-id 1.1.1.1

network 10.0.2.0 0.0.0.255 area 11

network 172.16.0.128 0.0.0.63 area 0

SPOKEB:

interface Tunnel0

ip ospf network broadcast

ip ospf priority 0

!

router ospf 10

router-id 1.1.2.1

network 10.0.3.0 0.0.0.255 area 12

network 172.16.0.128 0.0.0.63 area 0

SPOKEC:

interface Tunnel0

ip ospf network broadcast

ip ospf priority 0

!

router ospf 10

router-id 1.1.3.1

network 10.0.4.0 0.0.0.255 area 13

network 172.16.0.128 0.0.0.63 area 0

II-5 Next Hop Resolution Protocol – NHRP:

NHRP role is to discover address of other routers/hosts connected to a NBMA. Generally NBMA networks require tedious static configuration (static mapping between network layer address).

NHRP provides ARP like resolution and dynamically learn addresses of devices connected to the same NBMA networks from a next-hop server and then directly communicate them without passing through intermediate systems like in traditional Hub and Spoke topologies.

In our case the NHRP will facilitate building GRE tunnels. The HUB will maintain NHRP database of spokes mGRE virtual addresses associated to their public interface addresses. Each spoke registers its public address when it boots and queries the NHRP database for other spokes IP when needed.

On each tunnel interface, NHRP must be enabled and identified, it is recommended that the NHRP network-id match tunnel key.

NHRP participants can optionally be authenticated.

Because NHRP is a client-server protocol, the server (HUB) doesn’t need to know clients, nevertheless clients have to know the server.

Spokes explicitly set HUB as the next-hop server and statically tie the HUB virtual tunnel IP to the HUB physical interface s1/0. In the same manner spokes must map multicast forwarding to the HUB virtual mGRE IP address.

The same configuration is required in the HUB site but without any mapping or next-hop servers just map multicast forwarding to any new dynamically created NHRP adjacency.

HUB:

interface Tunnel0

ip nhrp authentication cisco

ip nhrp map multicast dynamic

ip nhrp network-id 1

ip nhrp cache non-authoritative

ip ospf network broadcast

SPOKEA:

interface Tunnel0

ip nhrp authentication cisco

ip nhrp map multicast 192.168.0.17

ip nhrp map 172.16.0.129 192.168.0.17

ip nhrp network-id 1

ip nhrp nhs 172.16.0.129

SPOKEB:

interface Tunnel0

ip nhrp authentication cisco

ip nhrp map multicast 192.168.0.17

ip nhrp map 172.16.0.129 192.168.0.17

ip nhrp network-id 1

ip nhrp nhs 172.16.0.129

SPOKEC:

interface Tunnel0

ip nhrp authentication cisco

ip nhrp map multicast 192.168.0.17

ip nhrp map 172.16.0.129 192.168.0.17

ip nhrp network-id 1

ip nhrp nhs 172.16.0.129

II-6 IPSec VPN:

The only thing to do after setting IPSec phase 1 and phase 2 parameters is to define an IPSec profile and assign it to the mGRE tunnel interface.

Table3: IPSec parameters

PHASE 1

IKE parameters

IKE seq 1 Auth. (def. rsa-sig) pre-shared Encr. (def. des) 3des DH (def. group1) 2 Hash (def. sha) sha Lifetime (def. 86400) 86400 Preshared Key cisco Addr. – PHASE 2

Profile IPSecprofile Transform set ESP-3DES-SHA esp-3des esp-sha-hmac – Mode Transport

!!IKE phase1 – ISAKMP

crypto isakmp policy 1

!! Symmetric key algorithm for data encryption

encr 3des

!! authentication type using static key between participants

authentication pre-share

!! asymmetric algorithm for establishing shared symmetric key across the network

group 2

!! preshared key and peer (dynamic)

crypto isakmp key cisco address 0.0.0.0 0.0.0.0

!!IKE phase2 – IPSEec

crypto ipsec transform-set ESP-3DES-SHA esp-3des esp-sha-hmac

mode transport

crypto ipsec profile IPSecProfile

set transform-set ESP-3DES-SHA

interface Tunnel0

!!encapsulate IPSec inside mGRE

tunnel protection ipsec profile IPSecProfile

III – MONITORING

HUB:

First of all, Make sure those s1/0 and tunnel interfaces are up/up, otherwise review your static routing statements.

HUB#sh ip int brief

Interface IP-Address OK? Method Status Protocol

FastEthernet0/0 unassigned YES NVRAM administratively down down

Serial1/0 192.168.0.17 YES NVRAM up up

Serial1/1 unassigned YES NVRAM administratively down down

Serial1/2 unassigned YES NVRAM administratively down down

Serial1/3 unassigned YES NVRAM administratively down down

Loopback0 10.0.1.1 YES NVRAM up up

Loopback1 1.1.0.1 YES manual up up

Tunnel0 172.16.0.129 YES NVRAM up up

HUB#

After spokes have registered to the HUB through NHRP protocol, the HUB have dynamically learned IP addresses of all participants of the mGRE (logical topology) and know how to reach them through the physical topology.

Spokes mGRE tunnel IP addresses are mapped to their physical public routable IP’s.

HUB#sh ip nhrp

172.16.0.130/32 via 172.16.0.130, Tunnel0 created 02:40:11, expire 01:22:18

Type: dynamic, Flags: unique registered

NBMA address: 192.168.0.13

172.16.0.131/32 via 172.16.0.131, Tunnel0 created 02:40:14, expire 01:21:45

Type: dynamic, Flags: unique registered

NBMA address: 192.168.0.9

172.16.0.132/32 via 172.16.0.132, Tunnel0 created 02:40:14, expire 01:21:54

Type: dynamic, Flags: unique registered

NBMA address: 192.168.0.6

HUB#

The HUB establishes mGRE tunnel with all spokes and initiate OSPF neighbor relationship with each one of them.

All routers will exchange routing information through the HUB.

HUB#sh ip ospf neigh

Neighbor ID Pri State Dead Time Address Interface

1.1.1.1 0 FULL/DROTHER 00:00:32 172.16.0.130 Tunnel0

1.1.2.1 0 FULL/DROTHER 00:00:35 172.16.0.131 Tunnel0

1.1.3.1 0 FULL/DROTHER 00:00:32 172.16.0.132 Tunnel0

HUB#

Now the HUB knows about all spokes local networks.

HUB#sh ip route

Codes: C – connected, S – static, R – RIP, M – mobile, B – BGP

D – EIGRP, EX – EIGRP external, O – OSPF, IA – OSPF inter area

N1 – OSPF NSSA external type 1, N2 – OSPF NSSA external type 2

E1 – OSPF external type 1, E2 – OSPF external type 2

i – IS-IS, su – IS-IS summary, L1 – IS-IS level-1, L2 – IS-IS level-2

ia – IS-IS inter area, * – candidate default, U – per-user static route

o – ODR, P – periodic downloaded static route

Gateway of last resort is 192.168.0.18 to network 0.0.0.0

1.0.0.0/32 is subnetted, 1 subnets

C 1.1.0.1 is directly connected, Loopback1

172.16.0.0/26 is subnetted, 2 subnets

C 172.16.0.128 is directly connected, Tunnel0

S 172.16.0.0 is directly connected, Serial1/0

10.0.0.0/8 is variably subnetted, 4 subnets, 2 masks

O IA 10.0.3.1/32 [110/11112] via 172.16.0.131, 02:16:51, Tunnel0

O IA 10.0.2.1/32 [110/11112] via 172.16.0.130, 02:16:51, Tunnel0

C 10.0.1.0/24 is directly connected, Loopback0

O IA 10.0.4.1/32 [110/11112] via 172.16.0.132, 02:16:51, Tunnel0

192.168.0.0/30 is subnetted, 1 subnets

C 192.168.0.16 is directly connected, Serial1/0

S* 0.0.0.0/0 [1/0] via 192.168.0.18

HUB#

HUB#ping

Protocol [ip]:

Target IP address: 10.0.2.1

Repeat count [5]:

Datagram size [100]:

Timeout in seconds [2]:

Extended commands [n]: y

Source address or interface: 10.0.1.1

Type of service [0]:

Set DF bit in IP header? [no]:

Validate reply data? [no]:

Data pattern [0xABCD]:

Loose, Strict, Record, Timestamp, Verbose[none]:

Sweep range of sizes [n]:

Type escape sequence to abort.

Sending 5, 100-byte ICMP Echos to 10.0.2.1, timeout is 2 seconds:

Packet sent with a source address of 10.0.1.1

!!!!!

Success rate is 40 percent (2/5), round-trip min/avg/max = 80/116/152 ms

HUB#

HUB#ping

Protocol [ip]:

Target IP address: 10.0.4.1

Repeat count [5]:

Datagram size [100]:

Timeout in seconds [2]:

Extended commands [n]: y

Source address or interface: 10.0.1.1

Type of service [0]:

Set DF bit in IP header? [no]:

Validate reply data? [no]:

Data pattern [0xABCD]:

Loose, Strict, Record, Timestamp, Verbose[none]:

Sweep range of sizes [n]:

Type escape sequence to abort.

Sending 5, 100-byte ICMP Echos to 10.0.4.1, timeout is 2 seconds:

Packet sent with a source address of 10.0.1.1

!!!!!

Success rate is 60 percent (3/5), round-trip min/avg/max = 104/128/144 ms

HUB#

SPOKEA:

Routing information is exchanged between all routers via the HUB, this is the only OSPF neighbor relationship spokes establish.

SPOKEA#sh ip ospf neigh

Neighbor ID Pri State Dead Time Address Interface

1.1.0.1 1 FULL/DR 00:00:32 172.16.0.129 Tunnel0

SPOKEA#

Nevertheless, advertised networks are directly reachable through the router that announced them.

SPOKEA#sh ip route

…

Gateway of last resort is 192.168.0.14 to network 0.0.0.0

1.0.0.0/32 is subnetted, 1 subnets

C 1.1.1.1 is directly connected, Loopback1

172.16.0.0/26 is subnetted, 1 subnets

C 172.16.0.128 is directly connected, Tunnel0

10.0.0.0/8 is variably subnetted, 4 subnets, 2 masks

O IA 10.0.3.1/32 [110/11112] via 172.16.0.131, 00:01:25, Tunnel0

C 10.0.2.0/24 is directly connected, Loopback0

O IA 10.0.1.1/32 [110/11112] via 172.16.0.129, 00:01:25, Tunnel0

O IA 10.0.4.1/32 [110/11112] via 172.16.0.132, 00:01:25, Tunnel0

192.168.0.0/24 is variably subnetted, 2 subnets, 2 masks

C 192.168.0.12/30 is directly connected, Serial1/0

S 192.168.0.17/32 is directly connected, Serial1/0

S* 0.0.0.0/0 [1/0] via 192.168.0.14

SPOKEA#

Spoke A knows about the next-hop server, the HUB, through a static map statement and will register by announcing to the HUB its physical public routable IP address assigned to its virtual mGRE IP.

SPOKEA#sh ip nhrp

172.16.0.129/32 via 172.16.0.129, Tunnel0 created 00:02:03, never expire

Type: static, Flags: authoritative used

NBMA address: 192.168.0.17

When spokeA wants to communicate with 10.0.4.1/32 it inspects the routing table and will find that 10.0.4.1 is reachable though the next-hop mGRE address 172.6.0.129.

SpokeA will ask the HUB about the public IP address that corresponds to 172.16.0.129, the HUB will reply with 192.168.0.6; only then, the spoke will send the traffic directly to SPOKEC.

According to the outputs of “debug tunnel” and a “traceroute” commands on spokeB, you can note that the next-hop of the mGRE network is the final destination and there is no intermediate hops, this is confirmed by the delivery through the physical topology where mGRE traffic is encapsulated and sent directly from 192.168.0.6 to 192.168.0.9 with no intermediate hops.

SPOKEB#debug tunnel

Tunnel Interface debugging is on

SPOKEB#traceroute 172.16.0.132

Type escape sequence to abort.

Tracing the route to 172.16.0.132

1 172.16.0.132 80 msec

*Mar 1 00:03:55.839: Tunnel0: GRE/IP to decaps 192.168.0.6->192.168.0.9 (len=84 ttl=253)

*Mar 1 00:03:55.843: Tunnel0: GRE decapsulated IP 172.16.0.132->172.16.0.131 (len=56, ttl=255) * 44 msec

SPOKEB#

Because DMVPN involves several concepts multipoint GRE, NHRP, dynamic routing and IPSec, troubleshooting issues can be very time consuming, so the best practice is to focus on each step and make sure things work well before configuring the next step.