The BIG lab - Part 7 - GRE Tunnels and recursive routing (2.20)


Now that our networks are all talking to each other, following the previous redistribution post, we can start playing around with them a bit more. So let's create a tunnel, actually we are going to create a few tunnels, but only one today.

Creating a tunnel is very straight forward, but there is also one major "gotcha". A tunnel must have a route to its destination. So if we have a tunnel from router A to router C via router B router A must know of a way to get to router C. This can either be static routes, or learned through an IGP (or as we will be doing, learned through redistribution). Nothing too complex so far, right? But what if router A learns of its route to router C through the tunnel - i.e. if the tunnel offers a better metric, well then we are going to start seeing problems.

But let's cross that bridge when, and hopefully when, we come to it.

For today we are going to create a tunnel from the loopback 0 on R7 (10.60.1.1) which is in our RIPv2 network and connect it to loopback 1 on R10 (172.20.30.1) , which is in our EIGRP network.

So firstly lets see if they still can see each other after yesterday's redistribution exercise:
R7#sh ip route | beg Gate
Gateway of last resort is not set

      1.0.0.0/24 is subnetted, 1 subnets
R        1.2.3.0 [120/4] via 10.20.3.1, 00:00:24, Serial0/1
      10.0.0.0/8 is variably subnetted, 17 subnets, 2 masks
R        10.1.1.0/24 [120/4] via 10.20.3.1, 00:00:24, Serial0/1
R        10.1.2.0/24 [120/4] via 10.20.3.1, 00:00:24, Serial0/1
R        10.1.3.0/24 [120/4] via 10.20.3.1, 00:00:24, Serial0/1
R        10.1.4.0/24 [120/4] via 10.20.3.1, 00:00:24, Serial0/1
R        10.1.5.0/24 [120/4] via 10.20.3.1, 00:00:24, Serial0/1
R        10.1.6.0/24 [120/3] via 10.20.3.1, 00:00:24, Serial0/1
R        10.10.1.0/24 [120/4] via 10.20.3.1, 00:00:24, Serial0/1
R        10.20.1.0/24 [120/2] via 10.20.3.1, 00:00:24, Serial0/1
R        10.20.2.0/24 [120/1] via 10.20.3.1, 00:00:24, Serial0/1
C        10.20.3.0/24 is directly connected, Serial0/1
L        10.20.3.2/32 is directly connected, Serial0/1
R        10.25.1.0/24 [120/3] via 10.20.3.1, 00:00:24, Serial0/1
R        10.30.1.0/24 [120/3] via 10.20.3.1, 00:00:24, Serial0/1
R        10.31.1.0/24 [120/3] via 10.20.3.1, 00:00:24, Serial0/1
R        10.35.1.0/24 [120/2] via 10.20.3.1, 00:00:24, Serial0/1
C        10.60.1.0/24 is directly connected, Loopback0
L        10.60.1.1/32 is directly connected, Loopback0
      172.20.0.0/16 is variably subnetted, 3 subnets, 2 masks
R        172.20.20.0/24 [120/3] via 10.20.3.1, 00:00:25, Serial0/1
R        172.20.30.0/24 [120/3] via 10.20.3.1, 00:00:25, Serial0/1
R        172.20.40.0/23 [120/3] via 10.20.3.1, 00:00:25, Serial0/1

So R7 can see R10's Lo1, and has learned it via RIP (metric 120).

R10#sh ip route | beg Gate
Gateway of last resort is not set

      1.0.0.0/24 is subnetted, 1 subnets
D EX     1.2.3.0 [170/2937856] via 10.31.1.1, 00:00:51, Serial1/0
      10.0.0.0/8 is variably subnetted, 16 subnets, 2 masks
D EX     10.1.1.0/24 [170/2937856] via 10.31.1.1, 00:00:51, Serial1/0
D EX     10.1.2.0/24 [170/2937856] via 10.31.1.1, 00:00:51, Serial1/0
D EX     10.1.3.0/24 [170/2937856] via 10.31.1.1, 00:00:51, Serial1/0
D EX     10.1.4.0/24 [170/2937856] via 10.31.1.1, 00:00:51, Serial1/0
D EX     10.1.5.0/24 [170/2937856] via 10.31.1.1, 00:00:51, Serial1/0
D        10.1.6.0/24 [90/3193856] via 10.31.1.1, 00:01:48, Serial1/0
D EX     10.10.1.0/24 [170/2937856] via 10.31.1.1, 00:00:51, Serial1/0
D        10.20.1.0/24 [90/3193856] via 10.31.1.1, 00:11:53, Serial1/0
D        10.20.2.0/24 [90/3705856] via 10.31.1.1, 00:11:53, Serial1/0
D EX     10.20.3.0/24 [170/2937856] via 10.31.1.1, 00:04:04, Serial1/0
D        10.25.1.0/24 [90/2681856] via 10.31.1.1, 00:11:53, Serial1/0
D        10.30.1.0/24 [90/2681856] via 10.31.1.1, 00:11:53, Serial1/0
C        10.31.1.0/24 is directly connected, Serial1/0
L        10.31.1.2/32 is directly connected, Serial1/0
D        10.35.1.0/24 [90/41536000] via 10.31.1.1, 00:11:53, Serial1/0
D EX     10.60.1.0/24 [170/2937856] via 10.31.1.1, 00:04:04, Serial1/0
      172.20.0.0/16 is variably subnetted, 9 subnets, 3 masks
C        172.20.20.0/24 is directly connected, Loopback0
L        172.20.20.1/32 is directly connected, Loopback0
C        172.20.30.0/24 is directly connected, Loopback1
L        172.20.30.1/32 is directly connected, Loopback1
D        172.20.40.0/23 is a summary, 00:11:55, Null0
C        172.20.40.0/24 is directly connected, Loopback2
L        172.20.40.1/32 is directly connected, Loopback2
C        172.20.41.0/24 is directly connected, Loopback3
L        172.20.41.1/32 is directly connected, Loopback3

R10 also knows of R7's Lo0, which it has learned from Eigrp External (D EX) with an AD of 170.

Note - if we know exactly what IP prefix we are looking for, then we can just do:

R10#sh ip route | i 10.60           

D EX     10.60.1.0/24 [170/2937856] via 10.31.1.1, 00:07:48, Serial1/0

Now let's start setting up our tunnels.

Creating a GRE tunnels on Cisco IOS

Creating a GRE tunnel only takes a couple of commands, and really is not much different than creating any non-physical interface:
R7#conf t
Enter configuration commands, one per line.  End with CNTL/Z.
R7(config)#interface tunnel0
R7(config-if)#ip address 10.120.1.1 255.255.255.0
R7(config-if)#tunnel source Lo0
R7(config-if)#tunnel destination 172.20.30.1
R7(config-if)#
*Nov  8 09:27:10.335: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel0, changed state to up
R7(config-if)#

R10#conf t
Enter configuration commands, one per line.  End with CNTL/Z.
R10(config)#interface tunnel 0
R10(config-if)#ip address 10.120.1.2 255.255.255.0
R10(config-if)#tunnel source lo1
R10(config-if)#tunnel destination 10.60.1.1
R10(config-if)#
*Nov  8 09:28:19.371: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel0, changed state to up
R10(config-if)#
We can test that our tunnel is up by pinging the tunnel interface IP on the other side:
R7#ping 10.120.1.2

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.120.1.2, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 40/43/44 ms

R10#ping 10.120.1.1

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.120.1.1, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 40/42/44 ms
GRE (Generic Routing Encapsulation) is the default for these tunnels:

R10#sh int tunnel 0
Tunnel0 is up, line protocol is up 
  Hardware is Tunnel
  Internet address is 10.120.1.2/24
  MTU 17916 bytes, BW 100 Kbit/sec, DLY 50000 usec, 
     reliability 255/255, txload 1/255, rxload 1/255
  Encapsulation TUNNEL, loopback not set
  Keepalive not set
  Tunnel source 172.20.30.1 (Loopback1), destination 10.60.1.1
   Tunnel Subblocks:
      src-track:
         Tunnel0 source tracking subblock associated with Loopback1
          Set of tunnels with source Loopback1, 1 member (includes iterators), on interface <OK>
  Tunnel protocol/transport GRE/IP
    Key disabled, sequencing disabled
    Checksumming of packets disabled
<output omitted>

But we can switch to a number of modes using the command "tunnel mode" and then the mode, useful if the ISP(s) do not like GRE for any reason, or if we want to ensure our tunnel is IPv6, or for setting up an MPLS tunnel:

R7(config-if)#tunnel mode ?
  aurp       AURP TunnelTalk AppleTalk encapsulation
  cayman  Cayman TunnelTalk AppleTalk encapsulation
  dvmrp    DVMRP multicast tunnel
  eon         EON compatible CLNS tunnel
  gre         generic route encapsulation protocol
  ipip        IP over IP encapsulation
  ipsec      IPSec tunnel encapsulation
  iptalk     Apple IPTalk encapsulation
  ipv6       Generic packet tunneling in IPv6
  ipv6ip    IPv6 over IP encapsulation
  mpls      MPLS encapsulations
  nos        IP over IP encapsulation (KA9Q/NOS compatible)
  rbscp     RBSCP in IP tunnel

It's important to note that both sides of the tunnel must match.  

So far our tunnel has been rock-solid, so we could just end this session here. But where would the fun in that be? Let's make some recursive routing
R5(config)#int lo0
R5(config-if)#ip address 10.1.7.1 255.255.255.0
R5(config-if)#int tunnel 0
*Nov  8 10:11:05.475: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel0, changed state to down
R5(config-if)#ip address 10.120.2.1 255.255.255.0
R5(config-if)#tunnel source lo0
R5(config-if)#tunnel destination 172.20.40.1
R5(config-if)#
*Nov  8 10:11:50.195: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel0, changed state to up

R10#conf t
Enter configuration commands, one per line.  End with CNTL/Z.
R10(config)#int tunnel 1
R10(config-if)#ip address 10.120.2.2 255.255.255.0
R10(config-if)#tunnel source lo2
R10(config-if)#tunnel destination 10.1.7.1
R10(config-if)#
OK, so far so good, the tunnel comes up... but then goes down, then then up and down, and you get the general idea :

*Nov  8 10:11:50.195: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel0, changed state to up
R5(config-if)#
*Nov  8 10:12:59.619: %DUAL-5-NBRCHANGE: EIGRP-IPv4 10: Neighbor 10.120.2.2 (Tunnel0) is up: new adjacency
*Nov  8 10:12:59.731: %ADJ-5-PARENT: Midchain parent maintenance for IP midchain out of Tunnel0 - looped chain attempting to stack
R5(config-if)#
*Nov  8 10:13:04.491: %TUN-5-RECURDOWN: Tunnel0 temporarily disabled due to recursive routing
*Nov  8 10:13:05.491: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel0, changed state to down
*Nov  8 10:13:05.491: %DUAL-5-NBRCHANGE: EIGRP-IPv4 10: Neighbor 10.120.2.2 (Tunnel0) is down: interface down
R5(config-if)#
*Nov  8 10:14:06.495: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel0, changed state to up
*Nov  8 10:14:06.783: %DUAL-5-NBRCHANGE: EIGRP-IPv4 10: Neighbor 10.120.2.2 (Tunnel0) is up: new adjacency
*Nov  8 10:14:06.839: %ADJ-5-PARENT: Midchain parent maintenance for IP midchain out of Tunnel0 - looped chain attempting to stack

R10(config-if)#
*Nov  8 10:13:14.063: %DUAL-5-NBRCHANGE: EIGRP-IPv4 10: Neighbor 10.120.2.1 (Tunnel1) is down: holding time expired
R10(config-if)#
*Nov  8 10:14:06.123: %DUAL-5-NBRCHANGE: EIGRP-IPv4 10: Neighbor 10.120.2.1 (Tunnel1) is up: new adjacency
R10(config-if)#
*Nov  8 10:14:21.215: %DUAL-5-NBRCHANGE: EIGRP-IPv4 10: Neighbor 10.120.2.1 (Tunnel1) is down: holding time expired
R10(config-if)#

The main error we need to look for is %TUN-5-RECURDOWN: Tunnel0 temporarily disabled to to recursive routing.

To fix this we start by shutting down the tunnel interface and give our routing protocols a few seconds to recover.

R10 learned the route to R5's 10.1.7.1 interface through EIGRP, and R5 learned of R10's Lo2 interface through EIGRP as well. But clearly this is causing problems with both EIGRP and the tunnel, in fact it's caused recursion. Where our tunnel from R7 to R10 caused us no issues going from EIGRP to RIP, a tunnel from R5 to R10 actually tries to form a new EIGRP adjacency, and hence will actually try to use the tunnel for the adjacency, thereby losing the actual route to the tunnel end-point.

So what can we do here? 

The easiest method is to add a static route, remember from the post on redistribution that routing just cares about metrics, with lower being the preferred metric.

Routing Protocol Administrative Distance
Connected interface 0
Static route 1
EIGRP Summary route 5
External BGP 20
Internal EIGRP 90
IGRP 100
OSPF 110
IS-IS 115
RIP 120
EGP 140
ODR 160
External EIGRP 170
Internal BGP 200
As R5 has two links to R10 we can create two static routes (with different metrics) to R10, but R10 only needs one static route:
R5(config)#ip route 172.20.40.0 255.255.255.0 s0/0 1
R5(config)#ip route 172.20.40.0 255.255.255.0 s0/2 10

R10(config)#ip route 10.1.7.1 255.255.255.255 s1/0
Note that we have different metrics on R5, so that if interface serial0/0 (preferred) goes down then serial 0/2 can be used, but at the same time will not interfere with out EIGRP operations.

Once we have done a "no shut" on our tunnel interface the tunnel reestablishes and our EIGRP operations are not affected:

R5(config)#int tunnel 0
R5(config-if)#no shut
*Nov  8 10:38:02.903: %LINK-3-UPDOWN: Interface Tunnel0, changed state to up
*Nov  8 10:38:03.907: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel0, changed state to up
R5(config-if)#exit
R5(config)#
*Nov  8 10:38:04.215: %DUAL-5-NBRCHANGE: EIGRP-IPv4 10: Neighbor 10.120.2.2 (Tunnel0) is up: new adjacency
R5(config)#exit
*Nov  8 10:38:10.471: %SYS-5-CONFIG_I: Configured from console by console
R5#sh ip route | beg Gate
Gateway of last resort is not set

      1.0.0.0/24 is subnetted, 1 subnets
R        1.2.3.0 [120/2] via 10.20.1.1, 00:00:25, Serial0/0
      10.0.0.0/8 is variably subnetted, 23 subnets, 2 masks
R        10.1.1.0/24 [120/2] via 10.20.1.1, 00:00:25, Serial0/0
R        10.1.2.0/24 [120/2] via 10.20.1.1, 00:00:25, Serial0/0
R        10.1.3.0/24 [120/2] via 10.20.1.1, 00:00:25, Serial0/0
R        10.1.4.0/24 [120/2] via 10.20.1.1, 00:00:25, Serial0/0
R        10.1.5.0/24 [120/2] via 10.20.1.1, 00:00:25, Serial0/0
D        10.1.6.0/24 [90/2681856] via 10.20.1.1, 01:20:52, Serial0/0
C        10.1.7.0/24 is directly connected, Loopback0
L        10.1.7.1/32 is directly connected, Loopback0
R        10.10.1.0/24 [120/2] via 10.20.1.1, 00:00:25, Serial0/0
C        10.20.1.0/24 is directly connected, Serial0/0
L        10.20.1.2/32 is directly connected, Serial0/0
C        10.20.2.0/24 is directly connected, Serial0/1
L        10.20.2.1/32 is directly connected, Serial0/1
R        10.20.3.0/24 [120/1] via 10.20.2.2, 00:00:05, Serial0/1
D        10.25.1.0/24 [90/2681856] via 10.20.1.1, 01:30:59, Serial0/0
D        10.30.1.0/24 [90/3193856] via 10.20.1.1, 01:31:00, Serial0/0
D        10.31.1.0/24 [90/3193856] via 10.20.1.1, 00:01:22, Serial0/0
C        10.35.1.0/24 is directly connected, Serial0/2
L        10.35.1.1/32 is directly connected, Serial0/2
R        10.60.1.0/24 [120/2] via 10.20.2.2, 00:00:06, Serial0/1
D        10.120.1.0/24 [90/28160000] via 10.120.2.2, 00:01:23, Tunnel0
C        10.120.2.0/24 is directly connected, Tunnel0
L        10.120.2.1/32 is directly connected, Tunnel0
      172.20.0.0/16 is variably subnetted, 5 subnets, 2 masks
D        172.20.20.0/24 [90/3321856] via 10.20.1.1, 00:01:23, Serial0/0
D        172.20.30.0/24 [90/3321856] via 10.20.1.1, 00:01:23, Serial0/0
D        172.20.40.0/23 [90/3321856] via 10.20.1.1, 01:30:58, Serial0/0
S        172.20.40.0/24 is directly connected, Serial0/0
D        172.20.41.0/24 [90/27008000] via 10.120.2.2, 00:01:23, Tunnel0
R5#

We can see on R5 that the network 172.20.40.0/24 is learnt through a static route (denoted by an S), and through EIGRP (D). It will be the static route that is the preferred route.

R10#sh ip route | beg Gate
Gateway of last resort is not set

      1.0.0.0/24 is subnetted, 1 subnets
D EX     1.2.3.0 [170/2937856] via 10.31.1.1, 00:01:03, Serial1/0
      10.0.0.0/8 is variably subnetted, 22 subnets, 2 masks
D EX     10.1.1.0/24 [170/2937856] via 10.31.1.1, 00:01:03, Serial1/0
D EX     10.1.2.0/24 [170/2937856] via 10.31.1.1, 00:01:03, Serial1/0
D EX     10.1.3.0/24 [170/2937856] via 10.31.1.1, 00:01:03, Serial1/0
D EX     10.1.4.0/24 [170/2937856] via 10.31.1.1, 00:01:03, Serial1/0
D EX     10.1.5.0/24 [170/2937856] via 10.31.1.1, 00:01:03, Serial1/0
D        10.1.6.0/24 [90/3193856] via 10.31.1.1, 00:01:03, Serial1/0
D        10.1.7.0/24 [90/3321856] via 10.31.1.1, 00:01:03, Serial1/0
S        10.1.7.1/32 is directly connected, Serial1/0
D EX     10.10.1.0/24 [170/2937856] via 10.31.1.1, 00:01:03, Serial1/0
D        10.20.1.0/24 [90/3193856] via 10.31.1.1, 00:01:03, Serial1/0
D        10.20.2.0/24 [90/3705856] via 10.31.1.1, 00:01:03, Serial1/0
D EX     10.20.3.0/24 [170/2937856] via 10.31.1.1, 00:01:03, Serial1/0
D        10.25.1.0/24 [90/2681856] via 10.31.1.1, 00:01:03, Serial1/0
D        10.30.1.0/24 [90/2681856] via 10.31.1.1, 00:01:03, Serial1/0
C        10.31.1.0/24 is directly connected, Serial1/0
L        10.31.1.2/32 is directly connected, Serial1/0
D        10.35.1.0/24 [90/41536000] via 10.31.1.1, 00:01:03, Serial1/0
D EX     10.60.1.0/24 [170/2937856] via 10.31.1.1, 00:01:03, Serial1/0
C        10.120.1.0/24 is directly connected, Tunnel1
L        10.120.1.2/32 is directly connected, Tunnel1
C        10.120.2.0/24 is directly connected, Tunnel0
L        10.120.2.2/32 is directly connected, Tunnel0
      172.20.0.0/16 is variably subnetted, 9 subnets, 3 masks
C        172.20.20.0/24 is directly connected, Loopback0
L        172.20.20.1/32 is directly connected, Loopback0
C        172.20.30.0/24 is directly connected, Loopback1
L        172.20.30.1/32 is directly connected, Loopback1
D        172.20.40.0/23 is a summary, 01:30:39, Null0
C        172.20.40.0/24 is directly connected, Loopback2
L        172.20.40.1/32 is directly connected, Loopback2
C        172.20.41.0/24 is directly connected, Loopback3
L        172.20.41.1/32 is directly connected, Loopback3
R10#

Again R10 has learned of the 10.1.7.1/32 host through our static command, but also of the 10.1.7/0/24 network through EIGRP. Again the static route will be the one preferred.

And that is how we set up a tunnel between two routers, and fix route recursion.

In part 8 will will look at how we can filter routes, as well as changing how routes are advertised.

CCIE #49337, author of CCNA and Beyond, BGP for Cisco Networks, MPLS for Cisco Networks, VPNs and NAT for Cisco Networks.

Related Posts

Previous
Next Post »