What is CEF, and why do we use it?
CEF stands for Cisco Express Forwarding. It is (as the name suggests) proprietary to Cisco. It operates at layer 3, and is designed to speed up packet switching, CEF supports ethernet, Frame Relay, PPP, ATM, FDDI, tunnels and HDLC.
CEF keeps two tables, the FIB (Forwarding Information Base), and the adjacency table, which contains the layer 2 information for each particular FIB entry. There are different adjacencies for different scenarios:
- Cache adjacency: This type of entry contains the correct outbound interface and the correct MAC address for its FIB entry. The MAC address is the IP address's MAC address if the destination's subnet is directly connected to the router, or is the MAC address of the router that the packet needs to be sent to if the destination's subnet is not directly connected to the router currently processing the packet.
- Receive adjacency: This type of entry handles packets whose final destinations include the router itself. This includes packets whose IP addresses are assigned to the router itself, broadcast packets, and multicasts that have set up the router itself as one of the destinations.
- Null adjacency: Handles packets destined to a NULL interface. Packets with FIB entries pointing to NULL adjacencies will normally be dropped.
- Punt adjacency: Deals with packets that require special handling or can not be switched by CEF. Such packets are forwarded to the next switching layer (generally fast switching) where they can be forwarded correctly.
- Glean adjacency: This adjacency is created when the router knows that either the destination IP's subnet is directly connected to the router itself and it does not know that destination device's MAC address, or the router knows the IP address of the router to forward a packet to for a destination, but it does not know that router's MAC address. Packets that trigger this entry will generate an ARP request.
- Discard adjacency: FIB entries pointing to this type of adjacency will be discarded.
- Drop adjacency: Packets pointing to this entry are dropped, but the prefix will be checked.
There are two forms of CEF; Central CEF and Distributed CEF (dCEF). Central CEF which is available on the lower to mid-range devices uses the Route Processor (RP), higher end models use dCEF which splits the load over line cards, avoiding having to use the RP.
How do we use CEF and what is reliant on it?
CEF is enabled by default, and has been since IOS 12.0, if for any reason CEF is not enabled then the IOS will revert to process switching, which is a bit slower than CEF.
We will go through some of the basics of CEF with a simple three router example:
So far just the interface IPs have been set on the interfaces.
Looking at R1 we can see that the routing table (RIB) looks good.
R1#sh ip route | beg Gate
Gateway of last resort is not set
10.0.0.0/8 is variably subnetted, 4 subnets, 2 masks
C 10.1.1.0/24 is directly connected, GigabitEthernet1/0
L 10.1.1.1/32 is directly connected, GigabitEthernet1/0
C 10.3.1.0/24 is directly connected, GigabitEthernet2/0
L 10.3.1.1/32 is directly connected, GigabitEthernet2/0
R1#
And it's ARP cache also looks good:
R1#sh arp
Protocol Address Age (min) Hardware Addr Type Interface
Internet 10.1.1.1 - ca00.0253.001c ARPA GigabitEthernet1/0
Internet 10.1.1.2 4 ca01.0253.001c ARPA GigabitEthernet1/0
Internet 10.3.1.1 - ca00.0253.0038 ARPA GigabitEthernet2/0
Internet 10.3.1.3 3 ca02.0253.001c ARPA GigabitEthernet2/0
R1#
We can also check that CEF is enabled, either through a sh run:
R1#sh run
Building configuration...
Current configuration : 1130 bytes
!
! Last configuration change at 12:37:25 UTC Sat Mar 15 2014
upgrade fpd auto
version 15.1
service timestamps debug datetime msec
service timestamps log datetime msec
no service password-encryption
!
hostname R1
!
boot-start-marker
boot-end-marker
!
!
!
no aaa new-model
!
ip source-route
no ip icmp rate-limit unreachable
ip cef
!
Or through the command "sh ip cef" which will show us the FIB table:
R1#sh ip cef
Prefix Next Hop Interface
0.0.0.0/0 no route
0.0.0.0/8 drop
0.0.0.0/32 receive
10.1.1.0/24 attached GigabitEthernet1/0
10.1.1.0/32 receive GigabitEthernet1/0
10.1.1.1/32 receive GigabitEthernet1/0
10.1.1.2/32 attached GigabitEthernet1/0
10.1.1.255/32 receive GigabitEthernet1/0
10.3.1.0/24 attached GigabitEthernet2/0
10.3.1.0/32 receive GigabitEthernet2/0
10.3.1.1/32 receive GigabitEthernet2/0
10.3.1.3/32 attached GigabitEthernet2/0
10.3.1.255/32 receive GigabitEthernet2/0
127.0.0.0/8 drop
224.0.0.0/4 drop
224.0.0.0/24 receive
240.0.0.0/4 drop
255.255.255.255/32 receive
R1#
Here we can see the FIB along with the different adjacency tables as well. We can confirm our adjacencies are as expected by looking directly at the adjacencies table:
R1#sh adjacency
Protocol Interface Address
IP GigabitEthernet1/0 10.1.1.2(9)
IP GigabitEthernet2/0 10.3.1.3(9)
R1#
We can create a dummy entry for 0.0.0.0/0 pointing to Null0 and see the change in our FIB:
R1#conf t
Enter configuration commands, one per line. End with CNTL/Z.
R1(config)#ip route 0.0.0.0 0.0.0.0 null0
R1(config)#exit
R1#
*Mar 15 12:53:14.443: %SYS-5-CONFIG_I: Configured from console by console
R1#sh ip cef
Prefix Next Hop Interface
0.0.0.0/0 attached Null0
0.0.0.0/8 drop
0.0.0.0/32 receive
10.1.1.0/24 attached GigabitEthernet1/0
10.1.1.0/32 receive GigabitEthernet1/0
10.1.1.1/32 receive GigabitEthernet1/0
10.1.1.2/32 attached GigabitEthernet1/0
10.1.1.255/32 receive GigabitEthernet1/0
10.3.1.0/24 attached GigabitEthernet2/0
10.3.1.0/32 receive GigabitEthernet2/0
10.3.1.1/32 receive GigabitEthernet2/0
10.3.1.3/32 attached GigabitEthernet2/0
10.3.1.255/32 receive GigabitEthernet2/0
127.0.0.0/8 drop
224.0.0.0/4 drop
224.0.0.0/24 receive
240.0.0.0/4 drop
255.255.255.255/32 receive
R1#
We can use the command "sh ip cef" along with a ip address, or ip and prefix and get different results:
R1#sh ip cef 0.0.0.0/0
0.0.0.0/0
attached to Null0
R1#sh ip cef 0.0.0.0
0.0.0.0/32
receive
R1#
We can use CEF for load balancing. If we create another route on R1 pointing to R2 for the 0.0.0.0/0 route, we can use CEF to load balance the two routes (though to be fair both routes will still exist in the sh ip route output if CEF is turned off).
R1(config)#no ip route 0.0.0.0 0.0.0.0 null0
R1(config)#ip route 0.0.0.0 0.0.0.0 10.1.1.2
R1(config)#do sh ip route | beg Gate
Gateway of last resort is 10.1.1.2 to network 0.0.0.0
S* 0.0.0.0/0 [1/0] via 10.1.1.2
10.0.0.0/8 is variably subnetted, 4 subnets, 2 masks
C 10.1.1.0/24 is directly connected, GigabitEthernet1/0
L 10.1.1.1/32 is directly connected, GigabitEthernet1/0
C 10.3.1.0/24 is directly connected, GigabitEthernet2/0
L 10.3.1.1/32 is directly connected, GigabitEthernet2/0
R1(config)#ip route 0.0.0.0 0.0.0.0 10.3.1.3
R1(config)#do sh ip route | beg Gate
Gateway of last resort is 10.3.1.3 to network 0.0.0.0
S* 0.0.0.0/0 [1/0] via 10.3.1.3
[1/0] via 10.1.1.2
10.0.0.0/8 is variably subnetted, 4 subnets, 2 masks
C 10.1.1.0/24 is directly connected, GigabitEthernet1/0
L 10.1.1.1/32 is directly connected, GigabitEthernet1/0
C 10.3.1.0/24 is directly connected, GigabitEthernet2/0
L 10.3.1.1/32 is directly connected, GigabitEthernet2/0
R1(config)#
We can check CEF for the route:
R1#sh ip cef 0.0.0.0/0
0.0.0.0/0
nexthop 10.1.1.2 GigabitEthernet1/0
nexthop 10.3.1.3 GigabitEthernet2/0
R1#
And we can see that CEF is happy. If we look at the details we can also see that CEF is load-balancing:
R1#sh ip cef 0.0.0.0/0 detail
0.0.0.0/0, epoch 0, per-destination sharing
recursive via 10.1.1.2
attached to GigabitEthernet1/0
recursive via 10.3.1.3
attached to GigabitEthernet2/0
R1#
Per-destination is the default load-balancing, we can switch to per-packet using the following interface commands:
R1#conf t
Enter configuration commands, one per line. End with CNTL/Z.
R1(config)#int g1/0
R1(config-if)#ip load-sharing per-packet
R1(config-if)#int g2/0
R1(config-if)#ip load-sharing per-packet
R1(config-if)#
And again we can see what CEF is now doing:
R1#sh ip cef 0.0.0.0/0 detail
0.0.0.0/0, epoch 0, per-packet sharing
recursive via 10.1.1.2
attached to GigabitEthernet1/0
recursive via 10.3.1.3
attached to GigabitEthernet2/0
R1#
The caveat here though is that per-packet load-balancing usually means packets are delivered out of order. So generally per-destination is preferred. None of this really goes to show how useful CEF is though. On high end routers and switches it does take a large pressure off of the Route Processor, and routing will still use process switching, so what else requires CEF?
QoS
QoS depends heavily on CEF, NBAR requires it as does Class-Based packet marking using the set command, class-based policing and Auto QoS.
MPLS
MPLS also requires CEF. In MPLS networks packets that are labelled will be switched according to the LFIB (Label Forwarding Information Base). IP packets will be switched according to the CEF table. If an IP packet exits the router with an MPLS label it is CEF that needs to label the packet.
When should we not use CEF?
Its safer to leave CEF on, chances are your router will perform slower, and functionality (as explained above) will be missing if you turn if off. Should you do absolutely have to, or want to, disable it, then globally the command is "no ip cef", or for an interface "no ip route-cache cef".
