Troubleshooting EIGRP Route Flapping
This section discusses how to troubleshoot consistent EIGRP route flapping. The most important tool for troubleshooting this problem is the show ip eigrp event command. This command reveals which neighbor is updating and the metric with which it's updating. See Figure 7-27 for the flowchart for troubleshooting the EIGRP route flapping problem.
When troubleshooting EIGRP route-flap problems, a difference exists between the route disappearing from the routing table and the route timer in the routing table showing 00:00:00, as highlighted in Example 7-46.
)
Figure 7-27 Flowchart for Troubleshooting EIGRP Route Flapping
Example 7-46 Example of Routing Table That Shows the Update Timer Always at 00:00:00
Router A# show ip route 150.150.0.0 Routing entry for 150.150.0.0/16 Known via "eigrp 1", distance 90, metric 304128, type internal Last update from 10.1.1.2 on Ethernet 0, 00:00:00 ago
When the route timer in the routing table always shows 00:00:00, it doesn't necessarily mean that the router is constantly taking the route out and reinstalling it. It simply means that one of the router's neighbors is constantly updating the router with the route. The neighbor updating the route is not necessarily the best path to the route, but it is one possible path. The router simply refreshes the timer because it got an update from one of the neighbors. To truly verify that the router is taking out the route from the routing table and reinstalling it, use the debug ip routing. Example 7-47 demonstrates the output from this command on Router B.
Example 7-47 debug ip routing Command Output Verifies Whether a Route Is Being Installed
Router B# debug ip routing RT: add 150.150.0.0/16 via 10.1.1.2, eigrp metric [90/304128] RT: delete route to 150.150.0.0 via 10.1.1.2, eigrp metric [90/304128]
This debug shows all the routes that the routing table takes out and installs, although the output of the debug might be overwhelming to the routers. You can also use an access list to the debug so that the output shows only the routes in question. For example, if you want to do the debug only on the route 192.168.1.0/24 in the routing table, use an access list, as configured in Example 7-48.
Example 7-48 Using Access Lists to Limit debug ip routing Information
Router B#debug ip routing 1 access-list 1 permit 192.168.1.0 0.0.0.255 access-list 1 deny any
As previously mentioned, your best tool in troubleshooting EIGRP route flap is the show ip eigrp event command. By default, the router keeps a log of all EIGRP events. However, the log size is only 500 lines, which covers only a few hundred milliseconds of EIGRP events. The show ip eigrp event command provides you with a glimpse of EIGRP events that includes the neighbors that are updating the router with the route identified and the metric with which the neighbor updates the router.
Consider the network shown in Figure 7–28.
)
Figure 7-28 EIGRP Network Susceptible to EIGRP Route Flap
In Figure 7-28, a route of 150.150.0.0/16 in network cloud X gets passed to Router A from Routers B, C, and D. Router A chooses Router C as the next hop to network 150.150.0.0/16 and puts Routers B and D as the feasible successors to the network 150.150.0.0/16. Example 7-49 shows the pertinent configuration for all four routers.
Example 7-49 Configurations for Routers A, B, C, and D in Figure 7-28
Router A# interface ethernet 0 ip address 10.1.1.1 255.255.255.0 interface serial 0 ip address 10.1.2.1 255.255.255.0 Router B# interface serial 0 ip address 10.1.2.2 255.255.255.0 interface serial 1 ip address 10.1.3.1 255.255.255.0 Router C# interface ethernet 0 ip address 10.1.1.2 255.255.255.0 interface serial 0 ip address 10.1.4.1 255.255.255.0 Router D# interface ethernet 0 ip address 10.1.1.3 255.255.255.0 interface serial 0 ip address 10.1.5.1 255.255.255.0
The problem happens in Router A where the route timer for the route 150.150.0.0/16 in the routing table is constantly at 00:00:00. By looking at Router C, the next hop to the route, you can see that the route is stable and is not flapping. The neighbor relationship in Router A is also stable, and the interfaces on Router A are stable with no signs of interface flapping. The next step is to look at the event log in EIGRP and see which neighbor is updating Router A constantly about the route 150.150.0.0/16. Example 7-50 shows the relevant information in the EIGRP event log on Router A.
Example 7-50 show ip eigrp event Command Output on Router A
Router A# show ip eigrp event 20:47:13.2 Rcv update dest/nh: 150.150.0.0/16 10.1.1.3 20:47:13.2 Metric set: 150.150.0.0/16 4872198 20:47:13.2 Rcv update dest/nh: 150.150.0.0/16 10.1.1.3 20:47:13.2 Metric set: 150.150.0.0/16 4872198
Other output in the event log exists, but only the important lines are shown here. To make sure that the router is constantly getting updates, the show ip eigrp event command has to be done several times in succession. Check whether the timer on the left side of the output is constantly changing. If the timer is constantly changing, this indicates that the EIGRP process is constantly calculating. The EIGRP event log is read upside down, with the most recent event at the top of the list and the oldest event at the bottom of the list. The event log in Example 7-50 shows that Router A is constantly getting updates from 10.1.1.3 (Router D) for the route 150.150.0.0/16. Notice that the next-hop router that updates Router A does not reset the route timer in Router A. Any feasible successor that updates a router about a route resets the route timer on the router. Therefore, the route timers are reset, but the route stays in the routing table so that the router won't drop any packets.
From the EIGRP event log, it's Router D that constantly sends updates to Router A. The next step is to go to Router D to investigate why it is updating Router A with updates. One possible reason that this update is constantly occurring is that there is a routing loop in Router D for 150.150.0.0/16 route with other routers in network X, causing the routes to be sent to each other. If a routing loop occurs in the network, you need a current network diagram to go hop by hop to each router to track the routing loop.
Another possibility might be that the LAN switch on Router A's Ethernet 0 might have a spanning tree problem that keeps looping the packets from Router D to Router A.
If no routing loop is in the network and no spanning tree problem is on the switch, the other possibility is that Router D might be running into an EIGRP bug in which it is constantly sending out updates to Router A for no reason. One of the possible bugs might be CSCdt15109, in which the router constantly sends out updates that is not changing. Cisco IOS Software Release 12.1.7 and later will have the bug fix for this issue; however, it is always recommended to consult with Cisco TAC to determine whether the problem is caused by a software bug.
In this example, Router D is running into the software bug previously mentioned. Notice that the problem is not on Router A, but on Router D. Router D constantly sends out updates to Router A, and Router A constantly refreshes its timer. Router A is simply a result of the problem caused by Router D. After a Cisco IOS Software upgrade on Router D, Router A stops refreshing its routing table timer, as indicated in Example 7-51. Also, performing the show ip eigrp event several times in succession shows that the timers on the event table are not changing. This also verifies that the EIGRP process is stable and is not receiving unnecessary updates from its neighbors.
Example 7-51 Output of Routing Table on Router A to Verify the Fix of the Problem
Router_A#show ip route 150.150.0.0 Routing entry for 150.150.0.0/16 Known via "eigrp 1", distance 90, metric 4269056, type internal Redistributing via eigrp 1 Last update from 10.1.1.2 on ethernet 0, 00:03:18 ago Routing Descriptor Blocks: *10.1.1.2, from 10.1.1.2, 00:03:18 ago , v ia ethernet0 Route metric is 4269056, traffic share count is 1 Total delay is 102000 microseconds, minimum bandwidth is 1544 Kbit Reliability 255/255, minimum MTU 1500 bytes Loading 1/255, Hops 4