Automating Management and Monitoring Tasks
This section will discuss a very robust set of tools that are built-in to many Cisco devices such as:
Smart Call Home
Tcl Shell
Embedded Event Manager (EEM)
These tools are designed to make life a bit easier for the network operations staff by leveraging on-box automation.
Smart Call Home
Cisco’s Smart Call Home is a feature that is built into a large number of Cisco devices that allows the devices to automatically reach out to Cisco TAC when there is an issue in your campus environment. Smart Call Home can report a wide variety of different events. For example:
Generic online diagnostics (GOLD)
Syslog events
Environment events and alarms
Inventory and configuration
Field notices
Product security incident response team (PSIRT) notifications
There are three primary ways that Smart Call Home can collect this information from the IOS: Alert Groups and Profiles, collecting show commands, and interaction with the CLI. This information is sent via one of three different transport modes: HTTP(S) direct, HTTP(S) via a transport gateway, or via email through a transport gateway. A transport gateway is a device that securely forwards Call Home messages that are sourced from devices within the network. The information that is gathered and sent to Cisco TAC is then stored in a database within Cisco’s data centers. Once the information is collected and stored in the database, you will be able to view the information from a web portal where you can manage all your devices. Smart Call Home allows TAC to do multiple things with the collected information:
Automatically create TAC service requests, based on issues with the device(s)
Notify the Cisco partner should they need to be contacted
Notify the device owner that there is something going on with the device(s)
This helps make your business more proactive, rather than reactive. An example of Smart Call Home would be if you have a Catalyst 4500 series switch and one of the power supplies failed in the middle of the night. Instead of having to wake up, open a TAC case, and upload the serial number of the switch and the configuration and go through troubleshooting steps, the switch would have used Smart Call Home to contact TAC and upload all the necessary information and a TAC case would have already been opened automatically. In turn, an RMA could be issued automatically for the failed part. This drastically reduces the amount of time and effort engineers have to spend, going through the motions of all the steps mentioned above in order to get a replacement power supply and bring the network back to 100 percent. In addition to this, there is an anonymous reporting feature that allows Cisco to receive minimal error and health information from various devices.
There are six basic steps to enable Cisco’s Smart Call Home feature. Those steps are as follows:
Enable Call Home
Configure contact email address
Activate CiscoTAC-1 profile
Set transport mode
Install security certificate
Send a Call Home inventory to start the registration process
Enabling Smart Call Home on an Cisco Catalyst Switch
The following example depicts the process for setting up Smart Call Home on a Catalyst switch.
Switch# configure terminal Switch(config)# service call-home Switch(config)# call-home Switch(cfg-call-home)# contact-email-addr neteng@yourcompany.com Switch(cfg-call-home)# profile CiscoTAC-1 Switch(cfg-call-home-profile)# active Switch(cfg-call-home-profile)# destination transport-method http Switch(cfg-call-home-profile)# exit Switch(cfg-call-home)# exit Switch(config)# crypto pki trustpoint cisco Switch(ca-trustpoint)# enrollment terminal Switch(ca-trustpoint)# revocation-check crl none Switch(ca-trustpoint)# exit Switch(config)# crypto pki authenticate cisco Enter the base 64 encoded CA certificate. End with a blank line or the word "quit" on a line by itself -----BEGIN CERTIFICATE----- MIICPDCCAaUCEDyRMcsf9tAbDpq40ES/Er4wDQYJKoZIhvcNAQEFBQAwXzELMAkG A1UEBhMCVVMxFzAVBgNVBAoTDlZlcmlTaWduLCBJbmMuMTcwNQYDVQQLEy5DbGFz cyAzIFB1YmxpYyBQcmltYXJ5IENlcnRpZmljYXRpb24gQXV0aG9yaXR5MB4XDTk2 MDEyOTAwMDAwMFoXDTI4MDgwMjIzNTk1OVowXzELMAkGA1UEBhMCVVMxFzAVBgNV BAoTDlZlcmlTaWduLCBJbmMuMTcwNQYDVQQLEy5DbGFzcyAzIFB1YmxpYyBQcmlt YXJ5IENlcnRpZmljYXRpb24gQXV0aG9yaXR5MIGfMA0GCSqGSIb3DQEBAQUAA4GN ADCBiQKBgQDJXFme8huKARS0EN8EQNvjV69qRUCPhAwL0TPZ2RHP7gJYHyX3KqhE BarsAx94f56TuZoAqiN91qyFomNFx3InzPRMxnVx0jnvT0Lwdd8KkMaOIG+YD/is I19wKTakyYbnsZogy1Olhec9vn2a/iRFM9x2Fe0PonFkTGUugWhFpwIDAQABMA0G CSqGSIb3DQEBBQUAA4GBABByUqkFFBkyCEHwxWsKzH4PIRnN5GfcX6kb5sroc50i 2JhucwNhkcV8sEVAbkSdjbCxlnRhLQ2pRdKkkirWmnWXbj9T/UWZYB2oK0z5XqcJ 2HUw19JlYD1n1khVdWk/kfVIC0dpImmClr7JyDiGSnoscxlIaU5rfGW/D/xwzoiQ -----END CERTIFICATE--- -----BEGIN CERTIFICATE----- MIIE0DCCBDmgAwIBAgIQJQzo4DBhLp8rifcFTXz4/TANBgkqhkiG9w0BAQUFADBfMQswCQ YDVQQGEwJVUzEXMBUGA1UEChMOVmVyaVNpZ24sIEluYy4xNzA1BgNVBAsTLkNsYXNzID Mg UHVibGljIFByaW1hcnkgQ2VydGlmaWNhdGlvbiBBdXRob3JpdHkwHhcNMDYxMTA4MDAwMD AwWhcNMjExMTA3MjM1OTU5WjCByjELMAkGA1UEBhMCVVMxFzAVBgNVBAoTDlZlcmlTa Wdu LCBJbmMuMR8wHQYDVQQLExZWZXJpU2lnbiBUcnVzdCBOZXR3b3JrMTowOAYDVQQLEzEo Yy kgMjAwNiBWZXJpU2lnbiwgSW5jLiAtIEZvciBhdXRob3JpemVkIHVzZSBvbmx5MUUwQwYD VQQDEzxWZXJpU2lnbiBDbGFzcyAzIFB1YmxpYyBQcmltYXJ5IENlcnRpZmljYXRpb24gQX V0aG9yaXR5IC0gRzUwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCvJAgIKXo1 nmAMqudLO07cfLw8RRy7K+D+KQL5VwijZIUVJ/XxrcgxiV0i6CqqpkKzj/i5Vbext0uz/o 9+B1fs70PbZmIVYc9gDaTY3vjgw2IIPVQT60nKWVSFJuUrjxuf6/WhkcIzSdhDY2pSS9KP 6HBRTdGJaXvHcPaz3BJ023tdS1bTlr8Vd6Gw9KIl8q8ckmcY5fQGBO+QueQA5N06tRn/Ar r0PO7gi+s3i+z016zy9vA9r911kTMZHRxAy3QkGSGT2RT+rCpSx4/VBEnkjWNHiDxpg8v+ R70rfk/Fla4OndTRQ8Bnc+MUCH7lP59zuDMKz10/NIeWiu5T6CUVAgMBAAGjggGbMIIBlz APBgNVHRMBAf8EBTADAQH/MDEGA1UdHwQqMCgwJqAkoCKGIGh0dHA6Ly9jcmwudmVya XNp Z24uY29tL3BjYTMuY3JsMA4GA1UdDwEB/wQEAwIBBjA9BgNVHSAENjA0MDIGBFUdIAAwKj AoBggrBgEFBQcCARYcaHR0cHM6Ly93d3cudmVyaXNpZ24uY29tL2NwczAdBgNVHQ4EFgQU f9Nlp8Ld7LvwMAnzQzn6Aq8zMTMwbQYIKwYBBQUHAQwEYTBfoV2gWzBZMFcwVRYJaW1h Z2 UvZ2lmMCEwHzAHBgUrDgMCGgQUj+XTGoasjY5rw8+AatRIGCx7GS4wJRYjaHR0cDovL2xv Z28udmVyaXNpZ24uY29tL3ZzbG9nby5naWYwNAYIKwYBBQUHAQEEKDAmMCQGCCsGAQU FBz ABhhhodHRwOi8vb2NzcC52ZXJpc2lnbi5jb20wPgYDVR0lBDcwNQYIKwYBBQUHAwEGCCsG AQUFBwMCBggrBgEFBQcDAwYJYIZIAYb4QgQBBgpghkgBhvhFAQgBMA0GCSqGSIb3DQEBB Q UAA4GBABMC3fjohgDyWvj4IAxZiGIHzs73Tvm7WaGY5eE43U68ZhjTresY8g3JbT5KlCDD PLq9ZVTGr0SzEK0saz6r1we2uIFjxfleLuUqZ87NMwwq14lWAyMfs77oOghZtOxFNfeKW/ 9mz1Cvxm1XjRl4t7mi0VfqH5pLr7rJjhJ+xr3/ <snip> <Full certificate is issued from link in the Smart Call Home Quick Start Guide> <snip> quit Certificate has the following attributes: Fingerprint MD5: EF5AF133 EFF1CDBB 5102EE12 144B96C4 Fingerprint SHA1: A1DB6393 916F17E4 18550940 0415C702 40B0AE6B % Do you accept this certificate? [yes/no]: yes Trustpoint CA certificate accepted. % Certificate successfully imported Switch(config)# end Switch# copy running-config startup-config
Once you complete the certificate import process, you must then initiate a call home to begin the registration process for the device. Before we begin the call home process, we will enable the debug event manager action cli command as the following snippet depicts. This will show the steps that the call-home feature is taking. It is important to remember that call-home uses embedded event manager (EEM) to function. The following example also shows the call-home command that is used to initiate the call-home and registration process on a Cisco Catalyst switch.
Switch# debug event manager action cli Debug EEM action cli debugging is on Switch# call-home send alert-group inventory profile CiscoTAC-1 Sending inventory info call-home message ... Please wait. This may take some time ... Switch# Dec 7 22:48:38.089: %HA_EM-6-LOG: callhome : DEBUG(cli_lib) : : CTL : cli_open called. Dec 7 22:48:38.089: %HA_EM-6-LOG: callhome : DEBUG(cli_lib) : : OUT : Switch> Dec 7 22:48:38.089: %HA_EM-6-LOG: callhome : DEBUG(cli_lib) : : IN : Switch>enable Dec 7 22:48:38.099: %HA_EM-6-LOG: callhome : DEBUG(cli_lib) : : OUT : Switch# Dec 7 22:48:38.099: %HA_EM-6-LOG: callhome : DEBUG(cli_lib) : : IN : Switch#show version Dec 7 22:48:38.120: %HA_EM-6-LOG: callhome : DEBUG(cli_lib) : : OUT : Cisco IOS Software, C3560CX Software (C3560CX-UNIVERSALK9-M), Version 15.2(3)E, RELEASE SOFTWARE (fc4) Dec 7 22:48:38.120: %HA_EM-6-LOG: callhome : DEBUG(cli_lib) : : OUT : Technical Support: http://www.cisco.com/techsupport Dec 7 22:48:38.120: %HA_EM-6-LOG: callhome : DEBUG(cli_lib) : : OUT : Copyright (c) 1986-2014 by Cisco Systems, Inc. Dec 7 22:48:38.120: %HA_EM-6-LOG: callhome : DEBUG(cli_lib) : : OUT : Compiled Sun 07-Dec-14 13:15 by prod_rel_team Dec 7 22:48:38.120: %HA_EM-6-LOG: callhome : DEBUG(c Translating "tools.cisco.com"... domain server (X.X.X.X)li_lib) : : OUT : Dec 7 22:48:38.120: %HA_EM-6-LOG: callhome : DEBUG(cli_lib) : : OUT : ROM: Bootstrap program is C2960X boot loader Dec 7 22:48:38.120: %HA_EM-6-LOG: callhome : DEBUG(cli_lib) : : OUT : BOOTLDR: C3560CX Boot Loader (C3560CX-HBOOT-M) Version 15.2(3r)E1, RELEASE SOFTWARE (fc1) Dec 7 22:48:38.120: %HA_EM-6-LOG: callhome : DEBUG(cli_lib) : : OUT : Dec 7 22:48:38.120: %HA_EM-6-LOG: callhome : DEBUG(cli_lib) : : OUT : Switch uptime is 1 day, 6 hours, 9 minutes Dec 7 22:48:38.120 [OK] i: %HA_EM-6-LOG: callhome : DEBUG(cli_lib) : : OUT : System returned to ROM by power-on Dec 7 22:48:38.120: %HA_EM-6-LOG: callhome : DEBUG(cli_lib) : : OUT : System restarted at 16:38:44 UTC Sun Dec 6 2015 Dec 7 22:48:38.120: %HA_EM-6-LOG: callhome : DEBUG(cli_lib) : : OUT : System image file is "flash:/c3560cx-universalk9-mz.152-3.E/c3560cx-universalk9-mz .152-3.E.bin" Dec 7 22:48:38.120: %HA_EM-6-LOG: callhome : DEBUG(cli_lib) : : OUT : Last reload reason: power-on Dec 7 22:48:38.120: %HA_EM-6-LOG: callhome : DEBUG(cli_lib) : : OUT : Dec 7 22:48:38.120: %HA_EM-6-LOG: callhome : DEBUG(cli_lib) : : OUT : Dec 7 22:48:38.120: %HA_EM-6-LOG: callhome : DEBUG(cli_lib) : : OUT : Dec 7 22:48:38.120: %HA_EM-6-LOG: callhome : DEBUG(cli_lib) : : OUT : This product contains cryptographic features and is subject to United Dec 7 22:48:38.120: %HA_EM-6-LOG: callhome : DEBUG(cli_lib) : : OUT : States and local country laws governing import, export, transfer and Dec 7 22:48:38.120: %HA_EM-6-LOG: callhome : DEBUG(cli_lib) : : OUT : use. Delivery of Cisco cryptographic products does not imply Dec 7 22:48:38.120: %HA_EM-6-LOG: callhome : DEBUG(cli_lib) : : OUT : third-party authority to import, export, distribute or use encryption. Dec 7 22:48:38.120: %HA_EM-6-LOG: callhome : DEBUG(cli_lib) : : CTL : 20+ lines read from cli, debug output truncated Dec 7 22:48:38.620: %HA_EM-6-LOG: callhome : DEBUG(cli_lib) : : IN : Switch#show inventory oid Dec 7 22:48:38.634: %HA_EM-6-LOG: callhome : DEBUG(cli_lib) : : OUT : NAME: "1", DESCR: "WS-C3560CX-8PC-S" Dec 7 22:48:38.638: %HA_EM-6-LOG: callhome : DEBUG(cli_lib) : : OUT : PID: WS-C3560CX-8PC-S , VID: V01 , SN: XXXXXXXXXXX Dec 7 22:48:38.638: %HA_EM-6-LOG: callhome : DEBUG(cli_lib) : : OUT : OID: 1.3.6.1.4.1.9.12.3.1.3.1593 Dec 7 22:48:38.638: %HA_EM-6-LOG: callhome : DEBUG(cli_lib) : : OUT : Dec 7 22:48:38.638: %HA_EM-6-LOG: callhome : DEBUG(cli_lib) : : OUT : Dec 7 22:48:38.638: %HA_EM-6-LOG: callhome : DEBUG(cli_lib) : : OUT : Switch# Dec 7 22:48:39.137: %HA_EM-6-LOG: callhome : DEBUG(cli_lib) : : IN : Switch#show env power Dec 7 22:48:39.155: %HA_EM-6-LOG: callhome : DEBUG(cli_lib) : : OUT : SW PID Serial# Status Sys Pwr PoE Pwr Watts Dec 7 22:48:39.155: %HA_EM-6-LOG: callhome : DEBUG(cli_lib) : : OUT : -- ------- ----------- ---------- --------------- ------- ------- ----- Dec 7 22:48:39.155: %HA_EM-6-LOG: callhome : DEBUG(cli_lib) : : OUT : 1 Built-in Good Dec 7 22:48:39.155: %HA_EM-6-LOG: callhome : DEBUG(cli_lib) : : OUT : Dec 7 22:48:39.155: %HA_EM-6-LOG: callhome : DEBUG(cli_lib) : : OUT : Switch# Dec 7 22:48:39.658: %HA_EM-6-LOG: callhome : DEBUG(cli_lib) : : CTL : cli_close called. Dec 7 22:48:39.658: Dec 7 22:48:39.658: tty is now going through its death sequence Switch#
Now that this step is complete, an email will be sent to the email address used in the CiscoTAC-1 profile as shown in Figure 7-3. In this case, that email address is neteng@yourcompany.com. Once that email is received, to complete the registration process you must follow the directions in the email. You must also have a valid contract associated to the device you are trying to register to the Smart Call Home portal. Following the link will redirect you to the Smart Call Home Web Portal as shown in Figure 7-4. Once logged into the portal, the device registration process can be completed.
)
Figure 7-3 Email from Cisco Smart Call Home Tool
)
Figure 7-4 Smart Call Home Web Portal
To verify that Smart Call Home is running on your device, issue the show call-home command from the privileged exec prompt. The following snippet displays the output from the show call-home command on a Cisco Catalyst Switch. There are many different options that can be configured with Smart Call Home. The following alert groups are enabled automatically when configuring Smart Call Home with the call-home send alert-group inventory profile CiscoTAC-1 command:
Configuration
Diagnostic
Environment
Inventory
Syslog
Switch# show call-home Current call home settings: call home feature : enable call home message's from address: Not yet set up call home message's reply-to address: Not yet set up vrf for call-home messages: Not yet set up contact person's email address: neteng@yourcompany.com contact person's phone number: Not yet set up street address: Not yet set up customer ID: Not yet set up contract ID: Not yet set up site ID: Not yet set up source ip address: Not yet set up source interface: Not yet set up Mail-server: Not yet set up Rate-limit: 20 message(s) per minute Available alert groups: Keyword State Description ------------------------ ------- ----------------------------- configuration Enable configuration info diagnostic Enable diagnostic info environment Enable environmental info inventory Enable inventory info syslog Enable syslog info Profiles: Profile Name: CiscoTAC-1 Switch#
Tcl Shell
Tcl Shell is a feature that is built into Cisco routers and switches that allows engineers to interact directly with the device by using various Tcl scripts. Tcl scripting has been around for quite some time and is a very useful scripting language. Tcl provides many ways to streamline different tasks that can help with day-to-day operations and monitoring of a network. Some of the following are tasks that can be automated by using these scripts:
Verify IP and IPv6 reachability, using ping
Verify IP and IPv6 reachability, using Traceroute
Check interface statistics
Retrieve SNMP information by accessing MIBs
Send email messages containing CLI outputs from Tcl scripts
Most often, basic Tcl scripts are entered line by line within the Tcl shell, although, for some of the more advanced scripting methods, you can load the script into the flash of the device you are working on and execute the script from there. These scripts have to be in a specific Tcl format as shown in the following examples. The following example illustrates how to enter the Tcl shell on a Cisco router and execute a simple ping script.
Router# tclsh
Router(tcl)# foreach address {
+>(tcl)# 192.168.0.2
>(tcl)# 192.168.0.3
+>(tcl)# 192.168.0.4
+>(tcl)# 192.168.0.5
+>(tcl)# 192.168.0.6
+>(tcl)# } { ping $address
+>(tcl)# }
Type escape sequence to abort.
Sending 5, 64-byte ICMP Echos to 192.168.0.2, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 4/4/4 ms
Type escape sequence to abort.
Sending 5, 64-byte ICMP Echos to 192.168.0.3, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 4/4/4 ms
Type escape sequence to abort.
Sending 5, 64-byte ICMP Echos to 192.168.0.4, timeout is 2 seconds:
.....
Success rate is 0 percent (0/5)
Type escape sequence to abort.
Sending 5, 64-byte ICMP Echos to 192.168.0.5, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/4 ms
Type escape sequence to abort.
Sending 5, 64-byte ICMP Echos to 192.168.0.6, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/3/4 ms
Router(tcl)# tclquit
Router#
An alternate to entering the DNS node names or IP addresses in a line-by-line fashion, you can also enter some of the script commands on a single line within the Tcl shell. For instance, the following example shows a similar ping script to the one entered before, but now it is executed on the same line within the Tcl shell.
Router# tclsh
Router(tcl)# foreach address {192.168.0.2 192.168.0.3 192.168.0.4} {ping $address}
Type escape sequence to abort.
Sending 5, 64-byte ICMP Echos to 192.168.0.2, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 4/4/4 ms
Type escape sequence to abort.
Sending 5, 64-byte ICMP Echos to 192.168.0.3, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 4/4/4 ms
Type escape sequence to abort.
Sending 5, 64-byte ICMP Echos to 192.168.0.4, timeout is 2 seconds:
.....
Success rate is 0 percent (0/5)
Router(tcl)# tclquit
Router#
To execute Tcl Scripts from the local flash memory, you would need to store the script in flash and then call the script by file name. Scripts can be stored on the device’s local flash, USB flash, or compact flash. Tcl scripts can be transferred into the IOS File System (IFS) by using SCP, TFTP, FTP, or RCP. From a security perspective, SCP is preferred due to its use of SSH. To execute a locally stored script, the source command from within the Tcl shell prompt can be used. The following example illustrates the steps to call a script named ping.tcl from the local flash on a device. This script is an example of the same ping script that was shown earlier in this chapter.
Router# tclsh Router(tcl)# source flash:ping.tcl Type escape sequence to abort. Sending 5, 64-byte ICMP Echos to 192.168.0.2, timeout is 2 seconds: !!!!! Success rate is 100 percent (5/5), round-trip min/avg/max = 4/4/4 ms Type escape sequence to abort. Sending 5, 64-byte ICMP Echos to 192.168.0.3, timeout is 2 seconds: !!!!! Success rate is 100 percent (5/5), round-trip min/avg/max = 4/4/4 ms Type escape sequence to abort. Sending 5, 64-byte ICMP Echos to 192.168.0.4, timeout is 2 seconds: ..... Success rate is 0 percent (0/5) Router(tcl)# tclquit Router#
Embedded Event Manager (EEM)
Embedded Event Manager (EEM) is a very flexible and powerful tool within Cisco IOS. EEM allows engineers to build software applets that can automate many tasks. EEM also derives some of its power from the fact that you can build custom scripts using Tcl so that they automatically execute, based on the output of an action or an event on a device. One of the main benefits of EEM is that it is all contained within the local device. There is no need to rely on an external scripting engine or monitoring device in most cases. Figure 7-5 illustrates some of the event detectors and how they interact with the IOS subsystem.
)
Figure 7-5 EEM Event Detectors
EEM Applets
EEM applets are comprised of multiple building blocks. In this chapter, we will focus on the two of the primary building blocks that make up EEM applets. Those building blocks are called events and actions. These EEM applets use a similar logic to the if-then statements found in some of the more common programming languages. For instance, if an event happens, then an action is taken. In the following example, we illustrate a very common EEM applet that is monitoring syslog messages on a router. This particular applet is looking for a specific syslog message, stating that the Loopback0 interface went down. The specific syslog message is matched using regular expressions. This is a very powerful and granular way of matching patterns. If this specific syslog pattern is matched (an event) at least once, then the following actions will be taken:
The Loopback0 interface will be shutdown and brought back up (shutdown, then no shutdown)
The router will generate a syslog message that says “I’ve fallen, and I can’t get up!”
An email message will be sent to the network administrator that includes the output of the show interface loopback0 command.
event manager applet LOOP0 event syslog pattern "Interface Loopback0.* down" period 1 action 1.0 cli command "enable" action 2.0 cli command "config terminal" action 3.0 cli command "interface loopback0" action 4.0 cli command "shutdown" action 5.0 cli command "no shutdown" action 5.5 cli command "show interface loopback0" action 6.0 syslog msg "I've fallen, and I can't get up!" action 7.0 mail server 10.0.0.25 to neteng@yourcompany.com from no-reply@yourcompany.com subject "Loopback0 Issues!" body "The Loopback0 interface was bounced. Please monitor accordingly. "$_cli_result"
Based on the output from the debug event manager action cli, you can see the actual actions taking place when the applet is running. The following example shows the applet being engaged when we issue the shutdown command on the Loopback0 interface. It also shows that there was an error when trying to connect to the SMTP server to send the email to the administrator. This is because the actual SMTP server we are using for this test is not configured. Notice that because we used the $_cli_result keyword in the configuration, it will include the output of any CLI commands that were issued in the applet. In this case, the output of the show interface Loopback0 command will be included in the debug and the mail message.
Switch# Switch# configure terminal Enter configuration commands, one per line. End with CNTL/Z. Switch(config)# interface loopback0 Switch(config-if)# shutdown Switch(config-if)# Dec 6 17:21:59.214: %LINK-5-CHANGED: Interface Loopback0, changed state to administratively down Dec 6 17:21:59.217: %HA_EM-6-LOG: LOOP0 : DEBUG(cli_lib) : : CTL : cli_open called. Dec 6 17:21:59.221: %HA_EM-6-LOG: LOOP0 : DEBUG(cli_lib) : : OUT : Switch> Dec 6 17:21:59.221: %HA_EM-6-LOG: LOOP0 : DEBUG(cli_lib) : : IN : Switch>enable Dec 6 17:21:59.231: %HA_EM-6-LOG: LOOP0 : DEBUG(cli_lib) : : OUT : Switch# Dec 6 17:21:59.231: %HA_EM-6-LOG: LOOP0 : DEBUG(cli_lib) : : IN : Switch#show interface loopback0 Dec 6 17:21:59.252: %HA_EM-6-LOG: LOOP0 : DEBUG(cli_lib) : : OUT : Loopback0 is administratively down, line protocol is down Dec 6 17:21:59.252: %HA_EM-6-LOG: LOOP0 : DEBUG(cli_lib) : : OUT : Hardware is Loopback Dec 6 17:21:59.252: %HA_EM-6-LOG: LOOP0 : DEBUG(cli_lib) : : OUT : MTU 1514 bytes, BW 8000000 Kbit/sec, DLY 5000 usec, Dec 6 17:21:59.252: %HA_EM-6-LOG: LOOP0 : DEBUG(cli_lib) : : OUT : reliability 255/255, txload 1/255, rxload 1/255 Dec 6 17:21:59.252: %HA_EM-6-LOG: LOOP0 : DEBUG(cli_lib) : : OUT : Encapsulation LOOPBACK, loopback not set Dec 6 17:21:59.252: %HA_EM-6-LOG: LOOP0 : DEBUG(cli_lib) : : OUT : Keepalive set (10 sec) Dec 6 17:21:59.252: %HA_EM-6-LOG: LOOP0 : DEBUG(cli_lib) : : OUT : Last input never, output never, output hang never Dec 6 17:21:59.252: %HA_EM-6-LOG: LOOP0 : DEBUG(cli_lib) : : OUT : Last clearing of "show interface" counters never Dec 6 17:21:59.252: %HA_EM-6-LOG: LOOP0 : DEBUG(cli_lib) : : OUT : Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0 Dec 6 17:21:59.252: %HA_EM-6-LOG: LOOP0 : DEBUG(cli_lib) : : OUT : Queueing strategy: fifo Dec 6 17:21:59.252: %HA_EM-6-LOG: LOOP0 : DEBUG(cli_lib) : : OUT : Output queue: 0/0 (size/max) Dec 6 17:21:59.252: %HA_EM-6-LOG: LOOP0 : DEBUG(cli_lib) : : OUT : 5 minute input rate 0 bits/sec, 0 packets/sec Dec 6 17:21:59.252: %HA_EM-6-LOG: LOOP0 : DEBUG(cli_lib) : : OUT : 5 minute output rate 0 bits/sec, 0 packets/sec Dec 6 17:21:59.252: %HA_EM-6-LOG: LOOP0 : DEBUG(cli_lib) : : OUT : 0 packets input, 0 bytes, 0 no buffer Dec 6 17:21:59.252: %HA_EM-6-LOG: LOOP0 : DEBUG(cli_lib) : : OUT : Received 0 broadcasts (0 IP multicasts) Dec 6 17:21:59.252: %HA_EM-6-LOG: LOOP0 : DEBUG(cli_lib) : : OUT : 0 runts, 0 giants, 0 throttles Dec 6 17:21:59.252: %HA_EM-6-LOG: LOOP0 : DEBUG(cli_lib) : : OUT : 0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort Dec 6 17:21:59.252: %HA_EM-6-LOG: LOOP0 : DEBUG(cli_lib) : : OUT : 0 packets output, 0 bytes, 0 underruns Dec 6 17:21:59.252: %HA_EM-6-LOG: LOOP0 : DEBUG(cli_lib) : : OUT : 0 output errors, 0 collisions, 0 interface resets Dec 6 17:21:59.252: %HA_EM-6-LOG: LOOP0 : DEBUG(cli_lib) : : OUT : 0 unknown protocol drops Dec 6 17:21:59.252: %HA_EM-6-LOG: LOOP0 : DEBUG(cli_lib) : : CTL : 20+ lines read from cli, debug output truncated Dec 6 17:21:59.252: %HA_EM-6-LOG: LOOP0 : DEBUG(cli_lib) : : IN : Switch#config terminal Dec 6 17:21:59.266: %HA_EM-6-LOG: LOOP0 : DEBUG(cli_lib) : : OUT : Enter configuration commands, one per line. End with CNTL/Z. Dec 6 17:21:59.266: %HA_EM-6-LOG: LOOP0 : DEBUG(cli_lib) : : OUT : Switch(config)# Dec 6 17:21:59.266: %HA_EM-6-LOG: LOOP0 : DEBUG(cli_lib) : : IN : Switch(config)#interface loopback0 Dec 6 17:21:59.277: %HA_EM-6-LOG: LOOP0 : DEBUG(cli_lib) : : OUT : Switch(config-if)# Dec 6 17:21:59.277: %HA_EM-6-LOG: LOOP0 : DEBUG(cli_lib) : : IN : Switch(config-if)#shutdown Dec 6 17:21:59.287: %HA_EM-6-LOG: LOOP0 : DEBUG(cli_lib) : : OUT : Switch(config-if)# Dec 6 17:21:59.287: %HA_EM-6-LOG: LOOP0 : DEBUG(cli_lib) : : IN : Switch(config-if)#no shutdown Dec 6 17:21:59.298: %HA_EM-6-LOG: LOOP0 : DEBUG(cli_lib) : : OUT : Switch(config-if)# Dec 6 17:21:59.298: %HA_EM-6-LOG: LOOP0: I've fallen and I can't get up! Dec 6 17:22:01.293: %LINK-3-UPDOWN: Interface Loopback0, changed state to up Dec 6 17:22:11.314: %HA_EM-3-FMPD_SMTP: Error occurred when sending mail to SMTP server: 10.0.0.25 : error in connecting to SMTP server Dec 6 17:22:11.314: %HA_EM-3-FMPD_ERROR: Error executing applet LOOP0 statement 7.0 Dec 6 17:22:11.314: %HA_EM-6-LOG: LOOP0 : DEBUG(cli_lib) : : CTL : cli_close called.
Another very useful aspect of EEM applets is that CLI patterns can be matched as an event. This means that when certain commands are entered into the router via CLI, they can trigger an EEM event within an applet. Then the configured actions will take place as a result of the CLI pattern being matched. The following example uses another common EEM applet to match the CLI pattern “wr mem”. Once the applet is triggered, the following actions will be invoked:
The router will generate a syslog message that says “Configuration File Changed!”
The startup-config will be copied to a TFTP server.
Generate a syslog message stating that the configuration has been successfully saved.
event manager environment filename Router.cfg event manager environment tftpserver tftp://10.1.200.29/ event manager applet BACKUP-CONFIG event cli pattern "write mem.*" sync yes action 1.0 cli command "enable" action 2.0 cli command "configure terminal" action 3.0 cli command "file prompt quiet" action 4.0 cli command "end" action 5.0 cli command "copy start $tftpserver$filename" action 6.0 cli command "configure terminal" action 7.0 cli command "no file prompt quiet" action 8.0 syslog priority informational msg "Configuration File Changed! TFTP backup successful."
As seen in the previous examples there are multiple ways to call out specific EEM environment values. The first example illustrated that you can use a single line to configure the mail environment and send messages with CLI output results. Using the event manager environment variables shown in the second example, you can statically set different settings that you can call on from multiple actions instead of calling them out individually on a single line. Although you can create custom names and values that are arbitrary and can be set to anything, it is good practice to use common and descriptive variables. Table 7-3 lists some of the most commonly used email variables in EEM.
Table 7-3 Common EEM Email Variables
EEM Variable |
Description |
Example |
_email_server |
SMTP server IP address or DNS name |
10.0.0.25 or MAILSVR01 |
_email_to |
Email address to send email to |
neteng@yourcompany.com |
_email_from |
Email address of sending party |
no-reply@yourcompany.com |
_email_cc |
Email address of additional email receivers |
elpdesk@yourcompany.com |
EEM and Tcl Scripts
Using an EEM applet to call Tcl scripts is another very powerful aspect of EEM. We have covered multiple ways to use EEM applets. In this section, we will discuss how to call a Tcl script from an EEM applet. The previous sections on EEM showed multiple ways of executing actions, based on the automatic detection of specific events when they are happening. This example shows how to manually execute an EEM applet that will, in turn, execute a Tcl script that is locally stored in the device’s flash memory. It is important to understand that there are many different ways to use EEM and that manually triggered applets are also a very useful tool. The following example depicts an EEM script that is configured with the event none command. This means that there is no automatic event that the applet is monitoring and that this applet will only run when it is triggered manually. To manually run an EEM applet, the event manager run command must be used as illustrated in second output.
event manager applet Ping event none action 1.0 cli command "enable" action 1.1 cli command "tclsh flash:/ping.tcl" Router# event manager run Ping Router# Dec 6 19:32:16.564: %HA_EM-6-LOG: Ping : DEBUG(cli_lib) : : CTL : cli_open called. Dec 6 19:32:16.564: %HA_EM-6-LOG: Ping : DEBUG(cli_lib) : : OUT : Router> Dec 6 19:32:16.568: %HA_EM-6-LOG: Ping : DEBUG(cli_lib) : : IN : Router>enable Dec 6 19:32:16.578: %HA_EM-6-LOG: Ping : DEBUG(cli_lib) : : OUT : Router# Dec 6 19:32:16.578: %HA_EM-6-LOG: Ping : DEBUG(cli_lib) : : IN : Router#tclsh flash:/ping.tcl Dec 6 19:32:16.711: %HA_EM-6-LOG: Ping : DEBUG(cli_lib) : : OUT : Type escape sequence to abort. Dec 6 19:32:16.711: %HA_EM-6-LOG: Ping : DEBUG(cli_lib) : : OUT : Sending 5, 100-byte ICMP Echos to 192.168.0.2, timeout is 2 seconds: Dec 6 19:32:16.711: %HA_EM-6-LOG: Ping : DEBUG(cli_lib) : : OUT : !!!!! Dec 6 19:32:16.711: %HA_EM-6-LOG: Ping : DEBUG(cli_lib) : : OUT : Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/4 ms Dec 6 19:32:16.711: %HA_EM-6-LOG: Ping : DEBUG(cli_lib) : : OUT : Type escape sequence to abort. Dec 6 19:32:16.711: %HA_EM-6-LOG: Ping : DEBUG(cli_lib) : : OUT : Sending 5, 100-byte ICMP Echos to 192.168.0.3, timeout is 2 seconds: Dec 6 19:32:16.711: %HA_EM-6-LOG: Ping : DEBUG(cli_lib) : : OUT : !!!!! Dec 6 19:32:16.711: %HA_EM-6-LOG: Ping : DEBUG(cli_lib) : : OUT : Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/1 ms Dec 6 19:32:16.711: %HA_EM-6-LOG: Ping : DEBUG(cli_lib) : : OUT : Type escape sequence to abort. Dec 6 19:32:16.711: %HA_EM-6-LOG: Ping : DEBUG(cli_lib) : : OUT : Sending 5, 100-byte ICMP Echos to 192.168.0.4, timeout is 2 seconds: Dec 6 19:32:16.711: %HA_EM-6-LOG: Ping : DEBUG(cli_lib) : : OUT : !!!!! Dec 6 19:32:16.711: %HA_EM-6-LOG: Ping : DEBUG(cli_lib) : : OUT : Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/3 ms Dec 6 19:32:16.711: %HA_EM-6-LOG: Ping : DEBUG(cli_lib) : : OUT : Type escape sequence to abort. Dec 6 19:32:16.711: %HA_EM-6-LOG: Ping : DEBUG(cli_lib) : : OUT : Sending 5, 100-byte ICMP Echos to 192.168.0.5, timeout is 2 seconds: Dec 6 19:32:16.711: %HA_EM-6-LOG: Ping : DEBUG(cli_lib) : : OUT : !!!!! Dec 6 19:32:16.711: %HA_EM-6-LOG: Ping : DEBUG(cli_lib) : : OUT : Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/4 ms Dec 6 19:32:16.711: %HA_EM-6-LOG: Ping : DEBUG(cli_lib) : : OUT : Type escape sequence to abort. Dec 6 19:32:16.711: %HA_EM-6-LOG: Ping : DEBUG(cli_lib) : : OUT : Sending 5, 100-byte ICMP Echos to 192.168.0.6, timeout is 2 seconds: Dec 6 19:32:16.711: %HA_EM-6-LOG: Ping : DEBUG(cli_lib) : : OUT : !!!!! Dec 6 19:32:16.711: %HA_EM-6-LOG: Ping : DEBUG(cli_lib) : : OUT : Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/1 ms Dec 6 19:32:16.711: %HA_EM-6-LOG: Ping : DEBUG(cli_lib) : : CTL : 20+ lines read from cli, debug output truncated Dec 6 19:32:16.711: %HA_EM-6-LOG: Ping : DEBUG(cli_lib) : : CTL : cli_close called.
For reference, see the following snippet for the exact content of the ping.tcl script used in the manually triggered EEM applet in the previous example. To see the contents of a TCL script that resides in flash, issue the more command followed by the file location and filename. The more command can be used to view all other text based files stored in the local flash as well.
Router# more flash:ping.tcl
foreach address {
192.168.0.2
192.168.0.3
192.168.0.4
192.168.0.5
192.168.0.6
} { ping $address}
EEM Summary
There are many ways to utilize EEM. From applets to scripting, the possibly use cases can only be limited by the engineer’s imagination. EEM provides on-box monitoring of various different components based on a series of events. Once an event is detected, an action can take place. This helps make some of the network monitoring more proactive, rather than reactive. This can also reduce the load on the network and improve efficiency from the monitoring system because now the devices can simply report when there is something wrong instead of continually asking the devices if there is anything wrong.