Troubleshooting Cisco Secure ACS on Windows

Cisco Secure Access Control Server, which is known as CS ACS, fills the server-side requirement of the Authentication, Authorization, and Accounting (AAA) client server equation. For many security administrators, the robust and powerful AAA engine, along with CS ACS's ability to flexibly integrate with a number of external user databases, makes the CS ACS software the first and sometimes only choice for an AAA server-side solution. This chapter explores CS ACS in detail and walks you through troubleshooting steps. The chapter focuses on the approach required to troubleshoot any issue efficiently, either with the CS ACS software itself or with the whole AAA process.

Overview of CS ACS

Before delving into the details of how an AAA request from a network access server (NAS) is processed by CS ACS, you need a good understanding of all the components that bring the Cisco Secure ACS into existence.

CS ACS Architecture

As shown in Figure 13-1, Cisco Secure ACS comprises a number of services.

• CSAdmin—This service provides the Web interface for administration of Cisco Secure ACS. Although it is possible, and sometimes desirable, to use the Command Line Interface (CLI) for CS ACS configuration, the Graphical User Interface (GUI) is a must because certain attributes may not be configured via CLI. In addition, with the GUI, the administrator has little or no chance to insert bad data, which could lead to database corruption, because the GUI has a sanity check mechanism for user data insertion. The web server used by CS ACS is Cisco proprietary and uses TCP/2002 rather than the standard port 80. Therefore, another web server may be running on the CS ACS server, but this is not recommended because of the security risk and other possible interference.
  

  Figure 13-1 Diagram of the Relationship Among Cisco Secure ACS Services

  Because CSAdmin service is coded as multi-threaded, it is possible to open multiple sessions from different locations to the CS ACS Server for configuration purposes, but CS ACS does not allow making the same profile or attribute changes by multiple administrators at the same time. For instance, group 200 may not be modified by two administrators at the same time. You need to create an admin account to allow remote access to CS ACS from another machine; you do not need the admin account, however, if you access it from the CS ACS server itself. To bring up the CS ACS GUI from a host other than CS ACS, point to the following location:

  http://<ip_address_of_CS ACS_server>:2002

  All the services except CSAdmin can be stopped and restarted from the GUI (System > Service Control>Stop/Restart). CSAdmin can be controlled via a Windows Services applet, which can be opened by browsing to Start > Programs > Administrative Tools > Services applet.

• CSAuth—CSAuth is the heart of CS ACS server, which processes the authentication and authorization requests from the NAS. It also manages the Cisco Secure CS ACS database.
• CSDBSync—CSDBSync is the database synchronization service, which allows the CS ACS database to be in sync with third-party relational database management system (RDBMS) systems. This feature is very useful when an organization has multiple data feed locations.
• CSLog—This is a logging service for audit-trailing, accounting of authentication, and authorization packets. CSLog collects data from the CSTacacs or CSRadius packet and CSAuth, and then scrubs the data so that data can be stored into comma-separated value (CSV) files or forwarded to an Open DataBase Connectivity (ODBC)-compliant database.
• CSMon—CSMon service is responsible for the monitoring, recording, and notification of Cisco Secure CS ACS performance, and includes automatic response to some scenarios. For instance, if either Terminal Access Controller Access Control System (TACACS+) or Remote Authentication Dial-In User Service (RADIUS) service dies, CS ACS by default restarts all the services, unless otherwise configured. Monitoring includes monitoring the overall status of Cisco Secure ACS and the system on which it is running. CSMon actively monitors three basic sets of system parameters:
  • Generic host system state—monitors disk space, processor utilization, and memory utilization.
  • Application-specific performance—periodically performs a test login each minute using a special built-in test account by default.
  • System resource consumption by Cisco Secure ACS—CSMon periodically monitors and records the usage by Cisco Secure ACS of a small set of key system resources. Handles counts, memory utilization, processor utilization, thread used, and failed log-on attempts, and compares these to predetermined thresholds for indications of atypical behavior.

  CSMon works with CSAuth to keep track of user accounts that are disabled for exceeding their failed attempts count maximum. If configured, CSMon provides immediate warning of brute force attacks by alerting the administrator that a large number of accounts have been disabled.

  By default CSMon records exception events in logs both in the CSV file and Windows Event Log that you can use to diagnose problems. Optionally you can configure event notification via e-mail so that notification for exception events and outcomes includes the current state of Cisco Secure ACS at the time of the message transmission. The default notification method is simple mail-transfer protocol (SMTP) e-mail, but you can create scripts to enable other methods. However, if the event is a failure, CSMon takes the actions that are hard-coded when the triggering event is detected. Running the CSSupport utility, which captures most of the parameters dealing with the state of the system at the time of the event, is one such example. If the event is a warning event, it is logged, the administrator is notified if it is configured, and no further action is taken. After a sequence of re-tries, CSMon also attempts to fix the cause of the failure and individual service restarts. It is possible to integrate custom-defined action with CSMon service, so that a user-defined action can be taken based on specific events.

The Cisco Secure ACS information is located in the following Windows Registry key as shown in Figure 13-2:

HKEY_LOCAL_MACHINE\SOFTWARE\CISCO

Figure 13-2 Cisco Secure ACS Registries Location

You can get to the screen shown in Figure 13-2 by browsing Start>Run>Type and entering "regedit" in the text box. Do not make any changes to Windows Registry settings related to CS ACS unless advised by a Cisco representative, as you may inadvertently corrupt your application. This chapter explains where the Registry entry should be added or modified.

The Life of an AAA Packet in CS ACS

This section builds on the knowledge that you have gained from the preceding section, to examine the life of an AAA packet within CS ACS when it hits the CS ACS server. When the packet reaches the CS ACS, the following events occur:

1. NAS interacts with CS ACS Server using CSTacacs or CSRadius Services. So, CSTacacs or CSRadius service receives the packet from the NAS.
2. Then NAS checking is performed with the IP address and shared secret and if successful, then CSTacacs or CSRadius performs the Network Access Restrictions (NAR) checking. If CSTacacs or CSRadius decides that it is a valid packet and passes the NAR test, the packet goes to the CSAuth Service.
3. The CSAuth checks the Proxy Distribution table and finds out if there is any matching string for the username in the Character String Column of the Proxy Distribution Table. If there is a match, and AAA proxy information is defined, then the authentication request is forwarded to the appropriate AAA server, and CS ACS at this stage acts as a middle man for AAA services. However, if there is no matching string found, ACS Local database performs the AAA services as described in the next step.
4. The CSAuth service looks up the user's information in its own internal database and if the user exists, it either allows or denies access based on password and other parameters. This status information, and any authorization parameters, are sent to the CSTacacs or CSRadius service, which then forwards the status information to the NAS.
5. If the user does not exist in the CS ACS local database, CS ACS marks that user as unknown and checks for an unknown user policy. If the unknown user policy is to fail the user, CS ACS fails the user. Otherwise, if external database is configured, CS ACS forwards that information to the configured external user database. Cisco Secure CS ACS tries each external user database until the user succeeds or fails.
6. If the authentication is successful, the user information goes into the cache of CS ACS, which has a pointer for using the external user database. This user is known as a dynamic user.
7. The next time the dynamic user tries to authenticate, Cisco Secure ACS authenticates the user against the database that was successful the first time. These cached user entries are used to speed up the authentication process. Dynamic users are treated in the same way as known users.
8. If the unknown user fails authentication with all configured external databases, the user is not added to the Cisco Secure user database and the authentication fails.
9. When a user is authenticated, Cisco Secure ACS obtains a set of authorizations from the user profile and the group to which the user is assigned. This information is stored with the username in the Cisco Secure user database. Some of the authorizations included are the services to which the user is entitled, such as IP over Point-to-Point Protocol (PPP), IP pools from which to draw an IP address, access lists, and password-aging information.
10. The authorizations, with the approval of authentication, are then passed to the CSTacacs or CSRadius modules to be forwarded to the requesting device.
11. If configured on the NAS, accounting starts right after the successful user authentication. Accounting can be configured for authorization as well. A START record from NAS is sent which follows the same paths as authentication requests on CS ACS with the addition of CSLog service involvement. For instance, if the radius protocol is used, packets go through CSRadius service first, then CSAuth. CSAuth then forwards the packet to the CSLog service. CSLog service decides if the accounting requests should be forwarded to another AAA server based on the Proxy Distribution Table, or should be processed locally. Additionally, if ODBC logging is configured for accounting, the packet is forwarded to the ODBC database. The same path is followed for the STOP record from the NAS, which completes the accounting record for a specific session.

CS ACS can integrate with a number of external user databases. Table 13-1 shows the components that are needed to integrate with those external user databases.

Table 13-1 Components Needed to Integrate with External Databases

CS ACS can be integrated with many external user databases; however, not every database supports every authentication protocol. Table 13-2 shows the protocols supported for specific databases.

Table 13-2 Protocols Supported on Various Databases