This chapter introduces general campus switching design considerations and describes modularity in switching designs. It includes the following sections:
Campus Design Methodology
Campus Design
Summary
Case Study and Simulation Exercise
Review Questions
The availability of multigigabit campus switches gives customers the opportunity to build extremely high-performance, high-reliability networksif they follow correct network design approaches. Unfortunately, some alternative network design approaches can result in a network that has lower performance, reliability, and manageability.
This chapter describes a hierarchical modular design approach called multilayer design. First, it addresses general campus switching design considerations. The differences between Layer 2 (L2) and Layer 3 (L3) switching, and where to use each, are also discussed.
When you finish this chapter, you will be able to understand campus network switch design fundamentals and describe the positioning of switches in campus network modules.
Campus Design Methodology
The multilayer approach to campus network design combines Layer 2 switching with Layer 3 switching to achieve robust, highly-available campus networks. This section discusses the factors you should consider for a Campus local-area network (LAN) design.
Designing an Enterprise Campus
Designing an Enterprise Campus network requires a broad view of the network's overall picture. The network designer must be familiar with both Enterprise Campus design methodologies and Enterprise Campus modules.
Campus design requires an understanding of the organizational network borders (geography) and the existing and planned application traffic flows. Physical characteristics of the network depend on the following criteria:
Selected transmission media
The type of technology (switched or shared)
The type of traffic forwarding (switching) in network devices (Layer 2 or Layer 3)
You should consider the following five factors when deploying the campus network:
Network geographyThe distribution of network nodes (for example, host or network devices) and the distances between them significantly affect the campus solutionespecially the physical transmission media.
Network applicationsIn terms of bandwidth and delay, the application requirements place stringent requirements on a campus network solution.
Data link layer technology (shared or switched)The dedicated bandwidth solution of LAN switching is replacing the traditional approach, in which all devices share the available bandwidth using hubs. The network designer must consider these options, especially when migrating or upgrading existing networks.
Layer 2 versus Layer 3 switchingThe network devices and their features determine the network's flexibility, but also contribute to the network's overall delay. Layer 2 switching is based on media access control (MAC) addresses, and Layer 3 switching is based on network layer addressesusually Internet Protocol (IP) addresses.
Transmission media (physical cabling)Cabling is one of the biggest long-term investments in network deployment. Therefore, transmission media selection depends not only on the required bandwidth and distances, but also on the emerging technologies that might be deployed over the same infrastructure in the future. The network designer must thoroughly evaluate the cost of the medium (including installation costs) and the available budget in addition to the technical characteristics, such as signal attenuation and electromagnetic interference. Two major cabling options exist: copper-based media (for example, unshielded twisted pair [UTP]) and optical fiber.
The following sections examine these factors.
Network Geography
The location of Enterprise Campus nodes and the distances between them determine the network's geography. When designing the Enterprise Campus network, the network designer's first step is to identify the network's geography. The network designer must determine the following:
Location of nodesNodes (end users, workstations, or servers) within an organization can be located in the same room, building, or geographical area.
Distances between the nodesBased on the location of nodes and the distance between them, the network designer decides which technology should be used, the maximum speeds, and so on. (Media specifications typically include a maximum distance, how often regenerators can be used, and so on.)
The following geographical structures can be identified with respect to the network geography:
Intra-building
Inter-building
Distant remote building
Distant remote building over 100 km
These geographical structures serve as guides to help determine Enterprise Campus transmission media and the logical modularization of the Enterprise Campus network. The following sections describe these geographical structures.
Intra-Building Structure
An intra-building campus network structure provides connectivity for the end nodes, which are all located in the same building, and gives them access to the network resources. (The access and distribution layers are typically located in the same building.)
User workstations are usually attached to the floor-wiring closet with UTP cables. To allow the most flexibility in the use of technologies, the UTP cables are typically Category 5 (CAT 5) or better. Wiring closets usually connect to the building central switch (distribution switch) over optical fiber. This offers better transmission performances and is less sensitive to environmental disturbances.
Inter-Building Structure
As shown in Figure 4-1, an inter-building network structure provides the connectivity between the individual campus buildings' central switches (in the distribution and/or core layers). Typically placed only a few hundred meters to a few kilometers apart, these buildings are usually in close proximity.
Figure 4-1 Inter-Building Network Structure
Because the nodes in all campus buildings usually share common devices such as servers, the demand for high-speed connectivity between the buildings is high. To provide high throughput without excessive interference from environmental conditions, optical fiber is the media of choice between the buildings.
Distant Remote Building Structure
When connecting distances that exceed a few kilometers (usually within a metropolitan area), the network designer's most important factor to consider is the physical media. The speed and cost of the network infrastructure depend heavily on the media selection.
Usually, the bandwidth requirements are higher than the physical connectivity options can support. In such cases, the network designer must identify the organization's critical applications and then select the equipment that supports intelligent network services, such as quality of service (QoS) and filtering capabilities that allow optimal use of the bandwidth.
Some companies might own their media, such as fiber or copper lines. However, if the organization does not own physical transmission media to certain remote locations, the Enterprise Network Campus must connect through the Enterprise Edge wide-area network (WAN) module using connectivity options from public service providers (such as metropolitan area network [MAN]).
Network Geography Considerations
Table 4-1 compares the types of connectivity, availability importance, required throughput, and expected cost for each geographical structure.
Table 4-1 Network Geography Considerations
Parameter |
Intra-building |
|
Inter-building |
Distant Remote Building |
Distant Over 100 km |
Connectivity type |
UTP |
Fiber |
Fiber MM/SM |
Fiber SM |
Copper/fiber |
Availability importance |
High |
Medium |
Medium |
Low |
Low |
Required throughput |
Medium |
High |
High |
Medium |
Low |
Cost |
$ |
$$ |
$$$ |
$$$$ |
$$$$$ |
MM = Multimode; SM = single-mode
Depending on the distances and environmental conditions that result from the respective geographical scopes, various connectivity options existranging from traditional copper media to fiber-based transmission media.
Typically, availability within a building is very important, and it decreases with distance between buildings. (This is because the physical buildings in the campus often form the core of the campus network; communication to buildings located farther from the core is not as important.)
The throughput requirements increase close to the network's core and close to the sites where the servers reside.
A quick review of Table 4-1 reveals a combination of a high level of availability, medium bandwidth, and a low price for the Enterprise Campus network when all nodes are located in the same building. The cost of transmission media increases with the distance between nodes. A balance between the desired bandwidth and available budget are usually required to keep the cost reasonable; bandwidth is often sacrificed.
Network Application Characterization
Application characterization is the process of determining the characteristics of the network's applications. Network designers should determine which applications are critical to the organization and the network demands of these applications to determine enterprise traffic patterns inside the Enterprise Campus network. This process should result in information about network bandwidth usage and response times for certain applications. These parameters influence the selection of the transmission medium and the desired bandwidth.
Different types of application communication result in varying network demands. The following sections review four types of application communication:
Client-client
Client-distributed server
Client-Server Farm
Client-Enterprise Edge
Client-Client Applications
From the network designer's perspective, client-client applications include those applications in which the majority of network traffic passes from one network edge device to another through the organization's network, as shown in Figure 4-2. Typical client-client applications include the following:
IP telephonyTwo peers establish communication with the help of a telephone manager workstation; however, the conversation occurs directly between the two peers when the connection is established.
File sharingSome operating systems (or even applications) require direct access to data on other workstations.
Videoconference systemsThis application is similar to IP telephony. However, the network requirements for this type of application are usually higher, particularly bandwidth consumption and QoS requirements.
Figure 4-2 Client-Client Application
Client-Distributed Server Applications
Historically, clients and servers were attached to a network device on the same LAN segment.
With increased traffic on the corporate network, an organization can decide to split the network into several isolated segments. As shown in Figure 4-3, each of these segments has its own servers, known as distributed servers, for its application. In this scenario, servers and users are located in the same virtual LAN (VLAN). Department administrators manage and control the servers. The majority of department traffic occurs in the same segment, but some data exchange (to a different VLAN) can happen over the campus backbone. For traffic passing to another segment, the overall bandwidth requirement might not be crucial. For example, Internet access must go through a common segment that requires less performance than the traffic to the local segment servers.
Figure 4-3 Client-Distributed Server Application
Client-Server Farm Applications
In a large organization, the organizational application traffic passes across more than one wiring closet, or VLAN. Such applications include
Organizational mail servers (such as Lotus Notes and Microsoft Exchange)
Common file servers (such as Novell, Microsoft, and Sun)
Common database servers for organizational applications (such as Sybase, Oracle, and IBM)
A large organization requires its users to have fast, reliable, and controlled access to the critical applications. To fulfill these demands and keep administrative costs down, the solution is to place the servers in a common Server Farm, as shown in Figure 4-4. The placement of servers in a Server Farm requires the network designer to select a network infrastructure that is highly resilient (providing security), redundant (providing high availability), and that provides adequate throughput. High-end LAN switches with the fastest LAN technologies, such as Gigabit Ethernet, are typically deployed in such an environment.
Figure 4-4 Client-Server Farm Application
Client-Enterprise Edge Applications
As shown in Figure 4-5, Client-Enterprise Edge applications use servers on the Enterprise Edge. These applications exchange data between the organization and its public servers.
The most important communication issue between the Enterprise Campus Network and the Enterprise Edge is not performance, but security. High availability is another important characteristic; data exchange with external entities must be in constant operation. Applications installed on the Enterprise Edge can be crucial to organizational process flow; therefore, any outages can increase costs.
Typical Enterprise Edge applications are based on web technologies. Examples of these application types, such as external mail servers and public web servers, can be found in any organization.
Figure 4-5 Client-Enterprise Edge Application
Organizations that support their partnerships through e-commerce applications also place their e-commerce servers into the Enterprise Edge. Communication with these servers is vital because of the two-way replication of data. As a result, high redundancy and resiliency of the network, along with security, are the most important requirements for these applications.
Application Requirements
Table 4-2 compares the types of applications and their requirements for the most important network parameters. The following sections discuss these parameters.
Table 4-2 Network Application Requirements
Parameter |
Client-Client |
|
Client-Distributed Servers |
Client-Server Farm |
Client-Enterprise Edge Servers |
Connectivity type |
Shared |
Switched |
Switched |
Switched |
Switched |
High availability |
Low |
Low |
Medium |
High |
High |
Total required throughput |
Low |
Medium |
Medium |
High |
Medium |
Total network cost |
Low |
Low |
Medium |
High |
Medium |
Connectivity
The wide use of LAN switching at Layer 2 has revolutionized local-area networking and has resulted in increased performance and more bandwidth for satisfying the requirements of new organizational applications. LAN switches provide this performance benefit by increasing bandwidth and throughput for workgroups and local servers.
NOTE
The shared media for client-client (also termed peer-to-peer) communication is suitable only in a limited scope, typically when the number of client workstations is very low (for example, in small home offices).
Throughput
The required throughput varies from application to application. An application that exchanges data between users in the workgroup usually does not require a high throughput network infrastructure. However, organizational-level applications usually require a high-capacity link to the servers, which is usually located in the Server Farm.
NOTE
Client-client communication, especially in the case of frequent file transfers, could be intensive, and the total throughput requirements can be high.
Applications located on servers in the Enterprise Edge are normally not as bandwidth- consuming (compared to the applications in the Server Farm) but may require high-availability and security features.
High Availability
High availability is a function of the application and the entire network between a client workstation and a server that is located in the network. Although network availability is primarily determined by the network design, the individual components' mean time between failures (MTBF) is a factor. It is recommended that you add redundancy to the distribution layer and the campus.
Cost
Depending on the application and the resulting network infrastructure, the cost varies from low in a client-client environment to high in a highly redundant Server Farm. In addition to the cost of duplicate components for redundancy, costs include the cables, routers, switches, software, and so forth.
Data Link Layer Technologies
Traditionally, network designers had a limited number of hardware options when purchasing a technology for their campus networks. Hubs were used for wiring closets, and routers were used to break the network into logical segments. The increasing power of desktop processors and the requirements of client/server and multimedia applications drove the need for greater bandwidth in traditional shared-media environments. These requirements are prompting network designers to replace hubs with LAN switches.
Key Point: Bandwidth Domains and Broadcast Domains
A bandwidth domain, which is known as a collision domain for Ethernet LANs, includes all devices that share the same bandwidth. For example, when using switches or bridges, everything associated with one port is a bandwidth domain.
A broadcast domain includes all devices that see each other's broadcasts (and multicasts). For example, all devices associated with one router port reside in the same broadcast domain.
Devices in the same bandwidth domain also reside in the same broadcast domain; however, devices in the same broadcast domain can reside in different bandwidth domains.
All workstations residing in one bandwidth domain compete for the same LAN bandwidth resource. All traffic from any host in the bandwidth domain is visible to all the other hosts. In the case of an Ethernet collision domain, two stations can cause a collision by transmitting at the same time. The stations must then stop transmitting and try again at a later time, thereby delaying traffic transmittal.
All broadcasts from any host residing in the same broadcast domain are visible to all other hosts in the same broadcast domain. Desktop protocols such as AppleTalk, Internetwork Packet Exchange (IPX), and IP require broadcasts or multicasts for resource discovery and advertisement. Hubs, switches, and bridges forward broadcasts and multicasts to all ports. Routers do not forward these broadcasts or multicasts to any ports. In other words, routers block broadcasts (which are destined for all networks) and multicasts; routers forward only unicast packets (which are destined for a specific device) and directed broadcasts (which are destined for all devices on a specific network).
Shared Technology
Shared technology using hubs or repeaters is based on all devices sharing a segment's bandwidth. Initially, the entire Ethernet segment was a single common busthe cable itself. With the introduction of hubs and new structured wiring, the physical network bus topology changed to a star topology. This topology resulted in fewer errors in the network because of the repeaters receiving an electrical signal and boosting the signal before forwarding it to all other segment participants (on all other repeater ports). All devices on all ports of a hub or repeater are on the same bandwidth (collision) domain.
Switched LAN Technology
Switched LAN technology uses the same physical star topology as hubs but eliminates the sharing of bandwidth. Devices on each port of a switch are in different bandwidth (collision) domains; however, all devices are still in the same broadcast domain. The LAN switches provide an efficient way of transferring network frames over the organizational network. In case of a frame error, the switch does not forward the frame as a hub or repeater would.
Comparing Switched and Shared Technologies
Table 4-3 presents some of the most obvious differences and benefits of switched technology compared to shared technology. It uses Fast Ethernet as an example.
Table 4-3 Switched Versus Shared Fast Ethernet Technologies
Parameter |
Switched |
Shared |
Bandwidth |
>10 Megabits per second (Mbps) |
<100 Mbps |
Range |
From 1 meter |
<500 meters |
Intelligent services |
Yes |
No |
High availability |
Yes |
No |
Cost |
$$ |
$ |
Bandwidth
The major drawback of shared technology is that all network devices must compete for the same bandwidth; only one frame flow is supported at a time. Bandwidth in shared technology is limited to the speed on a network segment (in this case, 100 Mbps for Fast Ethernet). Because of collisions, aggregate network bandwidth is less than this.
LAN switching technology supports speeds from Ethernet (10 Mbps) onward and enables multiple ports to simultaneously forward frames over the switch. Thus, the utilized aggregate network bandwidth could be much greater than with shared technology.
NOTE
A Layer 3 device separates network segments from each other into different broadcast domains. A traditional network's Layer 3 device was a router; in a modern network, the preference is for a Layer 3 switch.
Range
In a shared network, the network's diameter (the largest distance between two network devices) is constrained by the transmission media's physical characteristics because of the collision detection algorithmthe maximum distance between devices is limited to ensure that no collisions occur. In a shared environment, all devices reside in the same collision domain. The hub improves the frame's physical characteristics but does not check for frame errors. Every station on the segment must compete for resources and be able to detect whether two or more network stations are transmitting at the same time. The Ethernet standard for shared technology defines how long the sending device must possess the bus before it actually sends the data, so collisions can be detected. Because of this time limitation, the length or range of the segment is defined and never reaches more than 500 meters in the best-case scenario.
In a switched environment, devices on each port are in different collision domains. Collision detection is only a concern on each physical segment, and the segments themselves are limited in length. Because the switch stores the entire frame or part of it before forwarding it, the segments do not generate any collisions. The media that is used does not constrain the overall network's diameter.
Intelligent Services
The traditional shared technology is not capable of supporting new network features; this became important with the increasing number of organizational client/server and multimedia applications. LAN switches perform several functions at Layer 3, and even at higher Open System Interconnection (OSI) layers. Modern networks are required to support intelligent network services (such as QoS), security, and management; LAN switches have the ability to support these.
High Availability
Many organizational processes that run on the network infrastructure are critical for the organization's success. Consequently, high availability has become increasingly important. While shared networks do not offer the required capability, the LAN switches do.
Switches can be interconnected with multiple links without creating loops in the network (using the Spanning Tree protocol). Hubs cannot be interconnected with redundant links.
Cost
Considering all the benefits LAN switches offer, you might expect the cost per port to be much higher on switches than on hubs. However, with wide deployment and availability, the price per port for LAN switches is almost the same as it is for hubs or repeaters.
NOTE
All of the previously listed factors have mostly eliminated shared technologies; the majority of new networks use only switched technologies. Shared technologies are present in only some parts of existing networks and in smaller home offices.
Layer 2 and Layer 3 Switching Design Considerations
LAN switches have traditionally been only Layer 2 devices. Modern switches provide higher OSI level functionalities and can effectively replace routers in the LAN switched environment. Deploying pure Layer 2 or selecting Layer 3 switches in the enterprise network is not a trivial decision. It requires a full understanding of the network topology and customer demands.
Key Point: Layer 2 Versus Layer 3 Switching
The difference between Layer 2 and Layer 3 switching is the type of information that is used inside the frame to determine the correct output interface. Layer 2 switching forwards frames based on data link layer information (MAC address), while Layer 3 switching forwards frames based on network layer information (such as IP address).
When deciding on the type of LAN switch to use and the features to be deployed into a network, consider the following factors:
Network service capabilitiesThe network services the organization requires (QoS, and so on).
Size of the network segmentsHow the network is segmented, based on traffic characteristics.
Convergence timesThe maximum amount of time the network can be unavailable in the event of network outages.
Spanning-Tree Domain Considerations
Layer 2 switches use the Spanning Tree Protocol (STP) to ensure that only one active path exists between two switches. If a physical loop exists (for redundancy), STP puts ports on the switch in blocking state (thereby effectively disabling the ports, from a data perspective) to ensure a loop-free network. In the event of a failure, the blocked port is re-enabled (put into a forwarding state). An STP domain is a set of switches that communicates via STP. STP is illustrated in Figure 4-6.
Figure 4-6 STP
STP selects a root switch (or root bridge, according to IEEE 802.1d standard terminology) and determines whether any redundant paths exist. After the switch comes online, it takes up to 50 seconds before the root switch and redundant links are detected. At this time, the switch ports go through the listening and learning states; from there they progress to either the forwarding or blocking state. No ordinary traffic can travel through the network at this time.
NOTE
The default STP Forward Delay timer is 15 seconds; it determines how long the port stays in both the listening and learning states (for a total of 30 seconds). The Maximum Age timer defaults to 20 seconds; this is the time during which a switch stores a BPDU before discarding it, and therefore determines when the switch recognizes that a topology change has occurred. The addition of 30 seconds and 20 seconds composes the 50 seconds referred to previously.
When the primary link goes down and the redundant link must be activated, a similar event occurs. The time it takes for a redundant path to be activated depends on whether the failure is direct (a port on the same switch) or indirect (a port on another switch). Direct failures take 30 seconds because the switch bypasses the 20-second Maximum Age timer (and associated Blocking State for the port); from there it moves straight to the listening state (for 15 seconds), and then to the learning state (for 15 seconds). For indirect failures, the switch port must first wait 20 seconds (Maximum Age Timer) before it can transition to the listening state and then the learning state, for a total of 50 seconds. Thus, when a link fails, up to 50 seconds might pass before another link becomes available.
Cisco has implemented several features that have improved STP convergence. Recent standardization efforts have also proposed some new enhancements to the STP. Following is a brief description of the STP enhancements that result in faster convergence; this convergence is comparable to Layer 3 convergence and, in some instances, even exceeds it.
-
PortFastUsed for ports in which end-user stations and/or servers are directly connected. When PortFast is enabled, there is no delay in passing traffic because the switch immediately puts the port in the forwarding state (skipping the listening and learning states). Two additional measures that prevent potential STP loops are associated with the PortFast feature:
-
Bridge Protocol Data Unit (BPDU) GuardPortFast transitions the port into STP forwarding mode immediately upon linkup. Since the port still participates in STP, the potential of STP loop exists (if some device attached to that port also runs STP). The BPDU guard feature enforces the STP domain borders and keeps the active topology predictable. If the port receives a BPDU, the port is transitioned into errdisable state (meaning that it was disabled due to an error) and an error message is reported.
BPDU FilteringThis feature allows the user to block PortFast-enabled nontrunk ports from transmitting BPDUs. Spanning tree does not run on these ports.
UplinkFastIf the link to the root switch goes down and the link is directly connected to the switch, UplinkFast enables the switch to put a redundant path (port) into active state within a second.
BackboneFastIf a link on the way to the root switch fails but is not directly connected to the switch, BackboneFast reduces the convergence time from 50 seconds to between 20 and 30 seconds. When this feature is used, it must be enabled on all switches in the STP domain.
NOTE
Additional information regarding the errdisable state is available in Recovering From errDisable Port State on the CatOS Platforms, at http://www.cisco.com/en/US/tech/tk389/tk214/technologies_tech_note09186a0080093dcb.shtml.
In addition to features that enable faster convergence of the STP, features exist that prevent errors from resulting in unpredictable STP topology changes that could lead to STP loops. These features include the following:
STP Loop GuardWhen one of the blocking ports in a physically redundant topology stops receiving BPDUs, usually STP creates a potential loop by moving the port to forwarding state. With the STP Loop Guard feature enabled and if a blocking port no longer receives BPDUs, that port is moved into the STP loop-inconsistent blocking state instead of the listening/learning/forwarding state. This feature avoids loops in the network that result from unidirectional or other software failures.
BPDU Skew DetectionThis feature allows the switch to keep track of late-arriving BPDUs (by default, BPDUs are sent every 2 seconds) and notify the administrator via syslog messages. Skew detection generates a report for every port on which BPDU has ever arrived late (this is known as skewed). Report messages are rate-limited (one message every 60 seconds) to protect the CPU.
Unidirectional Link Detection (UDLD)If the STP process that runs on the switch with a blocking port stops receiving BPDUs from its upstream (designated) switch on that port, STP creates a forwarding loop or STP loop by eventually aging out the STP information for this port and moving it to the forwarding state. The UDLD is a Layer 2 protocol that works with the Layer 1 mechanisms to determine a link's physical status. If the port does not see its own device/port ID in the incoming UDLD packets for a specific duration of time, the link is considered unidirectional from the Layer 2 perspective. Once UDLD detects the unidirectional link, the respective port is disabled and the error message is generated.
Although spanning tree was previously considered to have very slow convergence (up to 50 seconds), the latest standard enhancements render its convergence comparable to (or even exceeding) that of routing protocols. The following enhancements are useful in environments that contain several VLANs:
Rapid STP (RSTP, defined in IEEE 802.1W)RSTP provides rapid convergence of the spanning tree by assigning port roles and determining the active topology. The RSTP builds upon the IEEE 802.1d STP to select the switch with the highest switch priority as the root switch and then assigns the port roles (root, designated, alternate, backup, and disabled) to individual ports. These roles assist in rapid STP convergence, which can be extremely fast (within a second) because of the topology knowledge.
Multiple STP (MSTP, sometimes referred to as MISTP [Multiple Instances of STP], defined in IEEE 802.1S)MSTP uses RSTP for rapid convergence by enabling several (topologically identical) VLANs to be grouped into a single spanning tree instance, with each instance including a spanning tree topology that is independent of other spanning tree instances. This architecture provides multiple forwarding paths for data traffic, enables load balancing, and reduces the number of spanning tree instances that are required to support a large number of VLANs.
Load Sharing Guidelines
Layer 2 and Layer 3 switches handle load sharing differently, as described in the following sections.
Layer 2 Load Sharing
Because Layer 2 switches are aware of only MAC addresses, they cannot perform any intelligent load sharing. In an environment characterized by multiple VLANs per access switch and more than one connection to the uplink switch, the solution is to put all uplink connections into trunks (Inter-switch link [ISL] or 802.1q). Each trunk carries all VLANs; however, without additional configuration, the STP protocol disables all nonprimary uplink ports. This configuration can result in a bandwidth shortage because the traffic for all the VLANs passes through the same link. To overcome this problem, the STP parameters must be configured to carry some VLANs across one uplink and the rest of the VLANs across the other uplink. For example, one uplink could be configured to carry the VLANs with odd numbers, while the other uplink is configured to carry the VLANs with even numbers. The top of Figure 4-7 illustrates this situation.
Figure 4-7 Layer 2 Versus Layer 3 Load Sharing
Layer 3 Load Sharing
Layer 3-capable switches can perform load sharing based on IP addresses. As illustrated in the lower portion of Figure 4-7, most modern Layer 3 devices with load sharing capability can balance the load per packet or per destination-source IP pair.
The advantage of Layer 3 IP load sharing is that links are used more proportionately than with Layer 2 load sharing, which is based on VLANs only. For example, the traffic in one VLAN can be very heavy while the traffic in another VLAN is very low; in this case, per-VLAN load sharing by using even and odd VLANs is not appropriate. Due to the dynamic nature of organizational applications, Layer 3 load sharing is more appropriate. Layer 3 allows for dynamic adaptation to link utilization and depends on the routing protocol design. Layer 3 switches also support Layer 2 load sharing, so they can still apply per-VLAN load sharing while connected to other Layer 2 switches.
Layer 2 Versus Layer 3 Switching
Table 4-4 compares Layer 2 and Layer 3 switching with respect to various campus network features. Considerations for deployment include
Pure Layer 2 switching throughout the network
Various combinations of Layer 2 and Layer 3 switching, including
Layer 3 switching in the distribution layer only
Layer 3 switching in the distribution and core layers
Layer 3 switching throughout the network
Table 4-4 Layer 2 Versus Layer 3 Switching
Parameter |
Layer 2 Everywhere |
Layer 3 in Distribution Only |
Layer 3 in Core and Distribution |
Layer 3 Everywhere |
Policy domain |
Layer 2 Access Control List (ACL) and QoS |
Layer 2 and Layer 3 ACL and QoS |
Layer 2 and Layer 3 ACL and QoS |
Layer 2 and Layer 3 ACL and QoS |
Load sharing |
Per VLAN |
Per VLAN Per destination |
Per VLAN Per destination |
Per VLAN Per destination |
Failure domain |
VLAN |
Access, core |
Access |
Segment |
Convergence |
STP |
Distribution: Routing protocol hold-timer (quick) Other: STP |
Core and distribution: Routing protocol hold-timer (quick) Access: STP |
Routing protocol hold-timer (quick) |
Cost |
$_ |
$$_ |
$$$_ |
$$$$ |
The following sections elaborate on the features in Table 4-4.
Policy Domain
The policy domain is the scope of the network that is affected by a certain policy. A network policy is a formal set of statements that define how network resources are allocated among devices. In addition to selected hosts or applications, the policies can be applied to individual users, groups, or entire departments. For example, policies can be based on the time of day or client authorization priorities. Network managers implement policies and policy statements and store them in a policy repository or on the device itself. The devices then apply the configured policies to network resources.
The size of the policy domain depends on the switching layer and on the mechanisms for policy implementation. In pure Layer 2 switching, the policy domain overlaps with the switching domain's boundaries; Layer 3 switching offers much more flexibility. In Layer 2 switching, the access control lists (ACLs) and various QoS mechanisms can only be applied to switched ports and MAC addresses; in the Layer 3 switching, the ACL and QoS mechanisms are extended to IP addresses, or even applications (for example, using Transmission Control Protocol [TCP] and User Datagram Protocol [UDP] ports).
Load Sharing
When multiple links exist, they can be used for redundancy and/or traffic load sharing. As discussed in the "Load Sharing Guidelines" section of this chapter, Layer 2 switches only offer load sharing by distributing VLANs across different uplink ports. Layer 3 switches, however, can perform load sharing between ports based on IP destinations.
Failure Domain
A failure domain defines the scope of the network that is affected by network failures. In a Layer 2-switched domain, a misconfigured or malfunctioning workstation can introduce errors that impact or disable the entire domain. Problems of this nature are often difficult to localize.
A failure domain is
Bounded by Layer 3 switching
Bounded by the VLAN when Layer 2 switching is deployed in an entire campus
Convergence
As discussed in the "Spanning-Tree Domain Considerations" section of this chapter, loop prevention mechanisms in a Layer 2 topology cause the STP to take between 30 and 50 seconds to converge. To eliminate STP convergence issues in the campus backbone, all the links connecting backbone switches must be routed links, not VLAN trunks. This also limits the broadcast and failure domains.
In the case where the Layer 3 switching is deployed everywhere, convergence is within seconds (depending on the routing protocol implemented) because all the devices detect their connected link failure immediately and act upon it promptly (sending respective routing updates).
In a mixed Layer 2 and Layer 3 environment, the convergence time not only depends on the Layer 3 factors (including routing protocol timers such as hold-time and neighbor loss detection), but also on the STP convergence.
Using Layer 3 switching in a structured design reduces the scope of spanning tree domains. It is common to use a routing protocol, such as Enhanced Interior Gateway Protocol (EIGRP) or Open Shortest Path First (OSPF), to handle load balancing, redundancy, and recovery in the backbone.
Cost
The cost of deploying Layer 3 switching in comparison to Layer 2 switching increases with the scope of Layer 3 switching deployment. Layer 3 switches are more expensive than their Layer 2 counterparts; for example, Layer 3 functionality can be obtained by adding cards and software to a modular Layer 2 switch.
Transmission Media
An Enterprise Campus can use various physical media to interconnect devices.
Selecting the type of cable is an important consideration when deploying a new network or upgrading an existing one. Cabling infrastructure represents a long-term investmentit is usually installed to last for ten years or more. In addition, even the best network equipment does not operate as expected with poorly chosen cabling.
A network designer must be aware of physical media characteristics because they influence the maximum distance between devices and the network's maximum transmission speed.
Twisted-pair cables (copper) and optical cables (fiber) are the most common physical transmission media used in modern networks.
Unshielded Twisted-Pair (UTP) Cables
UTP consists of four pairs of isolated wires that are wrapped together in plastic cable. No additional foil or wire is wrapped around the core wires (thus, they are unshielded). This makes these wires less expensive, but also less immune to external electromagnetic influences than shielded cables. UTP is widely used to interconnect workstations, servers, or other devices from their network interface card (NIC) to the network connector at a wall outlet.
The characteristics of twisted-pair cable depend on the quality of their material. As a result, twisted-pair cables are sorted into categories. Category 5 or greater is recommended for speeds of 100 megabits per second (Mbps) or higher. Because of the possibility of signal attenuation in the wires and carrier detection, the maximum cable length is usually limited to 100 meters . For example, if one PC starts to transmit and another PC is more than 100 meters away, the second PC might not detect the signal on the wire and therefore start to transmit, causing a collision on the wire.
One of the frequent considerations in the cabling design is electromagnetic interference. Due to high susceptibility to interference, UTP is not suitable for use in environments with electromagnetic influences. Similarly, UTP is not appropriate for environments that can be affected by the UTP's own interference.
NOTE
Some security issues are also associated with electromagnetic interferenceit is easy to eavesdrop on the traffic carried across UTP because these cables emit electromagnetic interference.
Optical Cables
Typical requirements that lead to the selection of optical cable as a transmission media include distances longer than 100 meters, and immunity to electromagnetic interference. There are different types of optical cable; the two main types are multimode (MM) and single-mode (SM).
Both MM and SM optical cable have lower signal losses than a twisted pair cable; therefore, optical cables automatically enable longer distances between devices. However, fiber cable has precise production and installation requirements, resulting in a higher cost than twisted pair cable.
Multimode fiber is optical fiber that carries multiple light waves or modes concurrently, each at a slightly different reflection angle within the optical fiber core. Because modes tend to disperse over longer lengths (modal dispersion), MM fiber transmission is used for relatively short distances. Typically, light emitting diodes (LEDs) are used with MM fiber. The typical diameter of an MM fiber is 50 or 62.5 micrometers.
Single-mode (also known as monomode) fiber is optical fiber that carries a single wave (or laser) of light. Lasers are typically used with SM fiber. The typical diameter of an SM fiber core is between 2 and 10 micrometers.
Copper Versus Fiber
Table 4-5 presents some of the critical parameters that influence the network transmission medium selection.
Table 4-5 Copper Versus Fiber Media
Parameter |
Copper |
Fiber |
Bandwidth |
Ethernet: <1 gigabits per second (Gbps) LRE: <15 Mbps |
<10 Gbps |
Range |
Ethernet: <100 m LRE: <1.5km |
MM: 550 m* SM: <100 km* |
Deployment area |
Wiring closet |
Inter-node and inter-building |
Other considerations |
Interference, grounding |
Coupling loss |
Installation cost |
$ |
$$$ |
* When using Gigabit Ethernet
NOTE
Table 4-5 lists Ethernet as a technology; this includes Ethernet, Fast Ethernet, and Gigabit Ethernet. Long Reach Ethernet (LRE) is also listed. This latter technology is Cisco proprietary and runs on voice-grade copper wires; it allows higher distances than traditional Ethernet and is used as an access technology in WANs. Chapter 5, "Designing WANs," further describes LRE.
The following sections elaborate on the parameters in Table 4-5.
Bandwidth
The bandwidth parameter indicates the required bandwidth in a particular segment of the network, or the connection speed between the nodes inside or outside the building.
Range
The range parameter is the maximum distance between network devices (such as workstations, servers, printers, and IP phones) and network nodes, and between network nodes.
Table 4-6 summarizes the bandwidth and range characteristics of the transmission media types.
Table 4-6 Transmission Media Types Bandwidth and Range Characteristics
Parameter |
Twisted-Pair |
MM Fiber |
SM Fiber |
Distance (range) |
Up to 100 meters |
Up to 2 kilometers (km) (Fast Ethernet) Up to 550 m (Gigabit Ethernet) |
Up to 40 km Up to 100 km (Gigabit Ethernet) |
Speed (bandwidth) |
Up to 1 Gpbs |
Up to 1 Gbps |
10 Gbps |
Cost |
Cheap to install |
Moderate |
High price |
Copper cables are typically used for connectivity of network devices to the wiring closet where
Distances are less than 100 meters
Speeds of 100 Mbps are satisfactory
Cost must be kept within reasonable limits
NOTE
Fast EtherChannel (FEC) and Gigabit EtherChannel solutions group several parallel links between LAN switches into a channel that is seen as a single link from the Layer 2 perspective. Two protocols have been introduced for automatic EtherChannel formation: the Port Aggregation Control Protocol (PagP), which is Cisco proprietary, and the Link Aggregation Control Protocol (LACP), which is standardized and defined in IEEE 802.3ad.
Deployment Area
Deployment area indicates whether wiring is required for wiring closet only (where users access the network), for inter-node, or even for inter-building connections.
Connection from the wiring closet to the building central node can use UTP. As for most inter-node and especially inter-building connections, MM, or even SM, fiber is probably needed if there are high-speed requirements.
Other Considerations
When deploying UTP in an area with high electrical or magnetic interferencefor example, in an industrial environmentyou must pay special attention to media selection. In such environments, the disturbances might interfere with data transfer and therefore result in an increased number of frame errors. Electrical grounding can isolate some external disturbance, but the wiring increases the costs. Fiber optic installation is the only reasonable solution for such networks.
Optical fiber requires a precise technique for cable coupling. Even a small deviation from the ideal position of optical connectors can result in either a loss of signal or a large number of frame losses. Careful attention during optical fiber installation is imperative because of the traffic's high sensitivity to coupling misalignment. In environments where the cable does not consist of a single fiber from point to point, coupling is required and loss of signal can easily occur.
Installation Cost
Along with the cost of the medium, you must also seriously consider installation cost. Installation costs are significantly higher than UTP installation costs because of strict requirements for optical cable coupling.
Cabling Example
Figure 4-8 illustrates a typical campus network structure. End devices such as workstations, IP phones, and printers are no more than 100 m away from the LAN switch. UTP wiring can easily handle the required distance and speed; it is also easy to set up, and the price/performance ratio is reasonable.
Figure 4-8 A Campus Network Uses Many Different Types of Cables
Optical fiber cables handle higher speeds and distances that can be required among switch devices. MM optical cable is usually satisfactory inside the building. Depending on distance, organizations use MM or SM optical for inter-building communication cable. If the distances are short (up to 500 m), MM fiber is a more reasonable solution for speeds up to 1 Gbps.
However, an organization can install SM fiber if its requirements are for longer distances, or if they are planning for future higher speeds (for example, 10 Gbps). The current specification provides Gigabit Ethernet connectivity on SM fiber up to 5 km; however, Cisco has already provided modules that support connectivity up to 10 km, and even up to 100 km.
NOTE
Selecting the less expensive type of fiber might satisfy a customer's current need, but this fiber might not meet the needs of future upgrades or equipment replacement. Replacing cable can be very expensive. Planning with future requirements in mind might result in higher initial costs, but ultimately lower costs.