Distribution/Regional Network Design
The role of the regional network is to route intra- and inter-regional traffic. The regional network generally is comprised of a DC as the hub and a number of access POPs as the spokes. Usually, two redundant routers in each regional network will connect to the backbone.
DCs may also provide services such as Web-caching, DNS, network management, and e-mail hosting. In some cases, the latter functionality may be extended into major POPs.
Placement of DCs is generally an economical choice based on the geographical proximity to a number of access sites. However, this does not mean that an access POP cannot be a mini-distribution center or transit for another access POP, but this is the exception rather than the rule.
When an access POP site provides such transit, and when that transit is the responsibility of the service provider, it should be considered part of the distribution network functionality.
Although the DC may be the center of a star topology from a network or IP perspective, this does not limit the choice of data-link or WAN connectivity to point-to-point links. Frame Relay or other cloud technologies can be—and often are—used to provide the connectivity from the customers, or from other distribution and access sites to the DC. Even within the DC, a provider may utilize Layer 2 aggregation equipment, such as a Frame Relay or ATM switch, or even an add/drop multiplexor.
A major DC typically consists of many routers, carrying either intra-regional or backbone-transit traffic. As more customers receive service from the DC, the higher the stakes become. Therefore, the backbone and intra-distribution network infrastructure must become more reliable.
A common option at major DCs is to provide dual aggregation LANs, dual backbone routers, and dual backbone WAN connections, as shown in Figure 4-3. This approach also can provide an element of load sharing between backbone routers. Of course, a single aggregation LAN and single backbone router will also serve this purpose. It is important to weigh the cost-versus-reliability issues, and bear in mind that most simple MTBF calculations consider hardware, but often ignore both software bugs and human error.
FDDI rings are a logical choice for the aggregation LAN because of their inherent fail-over mechanisms. However, with the development of low-cost/high-reliability LAN switches based on FDDI, Ethernet, or ATM technology—not to mention the ever-increasing intra-DC traffic levels—it is not uncommon to implement the dual aggregation LANs using switched media. IP routing circumvents LAN failure at either the single line card or the single switch level, as discussed in upcoming chapters.
Of course, many other critical reliability issues have not yet been considered. These include facilities, such as power supply and the choice of router and switching equipment.
NOTE
The distribution network is hierarchical. Router dist3 is located as an access POP, which services fewer customers, and therefore is not a resilient design.
The backbone/distribution/access hierarchy can be bypassed to achieve lower delays at the expense of reliability. Customer 4 may connect directly to router core2.sfo. However, if core2.sfo fails—albeit a rare event—customer 4 is effectively cut off from the network. Alternatively, customer 4 may have a backup connection via dist3.sfo.
This arrangement is satisfactory, provided that it does not confuse the role of each router. For example, directly connecting customer routers to the core router indicates that they may have to perform dial-up authentication, packet and router filtering, and packet classification. Not only will this occupy precious switching cycles on the core router, but it also could mean running a larger and possibly less reliable software image.
Other possible failure modes include the following:
Core1—All intra-network traffic is routed through core2. All traffic to other ISPs is also routed through core2, presumably to another NAP connected to a backbone router elsewhere in the network.
Ds1—Traffic destined for a remote distribution network is switched through ds2, as is traffic destined for other locations in the local distribution network.
Dist1—Customer 2 is re-routed through Dist2.
Dist3—Customer 3 is cut off.
It is worth noting that any resilience at Layer 3 results in routing complexity. This is examined in detail in Part II. As a matter of policy, the network service provider may choose not to allow customers to connect to core routers or even to dual distribution routers.
However, in the enterprise environment, reliability affects user satisfaction. In the commercial environment, this may affect their choice of provider. Policy that simplifies engineering must be carefully balanced against customer requirements.
Policy also must be balanced against the risk of human error. A resilient routing environment might be more reliable in theory, but in practice it might have a greater risk of human configuration error, and possibly algorithmic or vendor implementation flaws.