Multicast Design Solutions

The previous chapters of this book discuss some rather advanced types of multicast networks. Typically, as the requirements of a network grow in complexity, so do the number and type of protocols needed to meet those requirements. The chapters so far in this book and IP Multicast, Volume 1 have provided the tools needed to meet most of these requirements. However, there are some additional elements to consider when designing complex multicast networks.

Items to cogitate include desired replication points, multitenancy and virtualization properties, and scalability of the multicast overlay in relation to the unicast network. In addition, every network architect must be concerned with the redundancy, reliability, and resiliency of a network topology. Multicast network design must consider these elements as well. The best way to discuss these elements of multicast design is through the examination of specific network archetypes that employ good design principles.

This chapter examines several archetypical network design models. These models might represent a specific network strategy that meets a specific commercial purpose, such as a trade floor. The examined model may also be a general design for a specific industry, such as the deployment of multicast in a hospital environment. The intent is to provide a baseline for each type of design, while providing examples of best practices for multicast deployments.

This chapter looks at the following design models:

• Multicast-enabled hospital networks, with an emphasis on multicast in wireless access networks

• Multicast multitenancy data centers

• Software-defined networks with multicast

• Multicast applications in utility networks

• Multicast-enabled market applications and trade floors

• Multicast service provider networks

Multicast-Enabled Clinical Networks

A large hospital network design is very similar to most large campus network designs. Campus networks typically focus on providing services to users. A proper campus network design may be very familiar to long-time Cisco networking veterans who studied the three-tier (access/distribution/core) network hierarchy. Figure 5-1 shows an archetype of the three-tier model.

Figure 5-1 Three-Tier Network Hierarchy

Some newcomers to networking may be unacquainted with the access/distribution/core design model. Modern network devices and protocols allow architects to collapse much of the three-tier hierarchy into segments or, in many cases, single devices. For most campuses, even if the physical segmentation between the layers does not exist, there is at minimum a logical segmentation between the access and distribution/core layers of the design. For example, there is a routing and switching layer at the collapsed distribution/core of the network. Additional resources connect to this collapsed core, including data center or WAN connections. Figure 5-2 shows a basic campus network with collapsed distribution and core layers.

Figure 5-2 Campus Design with Collapsed Network Core

Because network access can be obtained by a myriad of device types, the access layer design usually includes a very robust wireless and wired infrastructure. Most isolation and segmentation in the access layer is provided by virtual local area network (VLAN) segmentation on Layer 2 switches. Layer 3 distribution routers provide inter-VLAN routing, virtual private network (VPN) services, and transit to centralized resources. Campus core switches and routers move traffic across all segments of the network quickly and efficiently.

In addition, many elements of hospital network design are completely unique. A typical campus network provides employees access to resources such as Internet services, centralized employee services (print, file, voice, and so on), bring-your-own-device (BYOD) services, and some minor guest services. A hospital network needs to deliver these to employees as well. However, the main purpose of a hospital is to provide services to patients and guests. Therefore, a hospital network needs to accommodate devices that service patients. Devices in a hospital network include badge systems, medical monitors, interpersonal communications, media services, and biomedical equipment. Data generated by these devices needs to be stored locally and remotely, and other data center type services are required as well. In addition, the hospital is likely to have extensive guest services, mostly focused on providing wireless guest Internet access.

Keeping all these devices secure and properly securing and segmenting data paths is critical. Government regulations, such as the well-known Health Insurance Portability and Accountability Act (HIPAA) in the United States, require that these security measures be comprehensive and auditable. This means hospital devices must be segregated from both users and personnel networks.

All these unique elements make network design very complex. For this reason, there is a great deal of variability in hospital network deployments. Regardless of this variability, almost every hospital network offers some level of multicast service. Most medical devices and intercommunications media can be configured with IP Multicast to make service delivery and data collection more efficient. Providing medical device and intercommunications services is the primary focus of this design discussion.

Accommodating Medical Device Communications Through Multicast

If you have ever been to a hospital, it is likely that you have noticed that many medical devices are used by doctors and staff to treat and monitor patients. What you may not have realized is that many of these devices are connected to the hospital campus network. These devices can include anything from a smart pump, to a heart monitor, to imaging equipment such as x-ray machines.

The benefits of connecting these devices to the network are enormous. The most important benefit is that data generated by these devices can be accessed remotely and, in many cases, in real time. In addition, network-connected biomedical devices—as well as non-medical devices such as badges—can be moved to any room on the hospital campus. When connected to a wireless network, these devices can be small and follow staff and patients about the hospital as required.

Generally, the way in which these devices communicate is largely controlled by the device manufacturer. However, most modern hospital devices can at a minimum communicate using Layer 2 Ethernet, and many communicate using Internet Protocol (IP) at Layer 3. Most medical devices of a particular kind need to connect to a centralized controller or database that also needs access to the same network resources.

Manufacturers use three prototypical communication models to accomplish this:

• Layer 2 only

• Layer 2 and Layer 3 hybrid

• Layer 3 only

The first type of model uses Layer 2 connections only. In this model, it may be a requirement that all devices be on the same Layer 2 domain (segment). In an Ethernet network, this means they must be on the same VLAN. Centralized device controllers and data collectors may also need to be on the same segment. Some manufacturers require that the same wireless network, identified by the same Service Set Identification (SSID), be used for the monitors and controllers alike. Figure 5-2 depicts this type of model, with all the networked devices and their centralized resources connected to the same SSID (OneHospital) and VLAN (VLAN 10).

Figure 5-3 Layer 2 Only Medical Device Communication Model

The second communications model uses a hybrid of Layer 2 and Layer 3 communications. In this case, devices may be on separate VLANs and may be able to function within a small routed domain. However, a specific device type and its corresponding controller or central station may need to connect to the same VLAN. The controllers might provide communications and data outside the VLAN to remote resources, such as either a local or remote data center. They may also have no Layer 3 capabilities. A wireless LAN controller (WLC) is likely needed to accommodate inter-VLAN or Layer 3 communication at scale in this model. Figure 5-4 shows such a hybrid communications model.

Figure 5-4 Layer 2 and Layer 3 Hybrid Device Communications Model

Finally, some devices are capable of fully routed Layer 3 IP communications. Controllers and central stations may be located remotely, or they may be centralized on the hospital campus. Devices may roam the network but are likely segmented for security. Figure 5-5 shows an example of this communications model. In this example, the devices use IP to speak to controllers in a localized data center. The device controller servers share this data with local and remote data center resources across the network.

Figure 5-5 Layer 3 Only Communications Model

Even though there are three distinct models, you cannot assume that all devices fit neatly within one model—or even within the examples as they are shown here. They are, after all, only models. It is also likely that a hospital campus network has to accommodate all three models simultaneously to connect all medical devices.

Medical devices that make full use of IP networking and multicast communications are very similar to other network devices. They need to obtain an IP address, usually via Dynamic Host Configuration Protocol (DHCP), and have a default gateway to communicate outside their local VLAN. One major difference, as shown in the models in this section, is that medical devices typically connect to some type of controller or station. This is where multicast becomes very practical: allowing these devices to communicate with the central station in a very efficient manner, as there are many devices that connect to a single controller. In these scenarios, either the central station or the device (or both) may be the multicast source. This is a typical many-to-many multicast setup. Figure 5-6 shows this communication in action for the Layer 3 model shown in Figure 5-5.

Figure 5-6 Medical Device Communication Flow Example

There are many devices connected to a hospital network—many more than in a typical campus network. Multicasting eases the operational burden on network resources, including access switching resources. There are a number of things to consider in the multicast design of the campus to accommodate the communication shown in Figure 5-6.

The first decision that must be made corresponds to the type of communication model in place. If all the devices are using a Layer 2 only communications model, then a complex multicast overlay design may not be required. In this case, the local network access points (APs) and switches may only need to support basic Internet Group Management Protocol (IGMP) and IGMP snooping to accommodate localized multicast inside the same Layer 2 domain. If Layer 3 services are required, Protocol Independent Multicast (PIM) must also be used.

The second decision to make involves the type of PIM implementation to deploy across the network. If there are many sources and many receivers (many-to-many multicast, a likely scenario at a hospital), it may be wise to consider deploying Bidirectional PIM (Bidir-PIM). Using Bidir-PIM dramatically improves network device resource consumption by eliminating steps in the source tree building process, as all source packets are forwarded from the source toward the rendezvous point (RP). It also makes multicast traffic more predictable. However, Bidir-PIM inevitably causes path selection that may not be ideal, and it may cause overconsumption of centralized resources, potentially including precious bandwidth at the distribution/core of the network (depending on RP placement).

Many medical devices use both multicast and unicast communications—but for different functions. Each function may have different security and routing requirements. These networks may require that an RP exist at each segment, with no centralized RP services so that domain security is tightly controlled. Bidir-PIM could be a poor choice in an environment where severe segmentation at both Layer 2 and Layer 3 are required. When multiple RPs need to coexist, a standard PIM Sparse-Mode (PIM-SM) implementation may be a better choice. This is essentially a multidomain configuration. If interdomain multicast forwarding is required, PIM-SM with Multicast Source Discovery Protocol (MSDP), and perhaps Border Gate Protocol (BGP), is ideal.

PIM Source-Specific Multicast (SSM) provides the benefits of both Bidir-PIM and PIM-SM without the need for a complex interdomain or multidomain design. SSM does not require an RP for pathing, nor does it require additional protocols for interdomain communications. The biggest drawback to using SSM is that many medical devices, especially those that are antiquated, may not support IGMPv3. Recall from IP Multicast, Volume 1 that IGMPv3 subscribes a host to a specific source/group combination and is required for the last-hop router (LHR) to join the appropriate source-based tree. In addition, SSM may not be able to scale with a very large many-to-many design.

Selecting the best PIM implementation for a hospital campus is an exercise in comparing the advantages and disadvantages of each with the capabilities of both the network devices and the devices connecting to the campus network. It is very likely that no single implementation will perfectly address all the needs of the network. Table 5-1 compares some of the advantages and disadvantages of the different PIM implementations in a hospital network.

Table 5-1 Comparison for PIM Implementation Selection for Hospital Networks

This table is useful for any campus design but considers the specific needs of a hospital campus. It is very likely that the deployment of more than one implementation is the best choice. For example, localized medical device multicast that does not leave the local VLANs can use a simple Bidir-PIM implementation with a centralized Phantom RP. VLANs with devices that need fully routed IP Multicast with possible interdomain communication can use a single Anycast RP solution, with PIM-SM providing the tree building. Core network services may use an SSM implementation to improve the reliability of the infrastructure by eliminating the need for RPs. All three PIM implementations can coexist within the same campus if properly configured as separate, overlapping, or intersecting domains. PIM-SM and PIM-SSM can be configured on the same router interfaces, whereas PIM-SM and Bidir-PIM are mutually exclusive.

This brings up an important point: When purchasing network equipment for a medical campus, make sure it optimizes support for these implementations. Whenever possible, select routers, switches, APs, WLCs, and other infrastructure that support the following:

• All versions of IGMP (versions 1, 2, and 3)

• The appropriate scale of IGMP joins and IGMP snooping for the number of expected devices (This is the most basic requirement at the edge of the network.)

• The appropriate PIM implementation(s) (Bidir-PIM, PIM-SM, and PIM-SSM)

• Optimal packet replication techniques to improve throughput and resource consumption

• Simplified multicast configuration and troubleshooting

• Multicast path optimization features, such as multipath multicast and spanning-tree optimization, or elimination, for all traffic flows

• Ability to secure multicast devices and domains

• Ability to provide quality of service (QoS) to multicast flows, including rate-limiting and/or storm control to limit overconsumption of resources by malfunctioning devices

Multicast Considerations for Wireless Networks

For wired switches, multicast is rudimentary, a basic component of Layers 2 and 3. A wired switch uses IGMP snooping, and each VLAN is configured with at least one routed interface that supports IGMP and multicast forwarding. This ensures that the devices connected to the switch port have access to the larger campus multicast overlay. Wireless Ethernet functions in a similar manner but introduces additional concerns. Ensuring that the wireless network efficiently and effectively transports all traffic is perhaps the most important consideration in designing multicast in a medical campus.

The first point to understand when considering multicast data transmission in wireless networking is that wireless networks use IP Multicast in the wireless management plane. A campus network is likely to have many APs. To ensure that these APs function in sync with one another, one or more WLCs are used. A WLC acts very much like the medical device central station discussed in the previous section. It connects to and oversees the configuration of the APs in the network. To enable multicast within the management plane, the switched management network must be properly configured for a multicast overlay. This may very well be an MVPN using multicast Virtual Route Forwarding (VRF) instances, as discussed in Chapter 3, “Multicast MPLS VPNs.”

More important is the flow of multicast traffic to and from wireless LAN (WLAN) connected devices. This is especially relevant when sources and receivers are connected via either the wired or wireless infrastructure and they must be able to complete a forwarding tree, regardless of the connection type or location. Cisco WLANs must work seamlessly with the physical wired LAN. Another way of saying this is that a campus WLAN must be a functional extension of the VLAN, making the two functionally equivalent.

In an archetypical campus, each SSID is equivalent to a VLAN and is essentially configured as such. The VLAN may only be delivered wirelessly, or it could be accessible through both the wireless and the wired infrastructure. Figure 5-4 (shown earlier) illustrates just such a network, where patient heart monitors and the central station for these monitors are all part of the same VLAN, VLAN 11. SSID MonitorTypeTwo extends the VLAN to the individual wireless client monitors.

This design works well in environments where multicast is rarely forwarded past the local VLAN and where individual monitor functions need to be on the same Layer 2 domain. However, many modern hospitals use segmentation that is based on the patient’s location rather than the type of device deployed. Figure 5-7 shows a design that uses a room grouping (perhaps based on the hospital floor, or even the type of room) to divide VLANs and SSIDs. In this design, the central monitor stations are part of the same VLAN, terminated on the access layer switches of the hospital.

Figure 5-7 SSIDs Deployed by Room Groups with Local Control Stations

It is also possible that a routed network separates the monitoring station servers, which are then accessed by hospital staff through an application. These stations are generally centralized in a local data center to prevent WAN failures from causing service interruption. Figure 5-8 depicts the same room grouping design, with centralized control stations. Wireless APs provide access to the monitors, while a fully routed Layer 3 network extends between the monitor devices and the data center, crossing the collapsed campus core.

Figure 5-8 SSIDs Deployed by Room Groups with Centralized Central Stations

Regardless of how the SSIDs are deployed, how does the AP know what to do with the multicast packets sourced from either the wireless clients or other sources on the network? This question is especially relevant in a Layer 2 and 3 hybrid design or in a fully routed Layer 3 device communications model, like the one shown in Figure 5-8. The APs need to be able to replicate and forward packets upstream to where Layer 3 is terminated.

Wireless networks are, of course, more than just a collection of APs, SSIDs, and VLANs. Campus WLANs also use WLCs to manage the APs and SSIDs of the network. WLCs in a hospital campus almost always function as local management, meaning that APs are not only configured by the WLC but also tunnel traffic to the WLC for advanced switching capabilities, such as multicast packet replication. The tunneling mechanism used in this type of campus is the IETF standard Control and Provisioning of Wireless Access Points (CAPWAP), as defined by IETF RFC 5415. Figure 5-9 shows this relationship between the APs and the WLC(s) in a campus with a fully routed Layer 3 device communications model.

Figure 5-9 CAPWAP Tunneling with Layer 3 Device Communications Model

Some years ago, Cisco introduced a new architecture called Unified Access (UA) for deploying wireless in general but especially for multicast and broadcast traffic. While not all UA features are adopted in every network, this book uses UA as the de facto standard for this discussion. Before UA was implemented in Cisco wireless APs and WLCs, multicast packets were forwarded by a WLC encapsulated inside unicast packets. The WLC sent these unicast packets to each AP that had clients subscribed to the group instead of performing proper multicast replication, potentially including an AP from where the source was connected. This is not particularly efficient, nor does it control packet looping—one of the primary functions of multicast forwarding in modern networks. Cisco UA instead supports multicast forwarding across the APs and WLCs. Let’s examine how this works.

A Cisco UA-capable WLAN controller appears to the routed network as a router joined to the multicast groups of downstream clients. When the controller receives a multicast packet from the network, it examines the group to see if any wireless clients are currently subscribed. If there are subscribed clients, the WLC encapsulates the original multicast packet inside a CAPWAP multicast packet that uses a specific group. APs are subscribed to this group, making the WLAN controller the source and the APs the clients. The WLC then sends a single multicast toward the routed network for standard multicast replication and forwarding. The APs receive the CAPWAP-encapsulated multicast via the network, strip the CAPWAP headers, and then forward the original multicast packet. The packet must be transmitted on all radios servicing an SSID. There is no other way to transmit a local wireless multicast. As with a Layer 2 switch, if the AP receives unassociated multicast packets, it sends those packets to the bit bucket. Figure 5-10 visualizes this multicast flow.

Figure 5-10 CAPWAP Multicast Forwarding from a Wired Source

To summarize, when a multicast packet is received from the routed network, there are three main steps in the forwarding process, as outlined in Figure 5-10:

• Step 1. The WLC receives a multicast packet from the routed network and encapsulates the packet in a CAPWAP multicast packet. APs have already joined the CAPWAP group, acting as clients. A bitmap is created to map SSIDs to CAPWAP packets. CAPWAP multicast packets are forwarded using the management interface.

• Step 2. The multicast-enabled network receives the CAPWAP multicast from the management interface and forwards the packet to the APs through the multicast overlay. This relieves the WLC of the burden of replication.

• Step 3. APs receive the CAPWAP multicast packet and read the WLC-assigned bitmap for the packet, indicating the SSID for which the original multicast is intended. The AP then strips the outer header and broadcasts on the corresponding SSID. APs may receive other multicast packets, but they will forward only one copy of the multicast packet received from its primary WLC. An AP drops all other packets.

When a wireless client is the source of the multicast stream, the packet flow is slightly different. First, unlike a Layer 2 switch, the AP does not replicate the packet locally. Instead, it must forward all multicast packets up the unicast CAPWAP tunnel to the WLC. The WLC handles the multicast from there. Remember that clients for a group may be located on the wired LAN, on the routed network, or on the same WLAN. The WLC needs to service all these paths. You can add an additional two steps to the forwarding process when this happens, as shown in Figure 5-11:

Figure 5-11 CAPWAP Multicast Forwarding from a Wireless Source

• Step 1. The AP receives a multicast packet from a wireless client—in this case, for group 239.1.1.110. The AP encapsulates the original multicast packet inside the CAPWAP unicast tunnel and forwards it to the WLC. This is the same process for any packet that needs additional routing in a wireless CAPWAP network.

• Step 2. The WLC receives the packet, strips the unicast CAPWAP header, and forwards the original multicast packet upstream, toward the routed network. The standard routed multicast overlay ensures that the packets are forwarded toward any wired subscribed clients, regardless of their location (assuming that a complete multicast tree can be built). The WLAN controller also replicates the original multicast packet, encapsulates it inside a single CAPWAP multicast packet with the appropriate bitmap ID, and sends it toward the routed network. From there, the process is the same as when the packet comes from a wired client, eventually being forwarded to the appropriate APs and then to the SSID. This means the originating SSID, including the originating client, gets a copy of the same packet. Wireless multicast clients are designed to account for this phenomenon and should not process the replicated packet.

One of the primary uses of a multicast design in a hospital is to enable a hospital badging system. Clinical badge systems are not like the standard swipe card badge systems that many users are familiar with. Badge systems for hospitals, such as those provided by system manufacturer Vocera, are more forward thinking. They use IP integration and Voice over IP (VoIP) to establish communications between badge users and other hospital systems and applications, much like the futuristic badges used on your favorite sci-fi TV series.

Clinical badge systems allow hospital staff to communicate quickly and efficiently across specific teams, hospital wards, or other organizations. Many have unique paging features and are also used to track the locations of wearers. This location tracking is integrated into hospital security systems for quick access to specific areas of the hospital. Other medical badge systems can be used to track location for system lockdowns, like those used to monitor infant location and prevent infants from leaving the wards to which they are assigned. If you have ever been to a hospital, you may have seen badge systems like these.

Because badges are worn by hospital staff, it is critical that wireless infrastructure be fast, resilient, and reliable. Medical devices also need similar network resiliency. That is the primary reason that WLCs are installed locally at the distribution or access layer of the network and use CAPWAP tunnels instead of remotely deployed WLCs. As mentioned earlier, these badge systems rely on multicast as well. This means the multicast infrastructure should also be fast, resilient, and reliable. Achieving this means that designing a robust Layer 2 and Layer 3 multicast implementation is critical; the implementation must follow the best practices described in IP Multicast, Volume 1.

In addition, for PIM-SM networks, the placement of the RP is a critical consideration. If the RP is located on a remote WAN separated segment, or if there are too many failure points between the WLC and the RP, there will likely be outages for multicast clients and sources. To reduce the number of failures, place the WLCs and RPs close to each other and near the multicast network, with few failure points between.

Two typical multicast domain models are used to deploy RPs:

• Domain model 1: The first model is used for isolated domains that do not extend beyond specific geographic locations. In this model, RPs are placed locally in each domain. Often the nearest Layer 3 switch that is acting as the first-hop router (FHR) and last-hop router (LHR) is also configured as the RP. This limits the failure scope of the domain to that specific Layer 2 or 3 domain.

• Domain model 2: The second model is used when multidomain or interdomain communications are required. In this model, multiple redundant RPs may be used, usually placed by domain. This is similar to the first model but with the addition of hospitalwide resources.

Figure 5-12 shows a comparison of the domain setup for the two models. Domain model 1 uses one controller and the distribution/core switch pair as the RP for each group of rooms. The Layer 3 FHR and RP are configured on the appropriate VLAN interfaces. Domain model 2 expands on Domain model 1 by adding a hospitalwide domain with Anycast RPs and a stack of hospitalwide WLCs in the local hospital data center.

Figure 5-12 Multicast Domain Models

No matter what domain model is used, domains must be secured and segmented properly. Wireless and multicast architects should be well versed in the operation of these networks and should design clinical multicast with resiliency and reliability as key foundational elements. They need to ensure that clinical staff do not experience interruptions to service, which produces a superior patient experience.