SD-WAN and ACI
Cisco’s Application Centric Infrastructure (ACI) allows the enterprise to introduce macro- and microsegmentation with automation and assurance within the data center. ACI uses a structured hierarchy including tenants, contexts, and endpoint groups (EPGs) to create macrosegmentation and microsegmentation. The EPG is similar to the SGT in the SDA and SD-WAN environments. It allows for policy enforcement based on logical group membership.
When SD-WAN and ACI are deployed, the individual macrosegmentation that is created within the DC ACI environment is extended to the remote site location. For instance, the enterprise may provide managed services to their end customers, internal or external, and want to ensure segmentation from the DC to the site. Having an SD-WAN service VPN for each ACI tenant will maintain that segmentation. While the current APIC and Catalyst SD-WAN Manager allow for integration via REST APIs, that integration only supports a dynamic application-aware routing policy signaling from ACI to SD-WAN. Therefore, all of the routing interconnectivity must be performed individually in both environments. For this reason, standardization again becomes important.
Imagine an enterprise environment without SD-WAN or ACI consisting of two data centers and multiple remote locations. This enterprise currently has all of its clients in a single global routing table without any segmentation. Now, they would like to migrate to full segmentation with both ACI and SD-WAN deployed. It will take some time to build the ACI environment and migrate the relevant services into each tenant. It will also take time to migrate each of the individual sites to SD-WAN. How is this performed without issues?
First, we must consider all of the possible traffic patterns. There is traffic from the nonmigrated data center environment to the nonmigrated remote locations through the service provider environment using the current CE equipment. This traffic will exist throughout all the migrations until both the SD-WAN and the ACI migrations are fully completed, although the amount of traffic will decrease with each migration window. As the migrations proceed, there will be traffic from the ACI environment through the SD-WAN environment to the remote locations. At first, this traffic will not exist at all and will increase as migrations occur. There will also be traffic between the nonmigrated data center environment and the new SD-WAN remote locations, as well as nonmigrated remote sites with the newly migrated ACI environment. Additionally, traffic will exist between migrated and nonmigrated sites, and an existing data center with the ACI environment.
All of these traffic patterns will exist in some amount from the beginning of the project until the end. Therefore, from a routing and switching perspective, there are four domains: the SD-WAN environment, the ACI environment, the existing data center, and the existing WAN environment. It is recommended to create an additional routing and switching layer within the data center that performs aggregation and routing between the domains. In Figure 3-6 notice a new aggregation layer has been inserted between the legacy WAN environment, the new SD-WAN devices, the legacy data center services infrastructure, and the new ACI environment. This new layer will allow the routing to drive traffic to the correct blocks based on the destination location—whether already migrated to the new environment or not.
)
Figure 3.6 SD-WAN–ACI Topology
When the environment is designed and implemented, the use of standardized VLANs will facilitate an easier migration per client. After an aggregation layer is created within the data center to facilitate interactions between the environments, the SD-WAN headend devices may be stood up appropriately. When this has happened, the ACI and SD-WAN environments are migrated at their own individual rates. This approach allows the WAN team to focus on just the remote location migrations while the data center team is able to focus on client services.
Consider the migration of Client A. At first, the services for the client exist in the existing data center environment, and the remote locations that service this client all use the global routing table with the service provider. When the aggregation layer and the SD-WAN headends are in place, the migration of the client is transparent to the client, with the exception of the required routing updates during maintenance windows. Perhaps the ACI environment is not built out while the SD-WAN environment is ready for production. The service VPN for this client is provisioned on the headends—for example, VPN 1201. As part of the provisioning, BGP peering between the headends and the aggregation layer on VLAN 1201 is created. Provisioning the new service VPN in the headends will have no effect on the traffic flows because there will be no routing advertisements at this point coming from the headends. Whenever a remote location is moved to VPN 1201, the headends will begin to advertise the remote site via BGP while the service provider will lose the routing advertisement from the remote location. This is the case with Client A.
At any specific remote location, there may be a different collection of service VPNs, that is, clients, from other locations. Because it is conceivable that the headend environment may not be provisioned for all clients or the enterprise wants to move to SD-WAN quickly, we may want to create a single-service VPN that may be used similarly to the existing function of the global routing table. That is, migration to SD-WAN is performed, but segmentation is not fully introduced. Perhaps the local network has not been configured for segmentation via VRF Lite or some other manner; this common service VPN allows for the entire site to move to SD-WAN while not affecting the local environment. When the local network is ready to migrate Client A to its own segmented environment, the Client A service VPN is provisioned on the SD-WAN Edges at the site. With the ensuing routing updates, the Client A traffic for this remote site now uses the SD-WAN environment and is advertised from the headends to the data center aggregation layer to all other environments.
This architectural design also works for the migration of services for Client A. When the ACI environment is ready for production, all of the logical ACI components are added into the ACI environment to support Client A. The required services for Client A are moved into the ACI environment, and the ACI L3Outs, or border leafs, advertise the services to the data center aggregation layer.
Therefore, while ACI and SD-WAN are integrated together, the migration of the services for a particular tenant in ACI and the migration of the tenant’s remote locations in SD-WAN may proceed at their own individual pace. The aggregation layer handles the routing between the various environments. As shown in Figure 3-7, the aggregation layer allows a remote site that has been migrated to SD-WAN already to interact with a remote site that has not. The reason is that the SD-WAN headends are advertising to the aggregation layer the SD-WAN remote site prefixes to the legacy WAN environment, and vice versa. With an L3 MPLS offering from the service provider, it is conceivable that these sites are able to send traffic directly to each other across the service provider. However, because the SD-WAN traffic is encrypted while the legacy traffic across the service provider is not, during this hybrid state of migrated and nonmigrated sites, the headends and the aggregation layer must be utilized to interconnect the domains.
)
Figure 3.7 SD-WAN–ACI Traffic Flows During Migration
Catalyst SD-WAN Manager and APIC Integration
Integration of the Catalyst SD-WAN Manager with the ACI APIC is performed on the APIC itself. A static user on the Catalyst SD-WAN Manager is required for the APIC to communicate with the Catalyst SD-WAN Manager. For security, auditing, and best-practice purposes, it is recommended to use a service account for the integration, as well as authentication via an external identity store. Doing so will facilitate proper user auditing, as well as the ability to manage the account via the appropriate operations processes.
Example 3-2 illustrates the configuration process required to integrate the APIC and the Catalyst SD-WAN Manager together.
Example 3-2 Catalyst SD-WAN Manager and APIC Integration Process
apic1#conf t apic1(config)#integrations-group MyExtDevGroupClassic apic1(config-integrations-group)#integrations-mgr External_Device Cisco/vManage apic1(config-integrations-mgr)#device-address 172.31.209.198 apic1(config-integrations-mgr)#user admin Password: Retype password: apic1(config-integrations-mgr)#
ACI and SD-WAN Segmentation
While ACI and SD-WAN both support the concepts of macro- and microsegmentation, microsegmentation propagation does not occur without additional configuration. Also, the macrosegmentation propagation must be handled in a systematic manner.
For macrosegmentation propagation, this is where the ACI Tenant to VLAN to Service VPN mapping is important. The VLAN used to interconnect the ACI L3Out, or border leaf, to the SD-WAN Edge is crucial to maintain the macrosegmentation.
Microsegmentation propagation of the ACI EPG to SD-WAN SGT values is more difficult and limited. The APIC must be integrated with ISE using pxGrid in the same manner as used for the ACI-SDA integration. This will allow ACI to advertise or receive EPGs to and from ISE; however, as with the ACI-SDA integration, this is limited to a single context. For the SD-WAN side, the headend SD-WAN Edges may use SXP with ISE in any of the service VPNs. The SGT information will be propagated along the data path of SD-WAN to the remote SD-WAN Edge, where policy enforcement or further propagation may occur.
ACI and SD-WAN Best Practices
For ACI and SD-WAN integration together, the two most important aspects are standardization of the handoff between them and the support for migrated and nonmigrated traffic flows. For the former, it is recommended to use a planned VLAN to Service VPN numbering. This approach prevents confusion later when some clients have migrated to ACI while others have not, or when some sites or clients have been migrated to SD-WAN while others have not. For the latter, the use of the aggregation layer with BGP allows the enterprise to connect the legacy and new environments together while using BGP to affect policy routing, if necessary.