This blog follows on from my last post and continues the discussion on how to integrate a single/pair of SD-WAN routers into our existing branch site topology. If you missed that last blog, then you can check it out here. Don’t worry I’ll be right here waiting for you.
Last time out we looked at options for completely replacing our existing WAN routers in our branch. This time we will look at situations where we need to keep these existing devices around. Reasons for this could be that they terminate old WAN transports (T1, xDSL) which are not supported on our vEdge appliances. This is not an issue for our cEdge devices (ASR, ISR4Ks and ISR1K), which can happily facilitate these types of handoffs. Another reason could be that the existing devices are hosting services (WAAS or Voice) that are not currently supported in our XE SD-WAN image. Watch this space as on this requirement may soon also go the way of the dinosaurs as Cisco continues to port over more and more functionality from IOS-XE into XE SD-WAN.
Single cEdge Retain Customer Edge (CE) router to support legacy WAN handoff with Layer 2 LAN
This deployment option allows us to support legacy WAN handoffs. Here we keep the existing WAN router (CE) for WAN termination but disable its LAN side connectivity. All site subnets which were previously hosted on the CE are migrated over to our SD-WAN router. A new /30 subnet (C) is provisioned for use between our CE and SD-WAN router. This cross connect can be either a physical cable between the two devices or we can use a logical sub interface that is switched by our southbound access switch. Subnet C will need to be accessible by remote SD-WAN sites, so they can build their IPSec tunnels across the private WAN transport to cEdge1’s subnet C IP address. To achieve this, we can either include subnet C in our BGP advertisements to the service provider or have our service provider update their static routes for this site to include the new subnet.
Subnets A and B have a /30 IPv4 prefix length that are assigned from the provider. Two default routes (one for each transport) point to the CE router subnet C address and ISP PE address are used to build our tunnels.
This site does not have a local firewall, so internet traffic is either back hauled to a central site or is tunneled from our cEdge to a cloud hosted firewall service such as Cisco Umbrella or Zscaler.
LAN side (VPN 1) routing (Subnet X) is performed by our cEdge, which hosts our client default gateway addresses. In the above diagram we only have a single subnet X at this site, but this can be increased with the creation of sub interfaces for each additional VLAN. These connected routes are redistributed into OMP and advertised up to our vSmart controllers and onto other sites in our environment.
This design is not highly available since a failure to our single cEdge will bring down WAN and internet services for this site. If this a small site then this kind of risk may be acceptable, in which case the customer can choose from a range of service contract levels to have the failed equipment replaced. (Hardware replacement SLA’s range from as little as 2 hours up to Next business day). For most customer sites these kinds of outages cannot be tolerated so we need to amend our design to incorporate high availability. To do this we have two options:
Option 1 which is the recommended design and shown in our next example, greatly simplifies the deployment and removes operational complexity. Option 2 requires us to leak underlay routes into our overlay and vise versa, which if not done with careful route redistribution can at best lead to routing asymmetry and at worst routing loops. This second option also obviously means that SD-WAN services are no longer provided in the event of a failure to our SD-WAN router. Although not recommended I have worked on many deployments that have chosen this path and worked well.
Dual cEdge Retain Customer Edge (CE) router to support legacy WAN handoff with Layer 2 LAN
This deployment option adds high availability to the design above by adding a second cEdge to the site. All site subnets which were previously hosted on the CE are migrated over to our SD-WAN routers. VRRP is configured to run between our cEdges with cEdge1 assuming the master role. As above a new subnet C is introduced to the site but this time it will need to be a /29 prefix to support all three of our devices (CE, cEdge1 & 2). The cross connect will need to be switched by our southbound access switch. Advertisement requirements for Subnet C are the same as above.
Subnet A is still /30 IPv4 prefix assigned from our private WAN provider. Subnet B (Internet Transport Provider) in this case would need to be a /29 or lower to support each cEdge having its own public address. If we only have a /30 to work with then we will need to utilize the TLOC extension feature (see previous blog post). Each cEdge has two default routes (one for each transport) pointing to their respective provider PE addresses are used to build our tunnels.
LAN side (VPN 1) routing (Subnet X) is performed by our two cEdges, which hosts our client default gateway addresses (VRRP virtual IP). In the above diagram we only have a single subnet X at this site, but this can be increased with the creation of sub interfaces for each additional VLAN. These connected routes are redistributed into OMP and advertised up to our vSmart controllers and onto other sites in our environment. To retain traffic symmetry for return traffic we will increase the OMP preference of routes advertised from our master VRRP router.
This design resolves the cEdge single point of failure issue outlined above.
Single cEdge Retain Customer Edge (CE) router to support site services with Layer 2 LAN
Finally, this last design shows how to integrate our SD-WAN router into a site while retaining WAAS and or Voice services on a separate device.
Here we deploy our cEdge alongside our services node (CE router). As in the examples above we deploy a new subnet C which cross connects our two routers in VPN 0 (Transport VPN) and provides our cEdge with access to the private WAN transport. Unlike above we also need a second new subnet (D) to cross connect our routers in VPN 1 (Service VPN). This cross connect could be made with a single cable between the devices with two logical sub interfaces making up our two connections.
Gateway services are kept on the CE router and a VRRP relationship is built between it and our new cEdge across our LAN subnet (X).
Northbound traffic (towards the WAN) from our site clients will flow through the services router as it is it holds the master VRRP role. Traffic can then have services applied before being routed (we learn SD-WAN routes from the IGP peering) across subnet D to our cEdges VPN1 interface, finally our packets are encapsulated, encrypted and forwarded out into our SD-WAN fabric.
Southbound traffic (from the WAN) will arrive at our cEdge before being policy routed across subnet D to our services router. Services will be applied before final delivery to the client in subnet X. The policy route (data policy) is needed to stop our cEdge from delivering the returning packets directly to clients out of its subnet X interface. This ensures packets are inspected and have services applied symmetrically (inbound and outbound).
In the event of a failure to our services router our cEdge will take on the master VRRP role and route our traffic directly across the SD-WAN fabric. In the diagram above our private WAN transport is connected through the services (CE router) so a failure to it will also result in our cEdge losing access to this transport. Public transport tunnels will still be available. This dependency would be removed by terminating the private transport directly on our cEdge.
OK, that seems like as good a place as any to break it off. Next up I’ll continue this discussion and take a look at some options to integrate a firewall into our branch site.
Until then may your overlays and underlays routing never get leaked and your WAN bandwidth be plentiful and lossless.
Written By: Chris Marshall, LookingPoint Senior Solutions Architect - CCIE #29940