ACI Transit Routing

ACI Transit Routing 

BashTG Contributor 2xCCIE/CCDE April, 2021

ACI fabric supports transit routing. This feature enables a border leaf to perform bidirectional redistribution between routing domains. Traffic can pass from one layer 3 domain to another layer 3 domain through ACI (the ACI acting as a transit between the two layer 3 domains). A transit route is defined to import traffic through a Layer 3 outside network (L3out) where it is to be imported. A different transit route is defined to export traffic through another L3out where it is to be exported.

The route-maps for import and export route controls are made up of prefix-list matches. Each prefix-list consists of bridge domain (BD) external subnet prefixes in the VRF and the export prefixes that need to be advertised outside. Route control policies are defined in an l3out and controlled by properties and relations associated with the l3Out. APIC uses the enforce route control property of the l3Out to enforce route control directions. The default is to enforce control on export and allow all on import. The default scope for every route is import. These are the routes and prefixes which form a prefix-based EPG.

The use case for the transit routing is to provide full connectivity between two or multiple isolated L3 domains. It also can act as a backup path between L3 domains. The layer 3 domains such as external pods, NSX-T data center, mainframes, service nodes, or WAN routers can peer with the ACI fabric to use its transit functionality between them. 

Since multiple l3out policies can be deployed on a single node or multiple nodes in the fabric, a variety of protocol combinations are supported. Every protocol combination can be deployed on a single node using multiple l3out policies or multiple nodes using multiple l3out policies. Deployments of more than two protocols in different l3out policies in the fabric are supported. 

Transit Routing between Two L3 domains connected With EBGP

In this section, transit routing between two L3 domains connected using EBGP to the ACI fabric (no direct connectivity between the two domains) will be demonstrated. 

The Setup uses an ACI fabric running 4.2(6h) and CSR 1000V (Cloud Services Router) 15.5(3)S3.

Assumptions – 

• The two L3 domains are Data Center Core and NSX virtual environment.
• EBGP between Core – ACI and NSX – ACI is already configured.
• The IP addressing and the logical setup is as shown below.
• Each L3 domain peer with two border leaf switches.
• L3 domains in the lab utilize only one CSR 1000V, in production environment redundant router should be used for resiliency.
• Contract between EPGs is not part of this demonstration. This solely focuses on control plane (routing).
• Uses BGP community string to control the redistribution of routes. There are other ways like IP prefix-list to identify and control routes for redistribution, community string is only one way. 
• Uses Route map at the L3Out level. It’s also possible to apply the route-map at the neighbor level for BGP peering.

                       Fig 1- IP addressing 

Expected result –

1. Only subnets (10.10.1.0/24 & 10.10.2.0/24) tagged with 64525:10 from the core domain will be advertised to the NSX domain through the ACI fabric.
2. Only subnets (10.20.3.0/24 & 10.20.4.0/24) tagged with 65525:20 from the NSX domain will be advertised to the core domain through the ACI fabric.

Step 1: Verify the EBGP neighborship and BGP routes

ACI 

 The ‘show ip bgp summary vrf BASH_B:BASH’ show established neighborship with both Core and NSX domains. 

L3 Domain – Core 

The ‘show ip bgp summary’ show an established neighborship with the ACI fabric but the ‘show ip route bgp’ shows no route learned from the BGP peering.

L3 Domain – NSX

The ‘show ip bgp summary’ show an established neighborship with the ACI fabric but the ‘show ip route bgp’ shows no route learned from the BGP peering.

Step 2: Verify the EBGP learned subnet has the community as expected on the ACI border leaf switches. 

AS expected, 10.10.1.0/24 & 10.10.2.0/24 from Core, and 10.20.3.0/24 & 10.20.4.0/24 have the community string attached.

Step 3: Configure the transit route profile so that the routes with the right community string will be advertised to the opposite L3 domain. 

Step 3.1: Configure the match rule for the community strings

• Match 64525:10 for routes from L3 domain Core
• Match 65525:20 for routes from L3 domain NSX

This is configured under Tenant > Policies > Protocol > Match Rules

  Fig 3 – Creating a match rule using community string and ip prefix-list (repeat for 65525:20)

Step 3.2: Configure the Route map for import and export route control 

This is configured under Tenant > Networking > L3Outs > [L3Outs name] > Route map for import and export route control

 Fig 4 – Route map for import and export route control (configure for both L3Outs)

Step 4: Apply the route control profile to the L3out external EPG (InstP).

The route control profile is applied to the external EPG to have the policy at the L3Out level. The above created profile is applied on the export direction for both respected L3Outs.

This is configured under Tenant > Networking > L3Outs > [L3Outs name] > External EPGs > [External EPGs name]

 Fig 5 – Apply the Route Control Profile (apply on both L3Outs) 

Step 5: Verify the routes tagged with the right community strings are routed between the two isolated L3 domains through the ACI fabric.

L3 domain – Core

The tagged routes from the NSX domain are seen on the Core domain as expected for proper functioning of the VMs on the NSX environment.

L3 domain – NSX

The tagged routes from the core domain are seen on the NSX domain as expected for proper functioning of the VMs on the NSX environment.

neXt – 

1. TROUBLESHOOTING ACI TRANSIT ROUTING
2. PBR (Policy Based Routing) in ACI
3. Someday was today; Multi-Cloud / Hybrid-Cloud World.