ACI VRF Route Leaking¶
Estimated time to read: 14 minutes
- Originally Written: February, 2024
Info
This post uses the Nexus as Code (NaC) project which makes it very easy to configure ACI fabrics through a YAML file. More details and examples can be found at https://developer.cisco.com/docs/nexus-as-code/#!aci-introduction
Overview¶
This post will provide a few designs when configuring VRF route leaking in ACI
- Leaking between two VRFs in one tenant
- Leaking between two VRFs in one tenant and peering with an external network
- Leaking between two VRFs in two different tenants
Note
The first scenario will show three options to configure route leaking:
- Under the EPG subnets menu
- Under the bridge domain subnets menu
- Under the VRF menu -
the newer, simpler method
The remaining scenarios will configure the route leaking under the VRF only as this is the simplest option.
Leaking between two VRFs in one tenant¶
In this example there is one tenant, conmurph-01, and two VRFs, BDs, EPGs, all called either vrf-provider or vrf-consumer.
Leaking between two VRFs in one tenant - Configured under EPG¶
One contract is provided by the vrf-provider EPG and consumed by the vrf-consumer EPG. Only one subnet on the consumer side is configured and this is under the bridge domain. This subnet is marked as Shared between VRFs. There is no subnet configured under the consumer EPG.
This will automatically leak the 192.168.102.0/24 subnet from the consumer VRF into the provider VRF.
If you need to leak from provider to consumer, a subnet (additional to the provider BD subnet) can be configured on the provider EPG and marked as Shared between VRFs. The BD subnet acts as the default gateway so we mark the provider EPG subnet as No default SVI gateway. This is also a more specific route (192.168.101.10/32) which is leaked into the consumer.
Nexus as Code Configuration¶
Configuration
---
apic:
tenants:
- name: conmurph-01
managed: false
vrfs:
- name: vrf-provider
- name: vrf-consumer
bridge_domains:
- name: 192.168.101.0_24
alias: vrf-provider
vrf: vrf-provider
subnets:
- ip: 192.168.101.254/24
- name: 192.168.102.0_24
alias: vrf-consumer
vrf: vrf-consumer
subnets:
- ip: 192.168.102.254/24
shared: true
application_profiles:
- name: network-segments
managed: false
endpoint_groups:
- name: 192.168.101.0_24
alias: vrf-provider
bridge_domain: 192.168.101.0_24
vmware_vmm_domains:
- name: hx-dev-01-vds-01
contracts:
providers:
- permit-all
subnets:
- ip: 192.168.101.10/32
shared: true
no_default_gateway: true
- name: 192.168.102.0_24
alias: vrf-consumer
bridge_domain: 192.168.102.0_24
vmware_vmm_domains:
- name: hx-dev-01-vds-01
contracts:
consumers:
- permit-all
filters:
- name: permit-all
entries:
- name: src-any-to-any-dst
ethertype: unspecified
contracts:
- name: permit-all
scope: tenant
subjects:
- name: permit-all
filters:
- filter: permit-all
Verify¶
show ip route vrf conmurph-01:vrf-provider output showing 192.168.102.0/24 is leaked into the provider VRF.
show ip route vrf conmurph-01:vrf-consumer output showing 192.168.101.10/32 is leaked into the consumer VRF.
Leaking between two VRFs in one tenant - Configured under BD¶
The next option is to use the same configuration for the leaked subnets but have this under the bridge domains instead of EPG.
Warning
With this option only the VRF providing the contract will import the routes. Therefore both VRFs need to provide and consume the contract in order to install the 192.168.101 and 192.168.102.0 subnets.
Nexus as Code¶
Info
The configuration below has the consumer contract in the 192.168.101.0_24 EPG and the provider contract in the 192.168.102.0_24 EPG commented out. This is to show that the routes are only leaked one way i.e. into the provider. You can uncomment these lines if you need the routes leaked in the opposite direction.
Configuration
---
apic:
tenants:
- name: conmurph-01
managed: false
vrfs:
- name: vrf-provider
- name: vrf-consumer
bridge_domains:
- name: 192.168.101.0_24
alias: vrf-provider
vrf: vrf-provider
subnets:
- ip: 192.168.101.254/24
shared: true
- name: 192.168.102.0_24
alias: vrf-consumer
vrf: vrf-consumer
subnets:
- ip: 192.168.102.254/24
shared: true
application_profiles:
- name: network-segments
managed: false
endpoint_groups:
- name: 192.168.101.0_24
alias: vrf-provider
bridge_domain: 192.168.101.0_24
vmware_vmm_domains:
- name: hx-dev-01-vds-01
contracts:
providers:
- permit-all
# consumers:
# - permit-all
- name: 192.168.102.0_24
alias: vrf-consumer
bridge_domain: 192.168.102.0_24
vmware_vmm_domains:
- name: hx-dev-01-vds-01
contracts:
consumers:
- permit-all
# providers:
# - permit-all
Verify¶
show ip route vrf conmurph-01:vrf-provider output showing 192.168.102.0/24 is leaked into the provider VRF.
show ip route vrf conmurph-01:vrf-consumer output showing 192.168.101.0/24 is NOT leaked into the consumer VRF.
In order to leak 101 into vrf-consumer the contract must be also provided in the 192.168.102.0_24 EPG and consumed in the 192.168.101.0_24 EPG. You can uncomment the consumers and providers lines in the Nexus as Code example above.
Leaking between two VRFs in one tenant - Configured under VRF¶
The final option is to move the route leaking configuration into the VRF under the Inter-VRF Leaked Routes for ESG section. This is probably the easiet way to configure VRF route leaking and it also makes it easy to leak external route prefixes.
There are two sub-sections under Inter-VRF Leaked Routes for ESG:
EPG/BD subnets: subnets within the ACI fabric e.g. the192.168.101.0which has been previously shownExternal Prefixes: subnets external to the fabric e.g. learned through an external OSPF peering. This will be shown in a later example.
Note
Don't forget to configure the scope for the contract as Tenant rather than VRF (or context in NaC)
Nexus as Code¶
Configuration
---
apic:
tenants:
- name: conmurph-01
managed: false
vrfs:
- name: vrf-provider
leaked_internal_prefixes:
- prefix: 192.168.101.0/24
destinations:
- tenant: conmurph-01
vrf: vrf-consumer
- name: vrf-consumer
leaked_internal_prefixes:
- prefix: 192.168.102.0/24
destinations:
- tenant: conmurph-01
vrf: vrf-provider
bridge_domains:
- name: 192.168.101.0_24
alias: vrf-provider
vrf: vrf-provider
subnets:
- ip: 192.168.101.254/24
- name: 192.168.102.0_24
alias: vrf-consumer
vrf: vrf-consumer
subnets:
- ip: 192.168.102.254/24
application_profiles:
- name: network-segments
managed: false
endpoint_groups:
- name: 192.168.101.0_24
alias: vrf-provider
bridge_domain: 192.168.101.0_24
vmware_vmm_domains:
- name: hx-dev-01-vds-01
contracts:
providers:
- permit-icmp
consumers:
- permit-icmp
- name: 192.168.102.0_24
alias: vrf-consumer
bridge_domain: 192.168.102.0_24
vmware_vmm_domains:
- name: hx-dev-01-vds-01
contracts:
consumers:
- permit-icmp
providers:
- permit-icmp
filters:
- name: permit-icmp
entries:
- name: icmp
ethertype: ip
protocol: icmp
contracts:
- name: permit-icmp
scope: tenant
subjects:
- name: permit-icmp
filters:
- filter: permit-icmp
Verify¶
show ip route vrf conmurph-01:vrf-provider output showing 192.168.102.0/24 is leaked into the provider VRF.
show ip route vrf conmurph-01:vrf-consumer output showing 192.168.101.0/24 is leaked into the consumer VRF.
Leaking between two VRFs in one tenant and peering with an external network¶
The next scenario will use the same tenant, conmurph-01, and two VRFs, vrf-provider and vrf-consumer, however the provider VRF will also have an OSPF peering with an external router. The subnet, 192.168.102.0/24, in the consumer VRF will be leaked into the provider VRF and advertised to the external router.
Additionally, this example will use the External Prefixes option under the Inter-VRF Leaked Routes for ESG section to leak external prefixes received from the router into specific VRFs.
Nexus as Code¶
Configuration
---
apic:
tenants:
- name: conmurph-01
managed: false
vrfs:
- name: vrf-provider
leaked_internal_prefixes:
- prefix: 192.168.101.0/24
destinations:
- tenant: conmurph-01
vrf: vrf-consumer
leaked_external_prefixes:
- prefix: 172.16.99.0/24
destinations:
- {tenant: conmurph-01, vrf: vrf-consumer}
- name: vrf-consumer
leaked_internal_prefixes:
- prefix: 192.168.102.0/24
destinations:
- tenant: conmurph-01
vrf: vrf-provider
public: true
bridge_domains:
- name: 192.168.101.0_24
alias: vrf-provider
vrf: vrf-provider
subnets:
- ip: 192.168.101.254/24
- name: 192.168.102.0_24
alias: vrf-consumer
vrf: vrf-consumer
subnets:
- ip: 192.168.102.254/24
application_profiles:
- name: network-segments
managed: false
endpoint_groups:
- name: 192.168.101.0_24
alias: vrf-provider
bridge_domain: 192.168.101.0_24
vmware_vmm_domains:
- name: hx-dev-01-vds-01
contracts:
providers:
- permit-icmp
consumers:
- permit-icmp
- name: 192.168.102.0_24
alias: vrf-consumer
bridge_domain: 192.168.102.0_24
vmware_vmm_domains:
- name: hx-dev-01-vds-01
contracts:
consumers:
- permit-icmp
providers:
- permit-icmp
filters:
- name: permit-icmp
entries:
- name: icmp
ethertype: ip
protocol: icmp
contracts:
- name: permit-icmp
scope: tenant
subjects:
- name: permit-icmp
filters:
- filter: permit-icmp
l3outs:
- name: l3out-to-external-network
vrf: vrf-provider
domain: conmurph-01.vrf-01
ospf:
area: 0
area_type: regular
node_profiles:
- name: border-leafs
nodes:
- node_id: 101
router_id: 101.2.1.1
router_id_as_loopback: true
- node_id: 102
router_id: 102.2.1.1
router_id_as_loopback: true
interface_profiles:
- name: l3out-to-external-network-int-profiles
ospf:
policy: l3out-to-external-network-ospf-pol
interfaces:
- node_id: 101
channel: hx-dev-01-fi-a
vlan: 30
svi: true
ip: 172.16.100.1/24
- node_id: 102
channel: hx-dev-01-fi-b
vlan: 30
svi: true
ip: 172.16.100.2/24
external_endpoint_groups:
- name: all-ext-subnets
contracts:
providers:
- permit-icmp
subnets:
- prefix: 172.16.99.0/24
shared_security: true
shared_route_control: true
policies:
ospf_interface_policies:
- name: l3out-to-external-network-ospf-pol
network_type: bcast
Verify¶
show ip route vrf conmurph-01:vrf-consumer output showing 172.16.99.0/24 is leaked into the consumer VRF. This is the external prefix.
show ip route vrf conmurph-01:vrf-provider output showing 192.168.102.0/24 is leaked into the provider VRF and 172.16.99.0/24 is learned from the external router.
show ip route output from the external router showing 192.168.102.0/24 is learned from the ACI fabric via OSPF. The router IDs from the ACI leaf switches, 101.0.0.0/32 and 102.0.0.0/32, are also learned through OSPF.
Leaking between two VRFs in two separate tenants¶
This final scenario will configure route leaking between two separate tenants (no L3out in this scenario for simplicity).
It's very similar to previous scenarios however it's important to note that the contract scope should be global, not tenant or context/VRF.
It's always best practice to have accurate names and descriptions for different objects. Therefore in this example the contracts have been named permit-icmp-to-tn-conmurph-01 and permit-icmp-to-tn-conmurph-02, rather than just permit-icmp.
As the name suggests, permit-icmp-to-tn-conmurph-01 allows traffic to the conmurph-01 tenant. It is provided by tenant conmurph-01 and consumed by tenant conmurph-02.
permit-icmp-to-tn-conmurph-02 is the opposite and allows traffic to conmurph-02. It is provided by tenant conmurph-02 and consumed by tenant conmurph-01.
Nexus as Code¶
Configuration
---
apic:
tenants:
- name: conmurph-01
managed: false
vrfs:
- name: vrf-provider
leaked_internal_prefixes:
- prefix: 192.168.101.0/24
destinations:
- tenant: conmurph-02
vrf: vrf-consumer
bridge_domains:
- name: 192.168.101.0_24
alias: vrf-provider
vrf: vrf-provider
subnets:
- ip: 192.168.101.254/24
application_profiles:
- name: network-segments
managed: false
endpoint_groups:
- name: 192.168.101.0_24
alias: vrf-provider
bridge_domain: 192.168.101.0_24
vmware_vmm_domains:
- name: hx-dev-01-vds-01
contracts:
providers:
- permit-icmp-to-tn-conmurph-01
imported_consumers:
- permit-icmp-to-tn-conmurph-02
imported_contracts:
- name: permit-icmp-to-tn-conmurph-02
tenant: conmurph-02
contract: permit-icmp-to-tn-conmurph-02
contracts:
- name: permit-icmp-to-tn-conmurph-01
scope: global
subjects:
- name: permit-icmp
filters:
- filter: permit-icmp
filters:
- name: permit-icmp
entries:
- name: icmp
ethertype: ip
protocol: icmp
- name: conmurph-02
managed: false
vrfs:
- name: vrf-consumer
leaked_internal_prefixes:
- prefix: 192.168.102.0/24
destinations:
- tenant: conmurph-01
vrf: vrf-provider
bridge_domains:
- name: 192.168.102.0_24
alias: vrf-consumer
vrf: vrf-consumer
subnets:
- ip: 192.168.102.254/24
application_profiles:
- name: network-segments
managed: false
endpoint_groups:
- name: 192.168.102.0_24
alias: vrf-consumer
bridge_domain: 192.168.102.0_24
vmware_vmm_domains:
- name: hx-dev-01-vds-01
contracts:
providers:
- permit-icmp-to-tn-conmurph-02
imported_consumers:
- permit-icmp-to-tn-conmurph-01
imported_contracts:
- name: permit-icmp-to-tn-conmurph-01
tenant: conmurph-01
contract: permit-icmp-to-tn-conmurph-01
contracts:
- name: permit-icmp-to-tn-conmurph-02
scope: global
subjects:
- name: permit-icmp
filters:
- filter: permit-icmp
filters:
- name: permit-icmp
entries:
- name: icmp
ethertype: ip
protocol: icmp
Verify¶
APIC showing the consumed contract interfaces for the EPGs in each tenant
show ip route vrf conmurph-01:vrf-provider output showing 192.168.102.0/24 is leaked into the provider VRF in tenant conmurph-01.
show ip route vrf conmurph-02:vrf-consumer output showing 192.168.101.0/24 is leaked into the consumer VRF in tenant conmurph-02.
