If two sites of a MPLS L3VPN are connected to different Autonomous Systems (e.g., because the sites are connected to different SPs). The PE routers attached to that VPN will then not be able to maintain IBGP connections with each other, or with a common route reflector. Rather, there needs to be some way to use EBGP to distribute VPN-IPv4 addresses.
There are a number of different ways of handling this case, let’s discuss first option which is called option A or back-to-back VRF exchange.
In this procedure, a PE router in one AS attaches directly to a PE router in another. The two PE routers will be attached by multiple sub-interfaces, at least one for each of the VPNs whose routes need to be passed from AS to AS. Each PE will treat the other as if it were a CE router. That is, the PEs associate each such sub-interface with a VRF, and use EBGP (or any PE/CE IGP) to distribute unlabeled IPv4 addresses to each other.
We will use following network topology to discuss this feature.
PE-CE Configuration with ISP-1
R7 (CE Router) - Customer A Site 1 ## EIGRP is the PE-CE routing protocol used for this customer ## interface FastEthernet0/0 ip address 192.168.47.7 255.255.255.0 ! interface FastEthernet0/1 ip address 192.168.7.7 255.255.255.0 ! router eigrp 100 network 192.168.0.0 0.0.255.255 R8 (CE Router) - Customer B Site 1 ## OSPF is the PE-CE routing protocol used for this customer ## interface FastEthernet0/0 ip address 172.16.48.8 255.255.255.0 ip ospf network point-to-point ip ospf 1 area 0 ! interface FastEthernet0/1 ip address 172.16.8.8 255.255.255.0 ip ospf network point-to-point ip ospf 1 area 0 ! router ospf 1 router-id 172.16.8.8 R4 (PE Router) - ISP 1 vrf definition Cust-A rd 100:1 ! address-family ipv4 route-target export 100:1 route-target import 100:1 exit-address-family ! vrf definition Cust-B rd 100:2 ! address-family ipv4 route-target export 100:2 route-target import 100:2 exit-address-family ! interface Loopback0 ip address 1.1.0.4 255.255.255.255 ip ospf 1 area 0 ! interface FastEthernet0/1 ip address 1.1.34.4 255.255.255.0 ip ospf network point-to-point ip ospf 1 area 0 mpls ip ! interface FastEthernet1/0 vrf forwarding Cust-A ip address 192.168.47.4 255.255.255.0 ! interface FastEthernet1/1 vrf forwarding Cust-B ip address 172.16.48.4 255.255.255.0 ip ospf network point-to-point ip ospf 100 area 0 ! router eigrp 1 ! address-family ipv4 vrf Cust-A autonomous-system 100 default-metric 10000 100 255 1 1500 redistribute bgp 100 network 192.168.47.0 exit-address-family ! router ospf 100 vrf Cust-B router-id 172.16.48.4 redistribute bgp 100 subnets ! router ospf 1 router-id 1.1.0.4 ! router bgp 100 bgp router-id 1.1.0.4 bgp log-neighbor-changes no bgp default ipv4-unicast neighbor 1.1.0.1 remote-as 100 neighbor 1.1.0.1 update-source Loopback0 ! address-family ipv4 exit-address-family ! address-family vpnv4 neighbor 1.1.0.1 activate neighbor 1.1.0.1 send-community extended exit-address-family ! address-family ipv4 vrf Cust-A redistribute connected redistribute eigrp 100 exit-address-family ! address-family ipv4 vrf Cust-B redistribute connected redistribute ospf 100 match internal external 1 external 2 exit-address-family ! R3 (P Router) - ISP 1 interface Loopback0 ip address 1.1.0.3 255.255.255.255 ip ospf 1 area 0 ! interface FastEthernet0/0 ip address 1.1.13.3 255.255.255.0 ip ospf network point-to-point ip ospf 1 area 0 mpls ip ! interface FastEthernet0/1 ip address 1.1.34.3 255.255.255.0 ip ospf network point-to-point ip ospf 1 area 0 mpls ip ! router ospf 1 router-id 1.1.0.3 R1 (PE Router) - ISP 1 - ASBR connected to ISP 2 vrf definition Cust-A rd 100:1 ! address-family ipv4 route-target export 100:1 route-target import 100:1 exit-address-family ! vrf definition Cust-B rd 100:2 ! address-family ipv4 route-target export 100:2 route-target import 100:2 exit-address-family ! interface Loopback0 ip address 1.1.0.1 255.255.255.255 ip ospf 1 area 0 ! interface FastEthernet0/0 ip address 1.1.13.1 255.255.255.0 ip ospf network point-to-point ip ospf 1 area 0 speed 100 duplex full mpls ip ! interface FastEthernet0/1 no ip address ! interface FastEthernet0/1.1 encapsulation dot1Q 12 vrf forwarding Cust-A ip address 12.12.12.1 255.255.255.0 ! interface FastEthernet0/1.2 encapsulation dot1Q 21 vrf forwarding Cust-B ip address 21.21.21.1 255.255.255.0 ! router ospf 1 router-id 1.1.0.1 ! router bgp 100 bgp router-id 1.1.0.1 bgp log-neighbor-changes no bgp default ipv4-unicast neighbor 1.1.0.4 remote-as 100 neighbor 1.1.0.4 update-source Loopback0 ! address-family ipv4 exit-address-family ! address-family vpnv4 neighbor 1.1.0.4 activate neighbor 1.1.0.4 send-community extended exit-address-family ! address-family ipv4 vrf Cust-A neighbor 12.12.12.2 remote-as 200 neighbor 12.12.12.2 activate exit-address-family ! address-family ipv4 vrf Cust-B neighbor 21.21.21.2 remote-as 200 neighbor 21.21.21.2 activate exit-address-family !
PE-CE Configuration with ISP-2
Both ASBR routers in ISP1 and ISP2 see each other as CE router and treat itself as PE router. RD (Route Distinguisher) value for each customer does not require to match in both ISP networks. Here, Both ISPs are using different RD (Route Distinguisher).
We have used BGP as PE-CE routing protocol here. Any IGP can be used as well.
R2 (PE Router) - ISP 2 - ASBR connected to ISP 1 vrf definition Cust-A rd 200:1 ! address-family ipv4 route-target export 200:1 route-target import 200:1 exit-address-family ! vrf definition Cust-B rd 200:2 ! address-family ipv4 route-target export 200:2 route-target import 200:2 exit-address-family ! interface Loopback0 ip address 2.2.0.2 255.255.255.255 ip ospf 1 area 0 ! interface FastEthernet0/0 ip address 2.2.25.2 255.255.255.0 ip ospf network point-to-point ip ospf 1 area 0 mpls ip ! interface FastEthernet0/1 no ip address ! interface FastEthernet0/1.1 encapsulation dot1Q 12 vrf forwarding Cust-A ip address 12.12.12.2 255.255.255.0 ! interface FastEthernet0/1.2 encapsulation dot1Q 21 vrf forwarding Cust-B ip address 21.21.21.2 255.255.255.0 ! router ospf 1 router-id 2.2.0.2 ! router bgp 200 bgp router-id 2.2.0.2 bgp log-neighbor-changes no bgp default ipv4-unicast neighbor 2.2.0.6 remote-as 200 neighbor 2.2.0.6 update-source Loopback0 ! address-family ipv4 exit-address-family ! address-family vpnv4 neighbor 2.2.0.6 activate neighbor 2.2.0.6 send-community extended exit-address-family ! address-family ipv4 vrf Cust-A neighbor 12.12.12.1 remote-as 100 neighbor 12.12.12.1 activate exit-address-family ! address-family ipv4 vrf Cust-B neighbor 21.21.21.1 remote-as 100 neighbor 21.21.21.1 activate exit-address-family R5 (P Router) - ISP 2 interface Loopback0 ip address 2.2.0.5 255.255.255.255 ip ospf 1 area 0 ! interface FastEthernet0/0 ip address 2.2.25.5 255.255.255.0 ip ospf network point-to-point ip ospf 1 area 0 mpls ip ! interface FastEthernet0/1 ip address 2.2.56.5 255.255.255.0 ip ospf network point-to-point ip ospf 1 area 0 mpls ip ! router ospf 1 router-id 2.2.0.5 R6 (PE Router) - ISP 2 vrf definition Cust-A rd 200:1 ! address-family ipv4 route-target export 200:1 route-target import 200:1 exit-address-family ! vrf definition Cust-B rd 200:2 ! address-family ipv4 route-target export 200:2 route-target import 200:2 exit-address-family ! interface Loopback0 ip address 2.2.0.6 255.255.255.255 ip ospf 1 area 0 ! interface FastEthernet0/0 no ip address ! interface FastEthernet0/1 ip address 2.2.56.6 255.255.255.0 ip ospf network point-to-point ip ospf 1 area 0 mpls ip ! interface FastEthernet1/0 vrf forwarding Cust-A ip address 192.168.69.6 255.255.255.0 ! interface FastEthernet1/1 vrf forwarding Cust-B ip address 172.16.106.6 255.255.255.0 ip ospf network point-to-point ip ospf 200 area 0 ! ! router eigrp 1 ! address-family ipv4 vrf Cust-A autonomous-system 100 default-metric 10000 100 255 1 1500 redistribute bgp 200 network 192.168.69.0 exit-address-family ! router ospf 200 vrf Cust-B router-id 172.16.106.6 redistribute bgp 200 subnets ! router ospf 1 router-id 2.2.0.6 ! router bgp 200 bgp router-id 2.2.0.6 bgp log-neighbor-changes no bgp default ipv4-unicast neighbor 2.2.0.2 remote-as 200 neighbor 2.2.0.2 update-source Loopback0 ! address-family ipv4 exit-address-family ! address-family vpnv4 neighbor 2.2.0.2 activate neighbor 2.2.0.2 send-community extended exit-address-family ! address-family ipv4 vrf Cust-A redistribute connected redistribute eigrp 100 exit-address-family ! address-family ipv4 vrf Cust-B redistribute connected redistribute ospf 200 match internal external 1 external 2 exit-address-family R9 (CE Router) - Customer A Site 2 ## EIGRP is the PE-CE routing protocol used for this customer ## interface FastEthernet0/0 ip address 192.168.69.9 255.255.255.0 ! interface FastEthernet0/1 ip address 192.168.9.9 255.255.255.0 ! router eigrp 100 network 192.168.0.0 0.0.255.255 R10 (CE Router) - Customer B Site 2 ## OSPF is the PE-CE routing protocol used for this customer ## interface FastEthernet0/0 ip address 172.16.106.10 255.255.255.0 ip ospf network point-to-point ip ospf 1 area 0 ! interface FastEthernet0/1 ip address 172.16.10.10 255.255.255.0 ip ospf network point-to-point ip ospf 1 area 0 ! router ospf 1 router-id 172.16.10.10
Verification & Testing
R7#sh ip route | beg Gateway Gateway of last resort is not set 192.168.7.0/24 is variably subnetted, 2 subnets, 2 masks C 192.168.7.0/24 is directly connected, FastEthernet0/1 L 192.168.7.7/32 is directly connected, FastEthernet0/1 D EX 192.168.9.0/24 [170/284160] via 192.168.47.4, 01:04:34, FastEthernet0/0 192.168.47.0/24 is variably subnetted, 2 subnets, 2 masks C 192.168.47.0/24 is directly connected, FastEthernet0/0 L 192.168.47.7/32 is directly connected, FastEthernet0/0 D EX 192.168.69.0/24 [170/284160] via 192.168.47.4, 01:04:34, FastEthernet0/0 Ping Customer-A Site 2 network R7#ping 192.168.9.9 source 192.168.7.7 Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 192.168.9.9, timeout is 2 seconds: Packet sent with a source address of 192.168.7.7 !!!!! Success rate is 100 percent (5/5), round-trip min/avg/max = 152/160/172 ms Traceroute Customer-A Site 2 network R7#traceroute 192.168.9.9 source 192.168.7.7 Type escape sequence to abort. Tracing the route to 192.168.9.9 VRF info: (vrf in name/id, vrf out name/id) 1 192.168.47.4 28 msec 36 msec 36 msec 2 1.1.34.3 [MPLS: Labels 302/104 Exp 0] 88 msec 100 msec 88 msec 3 12.12.12.1 [MPLS: Label 104 Exp 0] 64 msec 92 msec 60 msec 4 12.12.12.2 116 msec 120 msec 92 msec 5 2.2.25.5 [MPLS: Labels 501/604 Exp 0] 176 msec 168 msec 152 msec 6 192.168.69.6 [MPLS: Label 604 Exp 0] 128 msec 140 msec 140 msec 7 192.168.69.9 176 msec * 156 msec We can see from the above output there are two different LSP (Label Switch Path) are being used to provide end-to-end reachability. LSP-1: From R4 to R1 using VPN Label 104 R4#sh bgp vpnv4 unicast vrf Cust-A 192.168.9.0 BGP routing table entry for 100:1:192.168.9.0/24, version 10 Paths: (1 available, best #1, table Cust-A) Not advertised to any peer Refresh Epoch 1 200 1.1.0.1 (metric 3) from 1.1.0.1 (1.1.0.1) Origin incomplete, metric 0, localpref 100, valid, internal, best Extended Community: RT:100:1 mpls labels in/out nolabel/104 rx pathid: 0, tx pathid: 0x0 The traffic between R1 & R2 is normal Unlabeled IPv4 traffic R1#sh bgp vpnv4 unicast vrf Cust-A 192.168.9.0 BGP routing table entry for 100:1:192.168.9.0/24, version 10 Paths: (1 available, best #1, table Cust-A) Advertised to update-groups: 3 Refresh Epoch 1 200 12.12.12.2 from 12.12.12.2 (2.2.0.2) Origin incomplete, localpref 100, valid, external, best Extended Community: RT:100:1 mpls labels in/out 104/nolabel rx pathid: 0, tx pathid: 0x0 LSP-2: From R2 to R6 using VPN label 604 R2#sh bgp vpnv4 unicast vrf Cust-A 192.168.9.0 BGP routing table entry for 200:1:192.168.9.0/24, version 10 Paths: (1 available, best #1, table Cust-A) Advertised to update-groups: 1 Refresh Epoch 1 Local 2.2.0.6 (metric 3) from 2.2.0.6 (2.2.0.6) Origin incomplete, metric 30720, localpref 100, valid, internal, best Extended Community: RT:200:1 Cost:pre-bestpath:128:30720 0x8800:32768:0 0x8801:100:5120 0x8802:65281:25600 0x8803:65281:1500 0x8806:0:3232253193 mpls labels in/out nolabel/604 rx pathid: 0, tx pathid: 0x0 Finally unlabelled traffic from ISP 2 PE Router (R6) to CE Router (R9) R6#sh bgp vpnv4 unicast vrf Cust-A 192.168.9.0 BGP routing table entry for 200:1:192.168.9.0/24, version 3 Paths: (1 available, best #1, table Cust-A) Advertised to update-groups: 1 Refresh Epoch 1 Local 192.168.69.9 from 0.0.0.0 (2.2.0.6) Origin incomplete, metric 30720, localpref 100, weight 32768, valid, sourced, best Extended Community: RT:200:1 Cost:pre-bestpath:128:30720 0x8800:32768:0 0x8801:100:5120 0x8802:65281:25600 0x8803:65281:1500 0x8806:0:3232253193 mpls labels in/out 604/nolabel rx pathid: 0, tx pathid: 0x0
Similarly, end-to-end reachability can be verified for Customer B sites. R8#sh ip route | beg Gateway Gateway of last resort is not set 172.16.0.0/16 is variably subnetted, 6 subnets, 2 masks C 172.16.8.0/24 is directly connected, FastEthernet0/1 L 172.16.8.8/32 is directly connected, FastEthernet0/1 O E2 172.16.10.0/24 [110/1] via 172.16.48.4, 01:33:51, FastEthernet0/0 C 172.16.48.0/24 is directly connected, FastEthernet0/0 L 172.16.48.8/32 is directly connected, FastEthernet0/0 O E2 172.16.106.0/24 [110/1] via 172.16.48.4, 01:33:51, FastEthernet0/0 R8#ping 172.16.10.10 source 172.16.8.8 Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 172.16.10.10, timeout is 2 seconds: Packet sent with a source address of 172.16.8.8 !!!!! Success rate is 100 percent (5/5), round-trip min/avg/max = 220/236/248 ms R8#trace 172.16.10.10 source 172.16.8.8 Type escape sequence to abort. Tracing the route to 172.16.10.10 VRF info: (vrf in name/id, vrf out name/id) 1 172.16.48.4 32 msec 52 msec 64 msec 2 1.1.34.3 [MPLS: Labels 302/106 Exp 0] 140 msec 140 msec 136 msec 3 21.21.21.1 [MPLS: Label 106 Exp 0] 84 msec 104 msec 116 msec 4 21.21.21.2 136 msec 152 msec 132 msec 5 2.2.25.5 [MPLS: Labels 501/606 Exp 0] 248 msec 224 msec 232 msec 6 172.16.106.6 [MPLS: Label 606 Exp 0] 208 msec 196 msec 192 msec 7 172.16.106.10 236 msec * 240 msec R4#sh bgp vpnv4 unicast vrf Cust-B 172.16.10.0 BGP routing table entry for 100:2:172.16.10.0/24, version 12 Paths: (1 available, best #1, table Cust-B) Not advertised to any peer Refresh Epoch 1 200 1.1.0.1 (metric 3) from 1.1.0.1 (1.1.0.1) Origin incomplete, metric 0, localpref 100, valid, internal, best Extended Community: RT:100:2 mpls labels in/out nolabel/106 rx pathid: 0, tx pathid: 0x0 R1#sh bgp vpnv4 unicast vrf Cust-B 172.16.10.0 BGP routing table entry for 100:2:172.16.10.0/24, version 12 Paths: (1 available, best #1, table Cust-B) Advertised to update-groups: 3 Refresh Epoch 1 200 21.21.21.2 from 21.21.21.2 (2.2.0.2) Origin incomplete, localpref 100, valid, external, best Extended Community: RT:100:2 mpls labels in/out 106/nolabel rx pathid: 0, tx pathid: 0x0 R2#sh bgp vpnv4 unicast vrf Cust-B 172.16.10.0 BGP routing table entry for 200:2:172.16.10.0/24, version 12 Paths: (1 available, best #1, table Cust-B) Advertised to update-groups: 2 Refresh Epoch 1 Local 2.2.0.6 (metric 3) from 2.2.0.6 (2.2.0.6) Origin incomplete, metric 2, localpref 100, valid, internal, best Extended Community: RT:200:2 OSPF DOMAIN ID:0x0005:0x000000C80200 OSPF RT:0.0.0.0:2:0 OSPF ROUTER ID:172.16.106.6:0 mpls labels in/out nolabel/606 rx pathid: 0, tx pathid: 0x0 R6#sh bgp vpnv4 unicast vrf Cust-B 172.16.10.0 BGP routing table entry for 200:2:172.16.10.0/24, version 7 Paths: (1 available, best #1, table Cust-B) Advertised to update-groups: 1 Refresh Epoch 1 Local 172.16.106.10 from 0.0.0.0 (2.2.0.6) Origin incomplete, metric 2, localpref 100, weight 32768, valid, sourced, best Extended Community: RT:200:2 OSPF DOMAIN ID:0x0005:0x000000C80200 OSPF RT:0.0.0.0:2:0 OSPF ROUTER ID:172.16.106.6:0 mpls labels in/out 606/nolabel rx pathid: 0, tx pathid: 0x0
The major drawback of this design is scalability as it requires a separate VRF interface for each customer on ASBR PE Routers.
This issue is addressed in MPLS Inter-AS option B & C. We will discuss it later.