If two sites of a MPLS L3VPN are connected to different Autonomous Systems (e.g., because the sites are connected to different SPs). The PE routers attached to that VPN will then not be able to maintain IBGP connections with each other, or with a common route reflector. Rather, there needs to be some way to use EBGP to distribute VPN-IPv4 addresses.

There are a number of different ways of handling this case, let’s discuss first option which is called option A or back-to-back VRF exchange.

In this procedure, a PE router in one AS attaches directly to a PE router in another.  The two PE routers will be attached by multiple sub-interfaces, at least one for each of the VPNs whose routes need to be passed from AS to AS.  Each PE will treat the other as if it were a CE router.  That is, the PEs associate each such sub-interface with a VRF, and use EBGP (or any PE/CE IGP) to distribute unlabeled IPv4 addresses to each other.

We will use following network topology to discuss this feature.

PE-CE Configuration with ISP-1

R7 (CE Router) - Customer A Site 1

## EIGRP is the PE-CE routing protocol used for this customer ##

interface FastEthernet0/0
 ip address 192.168.47.7 255.255.255.0
!
interface FastEthernet0/1
 ip address 192.168.7.7 255.255.255.0
!
router eigrp 100
 network 192.168.0.0 0.0.255.255

R8 (CE Router) - Customer B Site 1

## OSPF is the PE-CE routing protocol used for this customer ##

interface FastEthernet0/0
 ip address 172.16.48.8 255.255.255.0
 ip ospf network point-to-point
 ip ospf 1 area 0
!
interface FastEthernet0/1
 ip address 172.16.8.8 255.255.255.0
 ip ospf network point-to-point
 ip ospf 1 area 0
!
router ospf 1
 router-id 172.16.8.8

R4 (PE Router) - ISP 1

vrf definition Cust-A
 rd 100:1
 !
 address-family ipv4
  route-target export 100:1
  route-target import 100:1
 exit-address-family
!
vrf definition Cust-B
 rd 100:2
 !        
 address-family ipv4
  route-target export 100:2
  route-target import 100:2
 exit-address-family
!
interface Loopback0
 ip address 1.1.0.4 255.255.255.255
 ip ospf 1 area 0
!
interface FastEthernet0/1
 ip address 1.1.34.4 255.255.255.0
 ip ospf network point-to-point
 ip ospf 1 area 0
 mpls ip
!
interface FastEthernet1/0
 vrf forwarding Cust-A
 ip address 192.168.47.4 255.255.255.0
!
interface FastEthernet1/1
 vrf forwarding Cust-B
 ip address 172.16.48.4 255.255.255.0
 ip ospf network point-to-point
 ip ospf 100 area 0
!
router eigrp 1
 !
 address-family ipv4 vrf Cust-A autonomous-system 100
  default-metric 10000 100 255 1 1500
  redistribute bgp 100
  network 192.168.47.0
 exit-address-family
!
router ospf 100 vrf Cust-B
 router-id 172.16.48.4
 redistribute bgp 100 subnets
!
router ospf 1
 router-id 1.1.0.4
!
router bgp 100
 bgp router-id 1.1.0.4
 bgp log-neighbor-changes
 no bgp default ipv4-unicast
 neighbor 1.1.0.1 remote-as 100
 neighbor 1.1.0.1 update-source Loopback0
 !
 address-family ipv4
 exit-address-family
 !
 address-family vpnv4
  neighbor 1.1.0.1 activate
  neighbor 1.1.0.1 send-community extended
 exit-address-family
 !
 address-family ipv4 vrf Cust-A
  redistribute connected
  redistribute eigrp 100
 exit-address-family
 !
 address-family ipv4 vrf Cust-B
  redistribute connected
  redistribute ospf 100 match internal external 1 external 2
 exit-address-family
! 

R3 (P Router) - ISP 1

interface Loopback0
 ip address 1.1.0.3 255.255.255.255
 ip ospf 1 area 0
!
interface FastEthernet0/0
 ip address 1.1.13.3 255.255.255.0
 ip ospf network point-to-point
 ip ospf 1 area 0
 mpls ip
!
interface FastEthernet0/1
 ip address 1.1.34.3 255.255.255.0
 ip ospf network point-to-point
 ip ospf 1 area 0
 mpls ip  
!
router ospf 1
 router-id 1.1.0.3

R1 (PE Router) - ISP 1 - ASBR connected to ISP 2

vrf definition Cust-A
 rd 100:1
 !
 address-family ipv4
  route-target export 100:1
  route-target import 100:1
 exit-address-family
!
vrf definition Cust-B
 rd 100:2
 !        
 address-family ipv4
  route-target export 100:2
  route-target import 100:2
 exit-address-family
!
interface Loopback0
 ip address 1.1.0.1 255.255.255.255
 ip ospf 1 area 0
!
interface FastEthernet0/0
 ip address 1.1.13.1 255.255.255.0
 ip ospf network point-to-point
 ip ospf 1 area 0
 speed 100
 duplex full
 mpls ip
!
interface FastEthernet0/1
 no ip address
!
interface FastEthernet0/1.1
 encapsulation dot1Q 12
 vrf forwarding Cust-A
 ip address 12.12.12.1 255.255.255.0
!
interface FastEthernet0/1.2
 encapsulation dot1Q 21
 vrf forwarding Cust-B
 ip address 21.21.21.1 255.255.255.0
!   
router ospf 1
 router-id 1.1.0.1
!
router bgp 100
 bgp router-id 1.1.0.1
 bgp log-neighbor-changes
 no bgp default ipv4-unicast
 neighbor 1.1.0.4 remote-as 100
 neighbor 1.1.0.4 update-source Loopback0
 !
 address-family ipv4
 exit-address-family
 !
 address-family vpnv4
  neighbor 1.1.0.4 activate
  neighbor 1.1.0.4 send-community extended
 exit-address-family
 !
 address-family ipv4 vrf Cust-A
  neighbor 12.12.12.2 remote-as 200
  neighbor 12.12.12.2 activate
 exit-address-family
 !
 address-family ipv4 vrf Cust-B
  neighbor 21.21.21.2 remote-as 200
  neighbor 21.21.21.2 activate
 exit-address-family
!             

PE-CE Configuration with ISP-2

Both ASBR routers in ISP1 and ISP2 see each other as CE router and treat itself as PE router. RD (Route Distinguisher) value for each customer does not require to match in both ISP networks. Here, Both ISPs are using different RD (Route Distinguisher).

We have used BGP as PE-CE routing protocol here. Any IGP can be used as well.

R2 (PE Router) - ISP 2 - ASBR connected to ISP 1

vrf definition Cust-A
 rd 200:1
 !
 address-family ipv4
  route-target export 200:1
  route-target import 200:1
 exit-address-family
!
vrf definition Cust-B
 rd 200:2
 !        
 address-family ipv4
  route-target export 200:2
  route-target import 200:2
 exit-address-family
!
interface Loopback0
 ip address 2.2.0.2 255.255.255.255
 ip ospf 1 area 0
!
interface FastEthernet0/0
 ip address 2.2.25.2 255.255.255.0
 ip ospf network point-to-point
 ip ospf 1 area 0
 mpls ip
!
interface FastEthernet0/1
 no ip address
!
interface FastEthernet0/1.1
 encapsulation dot1Q 12
 vrf forwarding Cust-A
 ip address 12.12.12.2 255.255.255.0
!
interface FastEthernet0/1.2
 encapsulation dot1Q 21
 vrf forwarding Cust-B
 ip address 21.21.21.2 255.255.255.0
!
router ospf 1
 router-id 2.2.0.2
!
router bgp 200
 bgp router-id 2.2.0.2
 bgp log-neighbor-changes
 no bgp default ipv4-unicast
 neighbor 2.2.0.6 remote-as 200
 neighbor 2.2.0.6 update-source Loopback0
 !
 address-family ipv4
 exit-address-family
 !
 address-family vpnv4
  neighbor 2.2.0.6 activate
  neighbor 2.2.0.6 send-community extended
 exit-address-family
 !
 address-family ipv4 vrf Cust-A
  neighbor 12.12.12.1 remote-as 100
  neighbor 12.12.12.1 activate
 exit-address-family
 !
 address-family ipv4 vrf Cust-B
  neighbor 21.21.21.1 remote-as 100
  neighbor 21.21.21.1 activate
 exit-address-family


R5 (P Router) - ISP 2

interface Loopback0
 ip address 2.2.0.5 255.255.255.255
 ip ospf 1 area 0
!
interface FastEthernet0/0
 ip address 2.2.25.5 255.255.255.0
 ip ospf network point-to-point
 ip ospf 1 area 0
 mpls ip
!
interface FastEthernet0/1
 ip address 2.2.56.5 255.255.255.0
 ip ospf network point-to-point
 ip ospf 1 area 0
 mpls ip  
!
router ospf 1
 router-id 2.2.0.5

R6 (PE Router) - ISP 2

vrf definition Cust-A
 rd 200:1
 !
 address-family ipv4
  route-target export 200:1
  route-target import 200:1
 exit-address-family
!
vrf definition Cust-B
 rd 200:2
 !        
 address-family ipv4
  route-target export 200:2
  route-target import 200:2
 exit-address-family
!
interface Loopback0
 ip address 2.2.0.6 255.255.255.255
 ip ospf 1 area 0
!
interface FastEthernet0/0
 no ip address
!
interface FastEthernet0/1
 ip address 2.2.56.6 255.255.255.0
 ip ospf network point-to-point
 ip ospf 1 area 0
 mpls ip
!
interface FastEthernet1/0
 vrf forwarding Cust-A
 ip address 192.168.69.6 255.255.255.0
!
interface FastEthernet1/1
 vrf forwarding Cust-B
 ip address 172.16.106.6 255.255.255.0
 ip ospf network point-to-point
 ip ospf 200 area 0
!
!
router eigrp 1
 !
 address-family ipv4 vrf Cust-A autonomous-system 100
  default-metric 10000 100 255 1 1500
  redistribute bgp 200
  network 192.168.69.0
 exit-address-family
!
router ospf 200 vrf Cust-B
 router-id 172.16.106.6
 redistribute bgp 200 subnets
!
router ospf 1
 router-id 2.2.0.6
!
router bgp 200
 bgp router-id 2.2.0.6
 bgp log-neighbor-changes
 no bgp default ipv4-unicast
 neighbor 2.2.0.2 remote-as 200
 neighbor 2.2.0.2 update-source Loopback0
 !
 address-family ipv4
 exit-address-family
 !
 address-family vpnv4
  neighbor 2.2.0.2 activate
  neighbor 2.2.0.2 send-community extended
 exit-address-family
 !
 address-family ipv4 vrf Cust-A
  redistribute connected
  redistribute eigrp 100
 exit-address-family
 !
 address-family ipv4 vrf Cust-B
  redistribute connected
  redistribute ospf 200 match internal external 1 external 2
 exit-address-family


R9 (CE Router) - Customer A Site 2

## EIGRP is the PE-CE routing protocol used for this customer ##

interface FastEthernet0/0
 ip address 192.168.69.9 255.255.255.0
!
interface FastEthernet0/1
 ip address 192.168.9.9 255.255.255.0
!
router eigrp 100
 network 192.168.0.0 0.0.255.255

R10 (CE Router) - Customer B Site 2

## OSPF is the PE-CE routing protocol used for this customer ##

interface FastEthernet0/0
 ip address 172.16.106.10 255.255.255.0
 ip ospf network point-to-point
 ip ospf 1 area 0
!
interface FastEthernet0/1
 ip address 172.16.10.10 255.255.255.0
 ip ospf network point-to-point
 ip ospf 1 area 0
!
router ospf 1
 router-id 172.16.10.10

Verification & Testing

R7#sh ip route | beg Gateway
Gateway of last resort is not set

      192.168.7.0/24 is variably subnetted, 2 subnets, 2 masks
C        192.168.7.0/24 is directly connected, FastEthernet0/1
L        192.168.7.7/32 is directly connected, FastEthernet0/1
D EX  192.168.9.0/24 [170/284160] via 192.168.47.4, 01:04:34, FastEthernet0/0
      192.168.47.0/24 is variably subnetted, 2 subnets, 2 masks
C        192.168.47.0/24 is directly connected, FastEthernet0/0
L        192.168.47.7/32 is directly connected, FastEthernet0/0
D EX  192.168.69.0/24 [170/284160] via 192.168.47.4, 01:04:34, FastEthernet0/0

Ping Customer-A Site 2 network

R7#ping 192.168.9.9 source 192.168.7.7
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.9.9, timeout is 2 seconds:
Packet sent with a source address of 192.168.7.7 
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 152/160/172 ms

Traceroute Customer-A Site 2 network

R7#traceroute 192.168.9.9 source 192.168.7.7
Type escape sequence to abort.
Tracing the route to 192.168.9.9
VRF info: (vrf in name/id, vrf out name/id)
  1 192.168.47.4 28 msec 36 msec 36 msec
  2 1.1.34.3 [MPLS: Labels 302/104 Exp 0] 88 msec 100 msec 88 msec
  3 12.12.12.1 [MPLS: Label 104 Exp 0] 64 msec 92 msec 60 msec
  4 12.12.12.2 116 msec 120 msec 92 msec
  5 2.2.25.5 [MPLS: Labels 501/604 Exp 0] 176 msec 168 msec 152 msec
  6 192.168.69.6 [MPLS: Label 604 Exp 0] 128 msec 140 msec 140 msec
  7 192.168.69.9 176 msec *  156 msec

We can see from the above output there are two different LSP (Label Switch Path) are being used to provide end-to-end reachability.

LSP-1: From R4 to R1 using VPN Label 104

R4#sh bgp vpnv4 unicast vrf Cust-A 192.168.9.0
BGP routing table entry for 100:1:192.168.9.0/24, version 10
Paths: (1 available, best #1, table Cust-A)
  Not advertised to any peer
  Refresh Epoch 1
  200
    1.1.0.1 (metric 3) from 1.1.0.1 (1.1.0.1)
      Origin incomplete, metric 0, localpref 100, valid, internal, best
      Extended Community: RT:100:1
      mpls labels in/out nolabel/104
      rx pathid: 0, tx pathid: 0x0

The traffic between R1 & R2 is normal Unlabeled IPv4 traffic

R1#sh bgp vpnv4 unicast vrf Cust-A 192.168.9.0
BGP routing table entry for 100:1:192.168.9.0/24, version 10
Paths: (1 available, best #1, table Cust-A)
  Advertised to update-groups:
     3         
  Refresh Epoch 1
  200
    12.12.12.2 from 12.12.12.2 (2.2.0.2)
      Origin incomplete, localpref 100, valid, external, best
      Extended Community: RT:100:1
      mpls labels in/out 104/nolabel
      rx pathid: 0, tx pathid: 0x0

LSP-2: From R2 to R6 using VPN label 604

R2#sh bgp vpnv4 unicast vrf Cust-A 192.168.9.0
BGP routing table entry for 200:1:192.168.9.0/24, version 10
Paths: (1 available, best #1, table Cust-A)
  Advertised to update-groups:
     1         
  Refresh Epoch 1
  Local
    2.2.0.6 (metric 3) from 2.2.0.6 (2.2.0.6)
      Origin incomplete, metric 30720, localpref 100, valid, internal, best
      Extended Community: RT:200:1 Cost:pre-bestpath:128:30720 0x8800:32768:0 
        0x8801:100:5120 0x8802:65281:25600 0x8803:65281:1500 
        0x8806:0:3232253193
      mpls labels in/out nolabel/604
      rx pathid: 0, tx pathid: 0x0

Finally unlabelled traffic from ISP 2 PE Router (R6) to CE Router (R9)
 
R6#sh bgp vpnv4 unicast vrf Cust-A 192.168.9.0
BGP routing table entry for 200:1:192.168.9.0/24, version 3
Paths: (1 available, best #1, table Cust-A)
  Advertised to update-groups:
     1         
  Refresh Epoch 1
  Local
    192.168.69.9 from 0.0.0.0 (2.2.0.6)
      Origin incomplete, metric 30720, localpref 100, weight 32768, valid, sourced, best
      Extended Community: RT:200:1 Cost:pre-bestpath:128:30720 0x8800:32768:0 
        0x8801:100:5120 0x8802:65281:25600 0x8803:65281:1500 
        0x8806:0:3232253193
      mpls labels in/out 604/nolabel
      rx pathid: 0, tx pathid: 0x0

Similarly, end-to-end reachability can be verified for Customer B sites.

R8#sh ip route | beg Gateway 
Gateway of last resort is not set

      172.16.0.0/16 is variably subnetted, 6 subnets, 2 masks
C        172.16.8.0/24 is directly connected, FastEthernet0/1
L        172.16.8.8/32 is directly connected, FastEthernet0/1
O E2     172.16.10.0/24 [110/1] via 172.16.48.4, 01:33:51, FastEthernet0/0
C        172.16.48.0/24 is directly connected, FastEthernet0/0
L        172.16.48.8/32 is directly connected, FastEthernet0/0
O E2     172.16.106.0/24 [110/1] via 172.16.48.4, 01:33:51, FastEthernet0/0

R8#ping 172.16.10.10 source 172.16.8.8
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 172.16.10.10, timeout is 2 seconds:
Packet sent with a source address of 172.16.8.8 
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 220/236/248 ms
R8#trace 172.16.10.10 source 172.16.8.8
Type escape sequence to abort.
Tracing the route to 172.16.10.10
VRF info: (vrf in name/id, vrf out name/id)
  1 172.16.48.4 32 msec 52 msec 64 msec
  2 1.1.34.3 [MPLS: Labels 302/106 Exp 0] 140 msec 140 msec 136 msec
  3 21.21.21.1 [MPLS: Label 106 Exp 0] 84 msec 104 msec 116 msec
  4 21.21.21.2 136 msec 152 msec 132 msec
  5 2.2.25.5 [MPLS: Labels 501/606 Exp 0] 248 msec 224 msec 232 msec
  6 172.16.106.6 [MPLS: Label 606 Exp 0] 208 msec 196 msec 192 msec
  7 172.16.106.10 236 msec *  240 msec


R4#sh bgp vpnv4 unicast vrf Cust-B 172.16.10.0
BGP routing table entry for 100:2:172.16.10.0/24, version 12
Paths: (1 available, best #1, table Cust-B)
  Not advertised to any peer
  Refresh Epoch 1
  200
    1.1.0.1 (metric 3) from 1.1.0.1 (1.1.0.1)
      Origin incomplete, metric 0, localpref 100, valid, internal, best
      Extended Community: RT:100:2
      mpls labels in/out nolabel/106
      rx pathid: 0, tx pathid: 0x0

R1#sh bgp vpnv4 unicast vrf Cust-B 172.16.10.0
BGP routing table entry for 100:2:172.16.10.0/24, version 12
Paths: (1 available, best #1, table Cust-B)
  Advertised to update-groups:
     3         
  Refresh Epoch 1
  200
    21.21.21.2 from 21.21.21.2 (2.2.0.2)
      Origin incomplete, localpref 100, valid, external, best
      Extended Community: RT:100:2
      mpls labels in/out 106/nolabel
      rx pathid: 0, tx pathid: 0x0

R2#sh bgp vpnv4 unicast vrf Cust-B 172.16.10.0
BGP routing table entry for 200:2:172.16.10.0/24, version 12
Paths: (1 available, best #1, table Cust-B)
  Advertised to update-groups:
     2         
  Refresh Epoch 1
  Local
    2.2.0.6 (metric 3) from 2.2.0.6 (2.2.0.6)
      Origin incomplete, metric 2, localpref 100, valid, internal, best
      Extended Community: RT:200:2 OSPF DOMAIN ID:0x0005:0x000000C80200 
        OSPF RT:0.0.0.0:2:0 OSPF ROUTER ID:172.16.106.6:0
      mpls labels in/out nolabel/606
      rx pathid: 0, tx pathid: 0x0

R6#sh bgp vpnv4 unicast vrf Cust-B 172.16.10.0
BGP routing table entry for 200:2:172.16.10.0/24, version 7
Paths: (1 available, best #1, table Cust-B)
  Advertised to update-groups:
     1         
  Refresh Epoch 1
  Local
    172.16.106.10 from 0.0.0.0 (2.2.0.6)
      Origin incomplete, metric 2, localpref 100, weight 32768, valid, sourced, best
      Extended Community: RT:200:2 OSPF DOMAIN ID:0x0005:0x000000C80200 
        OSPF RT:0.0.0.0:2:0 OSPF ROUTER ID:172.16.106.6:0
      mpls labels in/out 606/nolabel
      rx pathid: 0, tx pathid: 0x0

The major drawback of this design is scalability as it requires a separate VRF interface for each customer on ASBR PE Routers.

This issue is addressed in MPLS Inter-AS option B & C. We will discuss it later.