BGP route-reflector next-hop issue

Question

Hello,
I have a small GNS3 lab that is working with one exception: I cannot ping loopback0 on RRc2 and RRc3 from RRc1.
RRc1, RRc2 and RRc3 can all ping loopback0 on SmileyISP and RRc2 and RRc3 can ping each others loopback0
interfaces.
I am broken between the two route-reflectors: RRS1 and RRS2.
Given these conditions:
1) Do not configure any IGP.
2) No static routes
How do I get connectivity from RRc1's loopback0 interface to RRc2 loopback0 and RRc3 loopback0?
I used a route-map to set the next hop, but I am obviously doing something wrong.
I am providing relevant show command outputs, router configs, and the GNS3 topology.net config.
You will have to change the image and working directories to match your computer.
Not quite sure where I am going wrong.
Any help would be greatly appreciated.
Thanks.
-- Mark
RRc1#sh ip bgp
BGP table version is 53, local router ID is 172.16.1.1
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
              r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter,
              x best-external, a additional-path, c RIB-compressed,
Origin codes: i - IGP, e - EGP, ? - incomplete
RPKI validation codes: V valid, I invalid, N Not found
     Network          Next Hop            Metric LocPrf Weight Path
 *>i 1.1.1.0/24       10.1.25.5                0    100      0 100 i
 *>i 10.1.12.0/24     10.1.26.2                0    100      0 i
 *>i 10.1.13.0/24     10.1.12.1                0    100      0 i
 *>i 10.1.14.0/24     10.1.12.1                0    100      0 i
 *>i 10.1.15.0/24     10.1.12.1                0    100      0 i
 *>i 10.1.25.0/24     10.1.26.2                0    100      0 i
 * i 10.1.26.0/24     10.1.26.2                0    100      0 i
 *>                   0.0.0.0                  0         32768 i
 *>  172.16.1.0/24    0.0.0.0                  0         32768 i
 *>i 172.16.2.0/24    10.1.12.1                0    100      0 i
 *>i 172.16.3.0/24    10.1.12.1                0    100      0 i
RRc1#
RRc1#ping 172.16.2.1 so lo0
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 172.16.2.1, timeout is 2 seconds:
Packet sent with a source address of 172.16.1.1
Success rate is 0 percent (0/5)
RRc1#
RRc2#sh ip bgp
BGP table version is 31, local router ID is 172.16.2.1
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
              r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter,
              x best-external, a additional-path, c RIB-compressed,
Origin codes: i - IGP, e - EGP, ? - incomplete
RPKI validation codes: V valid, I invalid, N Not found
     Network          Next Hop            Metric LocPrf Weight Path
 *>i 1.1.1.0/24       10.1.15.5                0    100      0 100 i
 * i 10.1.12.0/24     10.1.12.2                0    100      0 i
 * i 10.1.13.0/24     10.1.13.1                0    100      0 i
 *>                   0.0.0.0                  0         32768 i
 *>i 10.1.14.0/24     10.1.13.1                0    100      0 i
 *>i 10.1.15.0/24     10.1.13.1                0    100      0 i
 * i 10.1.25.0/24     10.1.12.2                0    100      0 i
 * i 10.1.26.0/24     10.1.12.2                0    100      0 i
 * i 172.16.1.0/24    10.1.12.2                0    100      0 i
 *>  172.16.2.0/24    0.0.0.0                  0         32768 i
 *>i 172.16.3.0/24    10.1.14.4                0    100      0 i
RRc2#
SmileyISP#sh run
Building configuration...
Current configuration : 988 bytes
version 15.2
service timestamps debug datetime msec
service timestamps log datetime msec
hostname SmileyISP
boot-start-marker
boot-end-marker
no aaa new-model
ip cef
no ipv6 cef
multilink bundle-name authenticated
interface Loopback0
 ip address 1.1.1.1 255.255.255.0
interface FastEthernet0/0
 no ip address
 shutdown
 duplex half
interface FastEthernet1/0
 ip address 10.1.15.5 255.255.255.0
 speed auto
 duplex auto
interface FastEthernet1/1
 ip address 10.1.25.5 255.255.255.0
 speed auto
 duplex auto
router bgp 100
 bgp log-neighbor-changes
 network 1.1.1.0 mask 255.255.255.0
 network 10.1.15.0 mask 255.255.255.0
 neighbor 10.1.15.1 remote-as 200
 neighbor 10.1.25.2 remote-as 200
ip forward-protocol nd
no ip http server
no ip http secure-server
control-plane
line con 0
 logging synchronous
 transport preferred none
 stopbits 1
line aux 0
 stopbits 1
line vty 0 4
 login
end
RRS1#sh run
Building configuration...
Current configuration : 1594 bytes
! Last configuration change at 19:24:34 UTC Sat Feb 7 2015
version 15.2
service timestamps debug datetime msec
service timestamps log datetime msec
hostname RRS1
boot-start-marker
boot-end-marker
no aaa new-model
ip cef
no ipv6 cef
multilink bundle-name authenticated
interface FastEthernet0/0
 no ip address
 shutdown
 duplex half
interface FastEthernet1/0
 ip address 10.1.15.1 255.255.255.0
 speed auto
 duplex auto
interface FastEthernet1/1
 ip address 10.1.12.1 255.255.255.0
 speed auto
 duplex auto
interface FastEthernet2/0
 ip address 10.1.13.1 255.255.255.0
 speed auto
 duplex auto
interface FastEthernet2/1
 ip address 10.1.14.1 255.255.255.0
 speed auto
 duplex auto
router bgp 200
 bgp log-neighbor-changes
 network 10.1.13.0 mask 255.255.255.0
 network 10.1.14.0 mask 255.255.255.0
 network 10.1.15.0 mask 255.255.255.0
 neighbor RouteReflectors peer-group
 neighbor RouteReflectors remote-as 200
 neighbor RouteReflectors route-map NEXTHOP out
 neighbor RRClients peer-group
 neighbor RRClients remote-as 200
 neighbor RRClients route-reflector-client
 neighbor 10.1.12.2 peer-group RouteReflectors
 neighbor 10.1.13.3 peer-group RRClients
 neighbor 10.1.14.4 peer-group RRClients
 neighbor 10.1.15.5 remote-as 100
ip forward-protocol nd
no ip http server
no ip http secure-server
route-map NEXTHOP permit 10
 set ip next-hop peer-address
control-plane
line con 0
 logging synchronous
 transport preferred none
 stopbits 1
line aux 0
 stopbits 1
line vty 0 4
 login
end
RRS2#sh ru
Building configuration...
Current configuration : 1542 bytes
! Last configuration change at 19:42:06 UTC Sat Feb 7 2015
version 15.2
service timestamps debug datetime msec
service timestamps log datetime msec
hostname RRS2
boot-start-marker
boot-end-marker
no aaa new-model
ip cef
no ipv6 cef
multilink bundle-name authenticated
interface FastEthernet0/0
 no ip address
 shutdown
 duplex half
interface FastEthernet1/0
 ip address 10.1.12.2 255.255.255.0
 speed auto
 duplex auto
interface FastEthernet1/1
 ip address 10.1.25.2 255.255.255.0
 speed auto
 duplex auto
interface FastEthernet2/0
 ip address 10.1.26.2 255.255.255.0
 speed auto
 duplex auto
interface FastEthernet2/1
 no ip address
 shutdown
 speed auto
 duplex auto
router bgp 200
 bgp log-neighbor-changes
 network 10.1.12.0 mask 255.255.255.0
 network 10.1.25.0 mask 255.255.255.0
 network 10.1.26.0 mask 255.255.255.0
 neighbor RouteReflectors peer-group
 neighbor RouteReflectors remote-as 200
 neighbor RouteReflectors route-map NEXTHOP out
 neighbor RRClients peer-group
 neighbor RRClients remote-as 200
 neighbor RRClients route-reflector-client
 neighbor 10.1.12.1 peer-group RouteReflectors
 neighbor 10.1.25.5 remote-as 100
 neighbor 10.1.26.6 peer-group RRClients
ip forward-protocol nd
no ip http server
no ip http secure-server
route-map NEXTHOP permit 10
 set ip next-hop peer-address
control-plane
line con 0
 logging synchronous
 transport preferred none
 stopbits 1
line aux 0
 stopbits 1
line vty 0 4
 login
end
RRc1#sh run
Building configuration...
Current configuration : 1005 bytes
! Last configuration change at 18:43:57 UTC Sat Feb 7 2015
version 15.2
service timestamps debug datetime msec
service timestamps log datetime msec
hostname RRc1
boot-start-marker
boot-end-marker
no aaa new-model
ip cef
no ipv6 cef
multilink bundle-name authenticated
interface Loopback0
 ip address 172.16.1.1 255.255.255.0
interface FastEthernet0/0
 no ip address
 shutdown
 duplex half
interface FastEthernet1/0
 ip address 10.1.26.6 255.255.255.0
 speed auto
 duplex auto
interface FastEthernet1/1
 no ip address
 shutdown
 speed auto
 duplex auto
router bgp 200
 bgp log-neighbor-changes
 network 10.1.26.0 mask 255.255.255.0
 network 172.16.1.0 mask 255.255.255.0
 neighbor 10.1.26.2 remote-as 200
ip forward-protocol nd
no ip http server
no ip http secure-server
control-plane
line con 0
 logging synchronous
 transport preferred none
 stopbits 1
line aux 0
 stopbits 1
line vty 0 4
 login
end
RRc2#sh run
Building configuration...
Current configuration : 1005 bytes
! Last configuration change at 18:45:05 UTC Sat Feb 7 2015
version 15.2
service timestamps debug datetime msec
service timestamps log datetime msec
hostname RRc2
boot-start-marker
boot-end-marker
no aaa new-model
ip cef
no ipv6 cef
multilink bundle-name authenticated
interface Loopback0
 ip address 172.16.2.1 255.255.255.0
interface FastEthernet0/0
 no ip address
 shutdown
 duplex half
interface FastEthernet1/0
 ip address 10.1.13.3 255.255.255.0
 speed auto
 duplex auto
interface FastEthernet1/1
 no ip address
 shutdown
 speed auto
 duplex auto
router bgp 200
 bgp log-neighbor-changes
 network 10.1.13.0 mask 255.255.255.0
 network 172.16.2.0 mask 255.255.255.0
 neighbor 10.1.13.1 remote-as 200
ip forward-protocol nd
no ip http server
no ip http secure-server
control-plane
line con 0
 logging synchronous
 transport preferred none
 stopbits 1
line aux 0
 stopbits 1
line vty 0 4
 login
end
RRc3#wr term
Building configuration...
Current configuration : 1005 bytes
! Last configuration change at 18:31:12 UTC Sat Feb 7 2015
version 15.2
service timestamps debug datetime msec
service timestamps log datetime msec
hostname RRc3
boot-start-marker
boot-end-marker
no aaa new-model
ip cef
no ipv6 cef
multilink bundle-name authenticated
interface Loopback0
 ip address 172.16.3.1 255.255.255.0
interface FastEthernet0/0
 no ip address
 shutdown
 duplex half
interface FastEthernet1/0
 ip address 10.1.14.4 255.255.255.0
 speed auto
 duplex auto
interface FastEthernet1/1
 no ip address
 shutdown
 speed auto
 duplex auto
router bgp 200
 bgp log-neighbor-changes
 network 10.1.14.0 mask 255.255.255.0
 network 172.16.3.0 mask 255.255.255.0
 neighbor 10.1.14.1 remote-as 200
ip forward-protocol nd
no ip http server
no ip http secure-server
control-plane
line con 0
 logging synchronous
 transport preferred none
 stopbits 1
line aux 0
 stopbits 1
line vty 0 4
 login
end
autostart = False
version = 0.8.6
[127.0.0.1:7202]
    workingdir = C:\Users\Mark\AppData\Local\Temp
    udp = 10200
        image = C:\downloads\GNS3\c7200-adventerprisek9-mz.152-4.S5.image
        idlepc = 0x62f1e4ec
        ghostios = True
        console = 2005
        aux = 2100
        cnfg = configs\SmileyISP.cfg
        slot1 = PA-2FE-TX
        f1/0 = RRS1 f1/0
        f1/1 = RRS2 f1/1
        x = -24.0
        y = -259.0
        z = 1.0
        hx = -1.5
        hy = -24.0
        console = 2015
        aux = 2101
        cnfg = configs\RRc1.cfg
        slot1 = PA-2FE-TX
        f1/0 = RRS2 f2/0
        x = -292.0
        y = 200.0
        z = 1.0
        hx = -5.5
        hy = -25.0
[127.0.0.1:7200]
    workingdir = C:\Users\Mark\AppData\Local\Temp
    udp = 10000
        image = C:\downloads\GNS3\c7200-adventerprisek9-mz.152-4.S5.image
        idlepc = 0x62f1e4ec
        ghostios = True
        console = 2012
        aux = 2102
        cnfg = configs\RRS1.cfg
        slot1 = PA-2FE-TX
        f1/0 = SmileyISP f1/0
        f1/1 = RRS2 f1/0
        slot2 = PA-2FE-TX
        f2/0 = RRc2 f1/0
        f2/1 = RRc3 f1/0
        x = 197.0
        y = 6.0
        z = 1.0
        hx = 42.5
        hy = -20.0
        console = 2013
        aux = 2103
        cnfg = configs\RRS2.cfg
        slot1 = PA-2FE-TX
        f1/0 = RRS1 f1/1
        f1/1 = SmileyISP f1/1
        slot2 = PA-2FE-TX
        f2/0 = RRc1 f1/0
        x = -239.0
        y = 9.0
        z = 1.0
        hx = 1.5
        hy = -24.0
[127.0.0.1:7201]
    workingdir = C:\Users\Mark\AppData\Local\Temp
    udp = 10100
        image = C:\downloads\GNS3\c7200-adventerprisek9-mz.152-4.S5.image
        idlepc = 0x62f1e4ec
        ghostios = True
        console = 2009
        aux = 2104
        cnfg = configs\RRc3.cfg
        slot1 = PA-2FE-TX
        f1/0 = RRS1 f2/1
        x = 337.0
        y = 155.0
        z = 1.0
        hx = 17.5
        hy = -25.0
        console = 2008
        aux = 2105
        cnfg = configs\RRc2.cfg
        slot1 = PA-2FE-TX
        f1/0 = RRS1 f2/0
        x = 149.0
        y = 204.0
        z = 1.0
        hx = -13.5
        hy = -23.0
[GNS3-DATA]
    configs = configs
        text = ".1"
        x = 208.0
        y = -23.0
        text = "10.1.12.0/24"
        x = -19.0
        y = 5.0
        text = ".1"
        x = 153.0
        y = 25.0
        text = ".1"
        x = 259.0
        y = 33.0
        text = "10.1.13.0/24"
        x = 238.0
        y = 84.0
        rotate = 99
        text = "10.1.25.0/24"
        x = -188.0
        y = -124.0
        text = "l0: 172.16.2.1/24"
        x = 125.0
        y = 244.0
        text = "l0:172.16.1.1/24"
        x = -269.0
        y = 240.0
        text = "10.1.15.0/24"
        x = 116.0
        y = -127.0
        text = "10.1.14.0/24"
        x = 293.0
        y = 53.0
        rotate = 50
        text = ".1"
        x = 194.0
        y = 68.0
        text = "AS100"
        x = -20.0
        y = -342.0
        text = ".2"
        x = -148.0
        y = 46.0
        text = "AS200"
        x = 33.0
        y = 300.0
        text = "l0: 1.1.1.1/24"
        x = -42.0
        y = -306.0
        text = ".5"
        x = 50.0
        y = -213.0
        text = ".2"
        x = -248.0
        y = 60.0
        text = ".2"
        x = -174.0
        y = -52.0
        text = ".5"
        x = -54.0
        y = -209.0
        text = ".6"
        x = -232.0
        y = 189.0
        text = "l0:172.16.3.1/24"
        x = 299.0
        y = 194.0
        text = "10.1.26.0/24"
        x = -274.0
        y = 167.0
        rotate = 290
        text = ".3"
        x = 208.0
        y = 187.0
        text = ".4"
        x = 312.0
        y = 155.0
        type = ellipse
        x = 50.0
        y = -35.0
        width = 385.0
        height = 345.0
        fill_color = "#ffff7f"
        border_style = 2
        z = -1.0
        type = ellipse
        x = -171.0
        y = -346.0
        width = 359.0
        height = 200.0
        fill_color = "#aaff7f"
        border_style = 2
        z = -1.0
        type = ellipse
        x = -407.0
        y = -87.0
        width = 883.0
        height = 443.0
        border_style = 2
        z = -2.0
        type = ellipse
        x = -361.0
        y = -29.0
        width = 385.0
        height = 326.0
        fill_color = "#55aaff"
        border_style = 2
        z = -3.0

Answer 1

BD,
Ahh...
OK.  In the original article, the author states that the final piece with the route map
NEXTHOP was supposed to fix the reachability issue.  Obviously it doesn't.
After reading your last post, I looked more carefully at the output from 'sh ip bgp'
on each of the client routers and I realized that several of the next hop addresses were
wrong for some of the prefixes.
1) I completely removed the 'neighbor RouteReflectors route-map NEXTHOP out'
from both RR's.  Then I ran 'sh ip bgp' on the clients and noted a change in the next hop addresses.  Still wrong, but it changed.
2) I then tried next-hop-self from the RR's to the clients, but it did not change from where
it was after I completed step 1.  I am not sure why there was no change. (actually, see the very end of this post)
3) I then applied my version of the route map:  route-map NEXTHOP permit 10
                                                                             set ip next-hop peer-address
to the RR's with this: neighbor RRClients route-map NEXTHOP out
That fixed it.  All three clients have as their next hop for all prefixes their respective
RR's (which is what they should have for this topology).
I have full connectivity everywhere, even loopback to loopback between all clients.
1) THANK YOU for pointing me in the right direction.
2) If I may ask, why did next hop self fail?  More specifically, I saw no change at all
in the next hop for the advertised prefixes.  Is it because next-hop-self should be used
for eBGP peers and all of the RR's and clients are all within the same AS?