VPC+, aka L3 on back-to-back vPC domains

Hi,
Please consider this scenario, where L2 VLANS are spanning 2 data centers and where R1-R4 are L2/L3 N7K routers (replacing existing 6K).
(I wish VSS would be available also in N7K to make life 10x easier!!).
                      R1                  |                     R2
                       ||                    |                     ||
vPC peer-link  ||    =======MAN=======   ||  vPC peer-link
                       ||                    |                     ||
                      R3                  |                    R4
                               Site A         Site B
Attached to R1 and R3 there are (dual-attached via 6K access switches) servers that may need to communicate to other servers in the same VLAN on the other side of the MAN. Over the MAN the VLANs are trunked, so its fine. This traffic can go over R1 or R3 both for L2 (vPC) and for L3 (HSRP vPC enhancements).
Anway, there is also a global OSPF domain for inter-VLANs communication and for going outside the DCs via other routers attached to the above cloud.
/* Style Definitions */
table.MsoNormalTable
{mso-style-name:"Table Normal";
mso-tstyle-rowband-size:0;
mso-tstyle-colband-size:0;
mso-style-noshow:yes;
mso-style-priority:99;
mso-style-qformat:yes;
mso-style-parent:"";
mso-padding-alt:0cm 5.4pt 0cm 5.4pt;
mso-para-margin:0cm;
mso-para-margin-bottom:.0001pt;
mso-pagination:widow-orphan;
font-size:10.0pt;
font-family:"Times New Roman","serif";}
I've heard there is a kind of enhancement request (or bug?,
/* Style Definitions */
table.MsoNormalTable
{mso-style-name:"Table Normal";
mso-tstyle-rowband-size:0;
mso-tstyle-colband-size:0;
mso-style-noshow:yes;
mso-style-priority:99;
mso-style-qformat:yes;
mso-style-parent:"";
mso-padding-alt:0cm 5.4pt 0cm 5.4pt;
mso-para-margin:0cm;
mso-para-margin-bottom:.0001pt;
mso-pagination:widow-orphan;
font-size:10.0pt;
font-family:"Times New Roman","serif";}
CSCtc71813) to have this kind of back-to-back vPC scenario to handle transparently L3 data (peer-gateway command should deliver also control-plane L3-info??). There are 2 workarounds available for this design:
1.       Define an additional router-in-a-stick using an extra VDC on each 7k. In this case, for example for R1 we would use 3 VDCs: 1 VDC for admin, 1 VDC for L2, 1 VDC for R1.
2.       Define static routes to tell each 7k how to reach the other 7k L3 next-hops.
a) What is the best workaround to choose in order to smooth the upgrade later to the version of vPC that will handles this issue?
b) Are there any more caveats I dont see? I havent seeen any link in CCO, so I am unsure how to proceed the design.
c) I would be tempted to think that using additional static routes is a better choice because it would easier to remove them once vPC+ is there.
What static routes shall I add? R1 to R2, R1 to R4 and so on and so forth? I miss the details of this implementation.
d) How would vPC+ looks like once (when?) is there?
Thansk for your valuable input in advance.
G.

To expand on Lucian's comment, because I'm sure the next though will be...can I run OSFP over a vlan and just carry THAT over my VPC.  You don't want to do this either.
We don't support running routing protocols over VPC enabled VLANs.
What happens is that your 6500 will form routing adj with each Nexus 7000....lets say Nexus 7000-1 and 7000-2.  Note my picture below.
Lets say that R1 is trying to send to a network that is behind R2.  R1 is adj to 7000-1 and 7000-2...we have equal cost paths.  CEF chooses that 7000-1 to route the packet, however Etherchannel load balancing chooses the physical link to 7000-2.  7000-2 will need to switch the packet over the VPC peer-link to 7000-1.  7000-1 receives the packet and tries to send it out VPC member port to R2....however egress port drops the packet.  This happens because we don't allow packets received from VPC member link send over VPC peer-link to be sent out another VPC member link.
I'd suggest to run an L3 link from your 6500 to each Nexus 7000 if you do want to do L3 on it.

Similar Messages

  • Best Practice for VPC Domain failover with One M2 per N7K switch and 2 sups

    I Have been testing some failover scenarios with 4 nexus 7000 switches with an M2 and an F2 card in each. Each Nexus has two supervisor modules.
    I have 3 VDC's Admin, F2 and M2
    all ports in the M2 are in the M2 VDC and all ports on the F2 are in the F2 VDC.
    All vPC's are connected on the M2 cards, configured in the M2 VDC
    We have 2 Nexus representing each "site"
    In one site we have a vPC domain "100"
    The vPC Peer link is connected on ports E1/3 and E1/4 in Port channel 100
    The peer-keepalive is configured to use the management ports. This is patched in both Sups into our 3750s. (this is will eventually be on a management out of band switch)
    Please see the diagram.
    There are 2 vPC's 1&2 connected at each site which represent the virtual port channels that connect back to a pair of 3750X's (the layer 2 switch icons in the diagram.)
    There is also the third vPC that connects the 4 Nexus's together. (po172)
    We are stretching vlan 900 across the "sites" and would like to keep spanning tree out of this as much as we can, and minimise outages based on link failures, module failures, switch failures, sup failures etc..
    ONLY the management vlan (100,101) is allowed on the port-channel between the 3750's, so vlan 900 spanning tree shouldnt have to make this decision.
    We are only concerned about layer two for this part of the testing.
    As we are connecting the vPC peer link to only one module in each switch (a sinlge) M2 we have configured object tracking as follows:
    n7k-1(config)#track 1 interface ethernet 1/1 line-protocol
    n7k-1(config)#track 2 interface ethernet 1/2 line-protocol
    n7k-1(config)#track 5 interface ethernet 1/5 line-protocol
    track 101 list boolean OR
    n7k-1(config-track)# object 1
    n7k-1(config-track)# object 2
    n7k-1(config-track)# object 5
    n7k-1(config-track)# end
    n7k-1(config)# vpc domain 101
    n7k-1(config-vpc-domain)# track 101
    The other site is the same, just 100 instead of 101.
    We are not tracking port channel 101, not the member interfaces of this port channel as this is the peer link and apparently tracking upstream interfaces and the peer link is only necessary when you have ONE link and one module per switch.
    As the interfaces we are tracking are member ports of a vPC, is this a chicken and egg scenario when seeing if these 3 interfaces are up? or is line-protocol purely layer 1 - so that the vPC isnt downing these member ports at layer 2 when it sees a local vPC domain failure, so that the track fails?
    I see most people are monitoring upstream layer3 ports that connect back to a core? what about what we are doing monitoring upstream(the 3750's) & downstream layer2 (the other site) - that are part of the very vPC we are trying to protect?
    We wanted all 3 of these to be down, for example if the local M2 card failed, the keepalive would send the message to the remote peer to take over.
    What are the best practices here? Which objects should we be tracking? Should we also track the perr-link Port channel101?
    We saw minimal outages using this design. when reloading the M2 modules, usually 1 -3 pings lost between the laptops in the diff sites across the stretched vlan. Obviously no outages when breaking any link in a vPC
    Any wisdom would be greatly appreciated.
    Nick

    Nick,
    I was not talking about the mgmt0 interface. The vlan that you are testing will have a link blocked between the two 3750 port-channel if the root is on the nexus vPC pair.
    Logically your topology is like this:
        |                             |
        |   Nexus Pair          |
    3750-1-----------------------3750-2
    Since you have this triangle setup one of the links will be in blocking state for any vlan configured on these devices.
    When you are talking about vPC and L3 are you talking about L3 routing protocols or just intervaln routing.
    Intervlan routing is fine. Running L3 routing protocols over the peer-link and forming an adjaceny with an router upstream using L2 links is not recommended. Teh following link should give you an idea about what I am talking here:
    http://bradhedlund.com/2010/12/16/routing-over-nexus-7000-vpc-peer-link-yes-and-no/
    HSRP is fine.
    As mentioned tracking feature purpose is to avoid block hole of traffic. It completely depends on your network setup. Don't think you would be needing to track all the interfaces.
    JayaKrishna

  • Switching back to Domain-0 console

    I have been playing around with the xm console <domain> command to see if I can get an X11 or Windows desktop on my computer screen without going through vnc. I want to run a vm server on my laptop and be able to switch back and forth between Windows and Linux. ( I realize that this isn't the intent of the product but I thought it would be fun to try ).
    The two issues that I am having are
    1) it appears that the consoles are mapped to vnc. Is there a way to map these to something else so that I could get the Windows console or X11 desktop through the xm console command?
    2) once I connect to the xm console to a Linux machine, how do I disconnect from this console connection and go back to Domain-0? Do I need to install the xm package on this Linux instance to make this happen or is there a magic keystroke that can be interpreted as something for Domain-0 to switch the console back. The only way that I can see how to do this is reboot the guest vm which then releases the xm console.

    Q1:Is there a way to map these to something else so that I could get the Windows console or X11 desktop through the xm console command?
    A1:here are steps to get windwos/X11 desktop for guest OS.
    1)start your VM guest.
    2)ssh -X your_oracle_vm_server
    3)vncviewer :5900
    5900 is the first vnc server port default. you may try "vncviewer :5901" if you start other VM guest.
    Q2:once I connect to the xm console to a Linux machine, how do I disconnect from this console connection and go back to Domain-0?
    A2:Typing Control-] in the xm console will detach your screen from the guest console and go back to Domain-0.

  • What is the best way to connect a firewall cluster to a VPC domain

    Hi All
    Can anyone help me decide what is the best way to connect a firewall cluster to a VDC running in a pair of N7K's which is a VPC domain?  
    Can I configure a VLAN interface on each VDC and use HSRP?  I was planning on presenting one 10GB cable from each VDC to each firewall.  Would this work OK?  HSRP traffic will go across the VPC peer link correct?
    thanks all

    No, but the one caveat is vpc orphan ports. If the vpc link between the nexus switches fails for any reason, all the vpc ports on the vpc secondary switch will be forced down. So it's recommended to connect single port devices to the primary vpc switch so the connections stay up. But if you're ok with that, then I don't see any problems.
    You have a few options, one would be to run a separate link between your nexus switches for non-vpc vlans. These vlans would not be allowed over the vpc peer-link, or forwarded out vpc's.
    See here page 49 :
    http://www.cisco.com/c/dam/en/us/td/docs/switches/datacenter/sw/design/vpc_design/vpc_best_practices_design_guide.pdf

  • Firewall Connections to vPC Domain

    Hi all,
    What is the best way to connect a Firewall cluster (Checkpoint FW cluster) to a vPC Domain ?
    Current Topology is like as below. We are gonna replace Cat6Ks with N7Ks.
    FW#1(Active)  ----- keepalive for amongt FWs -------- FW#2 (Standby)
         I                                                                               I
         I                                                                               I
         I                                                                               I
         I                 VLAN 100 HSRP on Cat6K Side               I
         I                                                                               I
         I                                                                               I
      Cat6K#2 -------------------peer keepalive------------------------------Cat6K#2
               --------------------- peer link-----------------------------------
    I know my options are :
    Connect the FWs to an edge switch which supports etherchannel and connects to vPC domain through that port channel.
    Connect the FWs through two ports (LACP config) to both N7Ks.
    Setup a seperate STP link between N7Ks, configure VLAN 100 on this link and then keep running HSRP on VLAN 100 on both N7ks on this non vPC VLAN.
    Setup the links between N7Ks and FWs as routed links and run a dynamic routing protocol in between.
    Thanks in advance.
    Dumlu

    Hello all,
    How about the option 1?
    Our scenario is as below:
                       DMZ switch ----- PC
                        |             |
                        |             |
                        |             |
                      FW         FW   (Checkpoint with VRRP connecting to N7k using VLAN 16)
                        |             |
                        L2 Switch
                        | |           | |
                    N7k-1 ----  N7k-2   (Peer Link Between N7k)
                        | |           | |
                        | |           | |
                       Inside switch ---- Server (VLAN16)
    When user ping from DMZ switch PC to Server in the Inside switch, the packet loss and long response time happen intermittently.
    But when we ping from Inside switch with another VLAN (VLAN12) to the server, it's okay. VLAN12 and VLAN16's  gateway are on N7k with HSRP.
    So N7k's inter-vlan routing seems to be okay, but through FW has problem.
    L2 switch and Inside switch connect to N7k with vPC. ALL the PC/Server are in VLAN 16 and their default gateway is to N7k.
    When user ping from inside to DMZ we can see a icmp redirect message, and I don't know whether it could be the problem to cause the intermittent packet loss?
    Thanks.
    Peter

  • Can I disable spanning-tree in a vpc domain ?

    i have two N7718s in a vpc domain and each have a vpc connection to  300+ TORs(non cisco switch).
    each 7718 have 300+ trunk port and a trunk port carring 80 vlans . so the logical port number is 300*80 = 24000
    the problem is n7k r-pvst logical ports limit is 16000,it causes the vpc primary 7718 ping latancy time exceed 1000ms
    2 ways to solve this problem : use mst instead of rpvst or disable spanning-tree
    if i use mst , the logical ports limit is 90000, the problem will appear one day
    so i want to disable spanning-tree . 7718s' vpc link to TOR use lacp ,it will prevent some  layer2 loops. can i do it?

    I have the same problem. :)

  • "Peer-switch" command on vPC domain and spanning-tree priority interaction

    Hi guy,
    We have 2 N7K (N7KA and N7KB) which will be running vPC in hybird and pure vPC environment.
    I have a question about the Hybird and pure vPC environment. With the "peer-switch" command enable, should i tune the spanning-tree priority to be the same for all the vlan running on vPC on both N7KA and N7KB? This way, when i enter the "sh spanning-tree vlan X(vPC vlan) detail" command on N7K, it will list both N7K announc itself as "We are the root of the spanning tree".Also the switch running spanning-tree with N7K vPC vlan (Hybird), will see both N7K has the same priority (4096), and it is not desirable for a spanning-tree environment. Therefore, i used the "spanning-tree pseudo-information" on N7KB to tune the spanning-tree priority to "8192" and the switch running spanning-tree with N7K will list N7KB has a priority of 8192(perfect).
    However, I notice some strange "show" output on the switch running Port-channel with the N7KA and N7KB. The "Designated bridge" priority is flapping as show on the switch. It is constantly changing between "4096 and 8192" with the same vPC system wide mac address.
    Entering the "sh spanning-tree vlan X detail" command repeatly on switch with port-channel toward N7KA and N7KB.
    >>sh spanning-tree vlan 10 detail
    Port 65 (Port-channel1) of VLAN10 is root forwarding
    Port path cost 3, Port priority 128, Port Identifier 128.65.
    Designated root has priority 4106, address 0013.05ee.bac8
    Designated bridge has priority 4106, address 0013.05ee.bac8
    Designated port id is 144.2999, designated path cost 0
    Timers: message age 15, forward delay 0, hold 0
    Number of transitions to forwarding state: 1
    Link type is point-to-point by default
    BPDU: sent 5, received 603
    one sec later.
    >>sh spanning-tree vlan 10 detail
    Port 65 (Port-channel1) of VLAN10 is root forwarding Port path cost 3, Port priority 128, Port Identifier 128.65. Designated root has priority 4106, address 0013.05ee.bac8 Designated bridge has priority 8202, address 0013.05ee.bac8 Designated port id is 144.2999, designated path cost 0 Timers: message age 15, forward delay 0, hold 0 Number of transitions to forwarding state: 1 Link type is point-to-point by default BPDU: sent 5, received 603
    Configuration:
    N7KA
    spanning-tree vlan 1-10 priority 4096
    vpc domain 200
    peer-switch
    N7KB
    spanning-tree vlan 1-10 priority 4096spanning-tree pseudo-information vlan 1-10 designated priority 8192
    vpc domain 200
    peer-switch

    We have a issue similar to this in our environment. I am trying to upgrade the existing 3750 stack router with 2 Nexus 5596 running VPC between them. For the transition I have planned to create a channel between 3750 stack and 5596's. Once this environment is set, my plan is to migrate all the access switches to N5k.
    The issue is when I connect the 3750 port channel to both N5Ks, all the Vlans on 3750 started to flap. If I connect the port channel to only one N5K everything is normal; but when I connect the port channel to both N5K running VPC, vlans are flapping. Any idea what is going wrong here? Am I missing something?

  • Qustion have 2 nexus 3000 setup in vPC domain, want to add the 3rd one, How is this done?

    If some can supply a sample of what the 3rd Nexus Switch configuration.
    Nexus-1 Config
    vrf context management
    ip route 0.0.0.0/0 192.168.50.1
    vlan 1
    vlan 10
    name ISCSI_VLAN
    vpc domain 1
    peer-keepalive destination 192.168.50.242
    peer-gateway
    interface port-channel1
    switchport mode trunk
    switchport trunk allowed vlan 10
    spanning-tree port type network
    flowcontrol receive on
    flowcontrol send on
    no negotiate auto
    vpc peer-link
    interface Ethernet1/52/1 - 4
    switchport mode trunk
    switchport trunk allowed vlan 10
    flowcontrol receive on
    flowcontrol send on
    channel-group 1 mode active
    Nexsus-2 Config
    vrf context management
    vlan 1
    vlan 10
    name ISCSI_VLAN
    no vpc domain 1
    peer-keepalive destination 192.168.50.241
    peer-gateway
    vpc domain 1
    peer-keepalive destination 192.168.50.241 vrf management
    peer-gateway
    interface port-channel1
    switchport mode trunk
    switchport trunk allowed vlan 10
    spanning-tree port type network
    flowcontrol receive on
    flowcontrol send on
    no negotiate auto
    vpc peer-link
    interface Ethernet1/51/1 - 4
    switchport mode trunk
    switchport trunk allowed vlan 10
    flowcontrol receive on
    flowcontrol send on
    channel-group 1 mode active

    VPC can only have 2 members
    HTH

  • IGMP Querier in VPC domain

    Hi
    I am a little confuse with igmp querier on Nexus 5548UP
    I have the following configuration on both VPC peers:
    vlan configuration 208
      ip igmp snooping version 2
      ip igmp snooping querier 192.168.201.104
    But the question is shell I use the same IP for querier on both VPC peers  or diffrent ?
    Regs
    Marcin

    It doesn't matter. It's only a source address and nobody will send anything to that address. IGMP joins are sent to multicast address 224.1.1.1 .

  • Files in MobileME needed back in domain.sites2 file to update in iweb

    I have the files in the mobileme - MANY files but not one named domain.sites2 - How do I repackage to have them work in iweb again?

    If you didn't find your domain file in the your User/Library/Application Support/iWeb folder the download Find File to search for domain.sites2 on your HD. It will search areas that Spotlight can't.
    If you don't have that domain file any more and don't have a backup (read Time Machine or the like) you'll have to rebuild the site from scratch. Chapter 2.3 of tje iWeb FAQ.org site has tips on using some of the published files in the reconstruction of the site.
    OT

  • VPC N5k Switch Failure causes connectivity disruption

    Hello,
    I have configured enhanced vPC on 2 n5k and B22 FEXs (vPC from 5k to B22, and vPC from B22 to blade servers).
    Everything is running smoothly, except when I power off one of the 5k, the connectivity to the blade servers is lost, comes back up for a short while, loses connectivity again, and after a few minutes comes back up for good.
    From the logs I can see that all the port-channels (peer-link, to the FEXs and port-channels to other switches in network) get in down state, then fizical interfaces start coming backup in fabric mode, then port-channels, see FEXs starting to get online then all the port-channels go down again and then whole thing starts again.(all of this is hapenig with one of the 5k powerd off, same thing hapens with promary and secondary vpc).
    Connectivity is lost in the same way when the 5k is started again, but just once.
    I am running NX OS version 5.2.1N1.3.
    I have no idea what could cause this behavior.
    Any help would be appreciated.
    Regards,
    Bogdan

    Hi Reza,
    Below you can find my run-config.
    version 5.2(1)N1(3)
    feature fcoe
    install feature-set virtualization
    feature-set virtualization
    logging level feature-mgr 0
    hostname N5k_1
    feature npiv
    feature telnet
    cfs eth distribute
    feature udld
    feature interface-vlan
    feature lacp
    feature vpc
    feature lldp
    feature vtp
    feature fex
    fex 107
      pinning max-links 1
      description "FEX0107"
      fcoe
    fex 108
      pinning max-links 1
      description "FEX0108"
    slot 1
      port 31-32 type fc
    vpc domain 1
      role priority 1000
      peer-keepalive destination 1.1.1.2
      auto-recovery
    vsan database
      vsan 50 name "VSAN_A"
    fcdomain fcid database
    interface port-channel100
      description Po Synch N5k
      switchport mode trunk
      spanning-tree port type network
      logging event port link-status
      logging event port trunk-status
      speed 10000
      vpc peer-link
    interface port-channel107
      switchport mode fex-fabric
      fex associate 107
      vpc 107
    interface port-channel108
      switchport mode fex-fabric
      fex associate 108
      vpc 108
    interface port-channel111
      switchport mode trunk
    interface vfc111
      bind interface Ethernet107/1/1
      no shutdown
    vsan database
      vsan 50 interface vfc111
      vsan 50 interface fc1/31
      vsan 50 interface fc1/32
    interface fc1/31
      no shutdown
    interface fc1/32
      no shutdown
    interface Ethernet1/1
      description Synch N5k
      switchport mode trunk
      logging event port link-status
      logging event port trunk-status
      udld aggressive
      channel-group 100 mode active
    interface Ethernet1/2
      description Synch N5k
      switchport mode trunk
      logging event port link-status
      logging event port trunk-status
      udld aggressive
      channel-group 100 mode active
    interface Ethernet1/3
      description Synch N5k
      switchport mode trunk
      logging event port link-status
      logging event port trunk-status
      udld aggressive
      channel-group 100 mode active
    interface Ethernet1/4
      description Synch N5k
      switchport mode trunk
      logging event port link-status
      logging event port trunk-status
      udld aggressive
      channel-group 100 mode active
    interface Ethernet1/5
      switchport mode fex-fabric
      fex associate 107
      channel-group 107
    interface Ethernet1/6
      switchport mode fex-fabric
      fex associate 107
      channel-group 107
    interface Ethernet1/7
      switchport mode fex-fabric
      fex associate 108
      channel-group 108
    interface Ethernet1/8
      switchport mode fex-fabric
      fex associate 108
      channel-group 108
    interface Ethernet107/1/1
      switchport mode trunk
      channel-group 111 mode active
    interface Ethernet108/1/1
      switchport mode trunk
      channel-group 111 mode active
    version 5.2(1)N1(3)
    feature fcoe
    install feature-set virtualization
    feature-set virtualization
    logging level feature-mgr 0
    hostname N5k_2
    feature npiv
    feature telnet
    cfs eth distribute
    feature udld
    feature interface-vlan
    feature lacp
    feature vpc
    feature lldp
    feature vtp
    feature fex
    fex 107
      pinning max-links 1
      description "FEX0107"
    fex 108
      pinning max-links 1
      description "FEX0108"
      fcoe
    slot 1
      port 31-32 type fc
    vpc domain 1
      role priority 1000
      peer-keepalive destination 1.1.1.1
      auto-recovery
    vsan database
      vsan 51 name "VSAN_B"
    fcdomain fcid database
    interface port-channel100
      description Po Synch N5k
      switchport mode trunk
      spanning-tree port type network
      logging event port link-status
      logging event port trunk-status
      speed 10000
      vpc peer-link
    interface port-channel107
      switchport mode fex-fabric
      fex associate 107
      vpc 107
    interface port-channel108
      switchport mode fex-fabric
      fex associate 108
      vpc 108
    interface port-channel111
      switchport mode trunk
    interface vfc111
      bind interface Ethernet108/1/1
      no shutdown
    vsan database
      vsan 51 interface vfc111
      vsan 51 interface fc1/31
      vsan 51 interface fc1/32
    interface fc1/31
      no shutdown
    interface fc1/32
      no shutdown
    interface Ethernet1/1
      description Synch N5k
      switchport mode trunk
      logging event port link-status
      logging event port trunk-status
      udld aggressive
      channel-group 100 mode active
    interface Ethernet1/2
      description Synch N5k
      switchport mode trunk
      logging event port link-status
      logging event port trunk-status
      udld aggressive
      channel-group 100 mode active
    interface Ethernet1/3
      description Synch N5k
      switchport mode trunk
      logging event port link-status
      logging event port trunk-status
      udld aggressive
      channel-group 100 mode active
    interface Ethernet1/4
      description Synch N5k
      switchport mode trunk
      logging event port link-status
      logging event port trunk-status
      udld aggressive
      channel-group 100 mode active
    interface Ethernet1/5
      switchport mode fex-fabric
      fex associate 107
      channel-group 107
    interface Ethernet1/6
      switchport mode fex-fabric
      fex associate 107
      channel-group 107
    interface Ethernet1/7
      switchport mode fex-fabric
      fex associate 108
      channel-group 108
    interface Ethernet1/8
      switchport mode fex-fabric
      fex associate 108
      channel-group 108
    interface Ethernet107/1/1
      switchport mode trunk
      channel-group 111 mode active
    interface Ethernet108/1/1
      switchport mode trunk
      channel-group 111 mode active

  • VPC failover

    We have 2 nexus 5548UP running Layer 2.  I have setup vPC between both nexus switches and have a few questions on how vPC failover works.  Below is partial snippets of what I have configured for vPC,etc. During testing, I reload nx-1 and what I am seeing is that the vPC peer-link goes down as expected but all vpc port-channels are in a failed state on nx-2 until nx-1 is back online once vPC is formed and functioning everything looks good. I then reload nx-2 and the vPC port-channels go into fail state, etc. Am I missing something in my configuration?
    NX-1
    feature tacacs+
    feature udld
    feature lacp
    feature vpc
    feature fex
    feature vtp
    cfs eth distribute
    no ip domain-lookup
    vtp mode transparent
    vpc domain 300
    role priority 200
    peer-keepalive destination 10.100.5.12 source 10.100.5.11
    int po12
    description NETAPPA
    switchport mode trunk
    vpc 12
    switchport trunk allowed vlan 5,6,7,8
    spanning-tree port type edge trunk
    int po13
    description NETAPPB
    switchport mode trunk
    vpc 13
    switchport trunk allowed vlan 5,6,7,8
    spanning-tree port type edge trunk
    int po250
    description nx-2
    switchport mode trunk
    vpc peer-link
    switchport trunk native vlan 999
    switchport trunk allowed vlan 5,6,7,8
    spanning-tree port type network
    int e1/27
    description vpc peer link
    switchport mode trunk
    switchport trunk native vlan 999
    switchport trunk allowed vlan 5,6,7,8
    channel-group 250 mode active
    int e1/28
    description vpc peer link
    switchport mode trunk
    switchport trunk native vlan 999
    switchport trunk allowed vlan 5,6,7,8
    channel-group 250 mode active
    int e1/1
    description netappa:e1a
    switchport mode trunk
    switchport trunk native vlan 999
    switchport trunk allowed vlan 5,6,7,8
    channel-group 12 mode active
    int e1/2
    description netappb:e1a
    switchport mode trunk
    switchport trunk native vlan 999
    switchport trunk allowed vlan 5,6,7,8
    channel-group 13 mode active
    NX-2
    feature tacacs+
    feature udld
    feature lacp
    feature vpc
    feature fex
    feature vtp
    cfs eth distribute
    no ip domain-lookup
    vtp mode transparent
    vpc domain 300
    role priority 300
    peer-keepalive destination 10.100.5.11 source 10.100.5.12
    int po12
    description NETAPPA
    switchport mode trunk
    vpc 12
    switchport trunk allowed vlan 5,6,7,8
    spanning-tree port type edge trunk
    int po13
    description NETAPPB
    switchport mode trunk
    vpc 13
    switchport trunk allowed vlan 5,6,7,8
    spanning-tree port type edge trunk
    int po250
    description nx-1
    switchport mode trunk
    vpc peer-link
    switchport trunk native vlan 999
    switchport trunk allowed vlan 5,6,7,8
    spanning-tree port type network
    int e1/27
    description vpc peer link
    switchport mode trunk
    switchport trunk native vlan 999
    switchport trunk allowed vlan 5,6,7,8
    channel-group 250 mode active
    int e1/28
    description vpc peer link
    switchport mode trunk
    switchport trunk native vlan 999
    switchport trunk allowed vlan 5,6,7,8
    channel-group 250 mode active
    int e1/1
    description netappa:e1b
    switchport mode trunk
    switchport trunk native vlan 999
    switchport trunk allowed vlan 5,6,7,8
    channel-group 12 mode active
    int e1/2
    description netappb:e1b
    switchport mode trunk
    switchport trunk native vlan 999
    switchport trunk allowed vlan 5,6,7,8
    channel-group 13 mode active

    What Nexus OS are you running.. You could be seeing this bug
    http://tools.cisco.com/Support/BugToolKit/search/getBugDetails.do?method=fetchBugDetails&bugId=CSCtw76636

  • Nexus 7000 - Moving vPC keep alive

    We have two Nexus 7010 switches running a vPC domain between the two switches.  On one of the 7010B, the peer keep alive (from the mgmt VRF) is connected to a 3560B *and* that 3560B also has a data connection back to the same 7010B.  Everything is fine with that setup.
    Our second 7010A, the peer keep alive link is also connected to a coresponding 3560A switch.  However, that 3560A switch is not connected to 7010A.
    I want to move the uplink from the 3560A from where it is to the 7010A which will break the keep alive.  However, I will not be breaking the vPC peer link as it is a pair of 10G connections between the two 7010 switches.
    I have read that the vPC won't come up unless the peer keep alive is present, but it wasn't clear about taking down the keep alive link momentarily.  Moving the cable would be quick, but I know the mac table will need to update since 7010B switch will now see the keep alive across it's peer link instead of some other direction.
    Can I take the peer keep alive link down providing the peer link stays up?
    We are running kickstart and system version 5.0(3).
    Thanks!
    /alan

    Peer keepalive works on UDP port 3200 over IP with 1 sec interval and 5 sec timeout.
    Iit is not requirement to have peer-keepalive destination IP in same subnet but if you do not have it in same subnet then you need to make sure you route it properly and your IP routed infrastructure that carries keeplive satisfies above requirement to make sure not a single event cause on that IP infrastructure causes keeplives to loose packets since peer-keepalive is UDP it is not reliable delivery method.
    Recommendation in past i heard was to use your managemet ports as peer-keepalive. But one problem happens during ISSU with dual sup, the each supervisor reboots and after it comes up role of active and standby gets switch at the end. So If you did not connect two managment ports(one from each supervisor) to your management network then you will loose keepalives during software upgrade because supervisor switch over occurs and new maangement port becomes active.
    So second recomendation is to create one peer-keepalive vrf so that it will have its own address space, if you have M1 1 gig card in each switch then connect one cable between switch and assign IP address (like 1.1.1.1-2/30) and put it in peer-keepalive vrf. With this set up during ISSU you do not loose peer keepalives because line cards does not need to reboot and your peer-keepalive UDP traffic will not depend on any other switch or router.    

  • 5548 Config Sync issue - Suspended by vPC

    I have 2 UCS 6120 fabric interconnects which both have VPCs to 2 x 5548s.  First fabric interconnect uses Po260 & vPC 260 and second fabric interconnect uses Po261 & vPC 261.  I used config sync to add "spanning-tree port type edge trunk" to int po 260 & 261.  The commit worked properly, peers are in sync, etc.  The problem is when I committed the command, int po 260 & 261 on the secondary 5548 went into "suspended by vPC".  I can't figure out why they did this, the configurations are the same and all vPC consistency checks pass.  To fix the issue, all I had to do was bounce the port-channel on the secondary 5548 (shut/no shut) after which it came back online.  I only did this to Po260 so Po261 is still down so that I can troubleshoot further.  Please see below:
    vPC domain id                     : 70 
    Peer status                       : peer adjacency formed ok     
    vPC keep-alive status             : peer is alive                
    Configuration consistency status  : success
    Per-vlan consistency status       : success                      
    Type-2 consistency status         : success
    vPC role                          : secondary                    
    Number of vPCs configured         : 7  
    Peer Gateway                      : Disabled
    Dual-active excluded VLANs        : -
    Graceful Consistency Check        : Enabled
    Auto-recovery status              : Disabled
    vPC Peer-link status
    id   Port   Status Active vlans   
    1    Po255  up     1,10-13,26-29,151-156,180-181,200,318,331,399-417,       
                       419-422,424-431,433-436,438-443,446-448,450,452-45       
                       3,455-458,460-465,467-471,480-494,498-499,503,602-       
                       633,644-657,659,663-664,698-701,800,805,850-851,89       
                       0-891,899-904,906,908,912-950,952-958,975,987-988,    ....
    vPC status
    id     Port        Status Consistency Reason                     Active vlans
    171    Po171       up     success     success                    1,10-13,26-
                                                                     29,151-156,
                                                                     180-181,200
                                                                     ,318,331,39
                                                                     9-417,41....
    260    Po260       up     success     success                    10-13,26-29
                                                                     ,663-664,89
                                                                     0-891      
    261    Po261       down*  success     success                    -
    sh int po 261
    port-channel261 is down (suspended by vpc)
    Any help would be appreciated

    Yes, I did check that and all parameters match as follows:
    5548-2# sh vpc consistency-parameters int po 261
        Legend:
            Type 1 : vPC will be suspended in case of mismatch
    Name                        Type  Local Value            Peer Value            
    Shut Lan                    1     No                     No                   
    STP Port Type               1     Edge Trunk Port        Edge Trunk Port      
    STP Port Guard              1     None                   None                 
    STP MST Simulate PVST       1     Default                Default              
    lag-id                      1     [(7f9b,                [(7f9b,              
                                      0-23-4-ee-be-46, 8105, 0-23-4-ee-be-46, 8105,
                                       0, 0), (8000,          0, 0), (8000,       
                                      0-5-73-d4-d5-fc, 1, 0, 0-5-73-d4-d5-fc, 1, 0,
                                       0)]                    0)]                 
    mode                        1     active                 active               
    Speed                       1     10 Gb/s                10 Gb/s              
    Duplex                      1     full                   full                 
    Port Mode                   1     trunk                  trunk                
    Native Vlan                 1     10                     10                   
    MTU                         1     1500                   1500                 
    Admin port mode             1                                                 
    Allowed VLANs               -     10-13,26-29,663-664,89 10-13,26-29,663-664,89
                                      0-891                  0-891                
    Local suspended VLANs       -     -                      -      

  • Routing issue in Nexus 7009 due to vPC or hsrp

    we have two site's, on first site we have two nexus 7009 switches (Nexus A  & Nexus B)  and other site is remote site having two 6500 switches. (design attached)
    we are using hsrp on nexus switches and Active is Nexus A for all vlan’s 
    From one of my remote site user's (user's are in vlan 30 ) are not able to communicate with  nexus site vlan 20 specially if host in vlan 20 take forwarding path from nexus switch B,
    I can ping the vlan 20 both physical address's and gateway (vlan 20 configured in both nexus switch and using HSRP) from vlan 30 which configured on remote site 6500 switch
    ospf with area 0 is the  routing protocol running between both site.
    vlan 10 we are using as a management  vlan on both nexus switch  that building neighbore ship with WAN router, it's means wan router have two neighbors nexus A and nexus B, but nexus B building the neigbhorship via a Nexus A because from WAN router we have single link which is terminated on Nexus A,
    there is one layer 2 switch between nexus A and WAN router, nexus A site that switch port in vPC because we are planning to pull second link later to nexus B.
    All user's are connected with edge switch and edge switch have a redundant uplink to nexus A and B with vPC configured
    After troubleshooting we observe that if user in vlan 20 wants to communicate with vlan 30 (remote site), traffic is taking Nexus B is forwarding path, then gets drops.
    I run the tracert from pc its showing route till SVI on Nexus B  after that seems packets not finding route.  Even vlan 30 routes are available in the routing table of Nexus B. we don’t have any access-list and Firewall between this path.

    Hi,
    I suspect in your scenario that traffic is being dropped due to the characteristics of vPC, the routing table on Nexus-B may reflect the next-hop address for the destination IP, however if that next-hop address is the address of the Nexus-A off of VLAN 20 then it will be forwarded across the vPC peer-link, this breaks the convention.
    When you attach a Layer 3 device to a vPC domain, the peering of routing protocols using a VLAN also carried on the vPC peer-link is not supported. If routing protocol adjacencies are needed between vPC peer devices and a generic Layer 3 device, you must use physical routed interfaces for the interconnection.
    You can configure VLAN Interfaces for Layer 3 connectivity on the vPC peer devices to link to Layer 3 of the network for such applications as HSRP and PIM. However, Cisco recommend that you configure a separate Layer 3 link for routing from the vPC peer devices, rather than using a VLAN network interface for this purpose.
    Take a look at the following URL, this article helps to explain the characteristics of vPC and routing over the peer-link:
    http://bradhedlund.com/2010/12/16/routing-over-nexus-7000-vpc-peer-link-yes-and-no/
    Regards
    Allan.
    Hope you find this is helpful.
    Sent from Cisco Technical Support iPad App

Maybe you are looking for

  • How can i get acrobat to work with windows vista

    How can I get adobeexportpdf to work with windows vista?

  • ERROR  IDOC STATUS 51

    Hi all I am trying to post MATMAS IDOCS into R/3 system. The IDOCS are in destination R/3 in following status: IDOC STATUS 51 cAN SOMEONE SUGGEST ME A WAY TO SOLVE THIS PROBLEM. Thankyou

  • What is going on with Adobe Flash Player?

    All of a sudden videos are either disappearing from my website, leaving just blank boxes or grey boxes with error messages, saying that Flash player has crashed, and added videos don't appear?????? I've been reading all the forums to find out what's

  • Possible simple fix for itunes store issues

    Was having the same problem with my Macbook Pro and iPhone 3gs. Could not login/download app updates etc. despite having the same account for several years. Here's what worked for me...... Go to the apple store (not iTunes store) and log in to your a

  • SOA 11g in Production Mode

    Hi Everybody I am going to install SOA Suit 11g for a production mode, any helpfull tutorial or steps?? I googled the net, I didn't find , all are talking about development mode.. Any help please??