Nexus 5K OSPF with vPC

Hi,
I know it is well documented using IGP's, more specifically OSPF with 7K's and vPC's but when it comes to the same thing on 5K's I am still a little confused.
My topology is:
5K01 and 5K02 are connected and are vPC peers, I currently have a management network on VLAN 114, both 5k's have SVI's on this and are currently OSPF neighbors over their vPC using this vlan.
I have an MPLS router (service provider PE) which is 2 routers but clustered so logically in this instance it is one router, the 5 k's will be conecting to this PE router via some switches over a vPC and needs to become a OSPF neighbor to both the 5K's.
Looking at this post:
http://adamraffe.com/2013/03/08/l3-over-vpc-nexus-7000-vs-5000/
It suggests that I can just add VLAN 114 to the vPC up to tyhe PE and turn OSPF on on the interface on the PE, although this will not support Multicast and I don't really want to restrict myself as this may be a future requirement.
What I thought might be a better solution would be to designate a new vlan and allow it on the vPC up to the PE and use that for the OSPF neighborships between the 5K's and the PE and not allowing it over the vPC peer link - leaving the 5K's neighborship over vlan 114.
Can someone tell me what the best practice/supported topology is here and maybe provide some cisco links?
Thanks a lot in advance.

You have to be very careful when configuring L3 services and interfaces while using VPC.
Take a look at this document:
http://www.cisco.com/c/dam/en/us/td/docs/switches/datacenter/sw/design/vpc_design/vpc_best_practices_design_guide.pdf
Also, take a look at this post:
http://bradhedlund.com/2010/12/16/routing-over-nexus-7000-vpc-peer-link-yes-and-no/
You can create a vlan used exclusively for Nexus-to-Nexus iBGP peering. Use a new 'access' link between the two switches and place them on the new vlan. Make sure that this VLAN does not traverse the VPC peer link. Then, create SVIs on each switch for that VLAN and peer over that link. Then, you can create a L3 link on each nexus to peer with your eBGP neighbors.
The point you want to make sure you understand is the VPC loop prevention mechanism that says "If a packet is received on a VPC port, traverses the VPC peer link, it is not allowed to egress on a VPC port."

Similar Messages

Nexus 7000 with VPC and HSRP Configuration

Hi Guys,
I would like to know how to implement HSRP with the following setup:
There are 2 Nexus 7000 connected with VPC Peer link. Each of the Nexus 7000 has a FEX attached to it.
The server has two connections going to the FEX on each Nexus 7k (VPC). FEX's are not dual homed as far as I now they are not supported currently.
R(A)              R(S)
|                     |
7K Peer Link 7K
|                     |
FEX              FEX
Server connected to both FEX
The question is we have two routers connected to each of the Nexus 7k in HSRP (active and one is standby). How can I configure HSRP on the nexus switches and how the traffic will routed from the Standby Nexus switch to Active Nexus switch (I know HSRP works differently here as both of them can forward packets). Will the traffic go to the secondary switch and then via the peer link to the active switch and then to the active router ? (From what I read the packet from end hosts which will go via the peer link will get dropped)
Has anyone implemented this before ?
Thanks

Hi Kuldeep,
If you intend to put those routers on a non-vpc vlan, you may create a new inter-switch trunk between the N7K and allow that non-vpc vlan . However if those will be on a VPC vlan, best to create two links to the N7K pair and create a VPC, otherwise configure those ports as orphan ports which will leverage the VPC peer link .
HTH
Jay Ocampo

Ask the Expert: Different Flavors and Design with vPC on Cisco Nexus 5000 Series Switches

Welcome to the Cisco® Support Community Ask the Expert conversation. This is an opportunity to learn and ask questions about Cisco® NX-OS.
The biggest limitation to a classic port channel communication is that the port channel operates only between two devices. To overcome this limitation, Cisco NX-OS has a technology called virtual port channel (vPC). A pair of switches acting as a vPC peer endpoint looks like a single logical entity to port channel attached devices. The two devices that act as the logical port channel endpoint are actually two separate devices. This setup has the benefits of hardware redundancy combined with the benefits offered by a port channel, for example, loop management.
vPC technology is the main factor for success of Cisco Nexus® data center switches such as the Cisco Nexus 5000 Series, Nexus 7000 Series, and Nexus 2000 Series Switches.
This event is focused on discussing all possible types of vPC along-with best practices, failure scenarios, Cisco Technical Assistance Center (TAC) recommendations and troubleshooting
Vishal Mehta is a customer support engineer for the Cisco Data Center Server Virtualization Technical Assistance Center (TAC) team based in San Jose, California. He has been working in TAC for the past 3 years with a primary focus on data center technologies, such as the Cisco Nexus 5000 Series Switches, Cisco Unified Computing System™ (Cisco UCS®), Cisco Nexus 1000V Switch, and virtualization. He presented at Cisco Live in Orlando 2013 and will present at Cisco Live Milan 2014 (BRKCOM-3003, BRKDCT-3444, and LABDCT-2333). He holds a master’s degree from Rutgers University in electrical and computer engineering and has CCIE® certification (number 37139) in routing and switching, and service provider.
Nimit Pathak is a customer support engineer for the Cisco Data Center Server Virtualization TAC team based in San Jose, California, with primary focus on data center technologies, such as Cisco UCS, the Cisco Nexus 1000v Switch, and virtualization. Nimit holds a master's degree in electrical engineering from Bridgeport University, has CCNA® and CCNP® Nimit is also working on a Cisco data center CCIE® certification While also pursuing an MBA degree from Santa Clara University.
Remember to use the rating system to let Vishal and Nimit know if you have received an adequate response.
Because of the volume expected during this event, Vishal and Nimit might not be able to answer every question. Remember that you can continue the conversation in the Network Infrastructure Community, under the subcommunity LAN, Switching & Routing, shortly after the event. This event lasts through August 29, 2014. Visit this forum often to view responses to your questions and the questions of other Cisco Support Community members.

Hello Gustavo
Please see my responses to your questions:
Yes almost all routing protocols use Multicast to establish adjacencies. We are dealing with two different type of traffic –Control Plane and Data Plane.
Control Plane: To establish Routing adjacency, the first packet (hello) is punted to CPU. So in the case of triangle routed VPC topology as specified on the Operations Guide Link, multicast for routing adjacencies will work. The hellos packets will be exchanged across all 3 routers and adjacency will be formed over VPC links
http://www.cisco.com/c/en/us/td/docs/switches/datacenter/nexus5000/sw/operations/n5k_L3_w_vpc_5500platform.html#wp999181
Now for Data Plane we have two types of traffic – Unicast and Multicast.
The Unicast traffic will not have any forwarding issues, but because the Layer 3 ECMP and port channel run independent hash calculations there is a possibility that when the Layer 3 ECMP chooses N5k-1 as the Layer 3 next hop for a destination address while the port channel hashing chooses the physical link toward N5k-2. In this scenario,N5k-2 receives packets from R with the N5k-1 MAC as the destination MAC.
Sending traffic over the peer-link to the correct gateway is acceptable for data forwarding, but it is suboptimal because it makes traffic cross the peer link when the traffic could be routed directly.
For that topology, Multicast Traffic might have complete traffic loss due to the fact that when a PIM router is connected to Cisco Nexus 5500 Platform switches in a vPC topology, the PIM join messages are received only by one switch. The multicast data might be received by the other switch.
The Loop avoidance works little different across Nexus 5000 and Nexus 7000.
Similarity: For both products, loop avoidance is possible due to VSL bit
The VSL bit is set in the DBUS header internal to the Nexus.
It is not something that is set in the ethernet packet that can be identified. The VSL bit is set on the port asic for the port used for the vPC peer link, so if you have Nexus A and Nexus B configured for vPC and a packet leaves Nexus A towards Nexus B, Nexus B will set the VSL bit on the ingress port ASIC. This is not something that would traverse the peer link.
This mechanism is used for loop prevention within the chassis.
The idea being that if the port came in the peer link from the vPC peer, the system makes the assumption that the vPC peer would have forwarded this packet out the vPC-enabled port-channels towards the end device, so the egress vpc interface's port-asic will filter the packet on egress.
Differences: In Nexus 5000 when it has to do L3-to-L2 lookup for forwarding traffic, the VSL bit is cleared and so the traffic is not dropped as compared to Nexus 7000 and Nexus 3000.
It still does loop prevention but the L3-to-L2 lookup is different in Nexus 5000 and Nexus 7000.
For more details please see below presentation:
https://supportforums.cisco.com/sites/default/files/session_14-_nexus.pdf
DCI Scenario: If 2 pairs are of Nexus 5000 then separation of L3/L2 links is not needed.
But in most scenarios I have seen pair of Nexus 5000 with pair of Nexus 7000 over DCI or 2 pairs of Nexus 7000 over DCI. If Nexus 7000 are used then L3 and L2 links are required for sure as mentioned on above presentation link.
Let us know if you have further questions.
Thanks,
Vishal

Nexus 7000 and 2000. Is FEX supported with vPC?

I know this was not supported a few months ago, curious if anything has changed?

Hi Jenny,
I think the answer will depend on what you mean by is FEX supported with vPC?
When connecting a FEX to the Nexus 7000 you're able to run vPC from the Host Interfaces of a pair of FEX to an end system running IEEE 802.1AX (802.3ad) Link Aggregation. This is shown is illustration 7 of the diagram shown on the post Nexus 7000 Fex Supported/Not Supported Topologies.
What you're not able to do is run vPC on the FEX Network Interface that connect up to the Nexus 7000 i.e., dual-homing the FEX to two Nexus 7000. This is shown in illustrations 8 and 9 of under the FEX topologies not supported on the same page.
There's some discussion on this in the forum post DualHoming 2248TP-E to N7K that explains why it's not supported, but essentially it offers no additional resilience.
From that post:
The view is that when connecting FEX to the Nexus 7000, dual-homing does not add any level of resilience to the design. A server with dual NIC can attach to two FEX so there is no need to connect the FEX to two parent switches. A server with only a single NIC can only attach to a single FEX, but given that FEX is supported by a fully redundant Nexus 7000 i.e., SE, fabrics, power, I/O modules etc., the availability is limited by the single FEX and so dual-homing does not increase availability.
Regards

Routing issue in Nexus 7009 due to vPC or hsrp

we have two site's, on first site we have two nexus 7009 switches (Nexus A & Nexus B) and other site is remote site having two 6500 switches. (design attached)
we are using hsrp on nexus switches and Active is Nexus A for all vlan’s
From one of my remote site user's (user's are in vlan 30 ) are not able to communicate with nexus site vlan 20 specially if host in vlan 20 take forwarding path from nexus switch B,
I can ping the vlan 20 both physical address's and gateway (vlan 20 configured in both nexus switch and using HSRP) from vlan 30 which configured on remote site 6500 switch
ospf with area 0 is the routing protocol running between both site.
vlan 10 we are using as a management vlan on both nexus switch that building neighbore ship with WAN router, it's means wan router have two neighbors nexus A and nexus B, but nexus B building the neigbhorship via a Nexus A because from WAN router we have single link which is terminated on Nexus A,
there is one layer 2 switch between nexus A and WAN router, nexus A site that switch port in vPC because we are planning to pull second link later to nexus B.
All user's are connected with edge switch and edge switch have a redundant uplink to nexus A and B with vPC configured
After troubleshooting we observe that if user in vlan 20 wants to communicate with vlan 30 (remote site), traffic is taking Nexus B is forwarding path, then gets drops.
I run the tracert from pc its showing route till SVI on Nexus B after that seems packets not finding route. Even vlan 30 routes are available in the routing table of Nexus B. we don’t have any access-list and Firewall between this path.

Hi,
I suspect in your scenario that traffic is being dropped due to the characteristics of vPC, the routing table on Nexus-B may reflect the next-hop address for the destination IP, however if that next-hop address is the address of the Nexus-A off of VLAN 20 then it will be forwarded across the vPC peer-link, this breaks the convention.
When you attach a Layer 3 device to a vPC domain, the peering of routing protocols using a VLAN also carried on the vPC peer-link is not supported. If routing protocol adjacencies are needed between vPC peer devices and a generic Layer 3 device, you must use physical routed interfaces for the interconnection.
You can configure VLAN Interfaces for Layer 3 connectivity on the vPC peer devices to link to Layer 3 of the network for such applications as HSRP and PIM. However, Cisco recommend that you configure a separate Layer 3 link for routing from the vPC peer devices, rather than using a VLAN network interface for this purpose.
Take a look at the following URL, this article helps to explain the characteristics of vPC and routing over the peer-link:
http://bradhedlund.com/2010/12/16/routing-over-nexus-7000-vpc-peer-link-yes-and-no/
Regards
Allan.
Hope you find this is helpful.
Sent from Cisco Technical Support iPad App

Question re. behaviour of single homed FEX with vPC

Hi Folks,
I have been looking at configuring Nexus 5Ks with FEX modules. Referring to the Cisco documentation;
http://www.cisco.com/en/US/docs/switches/datacenter/nexus5000/sw/layer2/513_n1_1/b_Cisco_n5k_layer2_config_gd_rel_513_N1_1_chapter_01001.html
In figure 3. showing a single homed FEX with vPC topology, I'm curious what happens if one of the 5Ks fail. For example if the 5K on the left hand side of the diagram fails do the ports on the attached FEX that the server is attached to drop? If not I would assume that the server has no way of knowing that there is no longer a valid path through those links and will continue to use them?
Many thanks in advance,
Shane.

Hello Shane.
Depending of type of the failureboth n5k can tace corrective actions and end host will always know that one of the port-channel members is down.
For example if one 5k will crash or will be reloaded - all connected fexes alre will go offline. FEX are not standalone switches and cannot work without "master" switch.
Also links which will go from fex to the end-host will be in vpc mode which means that all vpc redundancy features/advantages will be present.
HTH,
Alex

Unstable vMotion behavior over DCI with vPC?

hi out there
I need some ideas to track a problem - we have a DC running with a wmware esxi4.1 cluster (2 x 2 sets of blade-servers - one set at each site) with a DR site which is interconnected with 4 10G fiber where we have established 2 x 2 port-channels (Cisco Nexus 5k with vPC) between - 1 vpc portchannel with 2 10g connections for iSCSI and 1 vpc portchannel also with 2 10g connections for "non-iSCSI traffic" - eg the rest - we have seperated the iscsi traffic fully from the rest of the network. We have hereby a "simple" Layer 2 DC interconnection with a latencey between the sites of ~ 1mSec - and no erros reported by any of the involved devices. The iscsi consist of two EMC VNX 5500 controlleres - one at each site with a "local" san array.
My problem is that from time to time when we issue a vMotion or clone of a vm between the sites we get either an extrem slow response (will probably end in a timeout) or the operations fails with a timeout - could be "disk clone failed. Canceling Storage vMotion. Storage vMotion clone operation failed. Cause: Connection timed out"
Any suggestions to track this? It is a bit hard to track the network connections since it is 10 gig (haven't got any sniffer equipment yet which can catch up with a 10 gig interface). Could there be some buffer allocation problems on the nexus switches (no errors logged - any suggestions on which debug level?)
best regards /ti

Hi - we have a similar setup but where we use nx5k to service the DCI and VPC as solely L2 and then run the L3 on the NX7k. You need to have all the same vlans on the vpc as far as I know. You can't fool it - but you might be able to tricks something with some q-in-q trunks between the 2 sets of nx7k's
best regards /ti

Nexus 5548UP - HSRP and vPC, tracking required?

Hi,
We've got two Nexus 5548UPs that are vPC and HSRP peers.
I've had some feedback that I should incorporate the tracking function to close the vPC down in the case of a layer 3 problem, the thing is I'm not sure it's required. I can see in this article it recommends implementing tracking when your L2 peer-link and L3 interfaces are on the same module (which it is in my case).. http://www.cisco.com/en/US/docs/switches/datacenter/sw/design/vpc_design/vpc_best_practices_design_guide.pdf
But in this article it says not to use tracking.. http://www.cisco.com/en/US/prod/collateral/switches/ps9441/ps9670/design_guide_c07-625857.pdf
Any one got any real world experience and can offer some feedback.. I don't mind putting it in just want to understand why.
Thanks,
Nick.

Hi Nick
there is two tracking can be use din nexus enviroment
HSRP tracking and vPC tracking
for using one line card for the vPC peer link vPC tracking is recomnded
HSRP tracking is used to track L3 uplinks to the core
Using vPC with HSRP/VRRP object tracking may leads to traffic blackholing in case object tracking is triggered
its better to use separate L3 inter switch link instead of using HSRP tracking
hope this help

Nexus 5548UP lacp with IBM AIX P740

We are currently working in a lab to configure our first pair of 5548UP(with a 2232PP on each) with FCoE to an IBM p740 AIX server. On the Nexus side I have confugured the physical port e100/1/1 (2232pp) to be in a channel-group that will be part of a vpc with the other 5548UP switch, I am trunking all vlans in the etherchannel with the IBM server. Everything is working, i have the port-channel UP, the vfc and vpc when I use ''channel-group x mode ON'' on the physical ports and on the server side the 803.2ad is not active, it's configured ''roundRobin''.
Suppose I want to use LACP for the connections with the server should I configure the Nexus switch port with ''channel-group x mode active'' or ''channel-group x mode passive''? When the server is configured with ''802.3ad'' enabled and when I have ''channel-group x mode active'' or ''passive'' it doesn't work, the port-channel x does not come UP.
Does anyone have an explenation of how I should be configure?
Thanks

Hello,
Using LACP passive or active mode should not make much of a difference as long as one of the sides of the port-channel actively negotiates LACP. In most cases, we just configure both sides to be LACP active.
When you say the port-channel is not working when using LACP, what are the symptoms?
Are the physical ports in "I" state?'
'show port-channel summary' will show this
What does the output of "show lacp counters interface port-channel X" show?
This command will tell you if the Nexus 5K/2K is receiving LACPDUs from the IBM device (and if we are sending them). I would suggest to collect the LACP counters to verify that we are receiving the LACP packets from IBM, otherwise Nexus will not bring up the port-channel. Here is an example output from my lab switch:
5548-2019# show lacp counters interface port-channel 500
                    LACPDUs         Marker      Marker Response    LACPDUs
Port              Sent   Recv     Sent   Recv     Sent   Recv      Pkts Err
port-channel500
Ethernet1/15       16401 16399    0      0        0      0        0
Ethernet1/16       16400 16399    0      0        0      0        0
Regards,
Steven

Problems in using Windows Explorer with VPC Virtual PC?

Has anybody experienced problems in using Windows Explorer with VPC Virtual PC?
Lacking any "forbidden" or "appropriate usage" guidelines, I regularly use Windows Explorer (Windows 2000) to transfer file from the desktop. I have occasionally sensed that this might be wrong. Today I inadvertently clicked the MAC harddrive instead of the Desktop (within Windows Explorer) and caused all manner of mischief.
Any other views please?

Let me correct this:
I regularly use Windows Explorer (Windows 2000) to transfer files from the "Mac" desktop
Any ideas please Virtual PC VPC users?

GLBP with vPC configuration acceptable?

Hello,
I'm reposting this discussion here.
I have designed GLBP with vPC configured in a pair of N7K switches. But in Cisco documentation, the best practice configuration uses HSRP in vPC environment. Customer doesn't feel comfortable with GLBP since Cisco's best practice using HSRP. Is there any potential issue using GLBP in vPC environment?
Thanks,
Jason

Hi, there is no need for GLBP with vPC. Both HSRP peers are active.
http://www.netcraftsmen.net/component/content/article/69-data-center/1260.html
"...since both peers forward. This behavior also provides HSRP load-balancing without needing to switch to GLBP."
Don't forget to rate all posts that are helpful.

N7k as redundant core with vpc to 4510/3750 as distribution switch

Hi - basic question here
Got 2 qty N7k as redundant core with vpc to 4510 and 3750 as redundand distribution switch running MST. I got stuck with some bad cabling design from our IDF to Datacenter so have 2 access switch whereby each one will have a etherchannel to both distribution 4510 and 3750. My question is this is a doable design as I am not sure about the vpc upstream on how it effects etherchannel with MST for my distribution and access.
Thanks

vPC will be considered as one logical link by both upstream and downstream connected devices
the question here are you going to run L3 between the distribution and Core devices ? ( this is recommended design ) if yes, then you do not need to worry about MST and VPC if you going to have it L3 from distribution devices up to the Core
one thing to consider is the distribution switch in your design has big difference in terms of backplane throughput i mean between the 4500 and 3750 !
if you can have both as 4500 will be better and more consistent design
Good luck
if helpful Rate

Nexus 5596 reloads with redundant PS after 1 PS breaks down

Hi All,
We have a Nexus N5K-C5596UP with redundant power supply. 2 x N55-PAC-1100W.
Last week 1 of those PS broke down. When this happened the chassis reloaded. I wouldn't expect this to happen.
Does anyone have an idea of why this happened?
Our NX-OS version is n5000-uk9.5.2.1.N1.1a.
Thanks,
Joris

Joris, good day.
I suppose it's your case: http://www.cisco.com/c/en/us/support/docs/field-notices/638/fn63893.html
Best regards,
Zakhar Belyakov.

Peer-Switch with vPC and non-vPC Vlan Port-Channels

Hi,
in a design guide i have noticed that it is best practice to split vPC and non-vPC vlans on different inter-switch port-channels. Now, if i want to use the Peer-Switch function, but the port-channel interface of the non-vPC-vlan channel moves into blocking state. The option spanning-tree pseudo-information has no influence. Is peer-switch possible in my kind of topology?
Greeting,
Stephan

I believe absolutly possible. specifically coz peer-switch and spt pseudo-info are specific and local to cisco fabric services running as part of vpc technology. Personally me has lab with vpc-domain compounded of 2 N5Ks. They are peer-switches with spt-pseudoinfo and they have MST running on non VPC links independantly from vpc.

Nexus 1000v integeration with SCVMM

Hi fellows,
Can some one please guide to a good step by step document on integeration of Nexus 1000v (appliance) with SCVMM, and Windows hypervisor?

Hello,
It is not officially released and work is still in progress.
You can find links to demo of current pre-release software here
http://blogs.cisco.com/datacenter/nexus-1000v-provides-cloud-switching-to-new-windows-server-2012-platform/
Padma

Nexus 5K OSPF with vPC

Similar Messages

Maybe you are looking for