Nexus-switches issues no arp-requests.

Hallo all,
I see a very strange behavior on my two nexus switches.
Both are Nexus 5548 with L3-daughter-cards. Both do l2 and l3-switching, ACL-filtering and other things. Furthermore I have a set of servers connected to both switches in a vPC-setup. All in all I do nothing special.
After reloading the primary switch (vpc-primary, root-bridge for all vlans and hsrp-active with preemption for all SVIs) the switche comes back online and after getting up all links and reconverging everthing the network breaks. After a lot of debugging and curses and connection tries and a few additional gray hairs later I have got it to work by pinging all ip-addresses from the switch that I have previously rebooted.
Later I do some tests to find out what was going wrong. I found out that if I clear the arp-cache I will get the same issue. Pinging from server A in one subnet to server B in another subnet doesn't lead to success, because the switch issues no arp-requests. To make it work just ping server B from the switch and all works fine. The switch does arp, the arp-table is updated and the pings from the server A will reach the server B.
Any ideas?
Regards
Thomas
^^°-°^^

I can post a extraction of the relevant config items.
   - --[ vpc-primary
cfs01# sh run
!Command: show running-config
!Time: Wed Nov 13 08:46:18 2013
version 5.2(1)N1(1b)
cfs eth distribute
vrf context CEPH
vrf context management
ip route 172.31.0.0/20 172.31.8.190
vlan 14
name 172.31.50.0/26_CN/NN/OSDs@DMZ
vlan 4080
name 172.31.48.64/26_NAS.Infrastr@DMZ
spanning-tree vlan 1-129,131-3967,4048-4093 priority 0
udld aggressive
vpc domain 1
role priority 1
peer-keepalive destination 172.31.8.179 source 172.31.8.178
peer-config-check-bypass
delay restore 150
peer-gateway
auto-recovery
ip arp synchronize
interface Vlan14
no shutdown
mtu 9216
description CN/NN/OSDs@DMZ
ip access-group acl-vl14-in in
vrf member CEPH
no ip redirects
ip address 172.31.50.61/26
no ip port-unreachable
hsrp version 2
hsrp 3
    authentication md5 key-string 3-14
    preempt delay minimum 30 reload 60
    priority 255
    ip 172.31.50.62
interface Vlan4080
no shutdown
mtu 9216
description NAS.Infrastr@DMZ
ip access-group acl-vl4080-in in
vrf member CEPH
no ip redirects
ip address 172.31.48.125/26
no ip port-unreachable
hsrp version 2
hsrp 3
    authentication md5 key-string 3-4080
    preempt delay minimum 30 reload 60
    priority 255
    ip 172.31.48.126
interface port-channel7
switchport mode trunk
switchport trunk native vlan 991
spanning-tree port type network
speed 10000
vpc peer-link
interface port-channel100
switchport mode trunk
switchport trunk native vlan 991
switchport trunk allowed vlan 2,14-19,991,4078-4080
speed 10000
vpc 100
interface port-channel102
switchport mode trunk
switchport trunk native vlan 991
switchport trunk allowed vlan 2,14,18,991,1299-1400
speed 10000
vpc 102
interface Ethernet1/1
no cdp enable
switchport mode trunk
switchport trunk native vlan 991
switchport trunk allowed vlan 2,14-19,991,4078-4080
channel-group 100 mode active
interface Ethernet1/3
no cdp enable
switchport mode trunk
switchport trunk native vlan 991
switchport trunk allowed vlan 2,14,18,991,1299-1400
channel-group 102 mode active
interface Ethernet1/29
description cfs02_Eth29
switchport mode trunk
switchport trunk native vlan 991
channel-group 7 mode active
interface Ethernet1/30
description cfs02_Eth30
switchport mode trunk
switchport trunk native vlan 991
channel-group 7 mode active
interface Ethernet1/31
description cfs02_Eth31
switchport mode trunk
switchport trunk native vlan 991
channel-group 7 mode active
interface Ethernet1/32
description cfs02_Eth32
switchport mode trunk
switchport trunk native vlan 991
channel-group 7 mode active
interface mgmt0
description oam01_Gi0/19
ip address 172.31.8.178/26
cfs01#
   - --[ vpc-secondary
cfs02# sh run
!Command: show running-config
!Time: Wed Nov 13 08:46:05 2013
version 5.2(1)N1(1b)
cfs eth distribute
vrf context CEPH
vrf context management
ip route 172.31.0.0/20 172.31.8.190
vlan 14
name 172.31.50.0/26_CN/NN/OSDs@DMZ
vlan 4080
name 172.31.48.64/26_NAS.Infrastr@DMZ
spanning-tree vlan 1-129,131-3967,4048-4093 priority 4096
udld aggressive
vpc domain 1
role priority 2
peer-keepalive destination 172.31.8.178 source 172.31.8.179
peer-config-check-bypass
delay restore 150
peer-gateway
auto-recovery
ip arp synchronize
interface Vlan14
no shutdown
mtu 9216
description CN/NN/OSDs@DMZ
ip access-group acl-vl14-in in
vrf member CEPH
no ip redirects
ip address 172.31.50.60/26
no ip port-unreachable
hsrp version 2
hsrp 3
    authentication md5 key-string 3-14
    priority 254
    ip 172.31.50.62
interface Vlan4080
no shutdown
mtu 9216
description NAS.Infrastr@DMZ
ip access-group acl-vl4080-in in
vrf member CEPH
no ip redirects
ip address 172.31.48.124/26
no ip port-unreachable
hsrp version 2
hsrp 3
    authentication md5 key-string 3-4080
    priority 254
    ip 172.31.48.126
interface port-channel7
switchport mode trunk
switchport trunk native vlan 991
spanning-tree port type network
speed 10000
vpc peer-link
interface port-channel100
switchport mode trunk
switchport trunk native vlan 991
switchport trunk allowed vlan 2,14-19,991,4078-4080
speed 10000
vpc 100
interface port-channel102
switchport mode trunk
switchport trunk native vlan 991
switchport trunk allowed vlan 2,14,18,991,1299-1400
speed 10000
vpc 102
interface Ethernet1/1
no cdp enable
switchport mode trunk
switchport trunk native vlan 991
switchport trunk allowed vlan 2,14-19,991,4078-4080
channel-group 100 mode active
interface Ethernet1/3
no cdp enable
switchport mode trunk
switchport trunk native vlan 991
switchport trunk allowed vlan 2,14,18,991,1299-1400
channel-group 102 mode active
interface Ethernet1/29
description cfs01_Eth29
switchport mode trunk
switchport trunk native vlan 991
channel-group 7 mode active
interface Ethernet1/30
description cfs01_Eth30
switchport mode trunk
switchport trunk native vlan 991
channel-group 7 mode active
interface Ethernet1/31
description cfs01_Eth31
switchport mode trunk
switchport trunk native vlan 991
channel-group 7 mode active
interface Ethernet1/32
description cfs01_Eth32
switchport mode trunk
switchport trunk native vlan 991
channel-group 7 mode active
interface mgmt0
description oam02_Gi0/19
ip address 172.31.8.179/26
cfs02#

Similar Messages

3750 Stack Switch Issue

Hi
I have Stack of 7 3750 Switches. In that Switch 1 is master with 15 Priority and other switches are Member but because of some reason switch 1 rebooted and then Switch 2 becomes Master because it’s having 14 Priority.
We have 2 uplinks which connected on switch 1 (1/1/1) and switch 6 (6/1/1) Tengig port. Also we have 4 AP connected on switch 6.
So we observed that the Switch 1 came up and after that 3 AP is disconnected from WLC. if we disconnect the uplink port on Switch 1 (1/1/1) AP will connect the WLC and using uplink 6/1/1 but if we connect uplink port 1/1/1 and disconnect the uplink port 6/1/1 all 4 AP will disconnect from Network.
We try to connect the Uplink 1/1/1 to Switch 1 other port 1/1/2 but face the same issue. So we reconnect the port 6/1/1 and used the same link which connect on 1/1/1 to switch 7 (7/1/1) and AP will connect on WLC. So as per this troubleshooting we thought some issue with Network Module or Switch of Switch1.
So we Swap the Network module between Switch1 and Switch7 and connect the uplink on 1/1/1 but face the same issue and if we connect the same uplink on Switch 7 Ap connect to WLC.
So we thought that Switch is having issue. But before raising an RMA for Switch we observer 1 more thing that. Earlier that when the Switch 1 is Master all 4 AP's connected to WLC and working fine but when the Switch1 became rebooted and Switch2 became Master issue started. So we rebooted the Switch 2 and make switch master and found that while connecting on port 1/1/1 of Switch port AP's are able to connect to WLC and Issue resolved.
So I wants to know that is there any requirement that If the high priority switch rebooted and after came up it should not be Master then connected uplink will not work.
Also let us know what’s happen if the high priority switch came up in network. Means it’s will became Master or Menber.

Just now i Found one bug which is matching on our issue.
3750x stack fails to update ARP table after reboot causing traffic loss
BUG ID:-CSCtz98066
Symptoms: When the master switch (Switch A) is reloaded or loses power and
rejoins the stack as a member switch, any traffic stream being sent through
Switch A is unable to be received by the destination because the newly joined
member is not able to establish an ARP entry for the next hop router/switch.
Debugs confirm that Switch A does not send a GARP/ARP for the next hop, though
traffic continues to be sent to the switch.
Conditions: The symptom is observed when only Switch A has a physical
connection between the source and destination router/L3 switch. The newly
elected master (Switch B) does not.
Workaround: Ping destination from Switch A, forcing ARP request/response.
Also affected version is 12.2(58)SE and15.0(1)SE2 and we are using IOS 12.2(58)SE.
So we are going to upgrade the devices by this weekend with IOS 15.0(2)SE5.
Will update you once we upgraded the switches.

Why the debug arp output the follow imformation "IP ARP throttled out the ARP Request for 10.170.254.13"

In the network,sometimes,I can't ping some servers,the getway is in the switch 4507,if I connect my computer in the vlan what the servers in,I will not ping the getway successful,and the computer can't learn the getway's MAC.In the same time I debug arp in the 4507,the output is :
"Jun 20 07:36:21.225: IP ARP throttled out the ARP Request for 10.170.254.46
Jun 20 07:36:21.225: IP ARP throttled out the ARP Request for 10.170.254.13
Jun 20 07:36:21.227: IP ARP: sent req src 10.170.252.30 b838.6168.3c7f,
dst 10.170.252.82 0000.0000.0000 Vlan252"
If I reload the 4507,I could ping the getway ,and the servers.
I think it's ARP attack,the machine who was the question one is send a lot of ARP request ,that let the 4507's ARP cache full ,and than overflow. My computer wants to request the getway's MAC ,the message is discard.SO my computer can't ping the getway.
Can someone tell me, am I right? It's very important for me.
and tell me ,how can i do ? I'll wait online.
thank you very very much.

Have a look at this document for troubleshooting ARP throttled issues
http://www.cisco.com/c/en/us/support/docs/ip/express-forwarding-cef/17812-cef-incomp.html
HTH

CSM reporting failed ARP request

Hi!!
We have a CSM on Catalyst 6509 SUP720, the IOS is 12.2(18)SXD7b.
We have a problem with load-balanced portal servers with the CSM. Checking Catalyst log we see the following messages:
Jun 7 06:17:42.145 UTC: %CSM_SLB-6-RSERVERSTATE: Module 4 server state changed: SLB-NETMGT: Server [ip address] failed ARP request
Jun 7 06:17:43.505 UTC: %CSM_SLB-6-RSERVERSTATE: Module 4 server state changed: SLB-NETMGT: Server [ip address]now responding to ARP requests
Jun 7 06:19:23.445 UTC: %CSM_SLB-6-GATEWAYSTATE: Module 4 gateway state changed: SLB-NETMGT: Gateway [ip address]failed ARP request
Jun 7 06:24:12.241 UTC: %CSM_SLB-6-GATEWAYSTATE: Module 4 gateway state changed: SLB-NETMGT: Gateway [ip address]now responding to ARP requests
At the begining, the message were on refer to failed ARP for the real servers, but now the CSM reports failed ARP request on the Gateways (Vlan Interfaces) too.
Any idea?? We don?t now if the problem is the network, or is the CSM Card or the IOS version.
The Catalyst 6500 is on Distribution Layer, connecting 4 Access switches (Enterasys) and 2 Core Cat6500 switches. The balanced servers are on a Access Enterasys N7 Switch.
Thanks in advance...
Pedro

Hi Gilles, thanks for your quick response.
But, as well as the problem with missing ARP?s, we have very slow responses and sites pages are shown with missing objets every time we point to the CSM virtual address with 2 internet proxi servers on a CSM Serverfarm, while if we point to the Real IP Address of any proxi-sever, all is fine and fast.
Now we are pointing directly to real servers (not to the CSM virtual), and all is fine, but it is not the final idea.
It sounds like a problem with the Channel betwen the CSM and the Switch.
We made traces monitoring the PortChannel 260, and we saw very ARP request but just a few replys.
Is very strange that also the CSM is reporting missing ARP?s on Gateways, because this gateway is an interface Vlan into the MSFC!! The only way to disappear the log missing ARP messages is configuring static arp on CSM.
What you think about?.
The version on CSM is 4.2(2)
Thanks!!
Pedro.

Issue in soap request SAP Host Control

Hi. I use SAP NW 7.4 SP2 and have issue in next request
POST / HTTP/1.1
SOAPAction: ""
Content-Type: text/xml; charset=UTF-8
Content-Length: 559
<SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/"
xmlns:ns0="urn:SAPHostControl">
<SOAP-ENV:Header/>
<SOAP-ENV:Body>
    <ns0:MoveIpAddress>
      <aInterfaceName/>
      <aAddress>IP</aAddress>
      <aAddressFamily/>
      <aTargetHost>IP</aTargetHost>
      <aTargetUser/>
      <aTargetPassword/>
      <aTargetInterfaceName>???</aTargetInterfaceName>
      <aConnectionToTarget>???</aConnectionToTarget>
    </ns0:MoveIpAddress>
</SOAP-ENV:Body>
</SOAP-ENV:Envelope>
maybe does anyone used this request, what I need write in aTargetInterfaceName && aConnectionToTarget aConnectionToTarget - http, tcp or https don't work
Thank you

Hi,
Run the program 'RSTXTRAN' using transaction SE38 and attach the Text object to your TR.
Last time I faced the same problem.
Everything gets transported except Text object.
Hope it will be useful to u.
Revert if not resolved.
Regards
Amit
Edited by: Amit Gupta on May 14, 2009 7:31 AM

How to show module and SFP information include hardware serial number on Nexus switch?

How to show module and SFP information include hardware serial number on Nexus switch? Thanks.

Hi,
The show inventory command will give you details of the Nexus chassis, power supplies, Supervisor, Fabric, I/O modules (including FEX if you have them) etc. To get the details of the installed SFP/SFP+ then you'll need to use the show interface transceiver command.
Regards

Solaris 9 and ARP request

Hello
my server is sending a lot of arp requests "who is ..." at first sight it looks quite ok, but ttl in arp cache is set on 20 min, but my server doesn't care he after getting an answer "...ip.ip.ip.ip is et.et.et.et.et.et..." is still asking " who is...."
is it ok?
and another wierd think: - maybe I am not smart enough but - shouldn't those two commends give the the same (+-) answers:
# arp -a
# ndd -get /dev/arp arp_cache_report
thenks for help
Agie

Solaris 9 - 9/05 HW (Update 9) is supported on SunFire V215, if you have a support contract you might will be able to download it from www.sun.com/OSC .
.7/M.

VPC between server and nexus switches

Hi all,
I would like to ask following question:
We are using virtual portchannels to connect e.g. ESXi-Servers with trunks on two Nexus switches. When the switch "rz2sw1" has gone faulty, we expect that the trunk connections between switch rz2sw2 would still work. That's the reason why they are built. But they didn't as you can see in the screenshot (attaced) former mail.
Do you know what can be problem.. thank you

It appears that the peer-link was down. If the peer-link is down and the keep-alive is still up, the secondary device will shutdown all of it's VPC links to ensure dual active (split brain) is avoided.
On a Nexus 7000, its recommended to use a front panel (data port) for the keep-alive instead of the management port. My assumption would be that you are using the MGMT0 for keep-alive and all front panel ports went off-line. If you use a front panel port for keep-alive and all line cards go off line on the primary, the secondary will take over as expected.

Fireworks CS3 Program Switching Issue

In the office we have 3 machines using Fireworks CS3 (part of Studio CS3). All the machines are experiencing the issue I am about to describe. Each machine is an Intel Core 2 Duo with 4GB ram. All 3 machines are using Windows Vista.
We all had the old Vista Aero problem with brushes, and we quickly overcame this by using the "disable desktop composition" fix.
However, one problem we have never been able to solve occurs when switching between ANY application and Fireworks. There is a delay of at least 20-30 seconds (sometimes more) when trying to swap back to Fireworks. We have being doing various Google & Adobe forum searches regularly over the past 2+ years, looking in vein for a fix, but as yet have still not managed to find a solution or even a record of someone else with the same issue (mainly because out searches are dominated with the old "disable desktop composition" fix). We assumed it was a problem fundemental to CS3 yet still hold out hope for a solution.
Normally it's not a problem, but when working on websites which involve a lot of graphics edits and/or switching in and out of Fireworks it can quickly add a very frustrating hour or more to a job.
Recently, I was forced to upgrade to a new computer with Windows 7, and transfered my old CS3 install to the new machine. I was surprised to find that I can switch in and out of Fireworks instantly, much to the frustration of the other guys in the office who are unfortunately stuck with Vista for the time being.
Is there a fix for this horrible switching delay on Vista and has anyone else here even sufered with the same issue?
TL:DR - This is not the desktop composition problem with brushes which seems to domnate results when we try to search for a fix, this is a very noticable delay when swapping in and out of Fireworks.

I'm affraid that didn't help. It actually slowed down the inital launch of the program and it took around 5 minutes for Fireworks to finally start. Once started the program switching issue was still present

Connecting two Nexus switches with link

Hi,
I would like to connect two nexus switches with a link. Like one c5020 and one c5548p. How do i need to configure the ports at both the switches which will be joined with a fiber cable.
For MDS switches such links becomes E-port and fabric is merged. Not sure what happenes at the Nexus swictches. Any input will be appreciated.
Thanks,

Do you have fc modules in each nexus? If so, they act much like the MDS 9148. All ports should be default auto speed and auto switchport mode. Insert fc-sfp's and a regular multimode fc cable between them. Default mode is fc switch.
If that doesn't work, configure the mode on each port as E.
Once done, the fc switch portion of each nexus will merge.
Dave

STB Sending ARP requests every 9 minutes

IS there a particular reason that I keep getting ARP requests from the STB even though it has been assigned an IP?
Solved!
Go to Solution.

So there is no way to route that correctly?
I mean, I havent blocked any of that traffic.
I used to use a different router with my old service and well you guys supply one.
Since I have a new router now, I wanted to check how well the firewall was set up and if there was any automatic anti-spoofing on the router or if I had to set it up myself.
So with feeling insecure I turned my personal Outpost firewall back on on my computer.
Which is catching that as a spoofing attack of course.
But now my question is, couldnt that traffic be routed appropriately?
IT doesnt seem like a big deal, just a little messy.
And this thread is easily applying to FIOS INTERNET more and more as we talk.
Form what I get its the STB's looking for the DVR.
What would happen if you blocked that info coming across the bridge or routed the info appropriately vs blocking it.
Does it have to come across the bridge and if so why?
Form the looks of things I could definitely keep that on the coax side of things vs having it run across the bridge.
My comp recognizes it as spoofing but the router doesnt of course because its all behind the router.
Maybe you could help me further or someone else in defining the design of that communication.
Again its harmless and not a big deal its more curiosity than anything else at this point.

ACI-How are the new advancements in ACI helping the drawbacks in nexus switching?

ACI-How are the new advancements in ACI helping the drawbacks in nexus switching?

ACI is a paradigm shift in data centre designs.
According to this solution overview ACI is the next generation of Software Defined Networking:
http://www.cisco.com/c/en/us/products/collateral/switches/nexus-9000-series-switches/guide-c07-731461.html
Having worked with Nexus switches for a couple of years now I haven't encountered any serious drawbacks with these devices.

6500 IOS HSRP Gateway not responding to ARP requests

WS-C6509, running Native IOS version 12.2(33)SXH4. Pair of 6500 configured with HSRP serving as Gateway to down stream clients
Ping requests from clients not having a Default Gateway (AIX Server Team recommendation) failing.
Packet capture show ARP requests being received by Gateway 6500, but Gateway will not respond for up to 30 seconds at which point ping requests will start working. But after a period of inactivity, the cycle starts over again - 30 second delay before traffic starts flowing.
Having a Default Gateway is a separate discussion. I'm just interested in being able to provide technical reason why this is occuring.
Anyone experience this?
Any suggestions for addtional troubleshooting measures?
Thanks in advance.

Looks to me like a timer problem - maybe a ARP timer on the clients? Check the ARP tables of your clients during your tests.
HTH

Css excessive arp requests

Hello all,
my CSS 11150 with WebNS 5.00 does excessive arp requests on its interfaces (up to 100 arps per second). The box seems to arp EVERYTHING especially in the 10.147.0.0 /16 subnet even if it is not used at all. My config is as follows:
ip no-implicit-service
ip opportunistic disable
ip route 0.0.0.0 0.0.0.0 10.147.1.1 1
circuit VLAN1
ip address 10.147.248.10 255.255.0.0
circuit VLAN2
ip address 10.145.45.254 255.255.255.128
service sunbl3s6-443
ip address 10.145.45.136
protocol tcp
port 443
keepalive type tcp
keepalive port 443
active
service sunbl3s6-80
ip address 10.145.45.136
protocol tcp
port 80
keepalive type tcp
keepalive port 80
active
service sunbl3s7-443
ip address 10.145.45.137
protocol tcp
port 443
keepalive type tcp
keepalive port 443
active
service sunbl3s7-80
ip address 10.145.45.137
protocol tcp
port 80
keepalive type tcp
keepalive port 80
active
owner unix-systems
content vrp-test-443
vip address 10.145.45.253
protocol tcp
port 443
balance aca
add service sunbl3s6-443
add service sunbl3s7-443
active
content vrp-test-80
vip address 10.145.45.253
protocol tcp
port 80
balance aca
add service sunbl3s6-80
add service sunbl3s7-80
active
group vrp-test
vip address 10.145.45.253
add destination service sunbl3s6-80
add destination service sunbl3s6-443
add destination service sunbl3s7-80
add destination service sunbl3s7-443
active
Does anybody have any hints?
Many thanks in advance
Uli

Hi,
I did a software upgrade yesterday and put ap0610405.adi.gz on the box. But the behaviour didn't change. We also checked the cabling for loops, that's also fine.
We have observed some further things:
The broadcasts are only on the 10.147.0.0 /16 subnet. As this is our local lan backbone we can't change it, I could only shift the frontend into another subnet and route it towards the backbone.
We have another two boxes (CSS11503 with 7.4) with a similar configuration - they also do excessive arp requests in the same subnet, the primary as well as the secondary. But the addresses being arped for are not necessarily the same.
I took some packet traces looking for broadcasts and multicasts that could inspire the boxes to arp for every address they see - nothing, the addresses being arped for are not seen in the seconds before the CSS arp request.
What could trigger arp requests for machines which never accessed or used the CSS services / rules??? I've never seen such a behaviour before...
Best Regards
Uli

Nexus switch for storage

we are currently evaluating solutions for our storage upgrade.
we required 10GbE connection from storage to our servers.
there are 2 proposals that uses nexus switch.
1 uses nexus 5548 while the other one uses 3524.
i understand that 5548 is of a higher grade than the 3524.
i have 7 dell servers that will be connected to the switch using 10GbE NIC.
which nexus switch is more suitable for my environment.
7 servers -> 2 10GbE iSCSI switch -> 1x iSCSI storage(20TB)

Well...you're on a Cisco forum, so we'd be remiss in not recommending Cisco gear.
The Nexus 5548UP is around 17k for the base model without any special layer 3 licenses.
http://www.softchoice.com/catalog/en-us/hubs-switches-cisco-nexus-5548up-switch-managed-N5K-C5548UP-FA-HV7684
If you don't use layer 3 on the Nexus (which I wouldn't), they've got pretty good port density for the cost. NXOS takes little bit of getting used to and, for us, the copper SFPs have been a royal pain in the ass, but otherwise, they're pretty solid devices.

Nexus-switches issues no arp-requests.

Similar Messages

Maybe you are looking for