To many ARP's
My BEFW11S4 wireless router is sending out arp's constantly on our network. How do I stop this. My firmware is up to date and I have a wireless G that doesn't do this. Also this same router loses it's connection and has to be reset at least once a day.
Thanks for any input, Bobby
what firmware are you using currently on the router ? Also is the router sending ARP's to a particular IP or the entire network ?
Similar Messages
-
On my 3850 (running 3.3.1) i have 1600+ entries in the arp table for a given vlan but I'm not acting as the gateway for the devices connecting to it (i'm trunked to the core which is acting as the gateway but I do have ip routing enabled on my 3850). I've put the nmsp attachment suppress command on all physical interfaces to resolve another issue I was having.
Is having all these arp entries expected behavior? I've tried to delete 1 ip in the table which I knew wasn't valid but my switch seems to ignore it as the entry is still there.
The reason I ask was due to a small unicast flooding issue I seemed to have (since gone away). I was told it may have been due to the switch having an arp entry for a mac addresses it didn't know and hence was flooding the switch. The person was surprised to see so many arp entries given i wasn't a gateway for this vlan.
ThanksHi,
If you issue "show running config all" command you can see all configuration lines of this switch including the default settings. Here is an example for one of the vlan interface configuration. As you can see "proxy-arp" is enabled globally & interface level by default.
3850-2#sh running-config all | in proxy
no ip arp proxy disable
3850-2#sh running-config all | be interface Vlan1410
interface Vlan1410
ip address 10.141.103.242 255.255.248.0
ip redirects
ip unreachables
ip proxy-arp
ip mtu 1500
ip load-sharing per-destination
ip cef accounting non-recursive internal
ip pim dr-priority 1
ip pim query-interval 30
ip mfib forwarding input
ip mfib forwarding output
ip mfib cef input
ip mfib cef output
ip route-cache cef
ip route-cache
ip split-horizon
ip igmp last-member-query-interval 1000
ip igmp last-member-query-count 2
ip igmp query-max-response-time 10
ip igmp version 2
ip igmp query-interval 60
ip igmp tcn query count 2
ip igmp tcn query interval 10
load-interval 300
carrier-delay 2
no shutdown
ipv6 nd reachable-time 0
ipv6 nd ns-interval 0
ipv6 nd dad attempts 1
ipv6 nd prefix framed-ipv6-prefix
ipv6 nd nud igp
ipv6 nd ra lifetime 1800
ipv6 nd ra interval 200
ipv6 redirects
ipv6 unreachables
snmp trap link-status
cts role-based enforcement
arp arpa
arp timeout 14400
spanning-tree port-priority 128
spanning-tree cost 0
hold-queue 75 in
hold-queue 40 out
no bgp-policy accounting input
no bgp-policy accounting output
no bgp-policy accounting input source
no bgp-policy accounting output source
no bgp-policy source ip-prec-map
no bgp-policy source ip-qos-map
no bgp-policy destination ip-prec-map
no bgp-policy destination ip-qos-map
This post explain "proxy-arp" behaviour well.
http://www.cisco.com/c/en/us/support/docs/ip/dynamic-address-allocation-resolution/13718-5.html
In your case all the SVI defined & end host gets default-gateway IP correctly, there is no need for "proxy-arp" enabled on SVI. You can safely disable it (globally or interface level) and check if that help to mitigate your arp cache issue.
3850-2(config)#ip arp proxy disable
or
3850-2(config)#int vlan 1410
3850-2(config-if)#no ip proxy-arp
HTH
Rasika
**** Pls rate all useful responses **** -
NAT ASA5512 8.6(1)2 in and out
Hello Everyone,
This is my first post so please forgive me if I miss something. I have an ASA5512 running 8.6(1)2 that I am trying to NAT a public IP address from my ISP to multiple phone systems on the inside of my network. One of these phone systems is at the same site as the ASA5512 and I have no problems getting this one to work with my current config. The problem comes when I apply the same type of NAT rule that works at the main site to allow NAT to the other sites. These sites are connected via a point-to-point system from our ISP. The point-to-point does not seem to be an issue as I can ping any device at our other sites and I can RDP into computers and servers at the others sites. I can also call internally between sites but when I try to call the other sites from my cell I cant get through. Also when I forward one of the extensions at the others sites to my cell and then call internally I do not get an outside line.
In the config below you can see that Ive applied the same NAT and ACL rules to the adminphonesystem and the deltaphonesystem objects. The adminphonesystem can make calls and recieve them with no issues. The deltaphonesystem cannot make or recieve calls from outside our network. Only internal calls are working for the deltaphonesystem. Ive done packet traces in every which way and corrected any issues that I have found with no fix to the problem. So I cleaned up my config and posted it here. Really hope someone can give me a few pointers in getting this problem solved.
On another note I have a Cisco ASA5505 with smartnet support. So i throw it in place of the 5512 and call cisco support. A tech calls me back and we get everything working perfectly on the 5505 with a few simple rules. I say thank you and have a nice. Then I throw the 5512 back in and replicate the rules from the 5505 that were working. Both of these units are using the new NAT setup that was released after 8.3. To my surprise the 5512 doesnt work even though I have the same rules as the 5505. If anyone can answer that side question please do.
ASA Version 8.6(1)2
hostname AdminASA
domain-name
enable password encrypted
passwd encrypted
names
interface GigabitEthernet0/0
shutdown
no nameif
security-level 0
no ip address
interface GigabitEthernet0/1
nameif Outside
security-level 0
ip address 76.320.333.43 255.255.255.224
interface GigabitEthernet0/2
nameif Inside
security-level 100
ip address 10.1.99.1 255.255.255.0
interface GigabitEthernet0/3
nameif P2P
security-level 100
ip address 10.2.99.2 255.255.255.0
interface GigabitEthernet0/4
shutdown
no nameif
no security-level
no ip address
interface GigabitEthernet0/5
shutdown
no nameif
no security-level
no ip address
interface Management0/0
nameif management
security-level 100
ip address 192.168.1.1 255.255.255.0
ftp mode passive
clock timezone MST -7
clock summer-time MDT recurring
dns server-group DefaultDNS
domain-name corp.centermh.org
same-security-traffic permit inter-interface
same-security-traffic permit intra-interface
object network DeltaNetwork
subnet 10.1.96.0 255.255.255.0
object network GunnisonNetwork
subnet 10.1.97.0 255.255.255.0
object network MiamiNetwork
subnet 10.1.98.0 255.255.255.0
object network NuclaNetwork
subnet 10.1.93.0 255.255.255.0
object network TellurideNetwork
subnet 10.1.94.0 255.255.255.0
object network AdminPhoneSystem
host 10.1.99.225
description Inside IP Address of Admin Phone System
object network DeltaPhoneSystem
host 10.1.96.225
description Internal IP Address of Delta Phone System
object network AdminPhonePublic
host 76.320.333.48
description Public IP Address of Admin Phone System
object network FastTrackPhone
host 234.213.124.81
description FastTrack SIP Trunk Authtication IP Address
object network FastTrackMonitor
host 290.230.195.8
description FastTrack Monitoring server
object network DeltaPhonePublic
host 76.320.333.51
description Public IP Address of Delta Phone System
object-group icmp-type ICMP-All
icmp-object echo
icmp-object echo-reply
icmp-object information-reply
icmp-object information-request
icmp-object time-exceeded
icmp-object timestamp-reply
icmp-object timestamp-request
icmp-object traceroute
icmp-object alternate-address
icmp-object conversion-error
icmp-object mask-reply
icmp-object mask-request
icmp-object mobile-redirect
icmp-object parameter-problem
icmp-object redirect
icmp-object router-advertisement
icmp-object router-solicitation
icmp-object source-quench
icmp-object unreachable
object-group protocol TCPUDP
protocol-object udp
protocol-object tcp
access-list global_access extended permit icmp object FastTrackMonitor any object-group ICMP-All
access-list Local_access_in extended permit ip any any
access-list MPLS_access_in extended permit ip any any
access-list CTN_access_in extended permit object-group TCPUDP object FastTrackPhone object DeltaPhoneSystem eq sip
access-list CTN_access_in extended permit icmp object FastTrackPhone object DeltaPhoneSystem object-group ICMP-All
access-list CTN_access_in extended permit object-group TCPUDP object FastTrackPhone object AdminPhoneSystem eq sip
access-list CTN_access_in extended permit icmp object FastTrackPhone object AdminPhoneSystem object-group ICMP-All
pager lines 24
logging enable
logging asdm informational
mtu Outside 1500
mtu Inside 1500
mtu P2P 1500
mtu management 1500
ip local pool vpnUsers 10.1.99.200-10.1.99.210 mask 255.255.255.0
icmp unreachable rate-limit 1 burst-size 1
no asdm history enable
arp timeout 14400
nat (Inside,Outside) source static DeltaPhoneSystem DeltaPhonePublic no-proxy-arp
nat (Inside,Outside) source static AdminPhoneSystem AdminPhonePublic no-proxy-arp
nat (P2P,Outside) after-auto source dynamic any interface
nat (Inside,Outside) after-auto source dynamic any interface
access-group Outside_access_in in interface Outside
access-group Inside_access_in in interface Inside
access-group P2P_access_in in interface P2P
access-group global_access global
route Outside 0.0.0.0 0.0.0.0 76.320.333.42 6
route P2P 10.1.93.0 255.255.255.0 10.2.99.1 1
route P2P 10.1.94.0 255.255.255.0 10.2.99.1 1
route P2P 10.1.95.0 255.255.255.0 10.2.99.1 1
route P2P 10.1.96.0 255.255.255.0 10.2.99.1 1
route P2P 10.1.97.0 255.255.255.0 10.2.99.1 1
route P2P 10.1.98.0 255.255.255.0 10.2.99.1 1
route P2P 10.2.93.0 255.255.255.0 10.2.99.1 2
route P2P 10.2.94.0 255.255.255.0 10.2.99.1 2
route P2P 10.2.95.0 255.255.255.0 10.2.99.1 2
route P2P 10.2.96.0 255.255.255.0 10.2.99.1 2
route P2P 10.2.97.0 255.255.255.0 10.2.99.1 2
route P2P 10.2.98.0 255.255.255.0 10.2.99.1 2
timeout xlate 3:00:00
timeout conn 1:00:00 half-closed 0:10:00 udp 0:02:00 icmp 0:00:02
timeout sunrpc 0:10:00 h323 0:05:00 h225 1:00:00 mgcp 0:05:00 mgcp-pat 0:05:00
timeout sip 0:30:00 sip_media 0:02:00 sip-invite 0:03:00 sip-disconnect 0:02:00
timeout sip-provisional-media 0:02:00 uauth 0:05:00 absolute
timeout tcp-proxy-reassembly 0:01:00
timeout floating-conn 0:00:00
dynamic-access-policy-record DfltAccessPolicy
user-identity default-domain LOCAL
http server enable
http 192.168.1.0 255.255.255.0 management
http 10.1.99.0 255.255.255.0 Inside
no snmp-server location
no snmp-server contact
snmp-server enable traps snmp authentication linkup linkdown coldstart warmstart
telnet timeout 5
ssh 10.1.99.0 255.255.255.0 Inside
ssh 192.168.1.0 255.255.255.0 management
ssh timeout 5
console timeout 0
dhcpd address 192.168.1.2-192.168.1.254 management
dhcpd enable management
threat-detection basic-threat
threat-detection statistics access-list
no threat-detection statistics tcp-intercept
ntp server 128.138.140.44 prefer
webvpn
anyconnect image disk0:/anyconnect-win-2.5.2014-k9.pkg 1
anyconnect enable
tunnel-group-list enable
username privilege 15
username privilege 15
class-map inspection_default
match default-inspection-traffic
policy-map type inspect dns preset_dns_map
parameters
message-length maximum client auto
message-length maximum 512
policy-map global_policy
class inspection_default
inspect dns preset_dns_map
inspect ftp
inspect h323 h225
inspect h323 ras
inspect ip-options
inspect netbios
inspect rsh
inspect rtsp
inspect skinny
inspect esmtp
inspect sqlnet
inspect sunrpc
inspect tftp
inspect sip
inspect xdmcp
inspect icmp
service-policy global_policy global
prompt hostname context
no call-home reporting anonymous
call-home
contact-email-addr
profile CiscoTAC-1
no active
destination address http https://tools.cisco.com/its/service/oddce/services/DDCEService
destination address email [email protected]
destination transport-method http
subscribe-to-alert-group diagnostic
subscribe-to-alert-group environment
subscribe-to-alert-group inventory periodic monthly 8
subscribe-to-alert-group configuration periodic monthly 8
subscribe-to-alert-group telemetry periodic daily
Cryptochecksum:
: endHi,
If I am not mistaken then atleast one big problem is the source interface in the other NAT configuration command
You have this
nat (Inside,Outside) source static DeltaPhoneSystem DeltaPhonePublic no-proxy-arp
Yet you have this "object network" and "route"
object network DeltaPhoneSystem
host 10.1.96.225
route P2P 10.1.96.0 255.255.255.0 10.2.99.1 1
So seems to me that your NAT configuration should be
nat (P2P,Outside) source static DeltaPhoneSystem DeltaPhonePublic no-proxy-arp
Just as a side note, I personally prefer to configure Static NAT with Network Object NAT. With those configurations your Static NAT configurations would look like this
object network DeltaPhoneSystem
host 10.1.96.225
nat (P2P,Outside) static 76.320.333.51
object network AdminPhoneSystem
host 10.1.99.225
nat (Inside,Outside) static 76.320.333.48
Also one very important note, if you are using multiple public subnets on your ASA "Outside" interface then the way this is implemented by your ISP has a lot of meaning.
If the ISP has configured one public subnet between its gateway device and your ASA and routed the other subnet(s) towards the ASAs "Outside" interface IP address then there is no problem.
If the ISP has configured both (or all) public subnets on their gateway interface (others as "secondary" subnets) then you will (to my understanding) run into a problem with ARP with nonconnected networks on the ASA.To correct this you would require you to either change the setup to the first option with the ISP or update your ASA software to 9.0(2) or possibly 9.1(2) to get access to the command "arp permit-nonconnected"
Here is the section from the patch notes that also explains the commands purpose
ARP cache additions for non-connected subnets
The ASA ARP cache only contains entries from directly-connected subnets by default. You can now enable the ARP cache to also include non-directly-connected subnets. We do not recommend enabling this feature unless you know the security risks. This feature could facilitate denial of service (DoS) attack against the ASA; a user on any interface could send out many ARP replies and overload the ASA ARP table with false entries.
You may want to use this feature if you use:
•Secondary subnets.
•Proxy ARP on adjacent routes for traffic forwarding.
We introduced the following command: arp permit-nonconnected.
Also available in 8.4(5).
If you want to take a look at a NAT 8.3+ document I made here on the CSC then follow this link
https://supportforums.cisco.com/docs/DOC-31116
Hopefully the above helps with your problem
Please do remember to mark the reply as the correct answer if it answered your question.
Ask more if needed
- Jouni -
Dear all,
We have issues with our mesh network, we have a 2 WLC 5508 ver7.0.220.0 (connected to 3560 with LAG) 16 APs1524, 60 Bridges 1300 ver12.3.8-JA2 configured like Workgroup bridge (mobile stations), 20 Switches IE3000 connected to WGB and services connected to SW IE3000.
Network is working relatively fine, links in our backhaul is OK SNR lower is 21 and max hops on MAP is 2, we have just one wlan configured, with passive client feature enabled, multicast feature is also enabled, in two times when one MAP is moved to another position (is mining environment) network present instability we have following symptoms:
Connectivity (through PRTG) with WGB is flapping so services behind were affected.
We understand if moved a MAP into Mesh probably network convergence, but when network is stable the symptom are maintained, the most strange is from the network we don’t have connectivity to 1300 however connectivity to Switch is OK, in fact we can connect to SWITCH and from it we can connect to 1300 when we are inside Bridge everything interfaces are up including radios, logs show some deauthenticate, but we have configuration without authentication just for now. In WCS we have a alarm CPU Receive Multicast Queue is full on Controller in the time that issue is arise. We make to following actions:
Enable Multicast, previously the final customer disable this feature.
Vlans were pruned to the WLC to only what it needs
Disable passive client (two weeks ago was enabled), in this point the services is more stable WGB is response from network and services is no affected, so we understand with the passive client is enabled the proxy ARP feature is disabled, so we make to captures in Switch IE300 for another issue one week ago, and we saw too many ARP broadcast from Gateway to all WGB and clients connected to WGB, so our doubts is this symptom is in relation with the message on WCS, version on 1300.
Any comment or action is welcome.
Thanks a lot!What kind of clients are behind the IE3000's? Are they passive?
The proper way to work with the passive clients is putting their MAC address in the Mac Filtering in the WLC and enabling the "passive client" in the WLAN.
Then you enable L2 bridge forwarding on the 1300 with the "bridge 1 addressxxxx.xxxx.xxxx forward FastEthernet0" command.
Then you add a static arp entry in the L3 switch that the SVI resides on.
When you don't have connectivity to 1300. Keep a ping going to it and then try to ping it from your IE3000. If it starts pinging from you computer immediately after that - it sounds like you have an ARP issue.
Is it possible to post the configs of the IE3000's and 1300's? -
I am carving up an internet Class C for customer. This class C is used by 3 distinct QA, Corporate and Production firewalls. I want to carve up IP space so there is a /26 for each environment. The issue I have is the firewalls may need communication with each other via the public IP space. Currently I don’t have any L3 switches in between the firewalls and the edge internet router. So with subnetting, it would seem I need to push everything through the internet router for the intra-firewall communication.
I would rather not push this traffic through the edge router, so I came up with an idea to allocate all firewall outside interface IP’s in the 4th (last remaining) /26. That way, I can allow firewalls to communicate over the primary interface IP’s, which will all be in the same subnet – without going through a routing “engine”/device.
For the actual environment subnets (NAT's on respective firewalls), I create a static route on the edge router pointing to each of the firewall’s primary IP’s for the respective environment routes (the first 3 - /26’s).
This is still a beta design, but I have done this before on small scale when ISP gave me 2 subnets for example, assuming I was going to put a router in between the customer firewall and ISP. I would use the “routed subnet” on the ASA interface, and then pull the NAT’s from the other subnet. The ISP would have to add a static route directing the NAT subnet to the “routed subnet” correct IP - which would be the firewall outside interface primary IP.
I recently found out that with ASA OS 8.4.3 and up, ASA will not proxy arp for IP’s not in its local interface subnet. This means the ISP/router will have to assign static ARP entries on the edge router. This can get messy after the first few NAT entries. So I am debating the design now. I think this kind of stuff going forward won’t be worthwhile with newer ASA 8.4.3 code.
Any ideas on how to communicate between different ASA’s, while still carving up the Class C into usable smaller subnets? The primary reason for doing this in the first place is to support routing on the edge router. I am thinking it might be time to ask for another Class C to do the routing functions, and keep the firewalls all at Layer 2 in one /24 - Class C?I recently found out that with ASA OS 8.4.3 and up, ASA will not proxy arp for IP’s not in its local interface subnet.
That is a surprise especially as using a different subnet than the one used to connect the ASA to the router for NAT is quite a common setup.
Anyway as we are brainstorming here are a couple of options that spring to mind. Please feel free to shoot them down
For both solutions you still have 4 x 26, the first 3 for each firewall to use as NAT and then the last /26 for the firewall interfaces + the ISP internal interface.
Option 1
======
when you allocate the IP to the firewall outside interfaces and the ISP internal interface they come out of the last /26 range but you use a /24 subnet mask. The router will arp out for all addresses within the /24 subnet but the firewalls should only answer via proxy arp for any statically mapped NAT entries that they have. They will answer because the /26 they use for NAT are within the range of their outside interface IP because that is using a /24.
Obviously because the interfaces are in the same /24 range they will be able to talk to each other wihout bouncing off the router.
Option 2
=======
pretty much the same as option 1 except the ISP router uses a /26 subnet and has routes for easch /26 NAT subnet pointing to the relevant firewall. This way you don't have as many arps being sent by the ISP router. The firewalls still have to use a /24 mask to enable them to talk with each other. And the firewalls and router still need to have IPs from the last /26.
Both would need testing and i may have missed something but i would have thought both would work.
Jon -
Slow ARP response for dial-in clients
I’ve been experiencing an intermittent issue with remote PC’s connecting to a Cisco AS5350 Universal Gateway - basically, a RAS server.
The issue as far as I’ve been able to pinpoint seems to be related to the amount of time it takes the dial-in client to register an ARP entry on the local network where the RAS server and other servers are connected. If I start an extend ping to one of the servers on the local network (not to the RAS server) once my dial-up connection has been established, I typically see anywhere between 3 and 18 ICMP request timeouts before I start receiving replies. And if at the same time I start an extended ping to the IP address of the RAS server, ICMP replies are received immediately with no request timeouts.
Topology:
Dial-in Client <===> AS5350 RAS <===> L2 Switch <===> Server
192.168.240.131 240.5 240.1 240.21
The switch that the AS5350 and the servers are connected to is a WS-C2960G-8TC-L layer-2 switch with a very basic config. Basically they only thing I’ve changed during the course of my troubleshooting is the STP mode, STP forward time and to enabled STP portfast on the uplinks to the AS5350 and the server… see configuration below:
Current configuration : 2721 bytes
version 12.2
no service pad
service tcp-keepalives-in
service tcp-keepalives-out
service timestamps debug datetime msec localtime show-timezone
service timestamps log datetime msec localtime show-timezone
service password-encryption
service sequence-numbers
hostname Switch
boot-start-marker
boot-end-marker
no aaa new-model
system mtu routing 1500
spanning-tree mode rapid-pvst
spanning-tree extend system-id
spanning-tree vlan 1 forward-time 5
vlan internal allocation policy ascending
interface GigabitEthernet0/1
description Uplink to Server
spanning-tree portfast
interface GigabitEthernet0/2
description Uplink to CLE-AS5350 RAS
speed 100
duplex full
spanning-tree portfast
interface GigabitEthernet0/3
interface GigabitEthernet0/4
interface GigabitEthernet0/5
interface GigabitEthernet0/6
interface GigabitEthernet0/7
interface GigabitEthernet0/8
interface Vlan1
ip address 192.168.240.1 255.255.255.0
ip http server
ip http secure-server
line con 0
exec-timeout 0 0
logging synchronous
line vty 0 4
login
line vty 5 15
login
end
For troubleshooting, I enabled “debug arp” on the switch and attempted a dial-up connection to the AS5350. Once the call was established and I received a DHCP lease (192.168.240.131), I started an extended ping to a server (192.168.240.21) on the network… see below:
Host Details:
192.168.240.1 (b4e9.b006.9e40) = Vlan1 on L2 switch.
192.168.240.21 (5cf9.dd48.76dd) = Server.
192.168.240.5 (000d.280c.fe1b) = Cisco AS5350 RAS server.
192.168.240.131 (0000.0000.0000) = PPP dial-in client on RAS server.
000292: *Mar 1 00:21:22.819 UTC: IP ARP: creating incomplete entry for IP address: 192.168.240.131 interface Vlan1
000293: *Mar 1 00:21:22.819 UTC: IP ARP: sent req src 192.168.240.1 b4e9.b006.9e40, dst 192.168.240.131 0000.0000.0000 Vlan1
000298: *Mar 1 00:21:27.013 UTC: IP ARP: rcvd req src 192.168.240.21 5cf9.dd48.76dd, dst 192.168.240.131 Vlan1
000299: *Mar 1 00:21:27.441 UTC: IP ARP: sent req src 192.168.240.1 b4e9.b006.9e40, dst 192.168.240.131 0000.0000.0000 Vlan1
000306: *Mar 1 00:21:32.441 UTC: IP ARP: sent req src 192.168.240.1 b4e9.b006.9e40, dst 192.168.240.131 0000.0000.0000 Vlan1
000314: *Mar 1 00:21:37.449 UTC: IP ARP: sent req src 192.168.240.1 b4e9.b006.9e40, dst 192.168.240.131 0000.0000.0000 Vlan1
000323: *Mar 1 00:21:42.440 UTC: IP ARP: sent req src 192.168.240.1 b4e9.b006.9e40, dst 192.168.240.131 0000.0000.0000 Vlan1
000329: *Mar 1 00:21:47.440 UTC: IP ARP: sent req src 192.168.240.1 b4e9.b006.9e40, dst 192.168.240.131 0000.0000.0000 Vlan1
000334: *Mar 1 00:21:52.439 UTC: IP ARP: sent req src 192.168.240.1 b4e9.b006.9e40, dst 192.168.240.131 0000.0000.0000 Vlan1
000344: *Mar 1 00:21:57.447 UTC: IP ARP: sent req src 192.168.240.1 b4e9.b006.9e40, dst 192.168.240.131 0000.0000.0000 Vlan1
000350: *Mar 1 00:22:02.447 UTC: IP ARP: sent req src 192.168.240.1 b4e9.b006.9e40, dst 192.168.240.131 0000.0000.0000 Vlan1
000358: *Mar 1 00:22:07.430 UTC: IP ARP: sent req src 192.168.240.1 b4e9.b006.9e40, dst 192.168.240.131 0000.0000.0000 Vlan1
000364: *Mar 1 00:22:12.438 UTC: IP ARP: creating incomplete entry for IP address: 192.168.240.131 interface Vlan1
000365: *Mar 1 00:22:12.438 UTC: IP ARP: sent req src 192.168.240.1 b4e9.b006.9e40,dst 192.168.240.131 0000.0000.0000 Vlan1
000372: *Mar 1 00:22:17.437 UTC: IP ARP: sent req src 192.168.240.1 b4e9.b006.9e40, dst 192.168.240.131 0000.0000.0000 Vlan1
000373: *Mar 1 00:22:17.446 UTC: IP ARP: rcvd rep src 192.168.240.131 000d.280c.fe1b, dst 192.168.240.1 Vlan1
The first line of the debug shows the switch creating an “incomplete entry” for the dial-in client (192.168.240.131).
For all subsequent ICMP requests, you can see that the dial-in client has a MAC address of 0000.0000.0000 – I guess you would call this an incomplete entry.
On the last line of the debug output, you can see that the dial-in client (192.168.240.131) finally gets the MAC address of the AS5350 (000d.280c.fe1b) assigned to it – this is when we start getting ICMP replies.
So during this capture, there were 12 ICMP request timeouts before the dial-in client started receiving replies.
Below is the current config on my Cisco AS5350 RAS server:
Current configuration : 6741 bytes
version 12.3
service timestamps debug datetime localtime show-timezone
service timestamps log datetime localtime show-timezone
service password-encryption
hostname AS5350
boot-start-marker
no boot startup-test
boot-end-marker
logging buffered 2048000 debugging
enable secret 5 *********************
resource-pool disable
calltracker enable
spe country usa
spe call-record modem
spe default-firmware spe-firmware-1
aaa new-model
aaa authentication login default group tacacs+ local
aaa authentication login NO_AUTHEN none
aaa authentication enable default group tacacs+ enable
aaa authentication ppp dialin if-needed local
aaa authorization exec default group tacacs+ local
aaa authorization commands 0 default group tacacs+ local none
aaa authorization commands 1 default group tacacs+ local none
aaa authorization commands 15 default group tacacs+ local none
aaa accounting exec default start-stop group tacacs+
aaa accounting commands 0 default start-stop group tacacs+
aaa accounting commands 1 default start-stop group tacacs+
aaa accounting commands 15 default start-stop group tacacs+
aaa accounting network default start-stop group tacacs+
aaa session-id common
ip subnet-zero
ip cef
ip dhcp excluded-address 192.168.240.1 192.168.240.127
ip dhcp excluded-address 192.168.240.150 192.168.240.254
ip dhcp pool LOCAL
network 192.168.240.0 255.255.255.0
default-router 192.168.240.1
lease 0 1
ip ssh time-out 10
ip ssh version 2
isdn switch-type primary-4ess
fax interface-type fax-mail
controller T1 3/0
shutdown
controller T1 3/1
framing esf
linecode b8zs
pri-group timeslots 1-24
description PRI on Copper
no crypto isakmp ccm
interface FastEthernet0/0
no ip address
shutdown
interface FastEthernet0/1
description Uplink to Switch – Gi0/2
ip address 192.168.240.5 255.255.255.0
duplex full
speed 100
interface Serial0/0
no ip address
shutdown
interface Serial0/1
no ip address
shutdown
interface Serial3/0:23
no ip address
shutdown
interface Serial3/1:23
description PRI on Copper
no ip address
encapsulation ppp
dialer rotary-group 2
dialer-group 2
isdn switch-type primary-4ess
isdn incoming-voice modem
isdn T306 60000
fair-queue
no cdp enable
interface Dialer2
ip unnumbered FastEthernet0/1
encapsulation ppp
dialer in-band
dialer idle-timeout 0
dialer-group 2
peer default ip address dhcp-pool LOCAL
fair-queue
no cdp enable
ppp authentication chap pap callin
ppp multilink
interface Group-Async0
no ip address
no group-range
interface Group-Async1
description Dial-up PRI modem lines
ip unnumbered FastEthernet0/1
encapsulation ppp
dialer in-band
dialer idle-timeout 0
async mode interactive
peer default ip address dhcp-pool LOCAL
fair-queue
ppp authentication chap pap callin
group-range 1/00 1/59
router eigrp 100
network 192.168.240.0
auto-summary
ip classless
ip route 0.0.0.0 0.0.0.0 192.168.240.1
ip tacacs source-interface FastEthernet0/1
no ip http server
no ip http secure-server
logging history debugging
logging trap debugging
logging x.x.x.x
access-list 101 deny eigrp any any
access-list 101 permit ip any any
access-list 101 remark dialer-list used for dialer-list 1
access-list 182 remark *** PERMIT SSH TO THIS DEVICE ***
access-list 182 permit tcp any any eq 22
access-list 182 deny ip any any log
dialer-list 1 protocol ip permit
tacacs-server host x.x.x.x
tacacs-server host x.x.x.x
tacacs-server directed-request
tacacs-server key 7 *******************
control-plane
voice-port 3/0:D
voice-port 3/1:D
dial-peer cor custom
ss7 mtp2-variant Bellcore 0
ss7 mtp2-variant Bellcore 1
ss7 mtp2-variant Bellcore 2
ss7 mtp2-variant Bellcore 3
line con 0
exec-timeout 0 0
logging synchronous
line aux 0
no exec
line vty 0 4
access-class 182 in
exec-timeout 30 0
logging synchronous
transport input ssh
escape-character BREAK
line 1/00 1/59
no modem callout
modem Dialin
rotary 1
transport input all
transport output all
autoselect during-login
autoselect ppp
scheduler allocate 10000 400
ntp clock-period 17180055
ntp server x.x.x.x
end
Cisco AS5350 IOS: c5350-ik9s-mz.123-11.T11.bin
Is anyone aware of an IOS bug or an error in my configurations that could be causing the delay in creating an ARP entry for the dial-in client?
I am open to any suggestions.
BTW, if I add static arp entries on the server, ICMP replies are typically received after one or two request timeouts.
However, I feel this is not a solution to the problem, only a band-aid fix.
arp -s 192.168.240.128 00-0d-28-0c-fe-1b
arp -s 192.168.240.129 00-0d-28-0c-fe-1b
arp -s 192.168.240.130 00-0d-28-0c-fe-1b
arp -s 192.168.240.131 00-0d-28-0c-fe-1b
arp -s 192.168.240.132 00-0d-28-0c-fe-1b
arp -s 192.168.240.133 00-0d-28-0c-fe-1b
arp -s 192.168.240.134 00-0d-28-0c-fe-1b
arp -s 192.168.240.135 00-0d-28-0c-fe-1b
arp -s 192.168.240.136 00-0d-28-0c-fe-1b
arp -s 192.168.240.137 00-0d-28-0c-fe-1b
arp -s 192.168.240.138 00-0d-28-0c-fe-1b
arp -s 192.168.240.139 00-0d-28-0c-fe-1b
arp -s 192.168.240.140 00-0d-28-0c-fe-1b
arp -s 192.168.240.141 00-0d-28-0c-fe-1b
arp -s 192.168.240.142 00-0d-28-0c-fe-1b
arp -s 192.168.240.143 00-0d-28-0c-fe-1b
arp -s 192.168.240.144 00-0d-28-0c-fe-1b
arp -s 192.168.240.145 00-0d-28-0c-fe-1b
arp -s 192.168.240.146 00-0d-28-0c-fe-1b
arp -s 192.168.240.147 00-0d-28-0c-fe-1b
arp -s 192.168.240.148 00-0d-28-0c-fe-1b
arp -s 192.168.240.149 00-0d-28-0c-fe-1b
Thank you for taking the time to read my post.
-BradHi Krishnamraj,
How many records are you gettnig from server..?? Are they very huge..??
Thanks,
Bhasker -
Sometimes Local Address not in ARP table and Ping fails (network problem?)
I see something like this on our network a couple of times a week.
The same replies have been received from different hosts.
ping fails
local subnet machine is not in arp table
ping fails
local subnet machine is not in arp table
traceroute may or maynot succeed
If traceroute succeeds an entry is in the arp table
if traceroute fails no entry will be in the arp table.
A netstat -s, ont the local host, doesn't show any thing strange except that udpNoPorts=10844982 (Unfortunately I don't know what udpNoPorts is)
The remote host IS UP.
Does anyone have an idea as to why this is happening?
Can our 100mb network, which is not that busy, be loosing that many ICMP or ARP messages?
This is a problem because I'm the guy getting paged if a system is down.
Local host is Solaris 7 on same subnet at IP 168.173.8.8
Remote hosts are usually NT boxes.
/usr/sbin/ping -svR stpaul_web2 56 3
----stpaul_web2.agribank.com PING Statistics----
3 packets transmitted, 0 packets received, 100% packet loss
/usr/sbin/arp stpaul_web2
stpaul_web2 (168.173.8.143) -- no entry
/usr/sbin/ping -svR stpaul_web2 56 3
----stpaul_web2.agribank.com PING Statistics----
3 packets transmitted, 0 packets received, 100% packet loss
/usr/sbin/arp stpaul_web2
stpaul_web2 (168.173.8.143) -- no entry
/usr/sbin/traceroute stpaul_web2
1 stpaul_web2.AGRIBANK.COM (168.173.8.143) 2995.868 ms 0.231 ms 0.211 ms
/usr/sbin/arp stpaul_web2
stpaul_web2 (168.173.8.143) at 0:1:2:cc:a3:51
Any help is greatly appreciated.
KsHi,
I Think you need to do Teaming on the servers.
++ configure etherchannel between switch and the server.
configuring etherchannel b/w 4503 and server:
================================
http://www.cisco.com/en/US/tech/tk389/tk213/technologies_configuration_example09186a008089a821.shtml
Sample NIC Teaming - HP NICs with Cisco Switches (EtherChannel) :
==============================================
http://support.citrix.com/article/CTX434260
There are several NIC teaming technologies available today from switch vendors. Cisco uses the term “EtherChannel.” Various switch vendors use various terms, and these may or may not provide the same exact functionality. Use of EtherChannel technology requires support from the server hardware vendor, NIC vendor, and Layer-2 switch vendor.
Hope this helps
Cheers
Somu
Rate helpful posts -
Hi we observed that if one of our Cluster is switching over( secondary is taking over the cluster IP adress) and sending Grat.ARP to announce the new MAC adress for the Cluster IP address, the Nexus does not update the ARP table based on a Grat.ARP.
I am not absolutely sure it seems if the Grat.ARP hits the HSRP active Switch than it works, if it hits HSRP standby switch it does not work.
Under vPC we have arp synchronize konfigured.
The only way how we can bring it to workmis to clear the arp table.
Any idea?
Thx
HubertHi Hubert,
ok I see; but what about the cluster? Is it somehow related to vmware? which OS are we talking about?
I insist on this point as usually server administrators are not too much into networking and they might give misleading info. I.E. the vmware heartbeat that control the IP floating mechanism (which happens in case of cluster failovers) relies on ARP probes and not on GARP. The 2 are pretty similar but a N7K ignores it if it is not destined to the local IP (which it is true for the active hsrp member only). It flags it as invalid packet and drops it... you can verify taking the following before and after the switchover from both n7k
show ip arp statistics vlan
often you see increasing value if you have many hosts in the vlan... so the outputs of this command is not definitive in all cases.
Riccardo -
Hi All,
I have devices which don’t send gratuitous arp when they plugged into a L3 switch. A problem occurs when one of this devices fails - and is replaced by another one (with another mac -- IP the same). The l3 switch doesn't update his ARP Table with the new mac and so the ping fails. When I clear the arp cache -> the arp table is updated by a new arp - request and the ping works.
The next workaround was to modify the ARP TIMEOUT to 60 sec. So when is swap an failed device - this takes longer then 60 sec - I thought that the arp cache is cleared in the meanwhile for this interface BUT it wasn't
How affect's the arp timeout - timer on the arp cache. I modified it many different values but the arp entry does not disappear??
thx
maxI believe that you are confusing the arp table and the cam table. You are correct that the associated entries in the cam for an interface are purged if the switch interface goes down. And in my experience the entries in the arp table are purged if the interface on which they are learned goes down. But frequently the device doing the arp is the layer 3 router and not the switch to which the PC or server was connected. So the old PC is removed, the replacement PC is booted up and configured, and the entry in the arp table does not change.
Max
I am surprised to read your post. In my experience setting the arp timeout has been effective. Would it be possible for you to recreate the issue and to post some outputs of events during the test? In particular I would be interested to see the output of debug arp and the output of show ip interface for the interface where the test device is connected (to verify the arp setting).
HTH
Rick -
Q: How many Users do you serve w. your OES11-Infrastructure?
Reason for this - at first glance - stupid- question is the following:
We migrated (or Novell forced us to) from our low power (two HL DL360 G3 and 2 DL360 g4 one G5) 6.5 Cluster to a very juicy (6xDL360 G8, tons of proccessors) OES 11 cluster. We serve
approx. 1600 - 1700 Clients. Today everything stopped some of the Cluster Nodes didn't respond. This never happend in our old Cluster!
The reason was or IS by "Enterprise" Design:
DMESG: ipv4: Neighbour table overflow. !!!! Are we the only ones in the (small Novell) World who serves more than 1000 Clients???? WHY is there a limitation in the IP Cluster stack???
Did one EVER tested that. Should we better migrate to M$???
ip neigh show | wc -l --> shows 1021 entrys
sysctl net.ipv4.neigh.default.gc_thresh3
net.ipv4.neigh.default.gc_thresh3 = 1024
Means one cluster Node can handle 1024 connections. What if on one Node runs the DNS server? Only 1020 of 1600 Clients get an answer! Cool enterprise solution boys!
VERY angry!On 03/25/2014 04:56 AM, jottschi wrote:
>
> Reason for this - at first glance - stupid- question is the following:
> We migrated (or Novell forced us to) from our low power (two HL DL360 G3
> and 2 DL360 g4 one G5) 6.5 Cluster to a very juicy (6xDL360 G8, tons of
> proccessors) OES 11 cluster. We serve
> approx. 1600 - 1700 Clients. Today everything stopped some of the
> Cluster Nodes didn't respond. This never happend in our old Cluster!
> The reason was or IS by "Enterprise" Design:
> DMESG: ipv4: Neighbour table overflow. !!!! Are we the only ones in the
> (small Novell) World who serves more than 1000 Clients???? WHY is there
> a limitation in the IP Cluster stack???
I'm sorry to hear about the lack of availability of the cluster nodes. I
am sure it is frustrating to have a system configured to be
highly-reliable (via clustering) to fail due to default settings. With
that said, this is a default Linux setting, and you're welcome to change it.
> Did one EVER tested that. Should we better migrate to M$???
I do not understand what you're asking.... did anybody test this on large
broadcast domains?
> ip neigh show | wc -l --> shows 1021 entrys
Wow, well that's quite a few. I think this number indicates something you
do not realize, though. While something like 'ss' or 'netstat' will tell
you how many layer three (IP) or four (TCP/UDP) connections your system
has, that number is not reflected by 'ip neigh', so usually the number of
clients your system has is limited in the list of neighbors by quite a
bit. For example, if I make a connection on my laptop (openSUSE, but same
default for my neighboring table) to Google, I can see that I have a fair
number of network connections total as shown:
Code:
me@mybox:~/Desktop> /usr/sbin/ss -planeto | grep -c ESTAB
61
but that's only tangentially related to the number of neighbors my laptop
sees:
Code:
me@mybox:~/Desktop> ip neigh
192.168.1.20 dev eth0 lladdr 00:1d:09:03:54:02 STALE
192.168.1.254 dev eth0 lladdr 00:1e:2a:74:66:35 STALE
192.168.1.1 dev eth0 lladdr f8:8f:ca:40:7a:1c REACHABLE
192.168.255.50 dev br8 lladdr ac:d3:58:ae:8e:ac STALE
How can that be? I have at least eight connections to unique remote
machines, so shouldn't I have at least eight listings in 'ip neigh'
output? The answer, of course, is 'no' because what you see from 'ip
neigh' are the cached addresses found via ARP, and ARP only matters within
the current broadcast domain, meaning on your local network before you hit
any routers. Even though I'm connected out to eight machines, all my
local routing table needs to care about (despite being VPN'd to a few
places and therefore having a few extra networks considered semi-local)
are the server, router, another router, and printer. Also, this number,
as you can see above, has entries become stale pretty quickly. I think
that time period is, again by default, something like thirty seconds,
since there is no guarantee from one second to the next that a given
server, workstation, or other node on the local network will still be
there. After something is stale the system just uses ARP again to see
what's still out there.
> sysctl net.ipv4.neigh.default.gc_thresh3
> net.ipv4.neigh.default.gc_thresh3 = 1024
Yes, this is a default for the Linux kernel in general. You're welcome to
tune it using sysctl, or probably via Yast so that it is stored to be used
after rebooting (and for the nice simple UI, if you're into that kind of
thing).
> Means one cluster Node can handle 1024 connections. What if on one Node
> runs the DNS server? Only 1020 of 1600 Clients get an answer! Cool
> enterprise solution boys!
No, it means that a given box, by default, can handle 1024 connections
from the local network. A typical Class C network, for example, only has
254 possible nodes, so only 253 connections. I've seen enterprises do
some supernetting to get things like 1022 nodes possible in a network, and
sometimes even fill that, but that's it. I have been on networks that had
addresses handed out from a Class B-like range (172.16.x.x) or even a
class A range (10.x.x.x) but those networks never had 65k or 10M boxes on
them because when you get too far beyond a thousand nodes you typically
suffer a bit of slowness. In your case you not only have more than 1024
nodes, but all of them are talking to the same server at the same time on
that same network.
Can Linux handle more? Sure, you're dealing with the same OS that runs
most supercomputers in the world, many of them being large clusters of
computers all working together in a single network, so obviously the
potential is there; however, there are reasons that one does not allow the
kernel to chew up all memory for a setting like "how many neighbors do you
have cached right now" when, for 99.9999% of the world, that max number
is around 200 since this specifically deals with neighbors on the same
segment or broadcast domain, not neighbors meaning anything reachable via
IP. Doing otherwise, for example setting it to 1,000,000, means that
somebody who was clever could more-easily waste your system's memory via a
denial of service (DoS) attack by flooding the network with bogus ARP
entries. Every one takes some memory, and every one needs to be managed
by the kernel until it is expired and removed, and again those networks
needing more than 1024 to be remembered within any single thirty-second
period are pretty rare. Your network must be pretty awesome to handle
that number of concurrent systems in the same logical network without
degradation.
Regarding running a DNS server, that is a good possibility as a service
that would experience this problem assuming all of the clients were on the
same network as the DNS server itself. While possible, the number of
networks where I've seen that has been pretty small overall since usually
those machines (servers) end up in a DMZ somewhere to prevent
unauthorized, or at least unaudited, access.
> VERY angry!
You're right in that this limitation should be documented, and made quite
obvious wherever it is documented. The migration guide would make sense,
as that would be a place where a lot of assumptions about defaults would
be questioned due to the OS change. If you have a specific page in the
documentation where you read through and expected limitations to be called
out, please either submit feedback, perhaps linking to this thread to
provide the background that you experienced, or post a link here an I'll
do the same.
If there are any other defaults that you feel should be changed,
particularly for your environment, sharing those may help as well.
In the meantime, you can change your current settings by running the
following as 'root' if you have not done so already:
Code:
cp /etc/sysctl.conf /etc/sysctl.conf-`date +%s`
cat << EOM >> /etc/sysctl.conf
# Setup higher threshold for arp
net.ipv4.neigh.default.gc_thresh3 = 4096
net.ipv4.neigh.default.gc_thresh2 = 2048
net.ipv4.neigh.default.gc_thresh1 = 1024
EOM
To apply the changes either run 'sysctl -p' (from memory I think that's
the right command) or else reboot the node, then perform again on other nodes.
Good luck.
If you find this post helpful and are logged into the web interface,
show your appreciation and click on the star below... -
I have a requirement to NAT a spare address on the same subnet range as one of the firewall interface - however, because this is not allocated to a physical interface, there is no mac entry in the arp cache. the other end of the link from the firewall is connected to a router which has no idea how to reach this "virtual address" - again because there is no entry in the arp cache
I have tried to put a static arp entry into the firewall but this doesn't appear to work either. Should I be using a mac address form a physical interface or can I create a dummy mac for this -
If the router can't see the ip address, then users will not be able to target this address - so that the firewall can NAT to the real outside address.
I have tried routes to null0 on the router and static arp entries on both devices but the user just times when trying to connect to 10.2.7.11 (nat to 10.2.32.11)
attached is a very basic visio diagram which I hope explains what I am trying to achieve.
any help would be appreciated.
many thanksAssuming your communications are always initiated from the inside, the first static statement above should suffice. When a session is built (initial syn in the TCP 3-way handshake) the xlate table will take care of the NAT on return path. I'm not sure of the effect of the second static, but I'd try temporaily removing it.
If you ever initiate from the outside (10.2.32.11/12), you would also need an access-list to allow moving from a lower security to higher security level.
Hope this helps. -
Hello all,
my CSS 11150 with WebNS 5.00 does excessive arp requests on its interfaces (up to 100 arps per second). The box seems to arp EVERYTHING especially in the 10.147.0.0 /16 subnet even if it is not used at all. My config is as follows:
ip no-implicit-service
ip opportunistic disable
ip route 0.0.0.0 0.0.0.0 10.147.1.1 1
circuit VLAN1
ip address 10.147.248.10 255.255.0.0
circuit VLAN2
ip address 10.145.45.254 255.255.255.128
service sunbl3s6-443
ip address 10.145.45.136
protocol tcp
port 443
keepalive type tcp
keepalive port 443
active
service sunbl3s6-80
ip address 10.145.45.136
protocol tcp
port 80
keepalive type tcp
keepalive port 80
active
service sunbl3s7-443
ip address 10.145.45.137
protocol tcp
port 443
keepalive type tcp
keepalive port 443
active
service sunbl3s7-80
ip address 10.145.45.137
protocol tcp
port 80
keepalive type tcp
keepalive port 80
active
owner unix-systems
content vrp-test-443
vip address 10.145.45.253
protocol tcp
port 443
balance aca
add service sunbl3s6-443
add service sunbl3s7-443
active
content vrp-test-80
vip address 10.145.45.253
protocol tcp
port 80
balance aca
add service sunbl3s6-80
add service sunbl3s7-80
active
group vrp-test
vip address 10.145.45.253
add destination service sunbl3s6-80
add destination service sunbl3s6-443
add destination service sunbl3s7-80
add destination service sunbl3s7-443
active
Does anybody have any hints?
Many thanks in advance
UliHi,
I did a software upgrade yesterday and put ap0610405.adi.gz on the box. But the behaviour didn't change. We also checked the cabling for loops, that's also fine.
We have observed some further things:
The broadcasts are only on the 10.147.0.0 /16 subnet. As this is our local lan backbone we can't change it, I could only shift the frontend into another subnet and route it towards the backbone.
We have another two boxes (CSS11503 with 7.4) with a similar configuration - they also do excessive arp requests in the same subnet, the primary as well as the secondary. But the addresses being arped for are not necessarily the same.
I took some packet traces looking for broadcasts and multicasts that could inspire the boxes to arp for every address they see - nothing, the addresses being arped for are not seen in the seconds before the CSS arp request.
What could trigger arp requests for machines which never accessed or used the CSS services / rules??? I've never seen such a behaviour before...
Best Regards
Uli -
Cisco ASA 5580 Arp Collision Errors
Dears,
I am receiving allot of Errors "%ASA-4-405001: received ARP collision from IP/MAC on interface dmz1 with existing ARP Entry IP/MAC
When i checked this MAC address in the same firewall it shows too many IP Addresses.
What could be the reason ?
Thanks...Hello Richard,
My first though is why is the ASA receiving this traffic is this is traffic that should not reach the default-gateway.
Anyway try the following
same-security-traffic permit intra-interface
Let me know how it goes
Julio -
Hellow Nexus Gurus,
I have had numerous instances where Broadcom NICs on Dell servers have started storming the LAN with directed ARP requests (unicast) for addresses off the subnet of the station sending the request. I've had stations send 2GB of ARP requests to the 7K in under a minute in some cases. Oddly this has not completely taken out the data center. It has only caused weird temporary outages to some servers throughout various subnets. I have no idea why the EDC wasnt taken out but I assume that many servers were saved due to the preconfigured COPP configuration.
class-map type control-plane match-any copp-system-class-redirect
match redirect arp-inspect
Can anyone explain the behavior above as well as what that class-map does?
Does anyone have a solution to prevent these unicast ARP storms in the future?
Any insight would be much appreciated.
/r
RobWe are seeing issues with our Broadcom NIC, Dell, Hyper V servers with NIC teaming to seperate Nexus 2248's where random virtual servers will stop responding, sometimes the eventually start responding again sometimes we move them to a different chassis. Is that the kind of "weird temporary outages" you were experiencing? And how did you find the ARP storms?
Thanks -
ARP table clash with checkpoint and ASA firewal issue
We are migrating DMZ segments from a checkpoint to a ASA 5585 firewall that we had connected to the same segments as the Checkpoint except on different IP addresses then the checkpoint interfaces. The Checkpoint interfaces are the default gateway for the servers. When I implemented the NATs entries below we experienced an arp table clash with the checkpoint and ASA firewall on the local segments that caused a application outage. What was determined was that the checkpoint firewall was showing that all the IP addresses in particular on vlan130 segment was associating the MAC address of the ASA interface instead of the real sever MAC address. I need assistance understanding the reason why the Checkpoint was pointing the ARP entries for many different address on VLAN130 to the ASA firewall MAC?
nat (any,internet-outside) source static any any destination static isxh2007_Xlate_167.9.6.21 isxh2007_10.121.201.86 unidirectional description To match chkpt NAT rule #5
nat (VLAN130,internet-outside) source static ISX_EDI_Hosts isxh2008_Xlat_167.9.6.22 unidirectional
nat (any,internet-outside) source static Private-Addresses ISX_OUTBOUND_NAT_167.9.6.1 destination static external_167.9.x external_167.9.x unidirectional
nat (any,any) source static Mars-Internal-All Mars-Internal-All destination static Private-Addresses Private-Addresses
nat (internet-dmz,internet-outside) source static acs-vmww2419.mars-ad.net acs-vmww2419_xlate_167.9.6.23
nat (internet-dmz,internet-outside) source static acs_vmww2420 acs_vmww2420_xlate_167.9.6.24
nat (internet-dmz,internet-outside) source static pass_reset_internal_10.121.201.50 pass_reset_external_167.9.6.25
nat (internet-dmz,internet-outside) source static HE-Portal-poland_10.121.120.10 ext_HE-Portal-poland_167.9.6.26
nat (any,internet-outside) source dynamic any ISX_OUTBOUND_NAT_167.9.6.1
isxasa04/wwy-legacy# sho interface
Interface TenGigabitEthernet0/8.129 "core-inside", is down, line protocol is down
MAC address 442b.0330.aba2, MTU 1500
IP address 10.121.129.X, subnet mask 255.255.255.0
Traffic Statistics for "core-inside":
241633 packets input, 12094352 bytes
44788 packets output, 3032584 bytes
109732 packets dropped
Interface TenGigabitEthernet0/9.130 "VLAN130", is down, line protocol is down
MAC address 442b.0330.aba3, MTU 1500
IP address 10.121.130.X, subnet mask 255.255.255.0
Traffic Statistics for "VLAN130":
1264203 packets input, 136452168 bytes
326080 packets output, 69216516 bytes
794035 packets dropped
Interface TenGigabitEthernet0/9.136 "VLAN136", is down, line protocol is down
MAC address 442b.0330.aba3, MTU 1500
IP address 10.121.136.X, subnet mask 255.255.255.0
Traffic Statistics for "VLAN136":
374547 packets input, 23696109 bytes
51186 packets output, 3324895 bytes
173500 packets dropped
Interface GigabitEthernet0/1 "internet-outside", is down, line protocol is down
MAC address 442b.0330.ab9b, MTU 1500
IP address 167.9.6.X, subnet mask 255.255.255.0
Traffic Statistics for "internet-outside":
352158 packets input, 17245425 bytes
76888 packets output, 3872904 bytes
12255 packets dropped
Interface GigabitEthernet0/2 "internet-dmz", is down, line protocol is down
MAC address 442b.0330.ab9c, MTU 1500
IP address 10.121.201.X, subnet mask 255.255.255.0
Traffic Statistics for "internet-dmz":
237795 packets input, 12460108 bytes
40787 packets output, 2775684 bytes
27378 packets dropped
Interface GigabitEthernet0/4 "VLAN140", is down, line protocol is down
MAC address 442b.0330.ab9e, MTU 1500
IP address 10.121.140.X, subnet mask 255.255.255.0
Traffic Statistics for "VLAN140":
386931 packets input, 18807725 bytes
48936 packets output, 3319712 bytes
114417 packets dropped
We crosschecked MAC addresses and this is what we found:
Checkpoint ARP table:
10.121.130.101 44:2b:3:30:ab:a3 3285
ASA ARP table:
isxasa04/wwy-legacy# sh arp | i 10.121.130.101
VLAN130 10.121.130.101 001a.4b06.dd45 10525
Server real address provided by processing:
0x001A4B06DD45
When we saw that the Checkpoints had a different/wrong entry we shut down all the physical ports on the new ASAs (except for failover and management);
Kevin cleared the ARP table on the Checkpoints and problem was solved;
Later I saw this:
isxasa04# sh int | i MAC
MAC address 442b.0330.ab9a, MTU not set
MAC address 442b.0330.ab9b, MTU not set
MAC address 442b.0330.ab9c, MTU not set
MAC address 442b.0330.ab9d, MTU 1500
MAC address 442b.0330.ab9e, MTU not set
MAC address 442b.0330.ab9f, MTU not set
MAC address 442b.0330.aba0, MTU not set
MAC address 442b.0330.aba1, MTU not set
MAC address 442b.0330.ab98, MTU not set
MAC address 442b.0330.ab99, MTU not set
MAC address 442b.0330.aba2, MTU not set
MAC address 442b.0330.aba3, MTU not setThe Asa is proxy Arping those macs. Turn off proxy arp and put in static arp entries until you completely shut down the checkpoint.
Sent from Cisco Technical Support iPad App
Maybe you are looking for
-
Execution Errors while running a mapping having external table operator
when I am executing a mapping with external table operator, I am getting the following error: ORA-29913: error in executing ODCIEXTTABLEOPEN callout ORA-29400: data cartridge error KUP-04040: file testdoc.txt in CONN_FLAT_FILE not found Here CONN_FLA
-
Manipulate the URL in HTTP receiver adapter possible?
hello together, i have an HTTP receiver adapter. To the parameters which i can tick in XI as default (sender party, sender service, ....) i want to add some parameters too. who can i do that? i have read the weblog from william li. i tried to change
-
How do I delete "Events from my Mac" from my iPad photo library?
I usually sync images from Aperture via iTunes. And though I've unchecked all the boxes so that no photos were to be synced, the pad still has a ton of photos that show up in the Events or Moments. I've now turned off Photo syncing in iTunes, which s
-
QuickView uses EXIF data for preview, even when bogus. A plist to turn that off?
We moved to MBPs with SnowLeopard recently and just found that many downloaded images usually from publicity or stock-photo sites - give distorted or no preview when we use QuickView (tapping the space-bar) on them in the Finder. Some investigation
-
How to disable a service in PI configuration?
Hi! Experts: We have a need to disable some services (such as principal propagation, etc. ) in the PI configuration. We looked into the exchangeProfile but found none. Where to display /edit the information like that? Thanks! Pints guaranteed.