%SW_MATM-4-MACFLAP_NOTIF: Host 00ff.ffff.ffff in vlan 112 is flapping between port Gi3/0/1 and port Fa2/0/46.....

Question

Hi Guys,
Iam seeing above issue on two of my switches connected to core switch ....i know there are quite of few discussion open on same issue but mine is diff....
i see same issue on two switches connected via core swicth on same vlan ( 112)....when i do mac address lookup it says the mac thats generating this error is invalid so cant track the source of this mac....also just saw on topoogy change notification on core traced it back to originating switch which is also generating this error but dnt see any change on the switch that is generated topology change notification....prob is vlan 112 all interface on both switches conected via core are generating this message so five interfaces each .....any expert advise on how to approach it as i cant get to source port generating this as nearly five ports in vlan 112 on bloth switches generating this error. thanks
Apr 15 15:56:08: %SW_MATM-4-MACFLAP_NOTIF: Host 00ff.ffff.ffff in vlan 112 is flapping between port Fa3/0/46 and port Gi3/0/1
Apr 15 15:56:50: %SW_MATM-4-MACFLAP_NOTIF: Host 00ff.ffff.ffff in vlan 112 is flapping between port Fa2/0/46 and port Gi3/0/1
Apr 15 15:56:51: %SW_MATM-4-MACFLAP_NOTIF: Host 00ff.ffff.ffff in vlan 112 is flapping between port Fa3/0/46 and port Gi3/0/1
Apr 15 15:58:29: %SW_MATM-4-MACFLAP_NOTIF: Host 00ff.ffff.ffff in vlan 112 is flapping between port Fa2/0/46 and port Gi3/0/1
Apr 15 15:59:27: %SW_MATM-4-MACFLAP_NOTIF: Host 00ff.ffff.ffff in vlan 112 is flapping between port Gi3/0/1 and port Fa2/0/46
Apr 15 15:59:45: %SW_MATM-4-MACFLAP_NOTIF: Host 00ff.ffff.ffff in vlan 112 is flapping between port Fa2/0/46 and port Gi3/0/1
Apr 15 16:00:14: %SW_MATM-4-MACFLAP_NOTIF: Host 00ff.ffff.ffff in vlan 112 is flapping between port Gi3/0/1 and port Fa3/0/46
Apr 15 16:00:36: %SW_MATM-4-MACFLAP_NOTIF: Host 00ff.ffff.ffff in vlan 112 is flapping between port Fa2/0/46 and port Gi3/0/1
Apr 15 16:02:40: %SW_MATM-4-MACFLAP_NOTIF: Host 00ff.ffff.ffff in vlan 112 is flapping between port Fa3/0/46 and port Gi3/0/1
Apr 15 16:03:22: %SW_MATM-4-MACFLAP_NOTIF: Host 00ff.ffff.ffff in vlan 112 is flapping between port Fa3/0/46 and port Gi3/0/1
Apr 15 16:03:31: %SW_MATM-4-MACFLAP_NOTIF: Host 00ff.ffff.ffff in vlan 112 is flapping between port Gi3/0/1 and port Fa2/0/46
Apr 15 16:04:03: %SW_MATM-4-MACFLAP_NOTIF: Host 00ff.ffff.ffff in vlan 112 is flapping between port Fa3/0/46 and port Gi3/0/1
Apr 15 16:04:34: %SW_MATM-4-MACFLAP_NOTIF: Host 00ff.ffff.ffff in vlan 112 is flapping between port Fa2/0/46 and port Gi3/0/1
Apr 15 16:04:41: %SW_MATM-4-MACFLAP_NOTIF: Host 00ff.ffff.ffff in vlan 112 is flapping between port Gi3/0/1 and port Fa2/0/46
Apr 15 16:05:05: %SW_MATM-4-MACFLAP_NOTIF: Host 00ff.ffff.ffff in vlan 112 is flapping between port Fa2/0/46 and port Gi3/0/1
Apr 15 16:05:13: %SW_MATM-4-MACFLAP_NOTIF: Host 00ff.ffff.ffff in vlan 112 is flapping between port Gi3/0/1 and port Fa3/0/46
sh spanning-tree vlan 112
VLAN0112
  Spanning tree enabled protocol rstp
  Root ID    Priority    8192
             Address     001e.13c1.5a70
             Cost        3004
             Port        109 (GigabitEthernet3/0/1)
             Hello Time   2 sec  Max Age 20 sec  Forward Delay 15 sec
  Bridge ID  Priority    49264  (priority 49152 sys-id-ext 112)
             Address     001f.261c.1d80
             Hello Time   2 sec  Max Age 20 sec  Forward Delay 15 sec
             Aging Time 300
  UplinkFast enabled but inactive in rapid-pvst mode
Interface        Role Sts Cost      Prio.Nbr Type
Fa2/0/46         Desg FWD 3019      128.104  P2p
Fa1/0/46         Desg FWD 3019      128.50   P2p
Gi3/0/1          Root FWD 3004      128.109  P2p
Fa3/0/46         Desg FWD 3019      128.158  P2p
Fa3/0/47         Desg FWD 3100      128.159  P2p
Fa3/0/48         Desg FWD 3019      128.160  P2p

Answer 1

ASAK Mohammed,
There are lots of thread discussing about this, you should do a search before creating a new post.
Anyway, this is how you approach these types of flapping:
1. Is the the given MAC flapping in the log flapping only 1 time or you see it multiple times over a reasonobly short time?
   If you see it only once or once every 2-3 hours this might be not an  issue worth being investigated. Sporadic one time flapping are expected  in L2 broadcast domain.
If you see it often continue to step 2.
2. Identify and locate the flapping mac in vlan 125: 3270.990a.a504
Is  the mac of a dual-homes server using some kind of load balancing  algorithm (active/active) for which the same address is used from both  NICs?
If yes, the message is not and issue but just an indication.  Fix this type of LB (make it active/standby or make sure the server  uses 2 different mac addresses, one per NIC) or if it is not possible  leave it like this.
3. Is the MAC a the wireless NIC of a PC?
Make sure that the user was not moving from one AP to another (flapping is normal in this case)
4.
See if you have increasing TCN's and check if they are coming from the same interface.
From  this point on you keep on troubleshooting STP until you find the  offending link (likely going up and down) or the switch. You also need  to check if STP in vlan112 is coherent with the actual L2 topology you  have.
=====================================================
2- Some more details information which might be helpfull to you.
http://www.cisco.com/en/US/products/hw/switches/ps663/products_tech_note09186a
00801434de.shtml#subtopic1k
Problem
The switch generates %SYS-3-P2_ERROR: Host xx:xx:xx:xx:xx:xx is flapping
between ports? messages, where xx:xx:xx:xx:xx:xx is a MAC address.
Description
This example shows the console output that you see when this error occurs:
%SYS-4-P2_WARN: 1/Host 00:50:0f:20:08:00 is flapping between port 1/2 and port
4/39
Use the steps and guidelines in this section in order to understand and
troubleshoot the cause of this error message.
The message indicates that your Catalyst 4500/4000 switch has learned a MAC
address that already exists in the content-addressable memory (CAM) table, on
a port other than the original one. This behavior repeatedly occurs over short
periods of time, which means that there is address flapping between ports..
If the message appears for multiple MAC addresses, the behavior is not normal.
This behavior indicates a possible network problem because the MAC addresses
move quickly from one port to another port before the default aging time. The
problem can be looping traffic on the network. Typical symptoms include:
·        High CPU utilization
·        Slow traffic throughout the network
·        High backplane utilization on the switch
For information on how to identify and troubleshoot issues with spanning tree,
refer to Spanning Tree Protocol Problems and Related Design Considerations
<http://www.cisco.com/en/US/tech/tk389/tk621/technologies_tech_note09186a00800
951ac.shtml> .
If the error message appears for one or two MAC addresses, locate these MAC
addresses in order to determine the cause. Issue the show cam mac_addr command
in order to identify from where these MAC addresses have been learned. In this
command, mac_addr is the MAC address that the error reports as flapping.
After you determine between which ports this MAC address is flapping, track
down the MAC address. Connect to the intermediate devices between your
Catalyst 4500/4000 and the device that has the problem MAC address. Do this
until you are able to identify the source and how this device connects to the
network.
Note: Because the MAC address is flapping between two ports, track down both
of the paths.
This example shows how to track both of the paths from which this MAC address
has been learned:
Note: Assume that you have received this message and you have begun to
investigate it.
%SYS-4-P2_WARN: 1/Host 00:50:0f:20:08:00 is flapping between port 1/2 and port
4/39
In order to track down how this MAC address was learned from both ports,
complete these steps:
1.     Consider port 1/2 first, and issue the show cam dynamic 1/2 command.
If you see the MAC address 00:50:0f:20:08:00 in the list of the MAC addresses
that have been learned on this port, determine if this is a single host that
is connected or if there are multiple hosts that are registered on that port.
2.     On the basis of whether there is a single or multiple hosts,
investigate the device:
o   If there is a single host (00:50:0f:20:08:00) that is connected, check the
other port that is registered and see if the host is dually attached to the
switch.
In this example, the other port is port 4/39.
o   If the host has connections to other devices that can eventually lead back
to this switch, try to track down the intermediate devices.
With Cisco devices, issue the show cdp neighbors mod/port detail command. The
output provides information about intermediate devices.
Here is sample output:
Cat4K> (enable) show cdp neighbors 1/2 detail
Port (Our Port): 1/2
Device-ID: brigitte
Device Addresses:
IP Address: 172.16.1.1
Novell address: aa.0
Holdtime: 171 sec
Capabilities: ROUTER
Version:
Cisco Internetwork Operating System Software
IOS (tm) 2500 Software (C2500-JS-L), Version 12.0(7)T,  RELEASE SOFTWARE (fc2)
Copyright (c) 1986-1999 by cisco Systems, Inc.
Compiled Mon 06-DEC-99 17:10 by phanguye
Platform: cisco 2500
Port-ID (Port on Neighbors's Device): Ethernet0
VTP Management Domain: unknown
Native VLAN: unknown
Duplex: half
System Name: unknown
System Object ID: unknown
Management Addresses: unknown
Physical Location: unknown
Cat4K> (enable)
3.     Establish a Telnet session with the device and follow the path of the
MAC address.
In this example, the IP address is 172.16.1.1.
Repeat the procedure for all MAC addresses that the error message reports as
flapping.
4.     Create a simple diagram of the source device with that MAC address and
of the physical connections (the Catalyst 4500/4000 ports) from which and to
which this MAC address is flapping.
The diagram enables you to determine if this is a valid port and path for your
network layout.
If you verify that both ports on which the MAC address is flapping provide a
path toward that network node, there is a possibility that you have a
spanning-tree failure issue. Refer to Spanning Tree Protocol Problems and
Related Design Considerations
<http://www.cisco.com/en/US/tech/tk389/tk621/technologies_tech_note09186a00800
951ac.shtml>  in order to isolate and troubleshoot this loop.
In large networks in which multiple hosts from multiple vendors are
interconnected, difficulty arises as you try to track down the host with use
of just the MAC address. Use the search utility for the IEEE OUI and
Company_id Assignments <http://standards.ieee.org/regauth/oui/index.shtml>  in
order to track down these MAC addresses. This list is the front end of the
database where IEEE has registered all MAC addresses that have been assigned
to all vendors. Enter the first three octets of the MAC address in the Search
for: field of this page in order to find the vendor that is associated with
this device. The first three octets in the example are 00:50:0f.
These are other issues that can cause this message to appear:
·        Server NIC redundancy problem?There is a server with a dual-attached
NIC that misbehaves and does not follow the standards. The server uses the
same MAC address for both ports that connect to the same switch.
·        Hot Standby Router Protocol (HSRP) flapping?Flapping HSRP can cause
these messages to appear in the Supervisor Engine console. If you notice that
HSRP implementation in your network is unstable, refer to Understanding and
Troubleshooting HSRP Problems in Catalyst Switch Networks
<http://www.cisco.com/en/US/tech/tk648/tk362/technologies_tech_note09186a00800
94afd.shtml>  in order to resolve the problem.
·        EtherChannel misconfiguration?A misconfigured EtherChannel connection
can also cause these symptoms. If ports that the flapping message reports are
members of the same channel group, check your EtherChannel configuration and
refer to Understanding EtherChannel Load Balancing and Redundancy on Catalyst
Switches
<http://www.cisco.com/en/US/tech/tk389/tk213/technologies_tech_note09186a00800
94714.shtml>  in order to troubleshoot the configuration.
·        Host reflects packets back onto the network?The reflection of packets
back onto the network by a host can also cause flapping. Typically, the root
cause of this packet reflection is a broken NIC or any failure of the physical
interface of the host that is connected to the port.
If the reflection of packets by the host is your root cause, obtain a sniffer
trace and examine the traffic that goes to and from the ports on which the
messages have appeared. If a host reflects packets, you typically see
duplicate packets in the trace. The duplicate packets are a possible symptom
of this flapping of the MAC address.
Refer to Configuring SPAN and RSPAN
<http://www.cisco.com/en/US/docs/switches/lan/catalyst4000/6.3and6.4/configura
tion/guide/span.html>  for details on how to configure a port for use with a
sniffer.
·        Software or hardware defect?If you have tried to troubleshoot the
flapping message with the instructions in this section but you still notice
the issue, seek further assistance from Cisco Technical Support
<http://www.cisco.com/en/US/support/tsd_cisco_worldwide_contacts.html> . Be
sure to mention and provide documentation of the information that you have
collected while you followed the steps. This information makes further
troubleshooting quicker and more efficient.
HTH
REgards
Inayath
*Plz rate all usefull posts.