Nexus 7000 - Adjacency table exhausted

Hi Experts,
Please give us some suggestions. I'm running a couple N7K as campus's core. Separate two LC on each box to TWO VDC e.g. Backbone & Wireless VDC. Wireless VPC is paring to 4 WLC, 2 links each. Between N7K and WLC running in layer 2 mode. We serve both IPv4 & IPv6.
Problem is one day on last week, the first day of university's semester. The N7K goes corrupt. After along while troubleshooting we found that due to this logs
N7K1 %IPFIB-SLOT4-4-CLP_FIB_ADJ_EXHAUSTED: Adjacency allocation failed on instance 6
We have tested for how adjacency table being accumulated. Then we found the following captured picture.
What I did wrong? It seem like N7K F2e Line-card be able to hold only 10,000+ MAC address (host)?
Please give us some suggestions.
Nipat CCIE#29422

The F2e cards can only support 16K MAC addresses per SoC (Switch on a Chip) and a total of 16K ARP entries total. The command "show hardware capacity forwarding" will show you the complete picture for layer 2. That's why we now have 7706s with F3 cards.

Similar Messages

PortChannel table in Nexus 7000/5000 through MIB

I ma trying to query   "CISCO-PORT-CHANNEL"   mib on Nexus 7000 for portChannel table and I am not getting any info.
Nexus OS versions : Nexus   7000 - System version: 5.1(5)
                                    Nuxus 5000 -    System version: 5.0(3)N1(1a)
Any pointers or other alternatives to query through MIB ?
Thanks,
Chandra.

Hi Chandra
Here is a list of MIBS supproted on n7k:
ftp://ftp-sj.cisco.com/pub/mibs/supportlists/nexus7000/Nexus7000MIBSupportList.html
You can use IF-MIB to poll general port-channel interface status and use following command to find port-channel interface indexes among other output:
sh interface snmp-ifindex
Also you can poll the CISCO-LAG-MIB for port-chanel details.
HTH,
Alex

GLBP on Nexus 7000's

We have GLBP configured on two Nexus 7000's using "load-balancing host-dependant" as our method of balancing. My question is, is there a quick way to determine which router each host is using. These are in production so any debugging is frowned on.
Thanks
David

Available command to verify GLBP operation is following
show glbp [group group-number]
Displays the GLBP status for all or one group.
show glbp capability
Displays the GLBP capability for all or one group.
show glbp interface interface-type slot/port
Displays the GLBP status for an interface.
show glbp interface interface-type slot/port [active] [disabled] [init] [listen] [standby]
Displays the GLBP status for a group or interface for virtual forwarders in the selected state.
show glbp interface interface-type slot/port [active] [disabled] [init] [listen] [standby] brief
Displays a brief summary of the GLBP status for a group or interface for virtual forwarders in the selected state.
But none of these will show you which host uses which AVF. In GLBP, Hosts still point to a default gateway IP address, but GLBP causes different hosts to send their traffic to one of up to four routers in a GLBP group. To do so, the GLBP Active Virtual Gateway (AVG) assigns each router in the group a unique virtual MAC address format 0007.B400.xxyy, where xx is the GLBP group number, and yy is a different number for each router (01, 02, 03, or 04). When a client ARPs for the (virtual) IP address of its default gateway, the GLBP AVG replies with one of the four possible virtual MACs. By replying to ARP requests with different virtual MACs, the hosts in that subnet will in effect balance the traffic across the routers, rather than send all traffic to the one active router. You can check ARP table of host and see the mac address of default gateway. But this is not an easy way.

Nexus 7000 route leak from GRT (default VRF) to other VRF's

Hello
We have a Nexus 7000 infrastructure whereby we have had multiple VDC's and VRF's deployed. A requirement has now come about whereby one of these VRF's needs to be able to see our GRT (default VRF) so we need to leak the GRT routes into the VRF and vice versa.
I have been doing a lot of reading and I am happy with the how this works with inter-VRF route leaking but I seem to missing a few things in respect of how this works with the GRT.
I have also read on another forum that this is not supported. See link below.
https://supportforums.cisco.com/document/133711/vrf-configuration-and-verification-nexus-7000
Does anyone have experience of this? I can also see how this works in IOS and I have GNS3 and got this working.
We use BGP currently so we are able to use MP-BGP if required.
Any help would be very useful.

Hi,
In Table 14 of the Cisco Nexus 7000 Series NX-OS Verified Scalability Guide the verified limit is specified as 1000 per system i.e., across all VDCs for NX-OS release 5.2, 6.0 and 6.1.
There is a footnote associated with this number which states:
With each new VDC configured, the number of configurable VRFs per system is reduced by two as each VDC has a default VRF and management VRFs that are not removable. For example, with 8 configured VDCs on Cisco NX-OS Release 5.2, you can configure up to 984 VRFs per system (either all in one VDC or across VDCs).
Regards

IP CEF hardware adjacency resources exhausted on Catalyst 4500

Hello,
I have a Cat 4510R with IOS 12.2(25)EW 5 and we are migrating an old network and connecting to this switch. We define a temporary VLAN xxx configured with an ip address 10.1.102.x /24 bits mask where the old network is connected. The old network has the ip subnet 10.1.x.x. The cat 4500 have another VLANs defined with ip address of 10.1.x.x /24 so we need to define a new VLAN for the old network and the IP subnets that not belong to this one, have to resolve the route through proxy arp. The switch have a default route through this VLAN xxx.
The problem we have is that the IP CEF table ("show ip cef details) is populated with too much entrys (internet hosts also are in the table) and sometimes appeared the error : Feb 20 11:05:38: %C4K_IOSIPROUTEMAN-3-ADJMANNOMOREADJS: AdjMan: hardware adjacency resources exhausted, performance may be degraded.
It's seems the problem is related to this configuration but I don't know how can i do to resolve it, without changing the IP subnetting in the old network, that includes all the servers. Now they have most of the hosts connected in the cat 4500 but, the servers, some few hosts, firewall and Internet connection are still in the old network.
Thanks a lot for your help.
Anacelia

Hello,
==================================================
Error Message C4K_IOSIPROUTEMAN-3-ADJMANNOMOREADJS:AdjMan:hardwar e adjacency resources exhausted, performance may be degraded.
Explanation Hardware adjacency resources have been exhausted. The Supervisor Engine III will forward packets to this adjacency in software. This action will likely represent a significant performance degradation.
Recommended Action Reduce the number of adjacencies loaded into the routing table, and then clear the IP routing table.
==================================================
In your case I believe the problem is with the default route you have pointing directly to VLAN999 interface. The network diagram provided shows legacy servers in the 10.1.0.0/16 network. You have a specific host route configured and a default route as follows:
ip route 0.0.0.0 0.0.0.0 Vlan999
ip route 10.1.1.50 255.255.255.255 Vlan999
So basically, every time a packet matches the longest prefix of 0.0.0.0 (or doesn't have another FIB entry) will be sent out the VLAN999 interface and a subsequent ARP will be generated to learn the MAC to forward to.
Preferrably the static routes would point to a next-hop-ip on a multiaccess interface. Is there are multiple Layer 3 devices you could point the routes to to avoid the excessive ARPing? If not can you limit the default route to point to the firewall and assign the /16 route to the interface.
http://www.cisco.com/en/US/products/hw/iad/ps397/products_tech_note09186a00800a73e9.shtml#topic7
Hope this helps.
Regards,
James

Nexus 7000 Platform Logging

Hello,
We recently had a power supply failure in one of our Nexus 7000s, and I noticed that the syslog for the Platform is only present in the default VDC, and not in any of the other VDCs syslogs. Is this by design, or is there a logging level I can turn up in another VDC to capture this log? Thanks for any input
syslog from default VDC -
2013 Mar 18 23:10:34 %PLATFORM-2-PS_CAPACITY_CHANGE: Power supply PS3 changed i
ts capacity. possibly due to power cable removal/insertion (Serial number xxxxxxxx)
nothing in the VDC where I would like to get the logging
default VDC logging level -
xxx7K02# show log level platform
Facility        Default Severity        Current Session Severity
platform                5                       5
0(emergencies)          1(alerts)       2(critical)
3(errors)               4(warnings)     5(notifications)
6(information)          7(debugging)
xxx7K02#
loggging from the specific VDC where we have management tools.
xxx-LOW# show log level platform
Facility        Default Severity        Current Session Severity
platform                5                       5
0(emergencies)          1(alerts)       2(critical)
3(errors)               4(warnings)     5(notifications)
6(information)          7(debugging)
xxx-LOW#

Hello Carl,
What version of code are you running on your Nexus 7k?
The expected behavior is:
"When a hardware issue occurs, syslog messages are sent to all VDCs."
http://www.cisco.com/en/US/docs/switches/datacenter/sw/nx-os/virtual_device_context/configuration/guide/vdc_mgmt.html#wp1170241
Dave

Dell Servers with Nexus 7000 + Nexus 2000 extenders

<< Original post by smunzani. Answered by Robert. Moving from Document section to Discussions>>
Team,
I would like to use some of the existing Dell Servers for new network design of Nexus 7000 + Nexus 2000 extenders. What are my options for FEC to the hosts? All references of M81KR I found on CCO are related to UCS product only.
What's best option for following setup?
N7K(Aggregation Layer) -- N2K(Extenders) -- Dell servers
Need 10G to the servers due to dense population of the VMs. The customer is not up for dumping recently purchased dell boxes in favor of UCS. Customer VMware license is Enterprise Edition.
Thanks in advance.

To answer your question, the M81KR-VIC is a Mezz card for UCS blades only. For Cisco rack there is a PCIe version which is called the P81. These are both made for Cisco servers only due to the integration with server management and virtual interface functionality.
http://www.cisco.com/en/US/prod/collateral/ps10265/ps10493/data_sheet_c78-558230.html
More information on it here:
Regards,
Robert

Ask the Expert: Basic Introduction and Troubleshooting on Cisco Nexus 7000 NX-OS Virtual Device Context

With Vignesh R. P.
Welcome to the Cisco Support Community Ask the Expert conversation.This is an opportunity to learn and ask questions of Cisco expert Vignesh R. P. about the Cisco® Nexus 7000 Series Switches and support for the Cisco NX-OS Software platform .
The Cisco® Nexus 7000 Series Switches introduce support for the Cisco NX-OS Software platform, a new class of operating system designed for data centers. Based on the Cisco MDS 9000 SAN-OS platform, Cisco NX-OS introduces support for virtual device contexts (VDCs), which allows the switches to be virtualized at the device level. Each configured VDC presents itself as a unique device to connected users within the framework of that physical switch. The VDC runs as a separate logical entity within the switch, maintaining its own unique set of running software processes, having its own configuration, and being managed by a separate administrator.
Vignesh R. P. is a customer support engineer in the Cisco High Touch Technical Support center in Bangalore, India, supporting Cisco's major service provider customers in routing and MPLS technologies. His areas of expertise include routing, switching, and MPLS. Previously at Cisco he worked as a network consulting engineer for enterprise customers. He has been in the networking industry for 8 years and holds CCIE certification in the Routing & Switching and Service Provider tracks.
Remember to use the rating system to let Vignesh know if you have received an adequate response.
Vignesh might not be able to answer each question due to the volume expected during this event. Remember that you can continue the conversation on the Data Center sub-community discussion forum shortly after the event. This event lasts through through January 18, 2013. Visit this forum often to view responses to your questions and the questions of other community members.

Hi Vignesh
Is there is any limitation to connect a N2K directly to the N7K?
if i have a an F2 card 10G and another F2 card 1G and i want to creat 3 VDC'S
VDC1=DC-Core
VDC2=Aggregation
VDC3=Campus core
do we need to add a link between the different VDC's
thanks

LMS 4.2.2 Interface utilisation on Nexus 7000

Hi All,
I'm trying to poll some interfaces for their utilization on a nexus 7000 through LMS 4.2.2.
When I create a poller fot the specific instances, the LMS recognises the instances, but after activating the poller I get the error "No Such Instance - The specified instance is not available".
No info is displayed when I generate an interface utilization report for the specific nexus.
When I activate the automonitor for interface utilization, the interfaces on the nexus are polled.
On the cisco website there are some features listed which LMS does not support on the Nexus 7000, but polling is not in that list (neither in the supported feature list).
Any tips?
Thanks for your help.
Joris

Any Idea..??

ESXi 4.1 NIC Teaming's Load-Balancing Algorithm,Nexus 7000 and UCS

Hi, Cisco Gurus:
Please help me in answering the following questions (UCSM 1.4(xx), 2 UCS 6140XP, 2 Nexus 7000, M81KR in B200-M2, No Nexus 1000V, using VMware Distributed Switch:
Q1. For me to configure vPC on a pair of Nexus 7000, do I have to connect Ethernet Uplink from each Cisco Fabric Interconnect to the 2 Nexus 7000 in a bow-tie fashion? If I connect, say 2 10G ports from Fabric Interconnect 1 to 1 Nexus 7000 and similar connection from FInterconnect 2 to the other Nexus 7000, in this case can I still configure vPC or is it a validated design? If it is, what is the pro and con versus having 2 connections from each FInterconnect to 2 separate Nexus 7000?
Q2. If vPC is to be configured in Nexus 7000, is it COMPULSORY to configure Port Channel for the 2 Fabric Interconnects using UCSM? I believe it is not. But what is the pro and con of HAVING NO Port Channel within UCS versus HAVING Port Channel when vPC is concerned?
Q3. if vPC is to be configured in Nexus 7000, I understand there is a limitation on confining to ONLY 1 vSphere NIC Teaming's Load-Balancing Algorithm i.e. Route Based on IP Hash. Is it correct?
Again, what is the pro and con here with regard to application behaviours when Layer 2 or 3 is concerned? Or what is the BEST PRACTICES?
I would really appreciate if someone can help me clear these lingering doubts of mine.
God Bless.
SiM

Sim,
Here are my thoughts without a 1000v in place,
Q1. For me to configure vPC on a pair of Nexus 7000, do I have to connect Ethernet Uplink from each Cisco Fabric Interconnect to the 2 Nexus 7000 in a bow-tie fashion? If I connect, say 2 10G ports from Fabric Interconnect 1 to 1 Nexus 7000 and similar connection from FInterconnect 2 to the other Nexus 7000, in this case can I still configure vPC or is it a validated design? If it is, what is the pro and con versus having 2 connections from each FInterconnect to 2 separate Nexus 7000? //Yes, for vPC to UCS the best practice is to bowtie uplink to (2) 7K or 5Ks.
Q2. If vPC is to be configured in Nexus 7000, is it COMPULSORY to configure Port Channel for the 2 Fabric Interconnects using UCSM? I believe it is not. But what is the pro and con of HAVING NO Port Channel within UCS versus HAVING Port Channel when vPC is concerned? //The port channel will be configured on both the UCSM and the 7K. The pro of a port channel would be both bandwidth and redundancy. vPC would be prefered.
Q3. if vPC is to be configured in Nexus 7000, I understand there is a limitation on confining to ONLY 1 vSphere NIC Teaming's Load-Balancing Algorithm i.e. Route Based on IP Hash. Is it correct? //Without the 1000v, I always tend to leave to dvSwitch load balence behavior at the default of "route by portID".
Again, what is the pro and con here with regard to application behaviours when Layer 2 or 3 is concerned? Or what is the BEST PRACTICES? UCS can perform L2 but Northbound should be performing L3.
Cheers,
David Jarzynka

Privilege Level for Tacacs Account in Nexus 7000

Hi,
I have configured the Tacacs (ACS 4.2v) on Nexus 7000 (as mentioned below) and works fine but unlike IOS (6509) It's doesn't prompt that you are in userexec mode (>) and then need to type enable and password for full privilege.
In n7k when I entered into "configure terminal" It won't allow me to access other commands.
How to login into level 15 privilege mode after authenticating from tacacs
(config)# show running-config tacacs+
tacacs-server key 7 "xxxxx"
tacacs-server host x.x.x.x key 7 "xxxx"
aaa group server tacacs+ TacServer
    server x.x.x.x (same ip as tacacs-server host)
    use-vrf management
    source-interface Vlan2
(config)# show running-config aaa
aaa authentication login default group TacServer
aaa authentication login console local
aaa user default-role
Here below are the commands accessible in "Terminal" currently
(config)# ?
no        Negate a command or set its defaults
username Configure user information.
end       Go to exec mode
exit      Exit from command interpreter
isb.n7k-dcn-agg-1-sw(config)#

Hi Jan.nielsen
Issue is resolved but by another way.
I have found the same resolution too of custom attirbute command but the Custom attribute Option for shell command wasn't available in ACS v4.2, so after enabling shell for users and by clicking exec--> Shell Exec and enabling priviledge level 15 in the same box of Shell options, It start working without any command

Nexus 7000 - unexpected shutdown of vPC-Ports during reload of the primary vPC Switch

Dear Community,
We experienced an unusual behavior of two Nexus 7000 switches within a vPC domain.
According to the attached sketch, we have four N7Ks in two data centers - two Nexus 7Ks are in a vPC domain for each data center.
Both data centers are connected via a Multilayer-vPC.
We had to reload one of these switches and I expected the other N7K in this vPC domain to continue forwarding over its vPC-Member-ports.
Actually, all vPC ports have been disabled on the secondary switch until the reload of the first N7K (vPC-Role: primary) finished.
Logging on Switch B:
20:11:51 <Switch B> %VPC-2-VPC_SUSP_ALL_VPC: Peer-link going down, suspending all vPCs on secondary
20:12:01 <Switch B> %VPC-2-PEER_KEEP_ALIVE_RECV_FAIL: In domain 1, VPC peer keep-alive receive has failed
In case of a Peer-link failure, I would expect this behavior if the other switch is still reachable via the Peer-Keepalive-Link (via the Mgmt-Port), but since we reloaded the whole switch, the vPCs should continue forwarding.
Could this be a bug or are there any timers to be tuned?
All N7K switches are running on NX-OS 6.2(8)
Switch A:
vpc domain 1
peer-switch
role priority 2048
system-priority 1024
peer-keepalive destination <Mgmt-IP-Switch-B>
delay restore 360
peer-gateway
auto-recovery reload-delay 360
ip arp synchronize
interface port-channel1
switchport mode trunk
switchport trunk allowed vlan <x-y>
spanning-tree port type network
vpc peer-link
Switch B:
vpc domain 1
peer-switch
role priority 1024
system-priority 1024
peer-keepalive destination <Mgmt-IP-Switch-A>
delay restore 360
peer-gateway
auto-recovery reload-delay 360
ip arp synchronize
interface port-channel1
switchport mode trunk
switchport trunk allowed vlan <x-y>
spanning-tree port type network
vpc peer-link
Best regards

Problem solved:
During the reload of the Nexus 7K, the linecards were powerd off a short time earlier than the Mgmt-Interface. As a result of this behavior, the secondary Nexus 7K received at least one vPC-Peer-Keepalive Message while its peer-link was already powerd off. To avoid a split brain scenario, the VPC-member-ports have been shut down.
Now we are using dedicated interfaces on the linecards for the VPC-Peer-Keepalive-Link and a reload of one N7K won't result in a total network outage any more.

Using SNMP to monitor Nexus 7000 Series Supervisor Module

Hello,
I got a Nexus 7000 supervisor module recently, I met a SNMP problem for this module
I would like to know which specific OIDs to use to monitor the following using SNMP on a Nexus 7000 supervisor module:
- Port status
- CPU total utilization
- Power Supply status
- Chassis Fan status
etc.
The Nexus is quite different from other Cisco devices - any help will be appreciated!

hope help, and
port status OID is ifOperStatus
CPU total utilization OID is 1.3.6.1.4.1.9.9.109.1.1.1.1.6.1
[root@NET-MONITOR-1 ~]#
[root@NET-MONITOR-1 ~]# snmpwalk -On -v 2c -c 360buy 172.17.0.253 ifDescr.83886080
.1.3.6.1.2.1.2.2.1.2.83886080 = STRING: mgmt0
[root@NET-MONITOR-1 ~]#
[root@NET-MONITOR-1 ~]# snmpwalk -On -v 2c -c 360buy 172.17.0.253 ifOperStatus.83886080
.1.3.6.1.2.1.2.2.1.8.83886080 = INTEGER: up(1)
[root@NET-MONITOR-1 ~]#
[root@NET-MONITOR-1 ~]# snmpwalk -On -v 2c -c 360buy 172.17.0.253 1.3.6.1.4.1.9.9.109.1.1.1.1.6.1
.1.3.6.1.4.1.9.9.109.1.1.1.1.6.1 = Gauge32: 21
[root@NET-MONITOR-1 ~]#

Query Nexus 7000 Enviroment Status

Hi,
I am trying to figure out how to query a Nexus 7010 Chassis about its enviroment. For our IOS Switches we use SNMP and OID 1.3.6.1.4.1.9.9.13.1 and the related sub OIDs. But this does not work on the Nexus 7010 with version 5.1. Is querying the information not supported or is there another OID?

Hi,I am trying to figure out how to query a Nexus 7010 Chassis about its enviroment. For our IOS Switches we use SNMP and OID 1.3.6.1.4.1.9.9.13.1 and the related sub OIDs. But this does not work on the Nexus 7010 with version 5.1. Is querying the information not supported or is there another OID?
Hi,
Check out the below link for nexus 7000 MIB reference ..
http://www.cisco.com/en/US/docs/switches/datacenter/sw/mib/quickreference/Cisco_Nexus_7000_Series_NX-0S_MIB_Quick_Reference_chapter1.html#con_40545
Hope to Help !!
Ganesh.H

Rule based span on Nexus 7000

Hi all,
I'm trying to configure rule based span on my Nexus 7000.
I want to monitor some vlans, but limit the traffic going to my monitor station by using frame-type ipv4 filter.
The link below explains how to configure it, but my nexus doesn't recognise the command "mode extended".
http://www.cisco.com/c/en/us/td/docs/switches/datacenter/sw/6_x/nx-os/system_management/configuration/guide/sm_nx_os_cg/sm_14span.html#wp1286697
Am I missing something? I'm running version 6.1.3.
Thanks,
Joris
NEXUS(config)# monitor session 1
NEXUS(config-monitor)# mode extended
^
% Invalid command at '^' marker.
NEXUS(config-monitor)# mode ?
*** No matching command found in current mode, matching in (exec) mode ***
connect Notify system on modem connection
restart Reenabling modem port

Hi Joris,
The rule based SPAN filtering was not introduced until NX-OS 6.2 so will not be available to you with NX-OS 6.1(3).
See the section SPAN in the NX-OS 6.2 release notes.
Regards

Nexus 7000 - Adjacency table exhausted

Similar Messages

Maybe you are looking for