Nexus 6K: Port-Channel Load-Balance

Hi all,
I configured "port-channel load-balance ethernet source-dest-mac" on Nexus 6001. But when I use "show run all | in load-balance", it displays module 1 and module 2 are still using source-dest-ip for port-channel load-balance. And for command "show port-channel load-balance" and "show port-channel load-balance forwarding-path interface", it still shows switch using MAC for hash algorithm. The NXOS is 6.0(2)N1(2a).
Does anybody know:
- What is the function of "port-channel load-balance ethernet source-dest-ip module" and in which situation this command will be effective?
- It shows "port-channel load-balance ethernet source-dest-ip module" command for both module 1 and 2. Module 1 is N6K Supervisor and module 2 is 4xQSFP Ethernet Module. Is it possible to set different load-balance algorithm  to these 2 modules?
# show run all | in load-balance
port-channel load-balance ethernet source-dest-mac
port-channel load-balance ethernet source-dest-ip module 1
port-channel load-balance ethernet source-dest-ip module 2
# show port-channel load-balance
Port Channel Load-Balancing Configuration:
System: source-dest-mac
Port Channel Load-Balancing Addresses Used Per-Protocol:
Non-IP: source-dest-mac
IP: source-dest-mac
# show port-channel load-balance forwarding-path interface port-channel 30 vlan 150 src-ip 172.25.228.6 dst-ip 172.25.226.97
Missing params will be substituted by 0's.
Load-balance Algorithm on switch: source-dest-mac
crc_hash: 977 Polynomial: CRC10b        Outgoing port id Ethernet1/2
Param(s) used to calculate load-balance:
        seed: 0x701
        dst-mac: 0000.0000.0000
        src-mac: 0000.0000.0000
# show module
Mod Ports Module-Type                         Model                  Status
1   48    Norcal 64 Supervisor                N6K-C6001-64P-SUP      active *
2   10    Nexus 4xQSFP Ethernet Module        N6K-C6001-M4Q          ok
Mod Sw              Hw      World-Wide-Name(s) (WWN)
1    6.0(2)N2(3)     1.0     --
2    6.0(2)N2(3)     1.0     --

Similar Messages

Nexus port channel load balance

Hi
I just want to clarify one setting for the port channel load balance on Nexus 6k switch. If I use the load balance option source-dest-ip-only, will following four converstions be load balanced?
10.10.10.1 -> 192.168.1.1
10.10.10.2 -> 192.168.1.1
10.10.10.1 -> 192.168.1.1
10.10.10.1 -> 192.168.1.2
Thanks. Leo

Hi Leo,
I think there may be typo in your question as I only see three conversations and not four. That aside I've not seen the Nexus port-channel load balancing sufficiently well documented to be able to give you the exact answer.
In their configuration guides Cisco only include the following statement:
Cisco NX-OS load balances traffic across all operational interfaces in a port channel by reducing part of the binary pattern formed from the addresses in the frame to a numerical value that selects one of the links in the channel.
There is other documentation that states the load balancing algorithm uses a CRC-8 based polynomial, but as we don't know exactly which parts of the frame are used in the calculation, I don't see it's possible to calculate the answer and so derive the link that will be used for a given conversation.
While I've not seen full documentation regarding the science used in the calculation, what Cisco have done is provide a command on the switch CLI that will allow you to determine which link of a port-channel will be used.
If you run the command show port-channel load-balance forwarding-path interface port-channel vlan src-ip dst-ip then one of the parts of the output is the member link of the port-channel that will be used for that flow.
You can find full details of the options for the show port-channel load-balance command in the command reference.
One other point to remember is that the load balancing across a port-channel is unidirectional, and the hashing might be completely different for the return flow of a conversation. For example it is entirely possible that traffic from A to B could use one link of a port-channel, while the return traffic from B to A for the same conversation could use a different link.
In general I would use the source-dest-port option for load balancing on the Nexus switches as this will obviously include the Layer-4 port numbers in the calculation, and so give you a better distribution of flows across all member links.
Regards

Nexus - port-channel load balancing

Port-channel load balancing is a global command or interface command in Nexus switch?
Thanks,
Manu

Hi,
It's a global command; port-channel load-balance ethernet.
You can find details in the Configuring Load Balancing Using Port Channels section of the Nexus 5500 Series NX-OS Interfaces Configuration Guide.
Regards

Port Channel Load-Balancing Algorithm (North Bound)

I'm trying to figure out what the load balancing algorithm for the 6100 and 6200 FIs for the Northbound connections. I can't find any documentation on how to change it.
The Nexus 7000s use an 8-bit hash, making it very easy to do something other than 2, 4, or 8 link port channel and get even (at least algorithmically) distribution.
Catalyst switches (not sure about Sup2T though) would use a 1, 2, or 3-bit index, which would skew traffic algorthmically if you used a non-power of 2.
Looking at the 5K documentation, it seems to use the Catalyst style (though haven't been able to confirm). My guess is that whatever is used for the 5Ks is used for the 6100/6200.
Design wise, this would mean you would want to use powers of 2 for your NB uplinks.

Hello Tony,
On UCS FI, it uses " sr-cdest-ip " as the load balancing algorithm and uses 8 parameters for hashing
6248-01-B(nxos)# sh port-channel load-balance
Port Channel Load-Balancing Configuration:
System: source-dest-ip
Port Channel Load-Balancing Addresses Used Per-Protocol:
Non-IP: source-dest-mac
IP: source-dest-ip source-dest-mac
6248-01-B(nxos)# show platform fwm info pc port-channel 1
dump pc info: if_index 369098752 dump_all 0 verbose 1
Po1: state 0x0 #pifs 1 fwimpd ctx 0x9666c1c
Po1: hash params - l2_da 1 l2_sa 1 l3_da 1 l3_sa 1
Po1: hash params - l4_da 1 l4_sa 1 xor_sa_da 1 hash_elect 1
I could not find an option to change these values.
Padma

Cisco MDS Port channel load balancing

A customer recently asked an interesting question about exchange based load balancing on an FC port channel. The platform is UCS with an 8 and 16 port channel per fabric interconnect on two separate UCS domains. The application is Oracles data warehousing which has been known to saturate 4 x 8gb fc links. Since the balancing method is exchange based what constitutes the start and end of an exchange? We are trying to avoid a condition where and intense read write conversation locks to a single link in the port channel and not spread across 8/16 links. Where can I find more information about exchange based routing protocol or how should I go about managing extreme io in a converged infrastructure.

The default loadbalance method on FI and MDS is src-dst-ox-id based. Note that loadbalancing is done by a device on *outgoing traffic*. FI and MDS do not have to negotiate anything here. Technically, one device can do src-dst-id based while the other can use src-dst-ox-id. However, in your case there is no reason for such a change.
As I wrote before, If all of the links in the port-channel are touching the max capacity, you should recommend your customer to increase links in the bundle (max 16) or upgrade to 16G links. If few of the links are heavily utilized while other links in the same port-channel are under utilized, you may want to check the application or HBA for capability of breaking down the large reads/writes under smaller exchanges. If no traffic is going on few of the links at all, then I would suspect UCS to FI pinning as well.

Port channel Load balancing in Storage VDC

Hi i am not able to find how to check PO load balancing for storage VDC, although i know by default for FCoE traffic on storage vdc it is OXID but whether it is src-dst l4port or src-dst ipl4port.

Hi,
From "Nexus 5500 to Nexus 7000 Multi-Hop FCoE Configuration Example" ,
Note: On Nexus 7000, by default the source-destination-oxid load balancing mechanism is used for FCoE traffic.
So let's see what is the default load balancing mechanism in Nexus 7000,
From "Nexus 7000 interface configuration guide" ,
The default load-balancing mode for Layer 3 interfaces is the source and destination IP address, and the default load-balancing mode for non-IP interfaces is the source and destination MAC address.
Which means src-dst ip.
So what I think is you need to have src-dst ip in default VDC for OXID load balancing in Nexus 7000.

3750X Port-Channel Load-Blanace method

I have a 3750X and I'm wondering what would be the best Port-Channel Load-Balancing method would be for my network.
switch(config)#port load ?
dst-ip       Dst IP Addr
dst-mac      Dst Mac Addr
src-dst-ip   Src XOR Dst IP Addr
src-dst-mac Src XOR Dst Mac Addr
src-ip       Src IP Addr
src-mac      Src Mac Addr
We have a few Layer 3 VLANs: Default, Servers, Clients, and Guests.
Some of our servers are LACP bundled.
So it would be mainly clients on the one layer 3 VLAN accessing the server on the other layer 3 VLAN.
We also have a few smaller switches that are trunked and LACP bundled back to the core 3750X.
Can anyone suggest which Load-Balancing method would be best for our situation?
Thanks!

The default load balancing method on the 3750 series is based on source-MAC address. This usually works fine in most cases. If you change it to something different than this, it will affect all your port-channels in that switch.
HTH

How can ftp service on non-standard port be load balanced using Cisco ACE.

How can ftp service on non-standard port be load balanced using Cisco ACE.For example ftp service required on tcp 2000 port

Hi Samarjit,
you can do this by specifying the port number in the class map that you create . Please find the below mentioend config guide where you can specify the tcp/udp port , range or ports or even the wild card to match the port.
http://www.cisco.com/en/US/docs/interfaces_modules/services_modules/ace/v3.00_A1/configuration/administration/guide/mapolcy.html#wp1318826
Regards
Abijith

Nexus 1000v - port-channel "refresh"

Hi All,
My question is, does anyone have any information on this 1000v command:
Nexus-1000v(config)# port-channel internal device-id table refresh
I am looking for a way for the port-channel interface to be automatically removed from the 1000v once the VEM has been deleted, currently the port-channel interface does not disappear when the VEM has been removed. This seems to be causing problems once the same VEM is re-added later on. Ports are getting sent into quarantine states and ending up in invalid states (eg. NoPortProfile state when there is actually a port-profile attached).
Anyway, if anyone can explain the above command or tell me how to find out more, it would be great, I can't find it documented anywhere and the context-sensitive help in the NXOS is vague at best.

Brendan,
I don't have much information on that command, but I do know it wont remove any unused port channels. They have to be manually deleted if they're no longer needed.
The port Channel ID will remain even after a VEM is removed in case the assigned VEM comes back. When a VEM is decommisioned permanently, I'll do a "no vem x" to also remove the Host entry for that VEM from the VSM. This way the module slot # can be re-assigned to the next new VEM inserted. After adding/removing VEMs just do a "show port-channel summary" to see any unused Port Channel IDs, and delete them. It's a quick & painless task.
I would hope this wouldn't be a common issue - how often are you deleting/removing VEMs?
Regards,
Robert

Nexus 5k port channel

I want to create a port channel on my nexus switch which will be connected to my server.
will someone please share with me the configuration for the same ?

Hi,
From server side create LACP port-channel and same from switch end.
Nexus-SW
=======
interface Ethernet1/1-2
description [connection to SRV-01]
switchport
switchport access vlan 10
channel-group 20 mode active
no shutdown
interface port-channel20
description [connection to SRV-01]
switchport
vpc 20
switchport access vlan 10

Cisco Nexus 7010 Port-Channel

Hi all,
Hi have install a cisco nexus 7010 with the following modules :
Mod Ports Module-Type Model Status
1 48 1000 Mbps Optical Ethernet XL Modul N7K-M148GS-11L ok
2 48 10/100/1000 Mbps Ethernet XL Module N7K-M148GT-11L ok
5 0 Supervisor Module-2 N7K-SUP2 active *
6 0 Supervisor Module-2 N7K-SUP2 ha-standby
9 48 10/100/1000 Mbps Ethernet XL Module N7K-M148GT-11L ok
10 48 1/10 Gbps Ethernet Module N7K-F248XP-25E ok
After trying to build a port-channel with a 1G port on slot 1 and a 1G port on slot 10 i get a messague with incompatible port.
Can any one tell me if its supported a port-channel with ports on the following modules :
N7K-M148GS-11L
N7K-F248XP-25E
Thanks
Patrício

Hi Particio,
The port-channel/vPC features require that all uplinks are connected to the same type of
module. You can refer to the following document.
http://www.cisco.com/en/US/partner/docs/switches/datacenter/sw/6_x/nx-os/interfaces/configuration/guide/if_portchannel.html#wp1888267
HTH
Regards
Inayath
*Plz rate if this is helpfull and mark as Answerd if this answerd your query.

Nexus 1000v port-channels questions

Hi,
I’m running vCenter 4.1 and Nexus 1000v and about 30 ESX Hosts.
I’m using one system uplink port profile for all 30 ESX Host; On each of the ESX host I have 2 NICs going to a Catalyst 3750 switch stack (Switch A), and another 2 NICs going to another Catalyst 3750 switch stack (Switch B).
The Nexus is configured with the “sub-group CDP” command on the system uplink port profile like the following:
port-profile type ethernet uplink
vmware port-group
switchport mode trunk
switchport trunk allowed vlan 1,800,802,900,988-991,996-997,999
switchport trunk native vlan 500
mtu 1500
channel-group auto mode on sub-group cdp
no shutdown
system vlan 988-989
description System-Uplink
state enabled
And the port channel on the Catalyst 3750 are configured like the following:
interface Port-channel11
description ESX-10(Virtual Machine)
switchport trunk encapsulation dot1q
switchport trunk native vlan 500
switchport trunk allowed vlan 800,802,900,988-991
switchport mode trunk
switchport nonegotiate
spanning-tree portfast trunk
end
interface GigabitEthernet1/0/18
description ESX-10(Virtual Machine)
switchport trunk encapsulation dot1q
switchport trunk native vlan 500
switchport trunk allowed vlan 800,802,900,988-991
switchport mode trunk
switchport nonegotiate
channel-group 11 mode on
spanning-tree portfast trunk
spanning-tree guard root
end
interface GigabitEthernet1/0/1
description ESX-10(Virtual Machine)
switchport trunk encapsulation dot1q
switchport trunk native vlan 500
switchport trunk allowed vlan 800,802,900,988-991
switchport mode trunk
switchport nonegotiate
channel-group 11 mode on
spanning-tree portfast trunk
spanning-tree guard root
end
Now Cisco is telling me that I should be using MAC pinning when doing a trunk to two different stacks , and that each interface on 3750 should not be configured in a port-channel like above, but should be configured as individual trunks.
First question: Is the above statement correct, are my uplinks configured wrong? Should they be configured individually in trunks instead of a port-channel?
Second questions: If I need to add the MAC pinning configuration on my system uplink port-profile can I create a new system uplink port profile with the MAC pinning configuration and then move one ESX host (with no VM on them) one at a time to that new system uplink port profile? This way, I could migrate one ESX host at a time without outages to my VMs. Or is there an easier way to move 30 ESX hosts to a new system uplink profile with the MAC Pinning configuration.
Thanks.

Hello,
From what I understood, you have the following setup:
     - Each ESX host has 4 NICS
     - 2 of them go to a 3750 stack and the other 2 go to a different 3750 stack
     - all 4 vmnics on the ESX host use the same Ethernet port-profile
          - this has 'channel-group auto mode on sub-group cdp'
     - The 2 interfaces on each 3750 stack are in a port-channel (just 'mode on')
If yes, then this sort of a setup is correct. The only problem with this is the dependance on CDP. With CDP loss, the port-channels would go down.
'mac-pinning' is the recommended option for this sort of a setup. You don't have to bundle the interfaces on the 3750 for this and these can be just regular trunk ports. If all your ports are on the same stack, then you can look at LACP. The CDP option would not be supported in the future releases. In fact, it is supposed to be removed from 4.2(1)SV1(2.1) but I still see the command available (ignore 4.2(1)SV1(4) next to it) - I'll follow up on this internally:
http://www.cisco.com/en/US/docs/switches/datacenter/nexus1000/sw/4_2_1_s_v_2_1_1/interface/configuration/guide/b_Cisco_Nexus_1000V_Interface_Configuration_Guide_Release_4_2_1_SV_2_1_1_chapter_01.html
For migrating, the best option would be as you suggested. Create a new port-profile with mac-pinning and move one host at a time. You can migrate VMs off the host before you change the port-profile and can remove the upstream port-channel config as well.
Thanks,
Shankar

FI 6248 Nexus 5548 Port-channel problem

                   Hi all
I have a strange behavior between 2 FI and 2 nexus 5k configured as VPC , From a remote location and from my desktop I can ping odd ip and not even , from an other desktop in the same LAN the reverse, for both the tracert are good.
The only solution is to shutdown ports 1/1-2 on nexus A or FI A and in that case both desktops can ping both IP.
All seems OK, no log error , no error on interface , I give below some commands
What thing to know is that the nexus b has a L3 card , not the Nexus A , In fact I am waiting for a spare L3 daughter card.
On both NexusI have such configuration and status
interface port-channel201
description ucs-Fab-a
switchport mode trunk
switchport trunk native vlan 999
spanning-tree port type edge trunk
vpc 201
UCSM-PARIS-A(SSI153000ST)
                       Eth1/1        129    S I s     UCS-FI-6248UP Eth1/11
UCSM-PARIS-A(SSI153000ST)
                       Eth1/2        174    S I s     UCS-FI-6248UP Eth1/12
sh port-channel summary
201   Po201(SU)   Eth      LACP      Eth1/1(P)    Eth1/2(P)
sh mac address-table dynamic | i 1010
* 1010     0025.b5c9.481e    dynamic   0          F    F Po201
* 1010     0025.b5c9.481f    dynamic   10         F    F Po201
sh ip arp
10.203.136.21   00:01:05 0025.b5c9.481f Vlan1010
10.203.136.22   00:12:43 0025.b5c9.481e Vlan1010
On both FI A have such configuration and status
terface port-channel201
description U: Uplink
switchport mode trunk
pinning border
switchport trunk native vlan 999
switchport trunk allowed vlan 1,104,999,1010-1013,1020-1021,1500-1502,1510-151
2,1550-1552,1560-1561,1600-1602,1610-1619
speed 10000
sh port-channel summary
201   Po201(SU)   Eth      LACP      Eth1/9(P)    Eth1/10(P)   Eth1/11(P)
                                     Eth1/12(P)
sh mac address-table dynamic
* 1010     0025.b5c9.481e    dynamic   0          F    F Po201
* 1010     0025.b5c9.481f    dynamic   0          F    F Po201
sh cdp nei
E2RTR001(SSI1608085E) Eth1/9          120     S I s       N5K-C5548UP   Eth1/1
E2RTR001(SSI1608085E) Eth1/10         120     S I s       N5K-C5548UP   Eth1/2
E2RTR002(SSI16080F17) Eth1/11         151     R S I s     N5K-C5548UP   Eth1/1
E2RTR002(SSI16080F17) Eth1/12         136     R S I s     N5K-C5548UP   Eth1/2
Any ideas, A configuration I forgot on FI ?
Regards .

In fact in the probem I described the path is Esx -> FAB A - > N5K-A (for layer 2) -> N5K-B (for L3) -> routing to the LAN
And what you told me is that such configuration is not supported with VPC configuration , so I need to wait to receive the L3 card for the N5k-A. and check later if I have the same problem.
thanks

Nexus 7010 port-channel to ESXi host running on Dell 910 and 810

Was wondering has anyone had any experience with configuring a port-channel going back to ESXi host. My basic config is below. My issue is with what to put in the spanning-tree port type. Do I use normal, edge, network, or edge trunk?? Yes I have read most of the documentation out there but am just a lil confused. I just wanna be right.
Also if anyone has any suggestions for the Vswitch config as well that would be much appreciated.
int PO 201
desc 01_6-1
switchport mode trunk
switchport access vlan 99
switchport trunk native vlan 777
switchport trunk allowed vlan 220,300,310,320-321,350,390-391
switchport trunk allowed vlan add 420,777
spanning-tree port type normal (what should go here)
speed auto
medium p2p
no shutdown
int e8/11
desc 01_6-1
channel-group 201 force
int e10/11
desc 01_6-1
channel-group 201 force
Thanks Guys

Short answer: You could use spanning tree port type edge (trunk; if it's going to be a trunk). The vswitch isn't really a switch, it doesn't participate in spanning tree, so it's fine to make it an edge port.
In my experience, many VMware guys don't like port channels as much as the other more host-centric NIC teaming methods. I think this has to do with ESX having a less than stellar implementation of etherchannel/LACP early on.

Nexus 7010 port-channel failed, not receiving any LACP BPDUs

Hi every one,
I'm having issue when trying to form LACP etherchannel links between N7k and server. Here's the setup:
SERVER eth0-------N7K-1 Eth102/1/9
SERVER eth1-------N7K-2 Eth102/1/9
Configuration:
N7K-1# sh run int eth102/1/9
interface Ethernet102/1/9
switchport
switchport access vlan 204
channel-group 321
no shutdown
N7K-1# sh run int po321
interface port-channel321
switchport
switchport access vlan 204
flowcontrol send on
vpc 321
N7K-2# sh run int eth102/1/9
interface Ethernet102/1/9
switchport
switchport access vlan 204
channel-group 321
no shutdown
N7K-2# sh run int po321
interface port-channel321
switchport
switchport access vlan 204
flowcontrol send on
vpc 321
vPC status
id Port Status Consistency Reason Active vlans
321 Po321 up success success 204
Scenario 1-Etherchannel only without protocol
Both links are up, problem only happened when link to N7K-2 restart. If link to N7K-1 restart no problem.
Scenario 2-Etherchannel with LACP
Both physical links NOT up. Logs showing this:
2013 Aug 20 17:04:31 N7K-1 %ETHPORT-5-IF_DOWN_ADMIN_DOWN: Interface Ethernet102/1/9 is down (Administratively down)
2013 Aug 20 17:04:32 N7K-1 %ETHPORT-5-IF_ADMIN_UP: Interface Ethernet102/1/9 is admin up .
2013 Aug 20 17:04:35 N7K-1 %ETHPORT-5-SPEED: Interface Ethernet102/1/9, operational speed changed to 1 Gbps
2013 Aug 20 17:04:35 N7K-1 %ETHPORT-5-IF_DUPLEX: Interface Ethernet102/1/9, operational duplex mode changed to Full
2013 Aug 20 17:04:35 N7K-1 %ETHPORT-5-IF_RX_FLOW_CONTROL: Interface Ethernet102/1/9, operational Receive Flow Control state changed to off
2013 Aug 20 17:04:35 N7K-1 %ETHPORT-5-IF_TX_FLOW_CONTROL: Interface Ethernet102/1/9, operational Transmit Flow Control state changed to on
2013 Aug 20 17:04:35 N7K-1 %ETHPORT-5-SPEED: Interface port-channel321, operational speed changed to 1 Gbps
2013 Aug 20 17:04:35 N7K-1 %ETHPORT-5-IF_DUPLEX: Interface port-channel321, operational duplex mode changed to Full
2013 Aug 20 17:04:35 N7K-1 %ETHPORT-5-IF_RX_FLOW_CONTROL: Interface port-channel321, operational Receive Flow Control state changed to off
2013 Aug 20 17:04:35 N7K-1 %ETHPORT-5-IF_TX_FLOW_CONTROL: Interface port-channel321, operational Transmit Flow Control state changed to on
2013 Aug 20 17:04:55 N7K-1 %ETH_PORT_CHANNEL-5-PORT_SUSPENDED: Ethernet102/1/9: Ethernet102/1/9 is suspended
2013 Aug 20 17:05:13 N7K-1 %LACP-FEX102-5-LACP_SUSPEND_INDIVIDUAL: LACP port Ethernet102/1/9(0x1f650200) of port-channel port-channel321(0x16000140) not receiving any LACP BPDUs suspending (individual) port
Scenario 3-Etherchannel with LACP to single 3750 switch
All working fine, fail over success as expected.
N7k is using NX-OS 6.0(4)
Anyone facing this issue?

Hi,
I had had this problem before.
Please, put the following command, if in your server side is using LACP:
interface Ethernet102/1/9
switchport
switchport access vlan 204
channel-group 321 mode active
no shutdown
N7K-1# sh run int po321
interface port-channel321
switchport
switchport access vlan 204
flowcontrol send on
vpc 321
N7K-2# sh run int eth102/1/9
interface Ethernet102/1/9
switchport
switchport access vlan 204
channel-group 321 mode active
no shutdown
N7K-2# sh run int po321
interface port-channel321
switchport
switchport access vlan 204
flowcontrol send on
vpc 321

Nexus 6K: Port-Channel Load-Balance

Similar Messages

Maybe you are looking for