NIC Teaming/Bonding with multiple switches.

Today I started setting up in my lab a network with redundant Cisco 2970 switches. The hosts are two HP DL145 servers with two broadcom GBe NICs. The systems are running Linux 2.6.15.1 and I have enabled bonding in the kernel as a module.
Each NIC is plugged into a different switch, and the switches are NOT connected together using ISL or any other method. For all intents they operate as single switches. Each switch is in turn connected to a switch on which my gateway is located.
I have the configuration working using mode 0 (balance-rr). Using tcpdump on the underlying slave interfaces, I can see that packets are in fact being sent out over both interfaces, and that odd sequence numbers are sent on if0 and even numbers are sent on if1. If I unplug one of the interfaces, the bonding driver marks it as down, and stops sending data on that interface. When plugged back in, data transmission is resumed. Furthermore, when using the bonding driver's arp monitoring of the default gateway, unplugging a switch from the upstream switch causes the interface connected to that switch to fail.
My question is if this is a "supported" configuration and/or if there is a better way to make this work.
Furthermore, my next test is to add a PIX 515UR to the mix, and figure out how to connect IT to both switches.
I cannot find any information about how to bond or team interfaces on the PIX, can it do something like this?

Similar Messages

Using NIC Teaming and a virtual switch for Windows Server 2012 host networking and Hyper-V.

Using NIC Teaming and a virtual switch for Windows Server 2012 host networking!
http://www.youtube.com/watch?v=8mOuoIWzmdE
Hi thanks for reading. Now I may well have my terminology incorrect here so I will try to explain as best I can and apologies from the start.
It’s a bit of both Hyper-v and Server 2012R2.
I am setting up a lab with Server 2012 R2. I have several physical network cards that I have teamed called “HostSwitchTeam” from those I have made several Virtual Network Adaptors such as below
examples.
New-VMSwitch "MgmtSwitch" -MinimumBandwidthMode weight -NetAdaptername "HostSwitchTeam" -AllowManagement $false
Add-VMNetworkAdapter -ManagementOS -Name "Vswitch" -SwitchName "MgmtSwitch"
Add-VMNetworkAdapter -ManagementOS -Name "Cluster" -SwitchName "MgmtSwitch"
When I install Hyper-V and it comes to adding a virtual switch during installation it only shows the individual physical network cards and the
HostSwitchTeam for selection. When installed it shows the Microsoft Network Multiplexor Driver as the only option.
Is this correct or how does one use the Vswitch made above and incorporate into the Hyper-V so a weight can be put against it.
Still trying to get my head around Vswitches,VMNetworkadapters etc so somewhat confused as to the way forward at this time so I may have missed the plot altogether!
Any help would be much appreciated.
Paul
Paul Edwards

Hi P.J.E,
>>I have teams so a bit confused as to the adapter bindings and if the teams need to be added or just the vEthernet Nics?.
Nic 1,2
HostVMSwitchTeam
Nic 3,4,5
HostMgmtSwitchTeam
>>The adapter Binding settings are:
HostMgmtSwitchTeam
V-Curric
Nic 3
Nic 4
Nic 5
V-Livemigration
HostVMSwitch
Nic 1
Nic 2
V-iSCSI
V-HeartBeat
Based on my understanding of the description , "HostMgmtSwitchTeam and
HostVMSwitch " are teamed NIC .
You can think of them as two physical NICs (do not use NIC 1,2,3,4,5 any more , there are just two NICs "HostMgmtSwitchTeam and
HostVMSwitch").
V-Curric,
V-Livemigration , V-iSCSI ,
V-HeartBeat are just VNICs of host (you can change their name then check if the virtual switch name will be changed )
Best Regards
Elton Ji
We
are trying to better understand customer views on social support experience, so your participation in this
interview project would be greatly appreciated if you have time.
Thanks for helping make community forums a great place.

NIC Teaming/Bonding

Do anyone has a doc or link for NIC teaming/bonding ?
Thanks in advance.
Regards,
Chandu

Hi Chandu,
NIC teaming is the process of grouping together several physical NICs into one single logical NIC, it can be used for network fault tolerance and transmit load balance. The process of grouping NICs is called teaming.
It has two purposes:
Fault Tolerance: By teaming more than one physical NIC to a logical NIC, high availability is maximized. Even if one NIC fails, the network connection does not cease and continues to operate on other NICs.
Load Balancing: Balancing the network traffic load on a server can enhance the functionality of the server and the network.
example: http://www.cisco.com/en/US/tech/tk389/tk213/technologies_configuration_example09186a008089a821.shtml
Hope it helps.
Regards
Dont forget to rate helpful posts.

Handshaking DMM with multiple Switch devices - DAQmx error

Hi.
I'm trying to create a handshaking loop with DMM (PXI-4071), SWITCH (PXI-2569) and MUX (PXI-2575). All three instruments are in segment 2 of PXI-1045 chassis (slots 8, 9 and 10) and I am using PXI trigger lanes to route triggers.
I followed the NI article 'Multi-module Scanning with National Instruments Switches' - I modified the NI SWITCH example 'niSwitchDMMSwitchHandshaking' to configure the other SWITCH but when I tried to run the example, I got an error:
0xbffa6b9a - No registered lines could be found between the device in the route. (pop-up screenshot is in the attachment). It is the niSwitch_InitiateScan function for the second Switch that returned the error.
Changing PIX trigger lanes has no effect.
I tried both CVI and LabVIEW examples with the same result.
I even tried to use two 2575 MUXes - same result.
Can anybody tell me what am I doing wrong?
Solved!
Go to Solution.
Attachments:
errror1.JPG ‏26 KB

Hi Pavel,
For the purposes of this post, I'll define
the measurement complete signal (sent by the DMM to the switch modules
after each measurement) as 'MC' and place it on TTL0.
For the
purposes of this post, I'll define the scan advanced signal (sent by the
switch module(s) to the DMM once the relays have settled) as 'SA' and place
it on TTL1.
You mentioned you're using NI-Switch, which is NI's IVI
compliant switch API. Since the IVI Foundation regulates the behavior
of IVI compliant software, we must adhere to their rules when
implementing our API. Unfortunately, the IVI switch standard doesn't
provide a method to control arbiting of triggers between multiple switch modules
simultaneously.
Let's
look at what happens when we setup a system with multiple switch
modules handshaking with a single DMM. The DMM is going to take a
measurement and then send MC on TTL0. Meanwhile, each switch is listening to TTL0,
waiting for the MC pulse. When the MC pulse is received, each switch sets
the relays according to its next scan list entry, waits for debounce,
and then sends SA on TTL1. The problem we run into here is that depending on the switch module, number of relays connected simultaneously, jitter, etc, it's possible that one module will send the SA trigger on TTL1 before the other. Since the IVI spec doesn't provide any way to implement a 'master' switch or an arbitor, it's impossible to implement a system such that only the last switch that settles sends a trigger. Therefore, what happens is we get a whole bunch of switch modules sending triggers at slightly different times onto TTL1. If one switch is driving TTL1 high while the others are driving TTL1 lo, it's remotely possible that we could damage the TTL circuitry on the PXI backplane.
To date, NI hasn't seen any failures due to simultaneously driving the TTL lines high and low at the same time with NI switch hardware, but it is theoretically possible that damage could occur. For this reason, NI implemented a change in DAQmx
9.0.0, 9.0.1, and
9.0.2 that prevents a user from setting up handshaking with multiple switch modules while using NI-Switch. What does DAQmx have to do with this, you might ask? A component installed with DAQmx is responsible for verifying the triggers are valid.
Customers with existing NI-Switch multi-module handshaking applications will find that upon upgrading to any of the above three versions of DAQmx, the error you observed will occur. We've evaluated this customer feedback and have decided to revert to the previous functionality in a yet-to-be released version of DAQmx. Please note that NI does not advise driving the same TTL line with multiple sources due to the chance that we'll double-drive the line. Therefore, it goes with out saying that NI does not advise using NI-Switch in multi-module handshaking applications. We do, however, still recommend NI-Switch for Syncrhonous triggering because the switches never send triggers (in synchronous mode, the DMM just waits a predefined amount of time before switching).
Note that if you use the DAQmx Switch API, we're no longer bound to the IVI spec, which means we have an arbitor that ensures only one switch module drives the SA trigger on TTL1. NI highly recommends that customers evaluate using the DAQmx switching API for multi-switch handshaking applications. An example DAQmx handshaking application for the DAQmx Switch API is located in Example Finder»Hardware Input and Output»DAQmx»Switches»Switch Scanning with DMM - Handshaking.vi.
Note that in DAQmx, we'll only have one scan list, regardless of the number of switches we have. Note that the syntax in DAQmx scanning is different than NI-Switch. I'll defer a detailed explanation of the differences to the DAQmx and NI-Switch Help (search for 'scan list'), but in short, we'll need to include the DAQmx Device name prior to each connection. For example, in NI-Switch, if we want to connect CH1 to Com1:
CH1->Com1;
In DAQmx, we'll need to include the Device name:
Dev1\CH1->Com1;
Note that to add additional switch modules in the DAQmx API, we simply
call the Set Topology and Reset VI multiple times:
DAQmx keeps the
session loaded in memory and as I noted above; we define which switch
does the action as part of the scan list entry.
If you'd still like to use NI-Switch, you could roll back to DAQmx 8.9.5 or previous, or if you want to stick with 9.0.x, I highly recommend that we daisy chain the triggers as follows:
DMM Measurement Complete to Advance Trigger on Switch1 via TTL0
Scan Advance from Switch1 to Advance Trigger on Switch2 via TTL1
Scan Advance from Switch2 to Trigger Source on DMM via TTL2
Note that we'll need an additional TTL line for each switch module. Also note that some switch modules allow front panel triggers, which reduces the number of TTL lines we'll need on the backplane, but which requires external wiring between switch modules.
Message Edited by Knights Who Say NI on 06-11-2010 05:25 PM
Message Edited by Knights Who Say NI on 06-11-2010 05:30 PM
Message Edited by Knights Who Say NI on 06-11-2010 05:30 PM
Message Edited by Knights Who Say NI on 06-11-2010 05:31 PM
-John Sullivan
Analog Engineer

NIC teaming/bonding for interconnection

Hi to all,
We are planning for a two node Oracle RAC implementation on Windows Server and we have some doubts about interconnect. Because redundancy is not as important, and because we are limited by money Gigabit switch will be used for interconnection and for public network.
Can we get in bandwidth (load balancing) if we use NIC Bonding on one switch. Which is best?
- To buy servers with two single 1Gb port NIC’s and bond this NIC’s for interconnection and one single port NIC for public network or
- One dual port NIC (bond this NIC’s) for interconnects and one single NIC for public network?

Nikoline wrote:
We are planning for a two node Oracle RAC implementation on Windows Server and we have some doubts about interconnect. Because redundancy is not as important, and because we are limited by money Gigabit switch will be used for interconnection and for public network.Keep in mind that the h/w architecture used, directly determines how robust RAC is, how well it performs, and how it meets redundancy, high availability and scalability requirements.
The wrong h/w choices can and will hurt all of these factors.
Can we get in bandwidth (load balancing) if we use NIC Bonding on one switch. No. Bonding usually does not address load balancing. You need to check exactly what the driver stack used for bonding supports.
The primary reason for bonding is redundancy. Not load balancing/load sharing.
Which is best?
- To buy servers with two single 1Gb port NIC’s and bond this NIC’s for interconnection and one single port NIC for public network or
- One dual port NIC (bond this NIC’s) for interconnects and one single NIC for public network?A server with dual 1Gb NICs will be bonded into a single bonded NIC. And that is what then must be used - the logical bonded NIC and not the individual NICs.
This means that 2 1Gb ports provide you with a single bonded port. And you need 2 ports for RAC - public and interconnect.
So you either need 4 physical ports in total to create 2 bonded ports, or 3 ports in total for 1 bonded port and 1 unbonded port.
And what about your cluster's shared storage?
This, together with the Interconnect, determines the robustness and performance of RAC. 1Gb Interconnect is already pretty much a failure in providing a proper Interconnect infrastructure for RAC.
Keep in mind that the Interconnect is used to share memory across cluster nodes (cache fusion). The purpose is to speed up I/O - making it faster for a cluster node to get data blocks from another node's cache instead of having to hit spinning rust to read that data.
Old style fibre channel technology for shared storage is dual 2Gb fibre channels. Which will be faster than your 1Gb Interconnect. How does it make sense to use an Interconnect that is slower (less bandwidth and more latency) than the actual I/O fabric layer?
Would you configure h/w for a Windows server (be that a database, web or mail server) where the logical I/O from the server's memory buffer cache for local disks is slower than actually reading that data off local disk?
Then why do this for RAC?
Last comment - over 90% of the 500 fastest and biggest clusters on this planet run Linux. 1% runs Windows. Windows is usually a poor choice when it comes to clustering. Even Oracle does not provide their Exadata Database Machine product (fastest RAC cluster in the world) on Windows - only on Linux and Solaris (and the latter only because Oracle now owns Solaris too).

Teaming servers to multiple switches

I am looking for a configuration guide / example that shows how to connect a Server with multiple NIC's to 2 different switches for failover. The servers could be Window's or Unix based. Thank you.

Actually, on the 3750s if stacked at least, I believe this is now possible to channel those ports.
I'm not sure if that carries over to the 2970s or not.
The "other company" called this "Split Multi Link Trunking". I'm not sure if Cisco calls it Split Etherchannel or not, but I'm 99% sure that that was one of the benefits of the stacked 3750s for uplinks, and there's no reason that it wouldn't apply to servers too, so long as they are capable of etherchannels.
...N

Nic teaming and hyper-v switches

I come from the ESX world but I am slowly falling in love with the simplicity of Hyper-v. I have a stack of dell c2100's I have been experimenting with. each have 2 1gb connections teamed to a cisco switch. when testing bandwidth with a file copy I
get around 240MBps. however if I add a hyper-v switch I max out at 90Mbps. worse than no teaming at all (112Mbps).
team is with integrated broadcom nics, LACP and I can confirm I get full bandwidth between 2 2012 r2 machines until adding a hyper-v switch. removing the switch lets me transfer at full bandwidth but then I cant use Hyper-v guests.
my goal will eventually be to add dual port 10gb cards to 5 of the C2100's and run them in a cluster to host all my VM's in HA. I don't want to waist my money on the switch and nics until I can get what i have working correctly.
HDD speed is also not the issue as each has 12 3tb WD re4 drives with 2 Intel 250GB ssd as cache. they easily hold 3000MBps sustained.

http://itproctology.blogspot.com/2008/05/hyper-v-tcpoffloading-poor-network.html
http://itproctology.blogspot.com/2011/03/tcp-checksum-offload-is-not-equal-to.html
Brian Ehlert
http://ITProctology.blogspot.com
Learn. Apply. Repeat.
Disclaimer: Attempting change is of your own free will.

NIC teaming and Hyper-V switch recommendations in a cluster

HI,
We’ve recently purchased four HP Gen 8 servers with a total of ten NICS to be used in a Hyper-V 2012 R2 Cluster
These will be connecting to ISCSI storage so I’ll use two of the NICs for the ISCSI storage connection.
I’m then deciding between to options.
1. Create one NIC team, one Extensible switch and create VNics for Management, Live Migration and CSV\Cluster - QOS to manage all this traffic. Then connect my VMs to the same switch.
2. Create two NIC teams, four adapters in each. Use one team just for Management, Live Migration and CSV\Cluster VNics - QOS to manage all this traffic.
Then the other team will be dedicated just for my VMs.
Is there any benefit to isolating the VMs on their own switch?
Would having two teams allow more flexibility with the teaming
configurations I could use, such as using Switch Independent\Hyper-V Port mode for the VM team? (I do need to read up on the teaming modes a little more)
Thanks,

I’m not teaming the ISCSI adapters. These would be configured with MPIO.
What I want to know,
Create one NIC team, one Extensible switch and create VNics for Management, Live Migration and CSV\Cluster - QOS to manage all this traffic. Then connect
my VMs to the same switch.
http://blogs.technet.com/b/cedward/archive/2014/02/22/hyper-v-2012-r2-network-architectures-series-part-3-of-7-converged-networks-managed-by-scvmm-and-powershell.aspx
What are the disadvantages to having this configuration?
Should RSS be disabled on the NICs in this configuration with DVMQ left enabled?
After reading through this post, I think I’ll need to do this.
However, I’d like to understand this a little more.
I have the option of adding an additional two 10GB NICS.
This would mean I could create another team and Hyper-V switch on top and then dedicate this to my VMs leaving the other team for CSV\Management and Live Migration.
How does this option affect the use of RSS and DVMQ?

NIC Teaming/Bonding Support by Oracle or NOT

hi:
I have customers using RedHat Linux ES3 Update 6 with NIC Bonding/Teaming on the Oracle 10G RAC's private/heartbeat LAN, I would like to hear from you all or Oracle Support whether SUCH CONFIGURATION IS SUPPORTED BY ORACLE OR NOT.
Thanks.
Bennie

hi:
I have customers using RedHat Linux ES3 Update 6 with NIC Bonding/Teaming on the Oracle 10G RAC's private/heartbeat LAN, I would like to hear from you all or Oracle Support whether SUCH CONFIGURATION IS SUPPORTED BY ORACLE OR NOT.
Thanks.
Bennie

Multiple Minis with Multiple Switched Monitors Losing Monitor 2

Hi.
I am working on an installation with six new Mac Mini's (Lion) and we are using dual screen HDMI outputs on both. The Monitor 2 (HDMI) output goes to the display screen "on stage" and the Monitor 1 (Thunderbolt to HDMI) output goes to a local HDMI switched monitor for local viewing and programming. The problem is that when we switch the local monitor from Mac 1 to Mac 2 and then back to Mac 1, Mac 1 will have lost its connection to Monitor 2, defaulting back to Single Monitor Output. Changing the monitor arrangement is not an option.
Is there any way around this?
Thanks.

When you switch the monitor away from a particular Mac, that Mac no longer receives a signal back from the monitor. The Mac then assumes it has been disconnected which in reality is true. The Mac therefore reconfigures itself accordingly.
In some circumstances like yours, and more commonly people using a TV as a display via an AV Receiver, this can be counter to what is desired. There is a solution but it is not free, Gefen make a device called a HDMI Detective which 'learns' the signal the TV sends, and then will keep sending it to the Mac even when the TV (or monitor) is disconnected. They also make a DVI version see http://www.gefen.com/kvm/product.jsp?prod_id=8005 and http://www.gefen.com/kvm/dproduct.jsp?prod_id=4715
It should be noted that if you are running audio via HDMI, then when the Mac loses the HDMI connection it also resets the sound to go via the built-in speaker. The Gefen HDMI Detective Plus also solves this.

SG 300 10 - Multicast with multiple switches in LAN

Hi there,
I have short question concerning Multicast in my LAN. My internet router acts as an IGMP Proxy for the LAN and distributes IPTV there. The router is connected to one of my three SG 300 10 Cisco switches (connected to one another via uplink port). Now I want all switches to send multicast traffic to only the recipients of such traffic (PCs with VLC and Network Media Players) connected to either one of them. How do I have to configure the switches to do so and not flood the LAN with multicast traffic, is it enough to enable IGMP Snooping on each switch?
Thanks in advance and best regards
Christoph

Hi Christopher, by the default the switch will treat multicast traffic like a broadcast until you enable the Bridge Multicast Filter and IGMP snooping.
Navigate to; Multicast -> Properties and enable the Bridge Multicast Filtering
Navigate IGMP snooping -> Enable
Multicast -> Multicast Router Port
Then make sure to specify the mrouter port (which must be a member of a static vlan, not a dynamic vlan)

NIC Teaming with CSS

Hi Gilles,
Is NIC teaming supported with CSS? How can I load balance two servers with each server having two NIC and using NIC teaming.
Thanks
Sushil

As far as I know there is no LAG on the css. I think the best solution is to use a switch in between the servers and the css.

IBM MCS Server RUnning Unity with NIC Teaming

All,
has anyone ever run NIC teaming on and IBM MCS Server with Unity before? At question is the fact that many severs create a virtual MAC address that is different from either of the actual MACs when you team NICs. If this is the case on the IBM Servers then we may need to request an updated license. The servers are branded as Cisco MCS-7835-I1
units.
Thanks in advance. All replies rated!

Hi
According to this doc: http://www.cisco.com/en/US/prod/collateral/voicesw/ps6790/ps5748/ps378/product_solution_overview0900aecd80091615.html
The 7825-I4 is an IBM IBM x3250-M2. You can use that model number to search the IBM drivers page for the things you need:
http://www-933.ibm.com/support/fixcentral/systemx/selectFixes
I think the NIC teaming comes with the Network Drivers.
Regards
Aaron
Please rate helpful posts...

NIC teaming in OVM

Hi folks,
Does Oracle VM support NIC Teaming(Bonding of more than 1 NICs in to 1 Logical NIC)? Does it require any driver or patch for the same.
I have SUN server x4710 which has 2 NIC cards.
Moreover If I ve VMs(guests) with IPs of different VLANs(Eg. 10.22.70.x and 202.49.214.x) residing on same VM server what would be the best practice?
Hope fully I have made the points clear..
Thanks in advance...

user10310678 wrote:
Does Oracle VM support NIC Teaming(Bonding of more than 1 NICs in to 1 Logical NIC)? Does it require any driver or patch for the same.Yes it does support bonding and no, it doesn't require any additional drivers or patches. Bonding is built into the kernel.
Moreover If I ve VMs(guests) with IPs of different VLANs(Eg. 10.22.70.x and 202.49.214.x) residing on same VM server what would be the best practice?I usually create a bridge per VLAN. That way, I can create a virtual interface to a guest that is already on a particular VLAN and the guest doesn't have to worry about VLANs. Also, it means you can control VLAN assignments outside the guest OS. See this wiki page for more info:
http://wiki.oracle.com/page/Oracle+VM+Server+Configuration-bondedand+trunked+network+interfaces

Correct binding order in a Cluster with logical switches, NIC teams, and vNICs on the host.

I have seen many recommendations to set the network binding order on you Hyper-V hosts to something similar to:
Management NIC
Cluster NICs
iSCSI NICS
However, all of these recommendations are for scenarios where the NICs are all physical NICs in the host.
Using Server 2012 R2, I am building converged networks with logical switches, NIC Teams, and vNICs on the host. So when I go set the network binding order, I now have all these components to deal with as well. For example, on a 4 adapter blade,
I might typically have the following items in the binding order drop-down.
4 - physical NICs (2- teamed for the 1 virtual switch, the other 2 used for iSCSI)
1 - Team interface (Datacenter_Switch)
5 - vNICs (Management, Cluster, LiveMigration, iSCSI-1, iSCSI-2)
So, should you only worry about order of the vNICS (placed at the top) and let the other components just fall to the bottom of the list? This seems to be likely to me, since the binding order applies to service access to the resources, and the other
components are not being directly accessed by network services?
Or, should the order start out with the physical resources needed to access the vNICs, followed by any intermediate resources (switches or team interfaces, then the vNICS themselves, to ensure that the resources are available to the subcompnents accessing
them?
Any help would be appreciated.
Thanks.
-Tim Reid

If by 'network binding order' you mean the order set in the Advanced Settings of the Network Connections of the Control Panel, then the most important one is to make sure the domain network is at the top of the list. Whichever network is at the top
of the list is used first for auth functions. So auth functions perform best when the proper network is placed first in the binding order. After that, I don't know that it makes much difference at all. (If it does, I'm sure my statement will
start a lively discussion. <grin>)
. : | : . : | : . tim

NIC Teaming/Bonding with multiple switches.

Similar Messages

Maybe you are looking for