NIC teaming/bonding for interconnection

Question

Hi to all,
We are planning for a two node Oracle RAC implementation on Windows Server and we have some doubts about interconnect. Because redundancy is not as important, and because we are limited by money Gigabit switch will be used for interconnection and for public network.
Can we get in bandwidth (load balancing) if we use NIC Bonding on one switch. Which is best?
- To buy servers with two single 1Gb port NIC’s and bond this NIC’s for interconnection and one single port NIC for public network or
- One dual port NIC (bond this NIC’s) for interconnects and one single NIC for public network?

Answer 1

Nikoline wrote:
We are planning for a two node Oracle RAC implementation on Windows Server and we have some doubts about interconnect. Because redundancy is not as important, and because we are limited by money Gigabit switch will be used for interconnection and for public network.Keep in mind that the h/w architecture used, directly determines how robust RAC is, how well it performs, and how it meets redundancy, high availability and scalability requirements.
The wrong h/w choices can and will hurt all of these factors.
Can we get in bandwidth (load balancing) if we use NIC Bonding on one switch. No. Bonding usually does not address load balancing. You need to check exactly what the driver stack used for bonding supports.
The primary reason for bonding is redundancy. Not load balancing/load sharing.
Which is best?
- To buy servers with two single 1Gb port NIC’s and bond this NIC’s for interconnection and one single port NIC for public network or
- One dual port NIC (bond this NIC’s) for interconnects and one single NIC for public network?A server with dual 1Gb NICs will be bonded into a single bonded NIC. And that is what then must be used - the logical bonded NIC and not the individual NICs.
This means that 2 1Gb ports provide you with a single bonded port. And you need 2 ports for RAC - public and interconnect.
So you either need 4 physical ports in total to create 2 bonded ports, or 3 ports in total for 1 bonded port and 1 unbonded port.
And what about your cluster's shared storage?
This, together with the Interconnect, determines the robustness and performance of RAC. 1Gb Interconnect is already pretty much a failure in providing a proper Interconnect infrastructure for RAC.
Keep in mind that the Interconnect is used to share memory across cluster nodes (cache fusion). The purpose is to speed up I/O - making it faster for a cluster node to get data blocks from another node's cache instead of having to hit spinning rust to read that data.
Old style fibre channel technology for shared storage is dual 2Gb fibre channels. Which will be faster than your 1Gb Interconnect. How does it make sense to use an Interconnect that is slower (less bandwidth and more latency) than the actual I/O fabric layer?
Would you configure h/w for a Windows server (be that a database, web or mail server) where the logical I/O from the server's memory buffer cache for local disks is slower than actually reading that data off local disk?
Then why do this for RAC?
Last comment - over 90% of the 500 fastest and biggest clusters on this planet run Linux. 1% runs Windows. Windows is usually a poor choice when it comes to clustering. Even Oracle does not provide their Exadata Database Machine product (fastest RAC cluster in the world) on Windows - only on Linux and Solaris (and the latter only because Oracle now owns Solaris too).