2 fabrics on a switch ?

Hi
I am new to SANs - done the course years ago and now have to put it into practice.
Problem is -
I have 2 existing fabrics - each with 1 9134 and 1 9124 hosts on the 9124
and storage on the 9134.
Now my designer (who doesnt know SANs ) wants me to connect 2 new 9124s
each with one interface on the 2 existing 9134s in the 2 fabrics above.
When i try attaching them the fabrics disappear or bits of the existing ones appear
in the new fabric,
First - what is the usual practice =
Can i do the above - can i have a number of fabrics on one switch
If i cant - what would be recommended - Just a different VSAN.
The only common vsan on the switches is vsan 1
Thats the default - is the clash causing my problem because i need to get rid of VSan 1
on the new switches
Any info gratefully received - i would like to go away and read this all up but as usual there
is no time for planning - i need to sort this fast
If you need any details let me know
Thanks in advance
Steve

Hi,
i my mind the goal of having 2 fabrics is to have real redundancy. In other words, the host is connected with one HBA to fabric 1 and with another HBA to fabric 2.
The same on the other end where the storage sits.
This is the typical san configuration that people use today. You are then using some form of multipathing software on the host and storage to deal with the
two path, load sharing, active passive, ect.
In this configuration you can shoutdown one path, i.e. the interfaces on the switches and simply do maintenance ect., upgrade switch software....
If there is a real failure in fabric a you are still running in fabric b and most likely your users are not even noticing.
If you merge the 2 fabrics you are taking this level of redundancy away.
Second abou the failures tha you see when interconnect the two fabrics you are most likely merging the two fabrics and that depends quite a bit on the
existing configurations of your switches.
If you want to do this you may have to talk to TAC first to get sorted out what is configured in one fabric and the other, i.e. if you use the same vsan numbers and have different devices in there you have a problem ect.
thanks...
Matthias

Similar Messages

Migrate fabric to new switch

I have two switches joined in a single fabric
MDS 9120 san-1a fabric policy switch
MDS 9124 san-1b added for additional port capacity
No longer require so many ports, so I've migrated all host connections to san-1b and want to retire san-1a, but it seems to host the fabric config. Is there a defined/recommended method for migrating the fabric config to the second switch?
paul

I assume you mean the zone and zoneset configuration right?
You can go go the san1-a switch and for each VSAN enabled. The config command is 'zoneset distribute full vsan xx'.
Once this is enabled for all VSANs, then on the same san1-a switch, reactivate the currently active zoneset for each VSAN.
This will push the complete zone database (fc-aliases too) to san-1b and it should show up in the config. You need to do this for each VSAN.
Once completed...issue 'copy run start' to save the updated running configuration to NVRAM.
Device-alias is distributed by CFS, so when if configured them on san-1a, when you issued the 'device-alias commit', it should have pushed them over to san-1b.
Not sure that other config statements would need to be moved over.
Hope this helps,
Mike

Fabric Resource & Virtual Switch help

Hi,
Apologies if this is a silly question. Basically we have a couple of Hypervisors HP Blade. The first Blade has a virtual network teamed NICs which give the guests network access.
When I provision an additional blade, I have to create another virtual switch rather than use the switch previously created. Am I correct in understanding that there needs to be one overall switch added in to each blade rather multiple VSwitches for
each blade? And if so how would I go about creating it?
Currently we have a VSwitch on blade 1. But looking Blade 2 nothing is available so I have to create a second. If we ever fail the clients presumably the guests loose connectivity because it's set up incorrectly? How do I go about creating it
so that it's done with the bigger picture in mind? Also how would I do this without affecting the current guests on blade 1?
Any advice would be appreciated and apologies if this question is somewhat convoluted!

Blades are independent installations of Windows Server/Hyper-V. How you configure one blade has absolutely nothing to do with how you configure the other blades from the Windows Server/Hyper-V standpoint. So, yes, if you want virtual switches
on each blade, you need to configure them on each blade because you are configuring them to the operating system installed on the blade.
. : | : . : | : . tim

Fabric Manager - switches status - No Traps.

Greetings,
When I am looking at the Fabric in Fabric Manager, the switches present information like name, etc.. and tere is a Status entry. I see Status of No Traps. What does it mean?
Stephen

If you get yellow text 'No Traps' in FM client means that the FM has not registered trap address on this switch successfully and hence not receiving traps from those switches. It may be that the trap address table is full (max is 10 entries) on the switch. If table is full, then delete any stale/unused addresses in the switch. The FMS's next periodic poll will add it's trap address to the switch and the 'No Traps' warning will go away after this. Try
refreshing the Switch table in the FM client to see an updated status.
And also check the Switches remain manageable even when "No Traps" is displayed. Then you are hitting bug CSCta72679. To workaround this Rediscovering the Fabric will change the status back to "ok" but this is only a temporary solution.

Multiple Fabrics - how many?

Hi,
We are configuring our MDS switches for new storage and there are a couple of options. I am not sure which is the best way to go.
We have 6 x 9124's. Our new storage has plenty of ports. Each of the 6 x 9124's will be connected to 2 ports on the storage (12 connections in total). I am going to treat the 6 x 9124s as pairs - so really it will be 3 pairs, and each pair is essentially a fabric A and B.
So the idea would be that when connecting a host to our SAN, we will pick one of the pairs and connect the host to both switches in that pair and zone it to use the storage ports connected to that same switch. This way, traffic will never actually need to "cross" between switches (and in fact it can't as there are no ISLs) - it will always be from the host to the storage all on the same switch. Each of the pairs of switches is acting as a fabric A and B.
In reality we will have 6 fabrics (as each switch will be a fabric). Connecting a new host to our storage will require configuration on two switches (fabrics), depending on which of the 3 pairs we choose to connect that host to. So really it will be 3 seperate "fabric A and B" setups, one for each pair.
This all sounds reasonable to me and should work quite well.
The only thing I am unsure about is whether there is any benefit in connecting ISLs between each of the "fabric A" switches and each of the "fabric B" switches - which would actually mean we really did have 2 fabrics rather than 6.
I am looking to understand the pros and cons of doing this. We have plenty of ports so the ports lost to the ISLs are not an issue. These ISLs would never be used for "storage traffic" to actually flow between switches as we would still zone it so that hosts always talked to the storage port(s) on the same switch. The only reason to create the ISLs would be to consolidate 6 fabrics down to 2 which may (?) make it easier to manage.
Are there real advantages to connecting the switches to consolidate down to 2 fabrics? I'm thinking that potentially it will be "nicer" to connect to DCNM-SAN and only see 2 fabrics rather than 6. I'm also thinking that the "maps" that DCNM produces will be easier to comprehend when there are only 2 - rather than having to look at 6 different maps to get a good picture of what we have. We will only be using the free DCNM-SAN (Essentials I think it is called).
Hopefully my question makes sense. I'm really looking to understand whether it is better to keep 6 seperate fabrics or whether it might make more sense to use ISLs to consolidate down to 2. Whether we keep 6 or go with 2, the concept would remain the same (each host connecting to a single pair of A/B switches and zones to the "local" storage ports) - so both solutions keep all storage traffic within the same switch. The ISLs would effectively only act as "management" connectors such that it could all be seen together within DCNM-SAN. Is there a disadvantage to connecting them together?
How would you do it?
Cheers,
David

Although I am not a SAN guy I would connect them together from the start. If the need ever arises to use a target on a different switch you would not face a SAN merge. Furthermore the zoning proces will be simpler you simply need to zone on FABRIC A and B and don't need to determine is the fabric A1 A2 or A3 to which the host is connected.

Fabric path vlan question

Setup as shown in the attached. S1-S4 are Fabric path spine switches. Will it work ? or I need to configure both vlan 10 and vlan 20 in "mode fabricpath" on all S1 to S4 even though only vlan 10 is required in S1-S2 and vlan 20 required in S3-S4.

Have you tried looking on Github? You might a Python script that you can re-use.

Uplink ports and actual switches sfp usages (please help~~)

Dear all,
I am reading around to see the differences between uplink ports and normal ethernet ports. It seems to be uplink port just reverse the rx,tx pins and remove the needs of using a crossover cable.
I have done a simple representation of using the below (normal ethernet ports)
PC
SWITCH
ROUTER
RX
straight
TX
straight
RX
RX
straight
TX
straight
RX
TX
straight
RX
straight
TX
TX
straight
RX
straight
TX
I assumed that the SWITCH ethernet port pins are naturally arranged as opposite from the PC so that a straight cable can be used and the same applies to a router port.
Q1) Hence from PC to Router ethernet port, I will need a cross cable, am I right ?
============================================================
Then I chance upon this website http://duxcw.com/faq/network/uplink.htm which states "A PC can be connected to an uplink port with a crossover cable"
As per my assumption above, a router has normal ethernet ports and uplink ports (for connecting to the wan/modem), since the normal ethernet ports of a router are in the order of (RX,RX,TX,TX) as shown above, I assumed the uplink port is (TX,TX,RX,RX).
PC
ROUTER normal ethernet port
PC
ROUTER uplink port
RX
cross
RX
RX
straight
TX
RX
cross
RX
RX
straight
TX
TX
cross
TX
TX
straight
RX
TX
cross
TX
TX
straight
RX
Q2) So how can the PC connect to the Router's Uplink port using a crossover cable ? Shouldn't it be using straight cable instead if it is trying to connect to a router's uplink port ?
============================================================
Q3) Where connecting switches (switch 1 to swtich2) using uplink port, can I say that a straight cable will be connected to switch 1 uplink port on 1 end, and a normal ethernet port in switch 2 on another end ?
SWITCH#1 ethernet port
SWITCH#1 Uplink port
SWITCH#2 ethernet port
TX
RX
straight
TX
TX
RX
straight
TX
RX
TX
straight
RX
RX
TX
straight
RX
============================================================
Q4) What is the purpose of an SFP port in a switch ? is it use as an uplink port ? Where does it actually uplink/connect to ?
Q5) If i connect a switch Uplink port to another switch Uplink port, does it means that I have to use a crossover cable ? does SFP fiber come with "crossover" type too ?
============================================================
Q6) If an uplink port in a switch is use to connect to a router (can we?), do i need to use a crossover cable then ? since a normal switch ethernet port will use a straight cable to connect to a router port ? as shown below
SWITCH
ROUTER
SWITCH UPLINK
ROUTER
TX
straight
RX
RX
cross
RX
TX
straight
RX
RX
cross
RX
RX
straight
TX
TX
cross
TX
RX
straight
TX
TX
cross
TX
============================================================
Q7) Last but not least, if a design connect two switches using SFP uplink ports on both side, what would be the main purpose ? to increase the amount of ports ? Why not stack them then ?
Sorry for the lengthy post, will really appreciate if some gurus can shed some light on the above.
Thanks a million.
Regards,
Noob

An uplink port doesn't necessarily run at wire speed but it is more likely to.
But this doesn't mean you don't get oversubscription from the switch to other switches.
It's quite a big subject to cover but i'll try and keep it brief.
Internally within a switch there is a crossbar switch fabric which I think we covered before. Think of it of as pathways between ports. On any switch there are a number of ports that are capable of certain speeds.
Put simply if you take the number of ports on the switch and multiply that by the fastest speed the ports can operate at, if that number is greater than the switch fabric then the switch is said to be oversubscribed.
It doesn't mean in practice it will be oversubscribed but the potential is there.
What can happen though with oversubscribed switches is that the uplinks ports are guaranteed not to be oversubscribed ie. they do not need to contend with the other ports on the switch. It's complicated but it is to do with the connectors per port group etc. to the switch fabric.
But even though the uplinks ports can run at wire speed so they are not oversubscribed, if they are uplinks to other switches there may very well be more traffic combined from the other ports on the switch to that uplink port so you still get an oversubscription ratio.
But what would make it a whole lot worse was if your uplink port was oversubscribed within the switch as well.
So there is oversubscription within the switch and this is where we talk about ports running at wire speed or not and then there is oversubscription between different layers of the network eg. access to distribution for example.
Jon

Firmware upgrade correct steps?

We are carrying out a firmware upgrade on our 9509’s from 3.3(1c) to 5.2(2) and I want to know if the steps below are correct?
Check hosts for paths to both Fabric's.
Take a backup of the config for all the switches.
On Fabric A upgrade switch 1 to 4.1(x)
Upgrade switch 2 in Fabric A to 4.1(x)
On Fabric A upgrade switch 1 to 5.0(x)
Upgrade switch 2 in Fabric A to 5.0(x)
On Fabric A upgrade switch 1 to 5.2(2)
Upgrade switch 2 in Fabric A to 5.2(2)
When happy, then do the same for Fabric B for the other switches.
Thanks

looks good, just follow the release notes to go to the correct intermediate code level so that upgrade is non disruptive

Adapter FEX and VM Fex

Hi!
I have a questions about Cisco Adapter FEX and VM Fex.
As i understand, Cisco Adapter FEX gives multiple vNIC on one mezzanine card, and each of them appears as virtual port on Fabric Interconnect.
Also, same thing for VM-FEX, each VM NIC appears as virtual port on Fabric Interconnect.
Am i right?
Thank you!

Adaptor FEX is I/O adaptor virtualization, which is agnostic to the OS and therefore works in virtualized as well as non virtualized environments.
VM-FEX (also called Hypervisor Bypass) is available for different Hypervisors
Cisco UCS Manager VM-FEX for Hyper-V CLI Configuration Guide, Release 2.2
Cisco UCS Manager VM-FEX for KVM CLI Configuration Guide, Release 2.2
Cisco UCS Manager VM-FEX for VMware CLI Configuration Guide, Release 2.2
Cisco UCS Manager VM-FEX for Hyper-V GUI Configuration Guide, Release 2.2
Cisco UCS Manager VM-FEX for KVM GUI Configuration Guide, Release 2.2
Cisco UCS Manager VM-FEX for VMware GUI Configuration Guide, Release 2.2
Cisco UCS Manager VM-FEX for Hyper-V CLI Configuration Guide, Release 2.1
Cisco UCS Manager VM-FEX for KVM CLI Configuration Guide, Release 2.1
Cisco UCS Manager VM-FEX for VMware CLI Configuration Guide, Release 2.1
Cisco UCS Manager VM-FEX for Hyper-V GUI Configuration Guide, Release 2.1
Cisco UCS Manager VM-FEX for KVM GUI Configuration Guide, Release 2.1
Cisco UCS Manager VM-FEX for VMware GUI Configuration Guide, Release 2.1
Cisco UCS Manager VM-FEX for KVM CLI Configuration Guide
Cisco UCS Manager VM-FEX for KVM GUI Configuration Guide
Cisco UCS Manager VM-FEX for VMware CLI Configuration Guide
Cisco UCS Manager VM-FEX for VMware GUI Configuration Guide
example VM-FEX for VMware ESX
VM-FEX (previously known as VN-link) is a method to extend the network fabric completely down to the VMs. With VM-FEX, the Fabric Interconnects handle switching for the ESXi host's VMs. UCSM utilizes the vCenter dVS Application Programming Interfaces (API) to this end. Therefore, VM-FEX shows as a dVS in the ESXi host.
There are many benefits to VM-FEX:
Reduced CPU overhead on the ESX host
Faster performance
VMware DirectPath I/O with vMotion support
Network management moved up to the FIs rather than on the ESXi host
Visibility into vSphere with UCSM

Do we need to create two zones for Two HBA for a host connected with SAN ?

Hi,While creating Zone , Do we need to create two zones for Two HBA for a host connected with SAN ? Or a zone is enough for
a host which having Two HBAs...We have two 9124s for our SAN fabric...
As I found like one zone below, I little bit confused that , if a host having two HBA connected with SAN, should I expect two zones for every Host?
from the zone set, I gave the command show zoneset
zone name SQLSVR-X-NNN_CX4 vsan 1
    pwwn 50:06:NN:NN:NN:NN:NN:NN
    pwwn 50:06:NN:NN:NN:NN:NN:NN
          pwwn 10:00:NN:NN:NN:NN:NN:NN
But I found only one zone for the server's HBA2:by the same time in the fabric I found switches A & B showing the WWNs of those HBAs on its
connected N port...Its not only for this server alone, but for all hosts..Can you help me to clarify on this please..that should we need to create one zone for
one HBA?

if u have two independent fabrics between hosts and storage, i think the below confs are recommended.
Scenario 1: 2 HBAs single port each ( redundancy across HBA / Storage port )
HBA1 - port 0 ---------> Fabric A ----------> Storage port ( FAx/CLx )
HBA2 - port 0 ---------> Fabirc B ----------> Storage port ( FAy/CLy )
Scenario 2: 2 HBAs of dual port each
HBA1 - port 0 -------> Fabric A ---------> Storage port ( FAx/CLx )
HBA2 - port0 ---------> Fabric A ---------> Storage port ( FAs/CLs )
HBA1 - port 1 --------> Fabric A --------> Storage port ( FAy/CLy )
HBA2 - port 1 ---------> Fabric B --------> Storage port ( FAt/CLt )
the zone which is in your output is VSAN 1. if its a production VSAN, Cisco doesn't recomends to use VSAN 1 ( default vsan ) for production.

Half of RAID not mounting

I have got an Xserve RAID RAID with 250 gig drives. Drives 1-7 RAID 5 connected to one server through a Qlogic SAN box 64 and working as expected. Drives 8-14 RAID 5 connected to another server also through the Qlogic switch. I cannot get 8-14 to mount.
I have rebuilt the parity information, restarted the controller, and even physically connected the RAID to the server with no luck.
I can see the Node World Wide Name of the RAID in System Profiler, but I see nothing in Disk Utility. Nothing appears out of the ordinary in RAID Admin.
I know the fiber connection points are working, as I can mount and work with my SAN (on other RAIDs). Firmware is 1.5.1 on all RAID units.
The only thing I have not tried is physically moving the drives to another unit a that requires taking down the SAN.
I do have all the data backed up on tape, but I don't really want to restore everything that way unless I have to.

Well, after several hours on the phone with Apple Enterprise support, the problem has been solved.
The problem was in fact LUN Masking. As we should all be aware LUN Masking was removed from RAID Admin 1.5.1 (it is still there in 1.5.0). 3 years ago when the Apple SE set up our SAN, he used LUN Masking instead of creating zones in the fibre switch. According to other information I received, this was an OK way to do it at the time.
Solving the problem was a little time-intensive, but it now works.
First I had to download RAID Admin 1.5.0 and install it on one of my clients. Using that, I had to remove LUN Masking from all of my RAIDs. This is where it got a little tricky, as I had to power down all my servers so LUNs did not show up on multiple computers.
Once the LUN Masking was removed, I created a new Fabric on my switch (thanks go to aaron on http://www.xsanity.com/article.php?story=20060312090411100&query=zoning). Then, after turning everything with a fibre port off, I updated the fabric zoning.
When I turned everything back on, not only did I have LUNs in the correct location, but my SAN worked, too!

Nexus 7K question regarding N7K-F132XP-15

Dear expert,
I work in Cisco lab and doing performance demo on mobile packet core solutions. We have a Next 7000 10 slot chassis with 3 32 port 10G modules(N7K-M132XP-12L). Since N7K-M132XP-12L limit total bandwidth to 80G, we want to get several new 32 port 1G and 10G F module(N7K-F132XP-15). I have a few questions through:
1, Can N7K-F132XP-15 and N7K-M132XP-12L co-exist in same chassis? My feeling is "yes" but want to confirm.
2, N7K-F132XP-15 can have 230G bandwidth to switch fabric given 5 switch fabric installed, how does 230G distribute to each goup of 8 ports(or 4 ports maybe)? I know for N7K-M132XP-12L, group of 4 ports share 10G, any such limitation for N7K-F132XP-15?
3, N7K-F132XP-15 appears to be less expensive than N7K-M132XP-12L even it has more bandwidth, do I loss something compare to N7K-M132XP-12L?
Thanks a lot
Ning Chen
Technical Marketing Engineer
Cisco

Hi Ning
I can help out on a few, so to answer:
1) Yes, you can have both N7K-F132XP-15 and N7K-M132XP-12L in the same chassis
2) I'm not sure on this one, will have to look into some specs on it. Perhaps someone else knows, feel free to chime in
3) Keep in mind the N7K-F132XP-15 (F1 module) is not very comparable to N7K-M132XP-12L(M1 module). F1 modules are L2 only, so no routing is done on the linecard itself. F1 can do fabricpath where as M1 cannot.
Depending on what you want to do with the module the M1 may be a better choice. If you are looking at pure L2 then F1 is fine, and then you can even take advantage of FabricPath, however if you want to do any L3 you need the M1 modules.
Hope that helps
Chad

UCS Basic Questions

Hi
I am new to Cisco UCS, I have a little idea about Nexus Series Switches.
I would like to know what exactly are the differentiating factors between -
UCS 6100 Fabric Interconnect
Nexus 5000 Series
MDS 9000 Series switches
All these can support FCoE, 1G / 10G Ports, I know 6100 Fabric Interconnect has built in UCS Manager
where I can place these in a real worl solution.
Second -
Can we make multiple Blades in a chassis, or Multiple Blades in Multiple Chassis conected to a Single / Multiple interconnected 6100 Fabric Interconnect / Nexus / MDS switch, to combine as a Single / One Server.
Appreciate your kind response.
Tahnks & Regards
Sunny

Sunny,
I hope the beow information helps clarify present questions and future knowledge you might get in the near future.
All the information is taken form the link that follows:
http://www.cisco.com/en/US/docs/solutions/Enterprise/Data_Center/VMDC/2.0/design_guide/archOver.html
Cisco Nexus 5000
The Cisco Nexus 5000 Series (http://www.cisco.com/en/US/products/ps9670/index.html) switches deliver high performance, standards-based Ethernet and FCoE that enable the consolidation of LAN, SAN, and cluster network environments onto a single Unified Fabric. Each switch contains a single unified crossbar fabric ASIC and multiple unified port controllers to support fixed ports and expansion modules.
Cisco UCS and UCSM
Cisco UCS 6100 Series Fabric Interconnects (http://www.cisco.com/en/US/partner/products/ps10276/index.html) is a family of line-rate, low-latency, lossless, 10-Gbps Ethernet and Fibre Channel over Ethernet interconnect switches.
Cisco UCS 5100 Series Blade Server Chassis (http://www.cisco.com/en/US/partner/products/ps10279/index.html) supports up to eight blade servers and up to two fabric extenders in a six rack unit (RU) enclosure.
Cisco MDS 9513
The Cisco MDS 9513 Multilayer Director allows you to deploy high-performance SANs using a high-performance, protocol-independent switch fabric. It provides uncompromising high availability, security, scalability, ease of management, and transparent integration of new technologies for extremely flexible data center SAN solutions. The Cisco MDS 9513 is compatible with first-, second-, and third-generation Cisco MDS 9000 Family switching modules.
For more information, see: http://www.cisco.com/en/US/products/hw/ps4159/index.html.
Cisco MDS 9134
The Cisco MDS 9134 Multilayer Fabric Switch is a 1RU chassis that provides line-rate 4-Gbps and 10-Gbps ports. It expands from 24 to 32 ports in 8-port increments and optionally activates 2 10-Gbps ports. The Cisco MDS 9134 offers non-blocking architecture, with all 32 4-Gbps ports and the 2 10-Gbps ports operating at line rate concurrently.
The 10-Gbps ports support a range of optics for connection to the Cisco MDS 9000 family core using 10-Gbps Inter-Switch Link (ISL) connectivity. The Cisco MDS 9134 can also be stacked using copper CX4 X2 transceivers to cost effectively offer up to 64-port densities. The Cisco MDS 9134 supports quick configuration and task wizards that allow it to be deployed quickly and easily in networks of any size. Powered by Cisco MDS 9000 NX-OS/SAN-OS Software, it includes advanced storage networking features and functions and is compatible with Cisco MDS 9500 Series Multilayer Directors and Cisco MDS 9200 Series Multilayer Fabric Switches, providing transparent, end-to-end service delivery in core-edge deployments.
For more information, see: http://www.cisco.com/en/US/products/hw/ps4159/index.html.
Cisco MDS 9148
The Cisco MDS 9148 Multilayer Fabric Switch is a one rack unit (1RU) top-of-rack (ToR) chassis that provides 48 line-rate 8-Gbps ports for storage networking deployments. It can expand from 16 to 48 ports in 8-port increments. The Cisco MDS 9148 delivers a non-blocking architecture, with all 48 1/2/4/8-Gbps ports operating at line-rate concurrently.
The Cisco MDS 9148 supports the Cisco Device Manager Quick Configuration Wizard, which allows it to be deployed quickly and easily in networks of any size. Powered by Cisco MDS 9000 NX-OS Software, it includes advanced storage networking features and functions and is compatible with Cisco MDS 9500 Series Multilayer Directors and Cisco MDS 9200 and other 9100 Series Multilayer Fabric Switches, providing transparent, end-to-end service delivery in core-edge deployments.
For more information, see: http://www.cisco.com/en/US/products/hw/ps4159/index.html.
PS: If you find it helpful, please mark it with a 5 stars grade!
-Kenny

Uplink failover scenarios - The correct behavior

/* Style Definitions */
table.MsoNormalTable
{mso-style-name:"Table Normal";
mso-tstyle-rowband-size:0;
mso-tstyle-colband-size:0;
mso-style-noshow:yes;
mso-style-priority:99;
mso-style-qformat:yes;
mso-style-parent:"";
mso-padding-alt:0in 5.4pt 0in 5.4pt;
mso-para-margin:0in;
mso-para-margin-bottom:.0001pt;
mso-pagination:widow-orphan;
font-size:11.0pt;
font-family:"Calibri","sans-serif";
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;}
Hello Dears,
I’m somehow confused about the failover scenarios related to the uplinks and the Fabric Interconnect (FI) switches, as we have a lot of failover points either in the vNIC , FEX , FI or uplinks.
I have some questions and I hope that someone can clear this confusion:
A-     Fabric Interconnect failover
1-      As I understand when I create a vNIC , it can be configured to use FI failover , which means if FI A is down , or the uplink from the FEX to the FI is down , so using the same vNIC it will failover to the other FI via the second FEX ( is that correct , and is that the first stage of the failover ?).
2-      This vNIC will be seen by the OS as 1 NIC and it will not feel or detect anything about the failover done , is that correct ?
3-      Assume that I have 2 vNICs for the same server (metal blade with no ESX or vmware), and I have configured 2 vNICs to work as team (by the OS), does that mean that if primary FI or FEX is down , so using the vNIC1 it will failover to the 2nd FI, and for any reason the 2nd vNIC is down (for example if uplink is down), so it will go to the 2nd vNIC using the teaming ?
B-      FEX failover
1-      As I understand the blade server uses the uplink from the FEX to the FI based on their location in the chassis, so what if this link is down, does that mean FI failover will trigger, or it will be assigned to another uplink ( from the FEX to the FI)
C-      Fabric Interconnect Uplink failover
1-      Using static pin LAN group, the vNIC is associated with an uplink, what is the action if this uplink is down ? will the vNIC:
a.       Brought down , as per the Network Control policy applied , and in this case the OS will go for the second vNIC
b.      FI failover to the second FI , the OS will not detect anything.
c.       The FI A will re-pin the vNIC to another uplink on the same FI with no failover
I found all theses 3 scenarios in a different documents and posts, I did not have the chance it to test it yet, so it will be great if anyone tested it and can explain.
Finally I need to know if the correct scenarios from the above will be applied to the vHBA or it has another methodology.
Thanks in advance for your support.
Moamen

Moamen
A few things about Fabric Failover (FF) to keep in mind before I try to address your questions.
FF is only supported on the M71KR and the M81KR.
FF is only applicable/supported in End Host Mode of Operation and applies only to ethernet traffic. For FC traffic one has to use multipathing software (the way FC failover has worked always). In End Host mode, anything along the path (adapter port, FEX-IOM link, uplinks) fails and FF is initiated for ethernet traffic *by the adapter*.
FF is an event which is triggered by vNIC down i.e a vNIC is triggered down and the adapter initiates the failover i.e it sends a message to the other fabric to activate the backup veth (switchport) and the FI sends our gARPs for the MAC as part of it. As it is adapter driven, this is why FF is only available on a few adapters i.e for now the firmware for which is done by Cisco.
For the M71KR (menlo's) the firmware on the Menlo chip is made by Cisco. The Oplin and FC parts of the card, and Intel/Emulex/Qlogic control that.
The M81KR is made by Cisco exclusively for UCS and hence the firmware on that is done by us.
Now to your questions -
>1-      As I understand when I create a vNIC , it can be configured to use FI failover , which means if FI A is down , or the uplink from the FEX to the >FI is down , so using the same vNIC it will failover to the other FI via the second FEX ( is that correct , and is that the first stage of the failover ?).
Yes
> 2-      This vNIC will be seen by the OS as 1 NIC and it will not feel or detect anything about the failover done , is that correct ?
Yes
>3-      Assume that I have 2 vNICs for the same server (metal blade with no ESX or vmware), and I have configured 2 vNICs to work as team (by the >OS), does that mean that if primary FI or FEX is down , so using the vNIC1 it will failover to the 2nd FI, and for any reason the 2nd vNIC is down (for >example if uplink is down), so it will go to the 2nd vNIC using the teaming ?
Instead of FF vNICs you can use NIC teaming. You bond the two vNICs which created a bond interface and you specify an IP on it.
With NIC teaming you will not have the vNICs (in the Service Profile) as FF. So the FF will not kick in and the vNIC will be down for the teaming software to see on a fabric failure etc for the teaming driver to come into effect.
> B-      FEX failover
> 1-      As I understand the blade server uses the uplink from the FEX to the FI based on their location in the chassis, so what if this link is down, > >does that mean FI failover will trigger, or it will be assigned to another uplink ( from the FEX to the FI)
Yes, we use static pinning between the adapters and the IOM uplinks which depends on the number of links.
For example, if you have 2 links between IOM-FI.
Link 1 - Blades 1,3,5,7
Link 2 - Blades 2,4,6,8
If Link 1 fails, Blade 1,3,5,7 move to the other IOM.
i.e it will not failover to the other links on the same IOM-FI i.e it is no a port-channel.
The vNIC down event will be triggered. If FF is initiated depends on the setting (above explanation).
> C-      Fabric Interconnect Uplink failover
> 1-      Using static pin LAN group, the vNIC is associated with an uplink, what is the action if this uplink is down ? will the vNIC:
> a.       Brought down , as per the Network Control policy applied , and in this case the OS will go for the second vNIC
If you are using static pin group, Yes.
If you are not using static pin groups, the same FI will map it to another available uplink.
Why? Because by defining static pinning you are purposely defining the uplink/subscription ratio etc and you don't want that vNIC to go to any other uplink. Both fabrics are active at any given time.
> b.      FI failover to the second FI , the OS will not detect anything.
Yes.
> c.       The FI A will re-pin the vNIC to another uplink on the same FI with no failover
For dynamic pinning yes. For static pinning NO as above.
>I found all theses 3 scenarios in a different documents and posts, I did not have the chance it to test it yet, so it will be great if anyone tested it and >can explain.
I would still highly recommend testing it. Maybe its me but I don't believe anything till I have tried it.
> Finally I need to know if the correct scenarios from the above will be applied to the vHBA or it has another methodology.
Multipathing driver as I mentioned before.
FF *only* applies to ethernet.
Thanks
--Manish

How to configure StorEdge 3510 as a SAN?

Hi all, just joined!
We've got a StorEdge 3510 set up as a DAS (private loop mode) connected to three servers. We've gone through hell and high water getting things configured (firmware upgrades were particularly troublesome), but now that we've got everything running, we're starting to think about future expansion. However, despite my experimentation, I can't work out how to set them up in a switched fabric.
We have a test 3510 set up as Point-to-Point, connected to an Emulex 355 FC switch. In Loop mode, the device shows up as a SCSI device, but I can't get Windows to see anything on the end of the HBA when it's in PTP. Half the time, the HBA won't even acknowledge a link.
I can't work out where to start with this. The switch, admittedly, came from Apple with an Xserve RAID, but we've had the 3510 plugged into it in Loop mode and visible, so I don't think there's any problem with Apple ROM-ing the switch.
I've tried Zoning the switch, and Smart-Setting the ports to be Switched Fabrics, but nothing seems to make the StorEdge visible to the host. I thought that the test box might be at fault initially, since it has a Dell-EMC FC HBA and it wouldn't surprise me if it's ROM'd to EMC boxes only, but it seems that another box with a generic QLogic card won't see the StorEdge either.
Full tech specs of the system:
-3510 dual-controller, firmware 4.23A, 12 146GB 10K disks
-Emulex 355 12-port FC SAN Switch
-Dell PowerEdge 1850 with Emulex LP1050Ex-E PCI-E HBA
-Dell PowerEdge 2850 with QLogic QLA210 PCI HBA
-Windows Server 2003 x64
Any advice or starting points would be massively appreciated!
Thanks,
Rob

You are right in changing the A3510 to point to point
From:
sccli> sho host
max-luns-per-id: 32
queue-depth: 1024
fibre-connection-mode: loop
inband-management: enabled
To:
sccli> sho host
max-luns-per-id: 32
queue-depth: 1024
fibre-connection-mode: point-to-point
inband-management: enabled
The 3510 is now configured to work in a SAN.
In loop mode you will see the enclosure as an ses device. In point to point you won't, you will just see the available LUNs.
In P2P you should see all your configured logical volumes or disks with there WWN names on the hosts side
From:
sccli> show logical
LD LD-ID Size Assigned Type Disks Spare Failed Status
ld0 5E714957 545.91GB Primary RAID5 5 2 0 Good
Write-Policy Default StripeSize 128KB
ld1 1F4ABC1D 545.91GB Secondary RAID5 5 2 0 Good
Write-Policy Default StripeSize 128KB
with
Serial Number: 07FE18
You should see something like (these names are taken from a Solaris luxadm probe)
Node WWN:206000c0ff07fe18 Device Type:Disk device
Logical Path:/dev/rdsk/c7t600C0FF00000000007FE181F4ABC1D00d0s2
Node WWN:206000c0ff07fe18 Device Type:Disk device
Logical Path:/dev/rdsk/c7t600C0FF00000000007FE185E71495700d0s2
I don't have any clue how this would show up under windows. Putting a fabric on your switch should make things more tidy, but shouldn't fundamentally change anything.

2 fabrics on a switch ?

Similar Messages

Maybe you are looking for