FI link to IOM

My FI (6248s) have 4 link-connections to IOM (2208XP) and the chassis discovery has been done. The above links are configured as server ports. So, do we leave them as individual server ports or do we use portchannel on the FI and IOM? In other words, what would be the best for redundancy if one of the links failed? I can’t seem find anything about the above config in Cisco docs, or may be, I’m looking at wrong place.
Thanks!

Configuring fabric port channel ( between FI and IOM ) provides you similar benefits of a regular port channel over individual links.
http://www.cisco.com/en/US/docs/unified_computing/ucs/sw/gui/config/guide/2.1/b_UCSM_GUI_Configuration_Guide_2_1_chapter_0110.html#concept_5D3D88341BFB43468B62B5A77876C67B
http://www.cisco.com/en/US/docs/unified_computing/ucs/sw/gui/config/guide/2.1/b_UCSM_GUI_Configuration_Guide_2_1_chapter_01100.html
Padma

Similar Messages

- No link between IOM port 3/2/2 and fabric interconnect B:1/18

Specifically I'm trying to find out what the 3/2/2 means in - No link between IOM port 3/2/2 and fabric interconnect B:1/18

This from the fault reference guide:
"No link between IOM port [chassisId]/[slotId]/[portId] and fabric interconnect [switchId]:[peerSlotId]/[peerPortId]"
http://www.cisco.com/en/US/docs/unified_computing/ucs/ts/faults/reference/UCS_SEMs.html#wp1306388

No link between IOM port 2/2/3 and fabric interconnect B

Hello
I am trying to remove an uplink from an IOM to the fabric interconnect as its not required. I have tried to reset the I/O module and re-acknowledge the chassis with no luck, it is still shown in the GUI with the error message as in the subject. We are currently running 2.0(1s). The kit isn't in production yet.
Thanks for any help

Yes i agress thats a pretty heavy approach to clean it. No if you left it there as it is, there would have been no operational issues. It would have just left this fault in the UCSM.
./Abhinav

Can't add a new vNIC for service profile

Hello,
I'm trying to add an additional vNIC for Service Profile.
B200-M3, VIC1240+expander, IOM 2208XP, 4 links between IOM and FI 6248, firmware 2.2(1f).
At the moment 7 vNICs are configured for this service profile. Service profile is associated with blade server.
When I try to add a new vNIC (it will be 8th vNIC), I see the error (see the attachment).
For some reasons separate vNICs are more useful for Hyper-V servers :)
All vNICs created via vNIC templates. All vNICs configured as VMQ, only iSCSI configured as Dynamic,
As I read the spec, VIC1240 can support up to 250 vNICs. What is wrong ? How to fix that ?

I have seen the VMQ limit being reached when applying the VMQ policy to the vNIC template. Can you try not selecting the VMQ policy for these new vNICs. If this works, you might be reaching the VMQ limits as described below:
Number of VMQ's:
128 is the Maximum number per vNIC
256 is the max number per Blade

Full width blade-Q

Hi, Cisco Folks:
Can someone please help me to clarify the following doubts?
Q1. What is the pinning order of B250-M1 to IOM if I have 2 Server Ports with a fully populated chassis of 4 B250-M1?
I know that for a half-width, the pinning order will look like:
                                                  Pinned Blade #
Server Link 1 or Port 1 of IOM       Blade 1,3,5,    7
Server Link 2 or Port 2 of IOM       Blade 2,4,6, & 8
OR
                Pinned Blade #
Server Link 1 or Port 1 of IOM      Blade 1,3,5,    7
Server Link 2 or Port 2 of IOM      Blade 2,4,6, & 8
Q1.1
If I have the Server Link 1 connects to Port 1 of IOM, but Server Link 2 connects to Port 3 of IOM, will this change the pinning order?
Q1.2
If I have the Server Link 1 connects to Port 1 of IOM, but Server Link 2 connects to Port 4 of IOM, will this change the pinning order?
How about other combinations Like Port 2 of IOM (Sever Link 1) and Port 3 of IOM (Server Link 2)?
Q2. If I only install a Single M81KR in B250-M1, do I have to care about which Adapter Slot to install my PALO Card? My concern here is really Slot 1 is tied to IOM A and Slot B is tied to IOM B, is there such concept at all??
If there is such concept, then I would make sure that for a 4 blades setup, I would have 2 blades with PALO inserted into Slot 1 and the other 2 blades with PALO inserted into Slot 2,
Thanks.
SiM

./Abhinav
Let me try out once I have the opportunity and update you.
I am beginning to understnad more now esp. with your deep technical expertise. I appreciate your input.
1 more question lingering on my mind:
If I have only 1 PALO installed in one of the 2 Mezzanine Slots of a Full-Width Blade, AM I really using only 1 Server Link at all assuming I have 2 Server Links i.e. I made use of 2 FEX Ports?
Example: B250-M2
2 Server Links Per IOM/FEX to Each FI
My full Width blade is in Slot 1 & 2 or the TOP of the Chassis.
For half-width, I know it is very clear as in:
2-Server Link     Blade (Assuming I have 8 blades)
Server Link 1     1, 3, 5, 7 - Blade 1,3,5 & 7 will make use of this 10G link only
Server Link 2     2, 4, 6, 8
4-Server Link     Blade (Assuming I have 8 blades)
Server Link 1     1, 5 - Blade 1 & 5 will make use of this 10G link only
Server Link 2     2, 6,
Server Link 3     3, 7
Server Link 4     4, 8
Please help me to explain further as I know Cisco blades, but there is a lot I need to learn everyday.
Appreciate it.
KP

Uplink failover scenarios - The correct behavior

/* Style Definitions */
table.MsoNormalTable
{mso-style-name:"Table Normal";
mso-tstyle-rowband-size:0;
mso-tstyle-colband-size:0;
mso-style-noshow:yes;
mso-style-priority:99;
mso-style-qformat:yes;
mso-style-parent:"";
mso-padding-alt:0in 5.4pt 0in 5.4pt;
mso-para-margin:0in;
mso-para-margin-bottom:.0001pt;
mso-pagination:widow-orphan;
font-size:11.0pt;
font-family:"Calibri","sans-serif";
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;}
Hello Dears,
I’m somehow confused about the failover scenarios related to the uplinks and the Fabric Interconnect (FI) switches, as we have a lot of failover points either in the vNIC , FEX , FI or uplinks.
I have some questions and I hope that someone can clear this confusion:
A-     Fabric Interconnect failover
1-      As I understand when I create a vNIC , it can be configured to use FI failover , which means if FI A is down , or the uplink from the FEX to the FI is down , so using the same vNIC it will failover to the other FI via the second FEX ( is that correct , and is that the first stage of the failover ?).
2-      This vNIC will be seen by the OS as 1 NIC and it will not feel or detect anything about the failover done , is that correct ?
3-      Assume that I have 2 vNICs for the same server (metal blade with no ESX or vmware), and I have configured 2 vNICs to work as team (by the OS), does that mean that if primary FI or FEX is down , so using the vNIC1 it will failover to the 2nd FI, and for any reason the 2nd vNIC is down (for example if uplink is down), so it will go to the 2nd vNIC using the teaming ?
B-      FEX failover
1-      As I understand the blade server uses the uplink from the FEX to the FI based on their location in the chassis, so what if this link is down, does that mean FI failover will trigger, or it will be assigned to another uplink ( from the FEX to the FI)
C-      Fabric Interconnect Uplink failover
1-      Using static pin LAN group, the vNIC is associated with an uplink, what is the action if this uplink is down ? will the vNIC:
a.       Brought down , as per the Network Control policy applied , and in this case the OS will go for the second vNIC
b.      FI failover to the second FI , the OS will not detect anything.
c.       The FI A will re-pin the vNIC to another uplink on the same FI with no failover
I found all theses 3 scenarios in a different documents and posts, I did not have the chance it to test it yet, so it will be great if anyone tested it and can explain.
Finally I need to know if the correct scenarios from the above will be applied to the vHBA or it has another methodology.
Thanks in advance for your support.
Moamen

Moamen
A few things about Fabric Failover (FF) to keep in mind before I try to address your questions.
FF is only supported on the M71KR and the M81KR.
FF is only applicable/supported in End Host Mode of Operation and applies only to ethernet traffic. For FC traffic one has to use multipathing software (the way FC failover has worked always). In End Host mode, anything along the path (adapter port, FEX-IOM link, uplinks) fails and FF is initiated for ethernet traffic *by the adapter*.
FF is an event which is triggered by vNIC down i.e a vNIC is triggered down and the adapter initiates the failover i.e it sends a message to the other fabric to activate the backup veth (switchport) and the FI sends our gARPs for the MAC as part of it. As it is adapter driven, this is why FF is only available on a few adapters i.e for now the firmware for which is done by Cisco.
For the M71KR (menlo's) the firmware on the Menlo chip is made by Cisco. The Oplin and FC parts of the card, and Intel/Emulex/Qlogic control that.
The M81KR is made by Cisco exclusively for UCS and hence the firmware on that is done by us.
Now to your questions -
>1-      As I understand when I create a vNIC , it can be configured to use FI failover , which means if FI A is down , or the uplink from the FEX to the >FI is down , so using the same vNIC it will failover to the other FI via the second FEX ( is that correct , and is that the first stage of the failover ?).
Yes
> 2-      This vNIC will be seen by the OS as 1 NIC and it will not feel or detect anything about the failover done , is that correct ?
Yes
>3-      Assume that I have 2 vNICs for the same server (metal blade with no ESX or vmware), and I have configured 2 vNICs to work as team (by the >OS), does that mean that if primary FI or FEX is down , so using the vNIC1 it will failover to the 2nd FI, and for any reason the 2nd vNIC is down (for >example if uplink is down), so it will go to the 2nd vNIC using the teaming ?
Instead of FF vNICs you can use NIC teaming. You bond the two vNICs which created a bond interface and you specify an IP on it.
With NIC teaming you will not have the vNICs (in the Service Profile) as FF. So the FF will not kick in and the vNIC will be down for the teaming software to see on a fabric failure etc for the teaming driver to come into effect.
> B-      FEX failover
> 1-      As I understand the blade server uses the uplink from the FEX to the FI based on their location in the chassis, so what if this link is down, > >does that mean FI failover will trigger, or it will be assigned to another uplink ( from the FEX to the FI)
Yes, we use static pinning between the adapters and the IOM uplinks which depends on the number of links.
For example, if you have 2 links between IOM-FI.
Link 1 - Blades 1,3,5,7
Link 2 - Blades 2,4,6,8
If Link 1 fails, Blade 1,3,5,7 move to the other IOM.
i.e it will not failover to the other links on the same IOM-FI i.e it is no a port-channel.
The vNIC down event will be triggered. If FF is initiated depends on the setting (above explanation).
> C-      Fabric Interconnect Uplink failover
> 1-      Using static pin LAN group, the vNIC is associated with an uplink, what is the action if this uplink is down ? will the vNIC:
> a.       Brought down , as per the Network Control policy applied , and in this case the OS will go for the second vNIC
If you are using static pin group, Yes.
If you are not using static pin groups, the same FI will map it to another available uplink.
Why? Because by defining static pinning you are purposely defining the uplink/subscription ratio etc and you don't want that vNIC to go to any other uplink. Both fabrics are active at any given time.
> b.      FI failover to the second FI , the OS will not detect anything.
Yes.
> c.       The FI A will re-pin the vNIC to another uplink on the same FI with no failover
For dynamic pinning yes. For static pinning NO as above.
>I found all theses 3 scenarios in a different documents and posts, I did not have the chance it to test it yet, so it will be great if anyone tested it and >can explain.
I would still highly recommend testing it. Maybe its me but I don't believe anything till I have tried it.
> Finally I need to know if the correct scenarios from the above will be applied to the vHBA or it has another methodology.
Multipathing driver as I mentioned before.
FF *only* applies to ethernet.
Thanks
--Manish

Fabric Port Channel

Hello,
Once in production, when adding cables in a Fabric PortChannel between an IOM and a FI, ¿is it needed to re-ack the chassis? (so that, we'll have downtime of the whole chassis). Or rather, as it is a Port-Channel the cables are detected automatically, so that it is not needed to re-ack, and so there's no downtime at all...which one is the correct statement?
Thanks,

Hello,
Let us consider following scenario on adding or removing links to existing deployment.
Initial deployment
UCSM 2.0, FI 62XX ,2208 IOM and 2 physical links with port channel configuration ( 2 link discovery policy )
Change
1) If additional links are added and if they need to be utilized and part of port channel, we need to re-ack the chassis.
2) If a link goes down,traffic is distributed across other links in port channel and does not require re-ack. Parameters like bandwidth, VIF ports that are calculated based on initial configuration remains the same.
3) If we physically remove the additional links between IOM / FI, re-acknowledment would create port-channel with original two links.Failure to re-ack will create over subscription ( bandwidth, available VIF ports ) of the network links.Re-ack is not mandatory but it creates appropriate configuration that matches required design criteria.
Caution
Adding or removing links from a fabric port channel is disruptive and may affect the available amount of VIF namespace.
http://www.cisco.com/en/US/docs/unified_computing/ucs/sw/gui/config/guide/2.0/b_UCSM_GUI_Configuration_Guide_2_0_chapter_0101.html#concept_5D3D88341BFB43468B62B5A77876C67B
Padma

Configuring additonal server ports

Hi, if i'm using 2 server ports per chassis, and wish to add another 2, besides plugging in the cables and settings the ports on the interconnect for "Configure as Server Port" is there anything else that is needed and is there any disruption to traffic ?

Hello David,
Re-acknowledging the chassis does not reboot the blades. The links between IOM and FI are flapped and are included to carry traffic between blades and FI.
---->>> What about changing from individual links to port-channel, will this require a re-acknowledgement as well ?
{Padma] Depends upon where we make the change.
If you change it under Equipment > Policies > , you need to manually re-acknowledge the chassis for the links to form a port-channel.
If the global policy is set to None for " Link Grouping Preference " and we are changing the policy for specific chassis ( Equipment > Chassis X > Connectivity Policy " to port-channel, then this configuration change automatically re-acknowledges the chassis. You will get a warning before saving the changes.
Fabric port channel is available when you have FI 62XX / IOM 220X / UCSM 2.0 hardware / software combination.
--->>> Using individual links what method does the FI use for sending traffic down 1 link as opposed to the other ?
[Padma] When using individual links between IOM and FI, traffic is distributed according to pre-defined configuration based on number of physical links
Supported physical links are 1,2 4 and 8.
Let me know how your hardware model and current number of links used between IOM and FI.
Padma

IOM connecting FI error - Fabric unsupported connection

Hi Guys,
Not sure if anyone face the same scenario with me.
I have a B series Chassis with two IOM card ( model : 2208XP) which is 8 external port / 10Gb.
while the bundle come with 8 10Gbase-CU cables, I connect the first 4 port on IOM-1 to Fi-A and first 4 port on IOM-2 to FI-B
IOM1      FI-A
1               Port 1
2               Port 2
3               Port 3
4               Port 4
IOM2      FI-B
1               Port 1
2               Port 2
3               Port 3
4               Port 4
However base on my observation the server port role works fine on IOM1 - FI-A
on IOM 2 we can only see fabric 2/1 to 2/3 ( 3 link) while the Led light on 2/4 is up but not shown in UCS manager.
There is also a major error message :
iom1/2 current connectivity does not match discovery policy or connectivity is unsupported
I also tried to access NXOS in the FI and notice that the port is up, show port-channel summary tell me 4 port channel are up.
Please refer to the attachment for screenshot.

Hi Manuel,
sorry for the late response. I tried to re-ack the chassis multiple time but not help.
Not sure why the fourth port is not able to configured into fex mode, although it is serverport in UCSM
UCS-FABRIC-A-B(nxos)# show fex
FEX         FEX           FEX                       FEX
Number    Description      State            Model            Serial
1        FEX0001                Online           N20-C6508   FCH17327EPD
UCS-FABRIC-A-B(nxos)#
UCS-FABRIC-A-B(nxos)# show int fex
     Fabric      Fabric       Fex                FEX
Fex Port      Port State    Uplink    Model         Serial
1   Eth1/23        Active     1         N20-C6508 FCH17327EPD
1   Eth1/24        Active     2         N20-C6508 FCH17327EPD
1   Eth1/25        Active     3         N20-C6508 FCH17327EPD
UCS-FABRIC-A-B(nxos)#
UCS-FABRIC-A-B(nxos)# show run int eth 1/23 - 26
!Command: show running-config interface Ethernet1/23-26
!Time: Wed Jan 8 03:27:19 2014
version 5.0(3)N2(2.11.3a)
interface Ethernet1/23
description S: Server
no pinning server sticky
switchport mode fex-fabric
fex associate 1 chassis-serial FOX1734GBBQ module-serial FCH17327EPD module-slot right
no shutdown
interface Ethernet1/24
description S: Server
no pinning server sticky
switchport mode fex-fabric
fex associate 1 chassis-serial FOX1734GBBQ module-serial FCH17327EPD module-slot right
no shutdown
interface Ethernet1/25
description S: Server
no pinning server sticky
switchport mode fex-fabric
fex associate 1 chassis-serial FOX1734GBBQ module-serial FCH17327EPD module-slot right
no shutdown
interface Ethernet1/26
description S: Server
no pinning server sticky
switchport mode trunk
no shutdown
UCS-FABRIC-A-B(nxos)#

UCS B series IOM firmware upgarde.

Dear Experts;
We have one BUG in our UCS B series server as below link.
https://tools.cisco.com/bugsearch/bug/CSCue49366/?reffering_site=dumpcr
The work around is to do the IOM firmware upgarde.
As teher are several Firmware are associated with the UCS system how to knwo the dependency and how to know which firmware is compatible with the uCS server.
Please assist me to get teh proper IOM firmware .I opened one TAC and they are telling TAC doesn't support this we have to contact cisco AS team and we don't have cisco AS team contract.
Regards
Debashis

Dear Walter;
Really appreciated your expert technical advice.
I have one small question as below if you can answer it..
As per the existing setup..
This UCS system is connected with the EMC storage Model:VMAX-1SE Version:5876.
We have no idea if UCS firmware version 2.1(3c) is compatible with the MC storage Model:VMAX-1SE Version:5876..
I have opened the TAC and also went through the Compatibility matrix where it mentioned as attached and to conatct EMC vendor.
We opened case with EMC they are telling contact Cisco and Cisco TAC telling contact EMC.
IS there any way we can know this?
Note as per the attachment:
5. Direct Connect FC/FCoE topologies are supported with this array.
6. Non-default adapter settings are typically required by storage vendors and found on their support list.
7.Multiple array and switch models supported with UCS by the storage vendor are on their support list.
12.iSCSI Boot is supported Both Appliance and Uplink ports are supported unless otherwise noted.
26. Supported for NFS/CIFS access via appliance ports on the Fabric Interconnect

UCS Chassis 5108 both IOMs inaccessible

Dear forum users,
Problem description: Sometime our UCS Chassis 5108 and both IOM are suddenly INACCESSIBLE. This happens frequently (Happend 5 times in just a month)
Temporary Solution: To unplug and re-plug Chassis power cords twice, then the Chassis back to normal
Once the Chassis is inaccessible state, below are what we witness on the Chassis:
- All the blades LEDs on chassis-1 shows amber lit
- 8 ports on IOM A and IOM B shows NO LED lit (Not even amber lit)
- Sudden increased in Fan speed on chassis-1
- FI ports which connected to Chassis-1, shows NO LED it
- Please find attached (Chassis-1_Fault.jpg) Fault message on chassis-1
We had tried the following:
- Tried re-sit the uplink cable
- Tried replacing the uplink cable, the port is still down
- Reboot FI
- Decommission the chassis-1
- Recommission the chassis-1
Below is our setup environment:
- 2x UCS 6248UP
- 2x UCS 5108
- 4x UCS 2208XP (2 IOM per chassis)
- 2x UCS B200 M2
- 5x UCS B250 M2
- Firmware 2.0(3b)
- Each IOM has 4 Uplink cables connect to FI
- Chassis Discovery Policy
1. Action (4 Links)
2. Link Grouping Preference (Port Channel)
We hope, someone should have experience with the same problem or have some advise to be shared.
Please let me know should you need any further details of the problem or other information.
Look forward to a positive reply.

Hello Mohamed,
Please open a TAC service request with UCSM and Chassis tech support log files.
Also, include approximate date & time of the event.
Padma

Network link LED on UCS blades

Hi,
we got a working UCS system (6248 FI , 5108 chassis with 2204 IOM and B200-M3 blades) all is working fine but the network link LED are off. Is this a normal behaviour ? Since according to documentation they should blink green.
regards

Hi Lukasz,
I'm assuming we are speaking about the network LED 's on the IOM which are off.
What about the LED's on the respective ports on the FI?
What's the color of the LED's on the power supplies?
Thanks,
Majid

Cisco UCS - IOM to 6248 (FC)

Hi All,
Just a quick one - is anyone using FC to connect the 6248s to their IOMs ? I've reached the dreaded limit of the twinax cables.
I can see it's supported but just want to check the configuration changes needed as I'm in a full production environment.
Is it as simple as plugging in the SFPs and configuring a server port or do I need to change the UP to FC only which would require a re-boot of the 6248.
Cheers
Steve.

Steve,
We just need to configure the port type as " Server " and use the optical / FET SFPs to connect to Fibre channel link.
http://www.cisco.com/en/US/docs/unified_computing/ucs/hw/chassis/install/install.html#wp1275791
http://www.cisco.com/en/US/prod/collateral/ps10265/ps11544/data_sheet_c78-675245.html
HTH
Padma

Download error in (osx) adobe desktop app (corrupted download link).

Here is a discription of the problem. Please consider that some of the wording might not be correct, as I do have to translate the error message from German into English.
Using OSX 10.9.2, when clicking inside the adobe desktop app (top of the screen bar) on the tap "apps", the following screen (screenshot) appears, which states, translated from top to bottom:
download error
download error. Please contact support.
(link) contact support
(link/button) download creative cloud -> This button unfortunatly leads to the following error page "http://www.adobe.com/special/errorpages/404.html"
All apps, like Bridge, Photoshop, Lightroom, etc. are installed and work just fine. So no problem here. I seem however unable to redownload the desktop app (in order to reinstall). As stated above the provided link inside the desktop app itself is coruppted and within the (online) web-based download centre (user logged in) I am only adviced to use the desktop app. This is a dead end and I do not know what to make of this error, let alone solving it. Please help!

I am sorry Romsinha but this doesn't really help.
I already restarted the desktop app and while I am obviously online and connected the problem (error message) remains the same. Information within the "home" tap is recieved/loaded (little blue spinning wheel) stating that various apps recently have been updated. Yet the same loading wheel within the "apps" tap results in an error. My best guess is that some internal link within the app is corrupted, leading to a source on a server that can not be reached.
UPDATE
I clean uninstalled adobe creative cloud as discribed in the article you provided (using the cleaner tool) and even uninstalled the browser plugin. After downloading and reinstalling creative cloud the problem however remains the same. "Apps" tap still shows the same problem. "Home" tap now displays the following:

Barra de Navegação cinza e preta encobrindo o link, como resolver?

Instalei o Kubuntu 14.04 e, após a atualização do sistema, na barra de navegação do Firefox os links ficam encobertos pelas cores cina e preto, impedindo a leitura e atrapalhando a navegação.
Mudei o esquema de cores padrão do sistema, e alterei o tema Gtk mas não resolveu.
Desinstalei e reinstalei o Firefox mas o problema persiste.
O que pode ser feito?

Hmmm,
# Na [[Location bar autocomplete|barra de endereços]], escreva '''about:config''' e pressione '''Enter'''. É possível que o aviso ''''Isso pode anular a garantia!'''' apareça.
# Clique no botão '''Serei cuidadoso, prometo!''', para acessar a página about:config.
# Procure por '''gfx.xrender.enabled''' e mude seu valor para '''false'''.
# Reinicie o Firefox.
Ou tente user o Tema [https://addons.mozilla.org/pt-br/firefox/addon/fxchrome/ FXChrome]

FI link to IOM

Similar Messages

Maybe you are looking for