Fabric Interconnect uplink failure

Hi, FI-A has 10GE uplink to LAN switch-A. Let's call this UplinkA FI-B has 10GE uplink to LAN switch-B. Let's call this UplinkB If UplinkA fails, but the server ports between FI-A and IOM-A is still up. Will traffic be sent to FI-A and then black hole there? I tried this in our setup and it seems to be the case. Then I try to apply Network control policy to the vnic.. In the policy, i specify {"CDP=disabled", Action on uplink fail=link down"} But it still did not work, when UplinkA is down, my ESX still see it's VMNIC as up and continue to send packet to FI-A and blackhole there. (1) Is Network Control policy required to detect FI uplink failure to work properly? (2) How do i see the operation status of the Palo vnic, i.e, whether it is up or down? Can't seem to find a place to see the status. Did i miss out anything? THanks Eng Wee

Hi Ferreira, Thanks for the reply. The uplink LANSwitch is actually a VSS. If i connect the two FI together using some 10GE ports, a layer 2 loop will be formed unless i can etherchannel (Multichassis Etherchannel) the uplink of FI-A and FI-B when connecting to the VSS. I think MEC is not supported in the FI. Alternate solution would be to have each FI to dual home to the LANSwitch. Each FI then creates a port-channel to the LANSwitch. In this way, there will be no L2 loop and no single point of failure. Unfortunately, my client only purchase 1x10GE uplink on each FI. How does end-host mode solve this problem? WHen UplinkA is down, vnic will go down? Rgds Eng Wee

Similar Messages

Fabric Interconnect Uplink port channeling...

Hi,
We setup port channel of two 10G uplink interfaces on each Fabric Interconnect. Everything is running as expected.
However we noticed that under 'Port-Channel # (Fabric #)' setting on LAN tap, 'Admin Speed' was set to '10Gbps'.
Should we leave this configuration as is or match to the maximum bandwidth of port Channel we set up (20Gbps)?
I only see four options on each port channel configuration, 1 Gbps, 10Gbps, 20Gbps and 40Gbps so I'm not sure if we should change this setting.
Thank you

Hello,
You do not need to change the speed ( admin ). 20 Gbps configuration is applicable for certain server models.
-------- NXOS output -----------------
FI-A(nxos)# sh run int port-channel 1
interface port-channel1
description U: Uplink
switchport mode trunk
speed 10000
FI-A(nxos)# show int port-channel 1
port-channel1 is up
Hardware: Port-Channel, address: 547f.ee67.e0fb (bia 547f.ee67.e0fb)
Description: U: Uplink
MTU 1500 bytes, BW 20000000 Kbit, DLY 10 usec,
     reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA
Port mode is trunk
full-duplex, 10 Gb/s
FI-A(nxos)# sh int bri
Port-channel VLAN Type Mode   Status Reason                    Speed Protocol
Interface
Po1          1     eth trunk up      none                       a-10G(D) lacp
There is cosmetic defect where we list the operational speed as 10 Gbps instead of 20Gbps
FI-A /eth-uplink/fabric # show port-channel detail
Port Channel:
    Port Channel Id: 1
    Name: FIA
    Admin State: Enabled
    Oper State: Up
   Speed: 10 Gbps
    Oper Speed: 10 Gbps
    State Reason:
    flow control policy: default
HTH
Padma

Fabric Interconnect Uplinks to Nexus 7710

Can someone please help suggest how many Fabric Interconnect switches I can uplink to pair of Nexus 7710's.

in most deployment, Fabric Interconnect is put as a pair for the redundancy. They are either vPCed to the Nexus switches or straight-through up to non-Nexus switches.
i guess we would like to see more background of your questions in term of traffic requirement, what kind of line cards used on the N7710 etc.
Regards,
Michael

How to recover from a software failure on the 6120 Fabric Interconnect

Hi all,
Had a software failure while upgrading the fabric interconnects. Does anyone know the process to recover from a software failure.
Ive copied up the kickstart and system images, from the loader prompt ive booted the kickstart image and then loaded the system image but the fabric interconnect still reboots to the "switch(boot)" prompt.
Thanks in Advance
Darren

Hi Robert,
Thanks for the tips, but unfortunately im still stuck on the switch(boot): prompt.
The bootflash: only has 2 files in it, the kickstart and system image.
Each time i get the FI to the boot prompt, i can ping the device, but not access via the GUI. (i dont have a local copy of UCSM installed)
What files should i have in the bootflash: for the FI to correctly load.
below is a log of the FI as it loads the kickstart image from the Loader prompt.
loader> boot bootflash:ucs-6100-k9-kickstart.4.1.3.N2.1.3c.bin
Booting kickstart image: bootflash:ucs-6100-k9-kickstart.4.1.3.N2.1.3c.bin....
....................................................................Image verif
ication OK
Starting kernel...
Usage: init 0123456SsQqAaBbCcUu
INIT: version 2.85 booting
Starting Nexus5010 POST...
Executing Mod 1 1 SEEPROM Test......done
Executing Mod 1 1 GigE Port Test.......done
Executing Mod 1 1 Inband GigE Test.....done
Executing Mod 1 1 NVRAM Test....done
Executing Mod 1 1 PCIE Test..............................done
Mod 1 1 Post Completed Successfully
Mod 2 Post Completed Successfully
POST is completed
Creating /callhome..
Mounting /callhome..
Creating /callhome done.
Callhome spool file system init done.
Checking all filesystems..... done.
Warning: switch is starting up with default configuration
Checking NVRAM block device ... done
The startup-config won't be used until the next reboot.
Loading system software
INIT: Sending processes the KILL signalkernel daemon...don
Cisco Nexus Operating System (NX-OS) Software
TAC support: http://www.cisco.com/tac
Copyright (c) 2002-2010, Cisco Systems, Inc. All rights reserved.
The copyrights to certain works contained in this software are
owned by other third parties and used and distributed under
license. Certain components of this software are licensed under
the GNU General Public License (GPL) version 2.0 or the GNU
Lesser General Public License (LGPL) Version 2.1. A copy of each
such license is available at
http://www.opensource.org/licenses/gpl-2.0.php and
http://www.opensource.org/licenses/lgpl-2.1.php
switch(boot)#
From here i can the load the system image, but it always fails and drops me back to the same prompt, output show below
Uncompressing system image: bootflash:/ucs-6100-k9-system.4.1.3.N2.1.3c.bin
Loading plugin 0: core_plugin...
plugin_link_to_exec_path: plugin_path = /isan/plugin/0, tar_log = /isan/plugin_extract_log/0
Loading plugin 1: eth_plugin...
plugin_link_to_exec_path: plugin_path = /isan/plugin/1, tar_log = /isan/plugin_extract_log/1
plugin_file_is_excluded_from_exec_path: /boot/ is excluded from linking
plugin_file_is_excluded_from_exec_path: /boot/lib/ is excluded from linking
plugin_file_is_excluded_from_exec_path: /boot/lib/libplugin_sysreg.so.0.0.0 is excluded from linking
plugin_file_is_excluded_from_exec_path: /boot/lib/libplugin_sysreg.so.0 is excluded from linking
plugin_file_is_excluded_from_exec_path: /boot/lib/libplugin_sysreg.so is excluded from linking
plugin_file_is_excluded_from_exec_path: /boot/etc/ is excluded from linking
plugin_file_is_excluded_from_exec_path: /boot/etc/plugin_exclude.conf is excluded from linking
plugin_file_is_excluded_from_exec_path: /boot/ is excluded from linking
plugin_file_is_excluded_from_exec_path: /boot/lib/ is excluded from linking
plugin_file_is_excluded_from_exec_path: /boot/lib/libplugin_sysreg.so.0.0.0 is excluded from linking
plugin_file_is_excluded_from_exec_path: /boot/lib/libplugin_sysreg.so.0 is excluded from linking
plugin_file_is_excluded_from_exec_path: /boot/lib/libplugin_sysreg.so is excluded from linking
plugin_file_is_excluded_from_exec_path: /boot/etc/ is excluded from linking
plugin_file_is_excluded_from_exec_path: /boot/etc/plugin_exclude.conf is excluded from linking
Loading plugin 2: fc_plugin...
plugin_link_to_exec_path: plugin_path = /isan/plugin/2, tar_log = /isan/plugin_extract_log/2
plugin_file_is_excluded_from_exec_path: /boot/ is excluded from linking
INIT: Sending processes the TERM signal /boot/lib/ is excluded from lin
INIT: (boot)# l
INIT: Sending processes the KILL signaldaemon...don
Cisco Nexus Operating System (NX-OS) Software
TAC support: http://www.cisco.com/tac
Copyright (c) 2002-2010, Cisco Systems, Inc. All rights reserved.
The copyrights to certain works contained in this software are
owned by other third parties and used and distributed under
license. Certain components of this software are licensed under
the GNU General Public License (GPL) version 2.0 or the GNU
Lesser General Public License (LGPL) Version 2.1. A copy of each
such license is available at
http://www.opensource.org/licenses/gpl-2.0.php and
http://www.opensource.org/licenses/lgpl-2.1.php
switch(boot)#
please let me know if you require any further information.
Thanks in Advance
Darren

EHM : 1 port uplink failure

Hi,
For now, to start a new UCS domain, my customer would want to configurate an Ethernet Uplink with only 1 port/FO. So, we are going to configurate a Po with 1 port.
In case of failure of this link, do all vNIC using this uplink fail ?
I mean, the vmnic from the WMare vswitch are going down if this Ethernet uplink failed ???
Thanks.
Nicolas.

Hi Nicolas,
If you have failover enabled on the UCS vNIC level, the vNICs would move to the other fabric interconnect. For example, if the vNIC is initially pointing to FI A and if the port-channel on FI A goes down, UCS vNIC failover capability would kick in and move the connection to FI B. The vmnic on ESX won't be aware of this.
If you do not have failover enabled, then the NIC would go down on the UCS (and on ESX)
Thanks,
Shankar

No link between IOM port 2/2/3 and fabric interconnect B

Hello
I am trying to remove an uplink from an IOM to the fabric interconnect as its not required. I have tried to reset the I/O module and re-acknowledge the chassis with no luck, it is still shown in the GUI with the error message as in the subject. We are currently running 2.0(1s). The kit isn't in production yet.
Thanks for any help

Yes i agress thats a pretty heavy approach to clean it. No if you left it there as it is, there would have been no operational issues. It would have just left this fault in the UCSM.
./Abhinav

Fabric Interconnect vNic Failover

We have the following Palo vNIC allocation for an ESXi host with 1000v in our vBlock:
VLANs on vNIC #
1: 14;
2: 41
3: 42
4 20,12,22,24;
5: 4,3,6
Now the statement has been made to us that this would not provide a redundant networking configuration in the event of a Fabric Interconnect failure, and there must be pairs of vNICs carrying a VLAN – one that traverses the left FIC and one the right FIC. The suggestion to us was to only use 2 vNICs and carry all VLANs.
My understanding is that with the M81KR vNICs, they can be set to ‘Fabric Failover Mode’ which will switch the accessible path in the event of a FIC failure seamlessly to the host. Removing the host managed failover and maintaining the HA within the fabric instead.
Is this a correct deployment when using Fabric Failover?

I think I have found the answer to my own question:
https://supportforums.cisco.com/docs/DOC-14992

Port-channel Problem between Fabric Interconnect and N7K vPC

Dear all,
I have a problem with Port-channel Uplink between Fabric Interconnect with N7K using vPC
This is my network topology for UCS Deployment
In N7K I has configured vPC for red link and green link, at Fabric Interconnect A I has configured Port-Channel with member is Port 1 and Port 2, uplink is red link. At Fabric Interconnect B, I has configured Port-Channel with member is Port 1 and Port 2, uplink is green link.
The show interface port-channel on N7K is good, every port-channel is up and have all member. But At Fabric Interconnnect, when I see on UCS Manager, the status of Port-Channel on Fabic A and Fabric B is fault with Additional Info: No operational member. Although all link is link up and I has status of Port-Channel is enable on UCS Manager. When I see the Properties of Port 1, Port 2 on Port-channel, I see the membership status is : individual. This mean port-channel is not up and no membership in this configuration. I want to using port-channel for load balance and plus more bandwidth for uplink to 20Gig. I don't understand why ?
Please help me resolve this problem, I has send the capture screen of UCS Manager when I show status of Port-channel and Port-member in port-channel in attach items.
Anyone can help me to resolve this, thanks you very much. Please reference attach items for more detail about fault.
Thanks,
Trung.

Thanks Matthew very much,
I has resolved this problem. The reason of problem is miss match protocol of port-channel between N7K and Fabric Interconnect. The Fabric Interconnect always use LACP protocol, but N7K using Port-channel mode on, that why the port-channel failed. I has configured LACP for port-channel in N7K, it has resolved the problems.
Thanks,
Trung.

UCS-Fabric Interconnect Replacement

Hello everybody,
I have a UCS domain with a failed Fabric Interconnect and i have to replace this Fabric. For me there is no action to do in the configuration of the new FI i just have to add it in the existing domain and give it the IP address, but i have a question about the firmware version. is the firmware automatically installing in the new Fabric or have i to install it manually ? the firmware version in my domain is 2.2(1b).
Thanks for the answer
Best Regards
François Fiocre

Hi Claudio
I assume you talk about 10G copper, not 1G copper;
and if 10G: CX-4 (Twinax) cables, or 10G Base-T ?
There is a 12-port 10GBASE-T module, but this is only supported on the Nexus 5596T so of no use on Nexus 5596UP switch, nor UCS FI.
Anyway, it is a not recommended to connect UCS FI Uplinks to a Nexus 22xx ? just think about the oversubscription ! you would like to have as much bandwidth Northbound as possible.

Connecting Fabric Interconnect with MDS

Dear Team,
Kindly share how to configure connectivity between Fabric Interconnect and MDS
what are the steps to be followed in establishing a successful connection between FI and MDS
We have a tape library from IBM and trying to connect the same to FI via MDS and want this Tape library to be detected on the guest VM running windows 2008 on Esxi( Service profile with 2 vhba already added),we are trying using Fibre connectivity.
Also please advice in which mode should we keep the FI (FC switching,end host etc)
Thanks and Regards
Jose

Hi Jose,
The common steps to configure FC on FI and MDS as following :
1. FI configured in NPV mode, ensure that all FC uplinks and vHBA of Blades have been assigned on correct VSAN
    Verify if vHBA Flogi into the FI
    show npv flogi-table
    show npv status
Example
FI-A(nxos)# sh npv flogi-table
SERVER                                                                  EXTERNAL
INTERFACE VSAN FCID             PORT NAME               NODE NAME       INTERFACE
vfc703    10   0x330001 20:00:00:cc:1e:dc:01:0a 20:00:00:25:b5:00:00:03 San-po102
vfc711    10   0x330002 20:00:00:cc:1e:dc:02:0a 20:00:00:25:b5:00:00:02 San-po102
Total number of flogi = 2.
FI-A(nxos)# sh npv status
npiv is enabled
disruptive load balancing is disabled
External Interfaces:
====================
Interface: san-port-channel 102, State: Up
        VSAN:   10, State: Up, FCID: 0x330000
Number of External Interfaces: 1
Server Interfaces:
==================
Interface: vfc703, VSAN:   10, State: Up
Interface: vfc711, VSAN:   10, State: Up
Number of Server Interfaces: 2
2. Enable NPIV mode on MDS, configure the link to FI as F port. Assign this port into correct VSAN
    FI will become Node proxy, so verify if all vHBA have been detect on Upstream NPIV switch
SW2(config)# sh npiv status
NPIV is enabled
SW2(config)# sh flogi database
INTERFACE        VSAN    FCID           PORT NAME               NODE NAME
San-po102        10    0x330000 24:66:00:2a:6a:05:e9:00 20:0a:00:2a:6a:05:e9:01
San-po102        10    0x330001 20:00:00:cc:1e:dc:01:0a 20:00:00:25:b5:00:00:03
                           [BLADE1-SAN-A]
San-po102        10    0x330002 20:00:00:cc:1e:dc:02:0a 20:00:00:25:b5:00:00:02
                           [BLADE2-SAN-A]
3. Configure zoning on the MDS (to make sure whether the issue on zoning or not, you can configure the default zone to permit)
Example : zone default-zone permit vsan 1
Verify the zoning on Upstream NPIV switch
SW2(config)# SH ZONESet active vsan 10
zoneset name ZONESET-VSAN10 vsan 10
zone name INE-VSAN10 vsan 10
* fcid 0x330001 [device-alias BLADE1-SAN-A]
* fcid 0x330002 [device-alias BLADE2-SAN-A]
* fcid 0x2a0000 [device-alias FC-SAN10]
4. Ensure that vHBA of BLADE had been discovered on the storage device. Configure LUN masking on the storage
I really don't have enough knowledge of VMware hahaha.....so can't help for troubleshooting the VM....Really sorry
Thanks,
Gofi

6120 Fabric Interconnect

Is it possible to connect both ethernet switch and FC switch from 6120 Fabric Interconnect using the 8 port extended module?

Dhinesh,
Which expansion module are you using? N10-E0440 or N10-E0080 (both of them have eight ports).
If its E0440 - Yes, you can use it for both Ethernet uplink as well as FC uplink.
E0080 - Is just FC ports.
(the ports on the module which have green color on them are FC ports).
Here is the URL to the Spec sheet for your refernce:
http://www.cisco.com/en/US/prod/collateral/ps10265/ps10276/spec_sheet_c17-665945.pdf
./Abhinav

FAS3220 and FAS2240 direct attach on fabric interconnect 6296UP

Hello everybody
I have the following architecture
1 Cisco Catalyst 6509-E
1 Cisco Catalyst 3560-x
2 Fabric Interconnect 6296UP
1 FAS 3220 (principal storage)
1 FAS2240 (vault backup)
1 chasis cisco UCS 5108
5 blade servers Cisco UCS B200M3
Firmware version: 2.2(3c)
Fabric Interconnect are direct attach on fabric interconnect 6296up and the ports are configurated as FC storage ports
My question is the following:
Can I zoning fabric interconnect with FAS3220 and FAS2240?
Can I present LUNS of FAS2240 and FAS3220 at the same time on the same boot policy, same san boot policy, and the same organization?
Thanks for your help

Well, even if that was possible. You're accounting for a critical failure - i.e. a failure of two redundant devices (both 3220s, which I would have assume would be running in a cluster ... or 7mode).
While just upstream you have a single cat 6500 - which would be a single point of failure.
Logic would dictate to account for a single device failure all across the path before looking into multi-device failure.

Monitoring traffic between UCS fabric extenders and fabric interconnect?

What is the best way to monitor the traffic between UCS fabric extenders (chassis) and fabric interconnect? Specifically I am looking for parameters to keep an eye on to determine when we may need to move from 2 links per fabric extender to 4 to handle greater IO needs.
Thanks.
- Klaus

One way you could monitor usage is by looking at the interface stats just as you would for any switch uplink.
Connect to the Cluster CLI
connect NXOS
Look at the input/output rate of your Server Uplink interfaces.
In my example I'm using fixed ports 1 through 4 on my FI's connecting to the IOM of the Chassis.
UCS-A(nxos)# show int eth1/1-4 | include rate
30 seconds input rate 17114328 bits/sec, 1089 packets/sec
30 seconds output rate 8777736 bits/sec, 693 packets/sec
    input rate 5.07 Mbps, 198 pps; output rate 1.03 Mbps, 99 pps
30 seconds input rate 2376 bits/sec, 0 packets/sec
30 seconds output rate 1584 bits/sec, 2 packets/sec
    input rate 1.58 Kbps, 0 pps; output rate 1.58 Kbps, 3 pps
30 seconds input rate 2376 bits/sec, 0 packets/sec
30 seconds output rate 31680 bits/sec, 20 packets/sec
    input rate 1.58 Kbps, 0 pps; output rate 30.89 Kbps, 18 pps
30 seconds input rate 2376 bits/sec, 0 packets/sec
30 seconds output rate 1584 bits/sec, 1 packets/sec
    input rate 1.58 Kbps, 0 pps; output rate 1.58 Kbps, 3 pps
If you notice your two links pushing near 10G consistently, it might be time to add another 2 link.
Other than this you can use SNMP to log the stats and look based on daily/weekly/monthly usage.
Robert

Failed:Error disabled - SFP vendor not supported on Fabric Interconnect 6248UP

We are connecting a FC-uplink on a Fabric Interconnect 6248UP to a MDS9124. On the Fabric Interconnect side we use a 8Gbps SFP (DSpSFP-FC8G-SW) on the MDS9124 side there is a 4 Gbps SFP.This should work when speed is at fixed speed 4 Gbps on both sides (I was told). When the SFP's and cabling is connected I get an error in the UCS manager on the FC-uplink port we are using:
Failed:Error disabled - SFP vendor not supported.
I cannot change the speed on the FC-Uplink ( I can only set user-label). The Fabric Interconnect is configured for FC on the last 8 ports and that is where the SFP for the storage is located (port 31).

I have a similar problem. I'm using port 1/41 and gets this output
show system firmware expand | head lines 10
UCSM:
   Running-Vers: 2.1(3a)
   Package-Vers: 2.1(3a)A
   Activate-Status: Ready
Catalog:
   Running-Vers: 2.1(3a)T
   Package-Vers: 2.1(3a)A
   Activate-Status: Ready
sh interface fc 1/41
fc1/41 is down (Error disabled - SFP vendor not supported)
   Hardware is Fibre Channel, SFP is Unknown(0)
   Port WWN is 20:29:00:2a:6a:7e:7b:00
   Admin port mode is F, trunk mode is off
   snmp link state traps are enabled
   Port vsan is 1
   Receive data field Size is 2112
   Beacon is turned off
   1 minute input rate 0 bits/sec, 0 bytes/sec, 0 frames/sec
   1 minute output rate 0 bits/sec, 0 bytes/sec, 0 frames/sec
     0 frames input, 0 bytes
       0 discards, 0 errors
       0 CRC, 0 unknown class
       0 too long, 0 too short
     0 frames output, 0 bytes
       0 discards, 0 errors
     0 input OLS, 0 LRR, 0 NOS, 0 loop inits
     0 output OLS, 0 LRR, 0 NOS, 0 loop inits
   last clearing of "show interface" counters never
show int fc 1/41 transceiver details
fc1/41 sfp is present but not supported
   name is CISCO-FINISAR
   part number is FTLX8571D3BCL-C2
   revision is A
   serial number is FNS17401NYG
   FC Transmitter type is Unknown(0)
   FC Transmitter supports Unknown(0) link length
   Transmission medium is Unknown(0)
   Supported speeds are - Min speed: -1 Mb/s, Max speed: -1 Mb/s
   Nominal bit rate is 10300 MBits/sec
   Link length supported for 50/125mm fiber is 80 m(s)
   Link length supported for 62.5/125mm fiber is 20 m(s)
   No tx fault, no rx loss, no sync exists, diagnostic monitoring type is 0x68
   SFP Diagnostics Information:
                                     Alarms                 Warnings
Is this a firmware problem?

Connecting Fabric Interconnects together with FC

Hi,
Is it possible to link two Fabric Interconnects together with FC uplinks?
If I have two sites and I have a pair of Fabric Interconnects in site A and a pair in site B. Is it possible to connect FI-A of site A to FI-A in site B via an FC upllink port? Or should I use a Cisco MDS or Brocade switch to make the connection?
In each site we have a server with FC connections and they need access the storage on the other side. Is it possible if I place the FI in FC switching mode?
Kind regards,
Gert

Hi Robert,
Thanks for the answer. On each site there is only one server with FC connections, so it's a high cost to buy MDS switches only for that. Is it possible to connect them via FC or should we really buy some FC switches. It's only for 1 server per site...
Thanks in advance and regards,
Gert

Fabric Interconnect uplink failure

Similar Messages

Maybe you are looking for