Fabric Interconnect To MDS 9148 FC Port Channel/Trunks

Is it possible to create both a Port Channel and Trunk on a Cisco MDS 9148? Yes correct?
Or can I only trunk between the 2 MDS switches and not the MDS to Fabric Interconnect?
These FC links would be connected to a 2248XP Fabric Interconnect carrying multiple VSANs from the MDS to the Fabric Interconnect.
Thanks

Hello,
Yes, you can have trunk links grouped into port channel ( F port-channel trunking ) between FI and MDS.
http://www.cisco.com/en/US/docs/unified_computing/ucs/sw/gui/config/guide/2.0/b_UCSM_GUI_Configuration_Guide_2_0_chapter_0101.html
Padma

Similar Messages

Connecting Fabric Interconnect with MDS

Dear Team,
Kindly share how to configure connectivity between Fabric Interconnect and MDS
what are the steps to be followed in establishing a successful connection between FI and MDS
We have a tape library from IBM and trying to connect the same to FI via MDS and want this Tape library to be detected on the guest VM running windows 2008 on Esxi( Service profile with 2 vhba already added),we are trying using Fibre connectivity.
Also please advice in which mode should we keep the FI (FC switching,end host etc)
Thanks and Regards
Jose

Hi Jose,
The common steps to configure FC on FI and MDS as following :
1. FI configured in NPV mode, ensure that all FC uplinks and vHBA of Blades have been assigned on correct VSAN
    Verify if vHBA Flogi into the FI
    show npv flogi-table
    show npv status
Example
FI-A(nxos)# sh npv flogi-table
SERVER                                                                  EXTERNAL
INTERFACE VSAN FCID             PORT NAME               NODE NAME       INTERFACE
vfc703    10   0x330001 20:00:00:cc:1e:dc:01:0a 20:00:00:25:b5:00:00:03 San-po102
vfc711    10   0x330002 20:00:00:cc:1e:dc:02:0a 20:00:00:25:b5:00:00:02 San-po102
Total number of flogi = 2.
FI-A(nxos)# sh npv status
npiv is enabled
disruptive load balancing is disabled
External Interfaces:
====================
Interface: san-port-channel 102, State: Up
        VSAN:   10, State: Up, FCID: 0x330000
Number of External Interfaces: 1
Server Interfaces:
==================
Interface: vfc703, VSAN:   10, State: Up
Interface: vfc711, VSAN:   10, State: Up
Number of Server Interfaces: 2
2. Enable NPIV mode on MDS, configure the link to FI as F port. Assign this port into correct VSAN
    FI will become Node proxy, so verify if all vHBA have been detect on Upstream NPIV switch
SW2(config)# sh npiv status
NPIV is enabled
SW2(config)# sh flogi database
INTERFACE        VSAN    FCID           PORT NAME               NODE NAME
San-po102        10    0x330000 24:66:00:2a:6a:05:e9:00 20:0a:00:2a:6a:05:e9:01
San-po102        10    0x330001 20:00:00:cc:1e:dc:01:0a 20:00:00:25:b5:00:00:03
                           [BLADE1-SAN-A]
San-po102        10    0x330002 20:00:00:cc:1e:dc:02:0a 20:00:00:25:b5:00:00:02
                           [BLADE2-SAN-A]
3. Configure zoning on the MDS (to make sure whether the issue on zoning or not, you can configure the default zone to permit)
Example : zone default-zone permit vsan 1
Verify the zoning on Upstream NPIV switch
SW2(config)# SH ZONESet active vsan 10
zoneset name ZONESET-VSAN10 vsan 10
zone name INE-VSAN10 vsan 10
* fcid 0x330001 [device-alias BLADE1-SAN-A]
* fcid 0x330002 [device-alias BLADE2-SAN-A]
* fcid 0x2a0000 [device-alias FC-SAN10]
4. Ensure that vHBA of BLADE had been discovered on the storage device. Configure LUN masking on the storage
I really don't have enough knowledge of VMware hahaha.....so can't help for troubleshooting the VM....Really sorry
Thanks,
Gofi

MDS Code Level for UCS FC Port Channel/Trunking

What is the required MDS code level needed to support FC Port-Channeling/Trunking on a 9222i, 9148, or 9509/9513 w/ Gen2 line cards to UCS 1.4(3)?
I know the fport-channel-trunk feature is needed and was released in MDS 4.2 code but I have seen documents that this level of code wont support these features when used to connect to Cisco UCS.
Can someone confirm? Is NXOS 5.0(4) or higher required?

Doug,
I wrote the doc almost a year ago and it doesn't have the latest interop matrix support information. The current MDS NX-OS support is located in this doc: http://www.cisco.com/en/US/docs/switches/datacenter/mds9000/interoperability/matrix/intmatrx.html
While, I wish this was also duplicated in UCS docs, they are more or less taking the stance that 3rd party (Cisco MDS is this case) interop is maintained by the 3rd party. EMC, Netapp, or even Cisco.
It would be a nightmare updating UCS docs with 3rd party qualifications and support.
The short answer to your question is:
UCS 1.4(1), 1.4(2), 1.4(3) Is supported by MDS9000: 3.3(4a), 3.3(5), 4.2(1a), 4.2(1b), 4.2(3a), 4.2(5), 5.0(1a), 5.0(4), 5.0(4c)
NPV was introduced in MDS 3.3. f port-channel/trunk was introduced in MDS 4.2.
As Ram mentioned above, 4.2(5) is what I would recommend if I wanted to implement f port-channel and/or f port-trunk. I have also had recent experience with 4.2(5) and it worked great with a large UCS f port-channel/trunk implementation
The specific switch limitations I mentioned in my doc still exist. Only certain models of MDS will support f port-trunk. Almost all models will support f port-channel.
Dave

FC Port Channel & Trunking

Hi, Cisco Experts:
Can I have a definitive answer to the following?
q1.
EMC mentioned that they do not support FC Port Channel & Trunking with regard to our UCS Fabric Interconnects/MDS/VNX5300?
Is that any truth in that?
We are trying a Cisco UCS solutions with MDS and EMC VNX in the mix-SAN Boot basically, ESXi 5.1 and all others.
q2.
From MDS all the ways up to EMC VNX, is there such concept too-FC Port Channel & Trunking, assuming we are doing FC Port Channel & Trunking from FInterconnect to MDS?
Please let me know.
Appreciate it.
SiM

Hi Sim,
We support FC port channel and trunking between UCS FI and MDS.
Is EMC saying about the port channel between the MDS and VNX? Is there any document EMC pointed out?
With respect to booting from SAN, yes, you can boot from SAN with MDS and EMC VNX as long as the OS is supported in the UCS system. You can refer to
http://www.cisco.com/en/US/products/ps10477/prod_technical_reference_list.html for the supported OS versions.
Hope this answers your question.
-Ganesh

F-port-channel-trunk disruptive or not?

Hi,
We have a N5K with NPIV configured and and MDS9148 we're gonna setup a port channel and have come to the conclusion that we need the F-port-channel-trunk feature enabled on the MDS side.
Question: Is enabling the F-port-channel-trunk feature on the MDS disruptive or non-disruptive?
--M

It's too late; I would say it is disruptive; however, the disruption is so short, that you didn't notice it. Similar without non F-port trunking channeling: using a link: the lost sessions are simply redistributed over the remaining links; new flogi is necessary.

MDS configuration for port channel

I've tried to set up the new FC port channel upload to a pair of MDS 9124s, but as I don't know enough about the MDS side I can't get the link to come up. Are there any references available anywhere that tell you blow by blow exactly how to configure the MDS side of the port channel? Or perhaps a sample working config?
Thanks

Simon,
A few items:
1) You are on 1.4.1i Balboa code
2) FI's are in FC switch mode as MDS 9124's don't support F-port channel (NPV/NPIV)
3) It's a lot easier the first time if you have matched speed sfp+ all around. Although, with the correct configs, unmatched speeds can be made to work.
If the FIs are in switch mode and your MDS is running a minimum 3.3 here are some configs that may help:
MDS side interfaces 4/11 and 4/12 are plugged into the UCS fc ports. First setup and ensure ISLs are working properly between UCS and MDS, then configure the port-channels
interface fc4/11
switchport rate-mode dedicated
switchport mode E
switchport trunk mode auto
no shutdown
interface fc4/12
switchport rate-mode dedicated
switchport mode E
switchport trunk mode auto
no shutdown
The UCS GUI Equipment -> Fabric Interconnects -> FI-A -> Set FC Switching Mode. This will cause both FI's to reboot into FC switch mode. This is VERY DISRUPTIVE, both FI's will reboot.
On the UCS CLI you should see this on the UCS fc ports connected to the MDS
cae-sj-ca3-A(nxos)# show running-config interface fc 2/1-2
!Command: show running-config interface fc2/1-2
!Time: Wed Oct 20 16:49:39 2010
version 4.2(1)N1(1.4)
interface fc2/1
switchport mode E
no shutdown
interface fc2/2
switchport mode E
no shutdown
Until you have VSAN trunks enabled, make sure the ports on the UCS and MDS are in the same VSAN. VSAN 1 for example.
Once you have working ISLs, then proceed to turn them into port-channels and enable VSAN trunking. Start with the MDS
Create a channel group, it should look like this
cae-sj-9506-1# show run interface port-channel 3
!Command: show running-config interface port-channel 3
!Time: Wed Oct 20 17:06:01 2010
version 5.0(1a)
interface port-channel 3
channel mode active
switchport mode E
switchport rate-mode dedicated
switchport trunk mode auto <-- VSAN trunking
Enable VSAN trunking on the MDS ISL interfaces, it'll look like this:
cae-sj-9506-1# show run interface fc 4/11-12
!Command: show running-config interface fc4/11-12
!Time: Wed Oct 20 17:07:05 2010
version 5.0(1a)
interface fc4/11
switchport rate-mode dedicated
switchport mode E
switchport trunk mode auto
no shutdown
interface fc4/12
switchport rate-mode dedicated
switchport mode E
switchport trunk mode auto
no shutdown
Add the channel group information to the MDS ISL interfaces:
Enter configuration commands, one per line. End with CNTL/Z.
cae-sj-9506-1(config)# interface fc 4/11-12
cae-sj-9506-1(config-if)# channel-group 3 force
fc4/11 fc4/12 added to port-channel 3 and disabled
please do the same operation on the switch at the other end of the port-channel,
then do "no shutdown" at both ends to bring it up
cae-sj-9506-1(config-if)# show run interface fc 4/11-12
!Command: show running-config interface fc4/11-12
!Time: Wed Oct 20 17:07:39 2010
version 5.0(1a)
interface fc4/11
switchport rate-mode dedicated
switchport mode E
switchport trunk mode auto
channel-group 3 force
no shutdown
interface fc4/12
switchport rate-mode dedicated
switchport mode E
switchport trunk mode auto
channel-group 3 force
no shutdown
Create the SAN port channel on the UCS side
SAN -> SAN Cloud -> Fabric A -> FC Port Channels -> Create Port Channel
After you create the SAN port channel in UCS, make sure and enable it. I also bounce the MDS port channel at this point.
If all is correct, in a few agonizing minutes, the port-channel will be formed and passing data.
cae-sj-ca3-A(nxos)# show interface san-port-channel 1
san-port-channel 1 is trunking
    Hardware is Fibre Channel
    Port WWN is 24:01:00:0d:ec:d3:5d:c0
    Admin port mode is E, trunk mode is on
    snmp link state traps are enabled
    Port mode is TE
    Port vsan is 1
    Speed is 8 Gbps
    Trunk vsans (admin allowed and active) (1,10,26,50,66,100-101,103,123,222,24
0)
    Trunk vsans (up)                       (1,10,50,100,103)
    Trunk vsans (isolated)                 (26,66,101,123,222,240)
    Trunk vsans (initializing)             ()
    5 minute input rate 2312 bits/sec, 289 bytes/sec, 2 frames/sec
    5 minute output rate 1440 bits/sec, 180 bytes/sec, 2 frames/sec
      2669 frames input, 194760 bytes
        0 discards, 0 errors
        0 CRC, 0 unknown class
        0 too long, 0 too short
      2677 frames output, 158316 bytes
        0 discards, 0 errors
    0 input OLS, 1 LRR, 0 NOS, 0 loop inits
Again, make sure you have the ISLs up an running first before configuring the port-channels. It makes troubleshooting much easier.
Let me know if you need any help.

Physical Connectivity from Fabric Interconnect to MDS and Failover please suggest

Dear Team
We have 2 FI and 2 MDS 1 SAN
Currenlty the connectivity is
2 direct physical connections from FI-A to MDS1
2 direct physical Connections from FI-B to MDS2
From MDS 1 connection to SAN Primary controller
From MDS 1 connection to SAN Secondary controller
From MDS 2 connection to SAN primary controller
From MDS 2 connection to SAN Secondary controller
Hope the above connectivty is fine?
we had looked at
http://www.cisco.com/en/US/prod/collateral/ps4159/ps6409/ps5990/white_paper_c11_586100.html (as team always preferred to go via cisco standards)
In FI we have to cluster the FI and Primary and Subordinate,likewise here for MDS to we need to? or will it be done via FIs ?
If not these 2 MDs they work independent but sharing the configs that happens among them?
Now what ever changes (Zone Creation)we make on MDS1 hope that will get relpicated to MDS2 via FIs?
If MDS1 Fails will all the configuration be available on MDS2 and still the infra will be smooth running ? and vice versa?
Any additional steps to be done to achieve it?
Which is the best way to achieve it please suggest
Thanks and Regards
Jose

Hi Jose.
Physically your connectivity is good.
On the UCS side, the 'Primary' and 'Subordinate' role refer only to the Management of the system and which device is actually running UCS Manager.
Each MDS device will have separate configuration (zoning). It's different between the 2 devices.
The blade itself will have a connection out each side, or 'SAN Fabric'.
e.g.
           +----+
       +---+    +-----+
       |   |    |     |
       |   |    |     |
       |   |SAN |     |
       | +->----<--+ |
       | |         | |
       | |         | |
   +---+-+-+    +--+--+--+
   |       |    |        |
   |MDS-A |    |MDS-B   |
   +--^-^--+    +---^-^--+
      | |           | |
   +--+-+--+    +---+-+--+
   |       |    |        |
   |UCS-A |    | UCS-B |
   +--+----+    +-----+--+
      |               |
      |               |
VSAN100    +-----+    | VSAN200
      |    |     |    |
      +---->     <----+
           |Blade|
           +-----+
The blade will have a HBA on FabricA (VSAN100) and FabricB (VSAN200)
Each HBA will have a different WWPN, and on the SAN array, each controller will have a WWPN
So on MDS-A, the zoning will be:
Blade WWPN A
Storage WWPN Primary A
Storage WWPN Secondary A
MDS-B, the zoning will be:
Blade WWPN B
Storage WWPN Primary B
Storage WWPN Secondary B
So the configuration is *not* synced between the two MDS devices, but they each have visibility to the blades vHBA device. At the blade level, the Multipathing software on the Operating System will handle any failover.
On the UCS, we would generally use a Port Channel up to the MDS.

About MDS 9222i config port-channel

dear everyone~
in project ,i need config MDS 9222i use 4 FCIP interface connect to 7606S 4 GE port for port-channel feature,but in MDS9222i i use port-channel commad ,cannot run。
how can i config it？
thinks~

Hi Sun,
Ethernet PortChannels are not supported on MSM-18/4 modules and 9222i IPS modules.
You can place the fcip interfaces in a fibre channel port-channel instead.
Regards,
David

Port-channel / Trunking / VSAN

Hi all,
I have a quick question...
I enabled trunking for all vsans and created a port-channel. I want to use this port-channel for all my vsans.
Does it matter in which vsan I add my port-channel?
If it doesn't matter why can I assign a port-channel to a vsan?
Cheers,
Mike

Mike,
It should be the same configuration on both sides for the port-channel to come up. The reason why you can assign a port-channel to a VSAN is that if you are using an E port then it depends on the matching port-channel VSAN to come up.
TE Port functionality is specific to MDS for enhanced ISL capabilities. E ports can be used for non-cisco switches to make the trunk work seamlessly as an E port.
You can refer to the following for more info:
http://www/en/US/docs/storage/san_switches/mds9000/sw/rel_3_x/configuration/guides/cli_3_4_x/trnk.html
http://www/en/US/docs/storage/san_switches/mds9000/sw/rel_3_x/configuration/guides/cli_3_4_x/pc.html
Hope this helps.
- Nikhil

Port-channel Problem between Fabric Interconnect and N7K vPC

Dear all,
I have a problem with Port-channel Uplink between Fabric Interconnect with N7K using vPC
This is my network topology for UCS Deployment
In N7K I has configured vPC for red link and green link, at Fabric Interconnect A I has configured Port-Channel with member is Port 1 and Port 2, uplink is red link. At Fabric Interconnect B, I has configured Port-Channel with member is Port 1 and Port 2, uplink is green link.
The show interface port-channel on N7K is good, every port-channel is up and have all member. But At Fabric Interconnnect, when I see on UCS Manager, the status of Port-Channel on Fabic A and Fabric B is fault with Additional Info: No operational member. Although all link is link up and I has status of Port-Channel is enable on UCS Manager. When I see the Properties of Port 1, Port 2 on Port-channel, I see the membership status is : individual. This mean port-channel is not up and no membership in this configuration. I want to using port-channel for load balance and plus more bandwidth for uplink to 20Gig. I don't understand why ?
Please help me resolve this problem, I has send the capture screen of UCS Manager when I show status of Port-channel and Port-member in port-channel in attach items.
Anyone can help me to resolve this, thanks you very much. Please reference attach items for more detail about fault.
Thanks,
Trung.

Thanks Matthew very much,
I has resolved this problem. The reason of problem is miss match protocol of port-channel between N7K and Fabric Interconnect. The Fabric Interconnect always use LACP protocol, but N7K using Port-channel mode on, that why the port-channel failed. I has configured LACP for port-channel in N7K, it has resolved the problems.
Thanks,
Trung.

Xconnect 20Gb Port-channel over MPLS Possible?

Can anybody confirm if you can establish an Xconnect between two 20Gb port-channels (2x 10Gig bundle). The core is 40Gig and running MPLS. We have many gigabit pseudowires running over it already.
Our intention is to terminate 10Gig server traffic on 4900M's and then have a 20Gb Port channel trunk link into a 6500 (which is part of the MPLS cloud). We would then issue the xconnect directly on the 20Gig port channel in order to traverse the MPLS Core.
Has anyone configured this before? Are there any caveats to watch out for? What sort of performance hit (if any does) xconnect functionality have on 10Gig Traffic.

Hi,
Yes it's supported but test it first (as always ;-) )
HTH
Laurent.

Fabric Interconnect Uplink port channeling...

Hi,
We setup port channel of two 10G uplink interfaces on each Fabric Interconnect. Everything is running as expected.
However we noticed that under 'Port-Channel # (Fabric #)' setting on LAN tap, 'Admin Speed' was set to '10Gbps'.
Should we leave this configuration as is or match to the maximum bandwidth of port Channel we set up (20Gbps)?
I only see four options on each port channel configuration, 1 Gbps, 10Gbps, 20Gbps and 40Gbps so I'm not sure if we should change this setting.
Thank you

Hello,
You do not need to change the speed ( admin ). 20 Gbps configuration is applicable for certain server models.
-------- NXOS output -----------------
FI-A(nxos)# sh run int port-channel 1
interface port-channel1
description U: Uplink
switchport mode trunk
speed 10000
FI-A(nxos)# show int port-channel 1
port-channel1 is up
Hardware: Port-Channel, address: 547f.ee67.e0fb (bia 547f.ee67.e0fb)
Description: U: Uplink
MTU 1500 bytes, BW 20000000 Kbit, DLY 10 usec,
     reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA
Port mode is trunk
full-duplex, 10 Gb/s
FI-A(nxos)# sh int bri
Port-channel VLAN Type Mode   Status Reason                    Speed Protocol
Interface
Po1          1     eth trunk up      none                       a-10G(D) lacp
There is cosmetic defect where we list the operational speed as 10 Gbps instead of 20Gbps
FI-A /eth-uplink/fabric # show port-channel detail
Port Channel:
    Port Channel Id: 1
    Name: FIA
    Admin State: Enabled
    Oper State: Up
   Speed: 10 Gbps
    Oper Speed: 10 Gbps
    State Reason:
    flow control policy: default
HTH
Padma

Port-channel L2 problem with Fabric Interconnect and Nexus 7010

Hi,
i using port-channel from both fabric interconnect to N7k with 3 cables per Fabric Interconnect.
but, my problem is when i creating port-channel, Fabric Interconnect don't support mode ON dan rate-mode share in Interface 10G Nexus 7010.
I was trying :
1. I using non dedicated port in Nexus 7010.
      - rate-mode share
      - channel-group 1 mode active
      - switchport mode trunk
when i using this option, the port-channel in Nexus 7010 was suspended
2. I using non dedicated port in Nexus 7010
     - rate-mode share
     - channel group 1 mode on
     - switchport mode trunk
when i using this option, the port-channel in Nexus 7010 was came up, but in Fabric interconnect was failed.
3. I using dedicated port in Nexus 7010
     - rate-mode share
     - channel group 1 mode active
     - switchport mode trunk
when i using this option, the port-channel in Nexus 7010 was suspended
4. I using dedicated port in Nexus 7010
     - rate-mode dedicated
     - channel group 1 mode active
     - switchport mode trunk
when i using this option, the port-channel in Nexus 7010 was came up and running well.
but, the problem is my costumer do not want using a dedicated rate-mode. if i using dedicated mode the only available port is 8 interfaces instead of 32 ports. i want to using rate-mode share in nexus 7010.
is there any way to configuring port-channel using mode on in fabric interconnect ? i was trying using CLI to create port-channel in Fabric interconect but i cannot configure the channel group protocol.
i attach the topology of N7K with Fabric interconnect.
regards,
Berwin H

Hi Manish,
the issue was solved, i was fix it last week.
the solution is:
i enable the license grace-priode (since my license is Enterprise so cannot create VDC) then i create a VDC (ex: VDC 2) so i allocate the interface on all module
/* Style Definitions */
table.MsoNormalTable
{mso-style-name:"Table Normal";
mso-tstyle-rowband-size:0;
mso-tstyle-colband-size:0;
mso-style-noshow:yes;
mso-style-priority:99;
mso-style-qformat:yes;
mso-style-parent:"";
mso-padding-alt:0cm 5.4pt 0cm 5.4pt;
mso-para-margin-top:0cm;
mso-para-margin-right:0cm;
mso-para-margin-bottom:10.0pt;
mso-para-margin-left:0cm;
line-height:115%;
mso-pagination:widow-orphan;
font-size:11.0pt;
font-family:"Calibri","sans-serif";
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-fareast-font-family:"Times New Roman";
mso-fareast-theme-font:minor-fareast;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:minor-bidi;}
N7K-M132XP-12 to VDC 2. after that i delete VDC 2 then all interface back to VDC 1 (default vdc). then i enable the rate-mode share in dedicated port and bundle into port-channel and its working.
i dont know why it must move to VDC first then it will working, maybe cisco can explain the reasons.
So here the result of my port-channel :
SVRN7KFARM-HO-01# show port-channel summary
Flags: D - Down        P - Up in port-channel (members)
        I - Individual H - Hot-standby (LACP only)
        s - Suspended   r - Module-removed
        S - Switched    R - Routed
        U - Up (port-channel)
Group Port-       Type     Protocol Member Ports
      Channel
1     Po1(SU)     Eth      LACP      Eth1/1(P)    Eth1/2(P)    Eth1/3(P)
                                     Eth1/4(P)    Eth1/25(P)
2     Po2(SU)     Eth      LACP      Eth1/9(P)    Eth1/10(P)   Eth1/11(P)
                                     Eth1/12(P)   Eth1/26(P)
3     Po3(SU)     Eth      LACP      Eth1/17(P)   Eth1/18(P)
4     Po4(SU)     Eth      NONE      Eth10/32(P) Eth10/34(P) Eth10/35(P)
                                     Eth10/36(P)
Thanks.
Berwin H

Interconnecting MDS 9148 with 2 different Fiber Providers

Hi
I would like discuss the following issue :
We have 2 HP SAN EVA 8400 ( one at the production site and one at our Disaster Recovery site ) replicating using a fiber channel service from ( 2* FC dark fiber from PROVIDER A )
They are linked together with 4 Cisco MDS 9148 Multilayer Fabric Switch (2 fabric ).
We have recently purchased a new fiber service ( 2* 1Gb FC managed service from PROVIDER B) in order to create a port channel with the 2 running ones and we have some problem.
We disconnected the PROVIDER A FC ( running for 10 years now ) and left the SAN replication during a couple of days running with the new PROVIDER B 1 Gb FC to make sure the link was stable and we had no issue.
Then we tried to connect the 2 FC together from the 2 providers and tried to create a Port channel.
However we discovered that to create a PortChannel, interfaces must have the same configurations ( speed, mode, rate )
In our case , the interface FC 1/1 using PROVIDER A link is running at 4 Gb and the interface FC 1/2 using PROVIDER B is running at 1 Gb
PROVIDER A dark fiber
PROVIDER B fiber managed service
fc1/1 is trunking
    Hardware is Fibre Channel, SFP is CWDM-1550
    Port WWN is 20:01:00:05:9b:72:b1:60
    Peer port WWN is 20:01:00:05:9b:72:b1:a0
    Admin port mode is E, trunk mode is on
    snmp link state traps are enabled
    Port mode is TE
    Port vsan is 1
    Speed is 4 Gbps
    Rate mode is dedicated
    Transmit B2B Credit is 32
    Receive B2B Credit is 32
    B2B State Change Number is 14
    Receive data field Size is 2112
    Beacon is turned off
    Trunk vsans (admin allowed and active) (1)
    Trunk vsans (up)                       (1)
    Trunk vsans (isolated)                 ()
    Trunk vsans (initializing)             ()
    5 minutes input rate 4416 bits/sec, 552 bytes/sec, 6 frames/sec
    5 minutes output rate 91616 bits/sec, 11452 bytes/sec, 11 frames/sec
      51537358280 frames input, 50955393731852 bytes
        0 discards, 0 errors
        0 CRC, 0 unknown class
        0 too long, 0 too short
      223256095067 frames output, 408667816663528 bytes
        32 discards, 0 errors
      34 input OLS, 36 LRR, 51 NOS, 0 loop inits
      53 output OLS, 35 LRR, 45 NOS, 0 loop inits
      32 receive B2B credit remaining
      32 transmit B2B credit remaining
      28 low priority transmit B2B credit remaining
    Interface last changed at Fri Dec 14 17:02:48 2012
fc1/2 is trunking
    Hardware is Fibre Channel, SFP is short wave laser w/o OFC (SN)
    Port WWN is 20:02:00:05:9b:72:b1:60
    Peer port WWN is 20:02:00:05:9b:72:b1:a0
    Admin port mode is E, trunk mode is on
    snmp link state traps are enabled
    Port mode is TE
    Port vsan is 1
    Speed is 1 Gbps
    Rate mode is dedicated
    Transmit B2B Credit is 32
    Receive B2B Credit is 32
    B2B State Change Number is 14
    Receive data field Size is 2112
    Beacon is turned off
    Trunk vsans (admin allowed and active) (1)
    Trunk vsans (up)                       (1)
    Trunk vsans (isolated)                 ()
    Trunk vsans (initializing)             ()
    5 minutes input rate 48 bits/sec, 6 bytes/sec, 0 frames/sec
    5 minutes output rate 48 bits/sec, 6 bytes/sec, 0 frames/sec
      39496053 frames input, 2696451760 bytes
        0 discards, 0 errors
        0 CRC, 0 unknown class
        0 too long, 0 too short
      167653989 frames output, 230540944084 bytes
        70 discards, 0 errors
      40377 input OLS, 345 LRR, 4248 NOS, 0 loop inits
      5186 output OLS, 1154 LRR, 39568 NOS, 0 loop inits
      32 receive B2B credit remaining
      32 transmit B2B credit remaining
      28 low priority transmit B2B credit remaining
    Interface last changed at Fri Dec 14 16:50:42 2012
So we tried to force the FC 1/1 to run at 1 Gb ( same speed as FC 1/2 ) to create the PortChannel, by doing the basic command on the interface : Switchport speed 1000
Since that , our SAN is logging every 10/15 minutes the following errors :
---------- Problem Found: CUST: CA informational: EVA8400_LIVE: Write resources on the inter site link have been reduced. at Fri 14 Dec 2012 17:21:04 GMT+01:00 ----------
---------- Problem Found: CUST: CA warning: EVA8400_DR: Excessive out of order message rate on the inter site link. at Fri 14 Dec 2012 17:27:04 GMT+01:00 ----------
---------- Problem Found: CUST: CA warning: EVA8400_DR: Excessive data exchange retry rate on the inter site link. at Fri 14 Dec 2012 17:41:34 GMT+01:00 ----------
---------- Problem Found: CUST: CA warning: EVA8400_LIVE: Excessive data exchange retry rate on the inter site link. at Fri 14 Dec 2012 17:41:49 GMT+01:00 ----------
We have the following questions :
Have you already seen that kind of situation ?
Is there a best practice to reduce the interface speed in order to create the PortChannel with identical interface setup ?
Your help and advice would be much appreciated.
Regards

the RTT delay by each link is different and this is causing packets to arrive out of order.
not sure if port channeling between two different links or different characteristics is good practive.

FC Port Channel between UCS and MDS.

Hi All,
I am new to Cisco Fabric Concepts. In my enviornment i have a F port channel(8 Port Group) created on MDS 9513 switch and this Fport
channel is connected to Cisco UCS 6296 FI. The Cisco UCS Blade servers are connected to the Fabric Interconnect .
On MDS NPIV is enabled.
Can anyone explain the below questions.
1. Why do we create an F port Channel Group and connect it to the UCS FI? Is this something similar to Brocade Edge to AG Switch Connectivity.
2. How to configure F port Channel Group in MDS . Can anyone explain with an example.
3. Do we need to make any Configuration on UCS FI ports for server connectivity and Channel port Connectivity if yes what are the steps required to
do the same.Does the WWPN shows up in the FLOGI Database if the connectivity and configuration looks good in UCS FI and MDS.
4. What happens when a VSAN on MDS switch is added to the Port Channel.
Thanks and Regards,
Santosh surya

Look at my remarks in
https://supportforums.cisco.com/discussion/12468266/fc-port-channels-between-mds-and-ucs-fi-best-practice
1. Why do we create an F port Channel Group and connect it to the UCS FI? Is this something similar to Brocade Edge to AG Switch Connectivity.
F port channel is proprietary; therefore any such F port channel between UCS FI and Brocade doesn't work.
2. How to configure F port Channel Group in MDS . Can anyone explain with an example.
see eg.
https://supportforums.cisco.com/sites/default/files/legacy/9/9/2/53299-UCS_1-4-1_F-port_channel-trunk-v1.pdf
3. Do we need to make any Configuration on UCS FI ports for server connectivity and Channel port Connectivity if yes what are the steps required to
see eg.
https://supportforums.cisco.com/sites/default/files/legacy/9/9/2/53299-UCS_1-4-1_F-port_channel-trunk-v1.pdf
Does the WWPN shows up in the FLOGI Database if the connectivity and configuration looks good in UCS FI and MDS.
flogi database is on the MDS, not FI; there are however UCS CLI commands, like "show npv ...."
4. What happens when a VSAN on MDS switch is added to the Port Channel.
If its not created on UCS, it will just not become the status "up"

Fabric Interconnect To MDS 9148 FC Port Channel/Trunks

Similar Messages

Maybe you are looking for