UCS Shortcomings?

Question

I regularly hear a few specific arguments critiquing the UCS that I would like someone who knows the UCS ecosystem well to clarify before my organization adopts it.
1. The Cisco UCS system is a totally proprietary and closed system, meaning:
     a) the Cisco UCS chassis cannot support other vendor’s blades.  For example, you can’t place an HP, IBM or Dell blade in a Cisco UCS  5100      chassis.
     b) The Cisco UCS can only be managed by the Cisco UCS manager – no 3rd party management tool can be leveraged.
     c) Two Cisco 6100 Fabric Interconnects can indeed support 320  server blades (40 chassis, as Cisco claims), but only with an unreasonable         amount of oversubscription. The more realistic number is two (2) 6100s for every four  (4) 5100 UCS chassis (32 servers), which will yield a more      reasonable oversubscription ratio of 4:1.
     d) A maximum of 14 UCS chassis can be managed by the UCS  manager, which resides in the 6100 Fabric Interconnects. This  creates islands of       management domains -- 14 chasses per island, which presents an interesting challenge if you indeed try to manage 40 UCS chassis (320 servers)        with the same pair of Fabric  Interconnects.
     e) The UCS blade servers can only use Cisco NIC cards (Palo).
     f) Cisco Palo cards use a proprietary version of interface virtualization and cannot support the open SR-IOV standard.
     g) The Cisco 5100 chassis can only be uplinked to the Fabric Interconnects, so any shop that already has ToR switches will have to replace them.
I would really appreciate it if anyone can give me bulleted responses to these issues. I already posted this question on Brad Hedlund's web blog -- he really knows his stuff. But I know there are a lot of knowledgeable professionals on here, too.
Thanks!

Answer 1

Robert, thank you very much for those most informative answers. I really appreciate it.
I have responded to your points in blue. Can you look them over right quick and give me your thoughts?
Thanks, again!
       a) the Cisco UCS chassis cannot support other vendor’s blades.  For   example, you can’t place an HP, IBM or Dell blade in a Cisco UCS    5100      chassis.
[Robert] - True.  This is standard in the industry.  You can't put IBM blades in an HP c7000 Chassis or vice-versa can you?
I believe the Dell blade chassis can support blades from HP and IBM. I would have to double-check that.
     b) The Cisco UCS can only be managed by the Cisco UCS manager – no 3rd party management tool can be leveraged.
[Robert]   - False.  UCS has a completely open API.  You can use any XML,  SMASH/CLP, IPMI, WS-MAN.  There are already applications that have  built-in support for UCS from vendors such as HP (OpenManage), IBM  (Tivoli), BMC Bladelogic, Altiris, Netcool etc.  There's even a  Microsoft SCOM plugin being deveoped. See here for more information: http://www.cisco.com/en/US/prod/ps10265/ps10281/ucs_manager_ecosystem.html
This is very interesting. I had no idea that the Cisco UCS ecosystem can be managed by other vendor management solutions. Can a 3rd party platform be used in lieu of UCS manager (as opposed to just using 3rd party plug-ins) ? Just curious...
       c) Two Cisco 6100 Fabric Interconnects can indeed support 320  server   blades (40 chassis, as Cisco claims), but only with an   unreasonable         amount of oversubscription. The more realistic   number is two (2) 6100s for every four  (4) 5100 UCS chassis (32   servers), which will yield a more      reasonable oversubscription  ratio  of 4:1.
[Robert]  Your oversubscription rate can vary from 2:1 all the way to 8:1  depending how many uplinks in use with the current IOM hardware.  With a  6120XP you can support "up to" 20 Chassis's (using single 10G uplinks  between Chassis and FI) assuming you're using the expansion slot for  your Ethernet & FC uplink connectivity.  You can support "up to" 40  Chassis using the 6140XP in the same regard.  Depending on your  bandwidth requirements you will might choose to scale this to at least 2  uplinks per IOM/Chassis (20GB redundant uplink). This would give you 2  uplinks from each Chassis to each Interconnect supporting a total of 80  servers with an oversubscription rate of 4:1. Choosing the  level of oversubscription requires an understanding of the underlying  technology - 10G FCoE. FCoE is a lossless technology which provides  greater efficiencies in data transmission than standard ethernet.  No  retransmissions & no dropped frames = higher performance &  efficiency.  Due to this efffciency you can allow for higher  oversubscription rates due to less chances of contention.  Of course  each environement is unique.  If you have some "really" high bandwidth  requirements you can increase the uplinks between the FI & Chassis.   For most customers we've found that 2 uplinks never becomes close to  saturated.  You best bet is to analyse/monitor the actual traffic and  decide what needs you require.
Actually, I should have checked this out for myself. What I posted was preposterous and I should have spotted that right off the bat. And you should have hung me out to dry for it! :-) Correct me if I'm wrong, but two (2) 6140 FICs can handle A LOT more than just 4 UCS blade chassis, as I stated earlier. In fact, if a 4:1 OS ratio is desired (as in my question), two 6140 FICs can handle 20 UCS chassis, not 4. Each chassis will have 4 total uplinks to the 6140s - 2 uplinks per FEX per FIC. That equates to 160 servers.
If 320 servers is desired, the OS ratio will have to go up to 8:1 - each chassis with 2 uplinks, one from each FEX to each FIC.
Is all this about OS ratios correct?
       d) A maximum of 14 UCS chassis can be managed by the UCS  manager,   which resides in the 6100 Fabric Interconnects. This  creates islands   of  management domains -- 14 chasses per island, which presents an   interesting challenge if you indeed try to manage 40 UCS chassis (320   servers)        with the same pair of Fabric  Interconnects.
[Robert]  False.  Since the release of UCS we have limited the # of Chassis  supported.  This is to ensure a controllable deployment in customer  environements.  With each version of software released we're increasing  that #.  The # of chassis is limited theoretically only by the # of  ports on the fabric interconnects (taking into account your uplink  configuration).  With the latest version 1.4, the supported chassis  count has been increased to 20.  Most customer are test driving UCS and  are not near this limiation.  For customers requiring more than this  about (or the full 40 Chassis limit) they can discuss this with their  Cisco Account manager for special consideration.
It's  always funny how competators comment on "UCS Management Islands".  If  you look at the competition and take into consideration Chassis, KVM,  Console/iLO/RAC/DRAC, Ethernet Switch and Fiber Switch management  elements UCS has a fraction the amount of management points when scaling  to beyond hundreds of servers.  
I understand. Makes sense.
     e) The UCS blade servers can only use Cisco NIC cards (Palo).
[Robert]  False.  Any UCS blade server can use either a Emulex CNA, Qlogic CNA,  Intel 10G NIC, Broadcom 10G NIC or ... our own Virtual Interface Card -  aka Palo.  UCS offers a range of options to suite various customer  preferences.
Interesting. I didnt know that.
     f) Cisco Palo cards use a proprietary version of interface virtualization and cannot support the open SR-IOV standard.
[Robert]  Palo is SR-IOV capable.  Palo was designed originally to not be SR-IOV  dependent by design. This removes dependencies on the OS vendors to  provide driver support.  As we have control over this, Cisco can provide  the drivers for various OS's without relying on vendors to release  patch/driver updates.  Microsoft, Redhat and VMware have all been  certified to work with Palo. 
Correct SR-IOV is a function of the NIC card and its drivers, but it does need support from the hypervisor. That having been said, can a non-Cisco NIC (perhaps one of the ones you mentiojned above) that supports SR-IOV be used with a Cisco blade server in the UCS chassis?
       g) The Cisco 5100 chassis can only be uplinked to the Fabric   Interconnects, so any shop that already has ToR switches will have to   replace them
[Robert]  Not necessaryily true.  The Interconnects are just that - they  interconnect the Chassis to your Ethernet & FC networks. The FI's  act as your access switches for the blades which should connect into  your distribution/Core solely due to the 10G interfaces requirements.   I've seen people uplink UCS into a pair of Nexus 5000's which in turn  connect to their Data core 6500s/Nexus 7000s.  This doesn't mean you  can't reprovision or make use of ToR switches, you're just freeing up a  heap of ports that would be required to connect a legacy non-unified I/O  chassis.
I understand what you mean, but if a client has a ToR design already in use, those ToRs must be ripped out. For example, lets say they had Brocade B-8000s at the ToR, it's not as if they can keep them in place and connect the UCS 5100 chassis to them. The 5100 needs the FICs.
Regards,
Joe