UCS Design - Where to begin?

Question

Hello all,
I recently was given a project to install UCS. It's new to me. We have purchased the following:
- 2 2960 switches
- 2 F5 Load Balancers
- 2 6120 Fabric Interconnects
- 2 MDS 9148's
- 1 5108 Blade Chassis
- EMC Storage (Celerra)
I understand physically how to connect them all together, but is there some kind of best practice document that tells me things such as how many VSAN's, zones to create, or where to port-channel connections together? The customer is open to any new ideas, so if anyone has a good design or some quick pointers they'd like to share, it would be greatly appreciated.
Thanks,
Bobby Grewal

Answer 1

Bobby,
I have yet to see a comprehensive guide that contains all the information you're looking for.  There are guides posted on this community & CCO which discuss various topics regarding UCS.  When talking about Best Practices the best method may be to keep recommendations in small digestable chunks. Since no two DC environments are identical, one man's best practice, may be another's man's bottleneck .  Alot of your design considerations also depend on applications, expected load and infrastructure capabilities.  'll shed some advice on a few topics you can take into consideration for your design.
Let's start from our blades and work our way up.
Chassis Uplinks
Assuming you have up to 8 blades in your single Chassis you'll need to first decide how many uplinks you want to use between your Chassis and Fabric Interconnects (FIs).  Supported configurations are 1, 2, or 4 uplinks which provide a max oversubscription of 2:1, 4:1 and 8:1.  Depending on your application bandwidth requirements you may be fine with 2 Uplinks per IOM.  Since you'll get the first 8 ports (6120) or 16 ports (6140) included with the base system, there's nothing stopping you from utilising all four.  People sometimes have reservations about oversubscription, but I can tell you a properly engineered system should be designed with overscription as part of the picture.  Keep in mind this connection a lossless 10G connection.  No TCP retransmissions or dropped frames, which make it much more efficient than standard ethernet.
Fabric Interconnect Uplinks - Ethernet
Here's where you will provide all the LAN connectivity for your UCS system into your distribution or core networking layer.  Again, the # of uplinks from each FI will depend on load, but most people start with at least 2 x 10G uplinks in a port channel to their upstream switches and add additional member interfaces as the links near saturation.  Without knowing what the upstream connecting switches your using (2960's or other) willl dictate how many and what toplogies you can take advantage of.  Example - If you had Nexus 5000 or Nexus 7000 switches upstream you could implement virtual port channels offering dual active links from each FI to each switch.  FI's act as your access layer switches, hopefully you have some distribution-level switches to connect into.
Fabric Interconnect Uplinks - Fibre Channel
As of the latest release of software (1.4.x) you can take advantage of SAN Port Channels and VSAN trunking.  Previously FC uplinks were treated individually, and allowed to carry only a single VSAN.  This new feature greatly increases the redudancy and efficiency of your SAN fabric being able to bundle 2 or 4 links towards your MDS core switches in a SAN port channel.  With the MDS 9148's support for 8GB FC, you can also minimize the amount of FC uplinks needed from your FI's assuming you have the six port 1/2/4/8 FC modules (Part ID N10-E0060) in your FIs.  If only the only option is 4GB FC due to cost or equipment you can use 2 or more FC uplinks to provide the necessary bandwidth. In terms of VSANs, I've seen two main designs.  One keeping a single VSAN on each Fabric Interconnect - with each FI uplinked to a single SAN Switch.  The other is creating both VSANs on each FI, with each FI uplinking to both SAN Switches.  Most people prefer to keep the SAN fabrics completely separate, but having the both VSANs exist on each FI ensures you maintain access to each fabric in the event a single FI should fail.  Zoning is another intersting topic.  Under legacy practices it was common to create a zone for each host.  This ensured maximum security for hosts & targets.  Since UCS allows us to pre-define WWN pools for quick deployments & provisioning, I see more & more customers implementing interface zoning. This allows you to zone the incoming FC interface coming from UCS with the designated storage targets.  The major advantage of this is once you configure this, you don't have to touch zoning to faciliate the deployment of any UCS hosts.  Drawback is that you now rely on the LUN masking to protect your data.  The management overhead required to deploy zones for each & every new host can be significant, so consider each benefit carefully.
EMC Celerra NAS
This is an interesting topic regarding 1.4 software capabilities.  If your EMC Celerra is going to be mainly for UCS access, you can direclty attach the NAS to the FI's.  This is a new feature called Direct Attach Storage. This allows you to connect a NAS such as the Celerra as an Appliance Port to the FI.  If the NAS is intended to server many other devices then you would be better off connecting it to your core network and allow UCS to access it via the uplinks.
I'll update this thread as I have other suggetions.
Regards,
Robert