PVLAN Config with UCS and 1000v Question ??

Question

We are attempting to setup a multi-tenant environment where the customer would like each tenant to have a single subnet which is segmented with private vlans. The design calls for three (3) UCS chassis'. Two (2) of the chassis' are fully populated with half-width blades and the third chassis has two half-width blades. The two chassis' that are fully populated will all be ESX hosts with Palo adapters and will operate with 1000v.
  As their current VLAN plan is to have approximately 10 private vlans per tenant/subnet, i'm concerned this design will not scale well at all within a UCS enviroment due to the limitations related to the total number of vnics/vhbas per chassis. I viewed a post where it was indicated that we could bypass the vnic limitation by simply trunking down all VLAN's to the VEM and configure all private-vlans on the 1000v only. This would allegedly alleviate the vnic limitation in a larger multi-tenant environment. Is this a valid and supported design/configuration and/or does this actually work? Or do we instead actually need to create a vnic for every private vlan we want to present to each ESX host as recommended/required in the config guides?

Answer 1

Hey Joey,
  Thanks for the response. I've added this same post to the TAC case but i'll update this discussion with the same for anyone else who may be interested.
  Our customer is still debating their requirement for PVLAN use within this Pod. However, if they choose to move forward, my primary concern is mostly with the UCS configuration related to PVLANs, how the UCS PVLAN configuration differs (if at all) with the integration of the 1000v and if this multitenant setup will cause the Fabric Interconnects to exceed their max VIF count. Related to the UCS configuration, it's my understanding that for every isolated private VLAN we would like to present to a blade, we need to create a separate vNIC in UCS. This customer is attempting to construct an environment based on the FlexPod model where multiple tenants would be present in the environment. Their idea, was to create a single subnet for each tenant and then isolate tier/purpose traffic via layer-2 PVLANs within each tenant subnet. This is where the need for the PVLANs come in. Simply a customer request in their design.
  So how this relates to my primary questions; if for every tenant we have to add 5+ vNICs as they are introduced into the Pod, my understanding is that this will easily cause us to have more than 120 VIFs per chassis in no time. It's my understanding that we have a total of (15*[number of I/O Module uplinks] - 2) VIFs available in total. (I'm assuming this is 118 total VIFs for two fully populated 2104's per chassis???) Currently, the design calls for a combination of ten (12) VIFs (10 vNICs and 2 vHBAs) as a standard on each of the eight blades in each chassis. On top of this would be when we begin adding tenant specific vNIC's for each tenant's PVLANs (If this is the proper/required config). However, I have later read that if we integrate the 1000v's into the environment, the necessity to create a new vNIC in UCS for every isolated PVLAN is no longer in play as all that is required is to trunk down all "parent" vlans to the VEMs and there, at the 1000v level only, we can perform the PVLAN config. It has been recommended to configure this environment similar to the strategy where the upstream switch does not understand/perform PVLANs. Is this correct or would we still need to add the vNIC's to the service profiles in UCS even when integrating the Nexus 1000v's?
  I have yet to really find a document that discusses the use of PVLANs within the UCS environment when implementing Nexus 1000v which would tie all of these questions together.
Thanks,
Eric