Exchange 2010 SP3 DAG Flip Flop File Share Witness Issue

Question

Hi All,
We are facing one strange issue with Exchange 2010 SP3.
Our environment is as below
a single DAG spanned accross 2 AD sites with 6 MBX server.
Each site have 3 MBx server.
File Share Witness is on Hub server in primary site.
Note: The DAG IP is from Primary DC VLAN and which cant be streched accross DC due to network restrictions/limitations.
It was working fine till date.
But, yesterday onwards we are facing issue.
All DB dismounted and DAG name disappear from EMC and while doing refresh its giving error as below
Exception type: Microsoft.Exchange.Data.DataValidationException
Exception message: Could not convert property WitnessServer to type UncFileSharePath. Error while converting string 'vesthub700.mydomain.com\DAG1.mydomain.com' to result type Microsoft.Exchange.Data.UncFileSharePath: "vesthub700.mydomain.com\DAG1.mydomain.com"
is not an acceptable path. You must use an absolute, local, long file path, that doesn't contain '~', or a UNC path. Parameter name: path
We are not able to execute any cmdlets related to databaseavailabilitygroup from EMS and its giving same error.
Failover Cluster DAG resource is Flip/Flip continuously.
Its only become stable after reboot of the hub server holding FSW and the 1st mailbox server.(Cluster Owner)
We are even not able to move cluster group to different server.
Performed ExBPA no such issues found which related to DAG.
Performed clsuter validation.
Only generic cluster events in Mailbox servers.
Observation:
Its only become stabe if we remove the msExchFileShareWitness Value for DAG1 using ADSIEdit.
And the problem recurs if we set it back again.
Any help or suggestion is appreciated.
Thanks,
soumen
Soumen Ghosh

Answer 1

Hi Belinda,
Thanks! I have checked the Mailbox Server Application Log and found few 2155 Event Id generated at the time of the issue occured.
Process Microsoft.Exchange.ServiceHost.exe () (PID=9312). Configuration object CN=DAG1,CN=Database Availability Groups,CN=Exchange Administrative Group (FYDIBOHF23SPDLT),CN=Administrative Groups,CN=xxxxxx,CN=Microsoft Exchange,CN=Services,CN=Configuration,DC=mydomain,DC=com
read from DCTDK35.mydomain.com failed validation. DataValidationException will be thrown. Set event logging level for Validation category to Expert to get additional events about each failure.
One update is that the DAG is stable now after we set an alternate FSW last night.
Regarding the 2nd IP assignment for DAG for the 2nd subnet, we have gone through technet and other sites which refering to add the same from EMC or EMS.
Is it sufficinet to simply adding the 2nd IP to DAG from EMC or EMS? Or do we need to take any other special consideration.
Also, adding another IP it will create one more IP address resource to Failover Cluster, but I belive at a time both the IP wont be online. Do we need to keep it offline manually or its automatically taken care by Exchange.
Please suggest.
Also, regarding executing the CollectOverMetrics.ps1 we are following
http://www.msexchange.org/articles-tutorials/exchange-server-2010/monitoring-operations/monitor-dag-database-failover.html
but failed to execute. Can you suggest the correct parameter.
Thanks,
Soumen
Soumen Ghosh