3 Node DAG - Failover??

I have a Dag setup,
Main Site:
MB1 (2010 SP1)
MB2 (2010 SP1)
3 node CAS array
DR Site:
MB_DR (2010 SP3) - Running as CAS
I am working through installing SP3, DR site has been completed, all the CAS servers have been completed. I set MB2 to maintenance mode using .\startdagmaintenance.ps1 - confirmed all mailboxes were off the server.
I started installing Windows patches and rebooted several times, then one round of patches took out the network interfaces, this meant the server lost access to the SAN and the cluster service failed. This caused the mailboxes all to initiate a failover
to the DR server!
Now couple of questions:
1) Since the DAG member was in maintenance mode should the fact the cluster service failed cause the DAG to initiate a failover? I have to patch MB1 and know the network is going to do the same thing so want to stop this happening again.
2) The DAG is in DAC mode, I didn't think that it should auto failover to the DR site?
Thanks
Richard

Hello Rhoderick
Thanks, I have been reading through your posts. Below is the output of running the Packets Received discarded script
Processing server: OLD 2003 to be decommisioned
InstanceName
CookedValue
hp nc371i multifunction gigabit server adapter _2
0
ms tcp loopback interface
0
Uptime In Days: 673
Install Date 20/12/2007 12:24:29
Processing server: CAS01 (this holds the file share Witness) - Virtual
vmxnet3 ethernet adapter _2
0
vmxnet3 ethernet adapter
143757
Uptime In Days: 6
Install Date 18/11/2011 12:07:20
Processing server: CAS02 - Virtual
vmxnet3 ethernet adapter _2
0
vmxnet3 ethernet adapter
0
Uptime In Days: 5
Install Date 18/11/2011 15:00:01
Processing server: CAS03 (Physical)
hp nc373i multifunction gigabit server adapter _38
0
hp nc373i multifunction gigabit server adapter _39
0
Uptime In Days: 3
Install Date 10/01/2012 13:44:22
Processing server: MB_DR (Physical)
xsigo vnic dag_hb_254 [psdrd01]
0
hp nc373i multifunction gigabit server adapter _42
0
hp nc373i multifunction gigabit server adapter _43
0
xsigo vnic drmapiv253 [psdrd01]
0
xsigo vnic driscsi237 [psdrd01]
0
Uptime In Days: 20
Install Date 12/01/2012 11:10:13
Processing server: MB1 (Physical)
hp nc373i multifunction gigabit server adapter _42
0
hp nc373i multifunction gigabit server adapter _43
3171
xsigo vnic iscsi_b1 [psd02]
0
xsigo vnic hbv251_a [psd01]
0
xsigo vnic iscsi_a1 [psd01]
0
xsigo vnic hbv251_b [psd02]
0
Uptime In Days: 25
Install Date 28/11/2011 15:27:52
Processing server: MB2 (Physical)
hp nc373i multifunction gigabit server adapter _43
0
xsigo vnic hbv251_a [psd01]
0
xsigo vnic iscsi_a1 [psd01]
0
xsigo vnic iscsi_b1 [psd02]
0
xsigo vnic hbv251_b [psd02]
0
Uptime In Days: 2
Install Date 29/11/2011 09:46:09
The Results of the Check Event 1135 is
MB_DR :- 50
MB1:- 103
MB2:- 102
There has been an issue for a while (I have only worked here 2 months, but logs show going back a year and increasing frequency where the mailboxes are failing over to the other node, normally 8 databases are split 4 on each MB1/MB2 and the Mailboxes usually
move to MB2. I have increased the timeout values of the cluster and we haven't seen the Databases move in over a week (except when patching MB2).
It looks like there are lots of errors for CAS01's Production vNIC - could this be an issue seeing as it is the FSW?
Thanks for your help, if I can get this cracked it could help me pass my probation period :)

Similar Messages

How does Dynamic Quorum work for a two Node DAG

Hi All,
I have a two node DAG with a FS witness server. One of the node is 'down' (I have kept it like that), cluster has quorum and all services are online.
What I'm trying to understand is if a node's State=Down, isn't the Dynamic Quorum Group Manager suppose to trigger and set the DynamicWeight to '0' for that server.
In my case its not doing so, please let me know if this the way it is, or something is not quite right and I need to fix it.
Troubleshooting info below:
PS C:\Windows\system32> Get-ClusterNode | ft name, dynamicweight, state, nodeweight,id -AutoSize
Name DynamicWeight State NodeWeight Id
exch1 1 Down 1 1
exch2 1 Up 1 2
PS C:\Windows\system32> (Get-Cluster).WitnessDynamicWeight
1
PS C:\Windows\system32> Get-ClusterResource
Name State OwnerGroup ResourceType
Cluster IP Address Online Cluster Group IP Address
Cluster Name Online Cluster Group Network Name
File Share Witness (\\fs1... Online Cluster Group File Share Witness
(Validation test)
Validate Quorum Configuration
Description: Validate
that the current quorum configuration is optimal for the cluster.
Validating cluster quorum settings.
Witness Type: File Share Witness
Witness Resource: \\fs1.contoso.com\dag1.contoso.com
Cluster managed voting: Enabled
Voter Name
State
Assigned Vote
Current Vote
File Share Witness (\\fs1.contoso.com\dag1.contoso.com) (\\fs1.contoso.com\dag1.contoso.com)
Online
1
1
exch1
Down
1
1
exch2
Up
1
1
This quorum model will be able to sustain failures of 1 node(s) if the file share witness remains available
and 0 node(s) when the file share witness goes offline or fails.
This quorum configuration can be changed using the Configure Cluster Quorum wizard. This wizard can be started from the Failover
Cluster Manager console by selecting the cluster name in the left hand pane, then in the right "actions" pane selecting "More Actions..." and then selecting "Configure Cluster Quorum Settings...".
When all servers were up
node/2+1 = 2/2+1=2 required for quorum and we have 3 votes
When 1 server gone 1/2+1=1 quorum should recalculate to this. But its still considering 3 votes out of 1down server+1up server+1witness. Ideally I should be able to loose the witness too aftersome time and still maintain quorum(unlike what
the validation test is saying).
Regards,
Satyajit
Please “Vote As Helpful”
if you find my contribution useful or “Mark As Answer” if it does answer your question. That will encourage me - and others - to take time out to help you.

Hi Simon,
Thanks for your response.
I have done some study, testing and what I figured out is this.
Split brain syndrome is prevented by always requiring a majority of the DAG members (and in the case of DAGs with an even number of member, the DAG witness server) to be available and interacting for the DAG to be operational.
All DAGs with an even number of members must use a witness server.
Hence a 3 node cluster behaves differently than a 2 node. Exchange 2013 DAG kind of forces you to have a witness server always.
You can specify only a name for the DAG and leave the Witness server and
Witness directory fields empty. In this scenario, the task will search for a Client Access server that doesn't have the Mailbox server role installed. It will automatically create the default witness directory and share on that Client Access
server and configure the DAG to use that server as its witness server.
You can
'overridde the quorum configuration using Windows2012 Failover Cluster Manager', however using it to modify a DAG is not recommended.
If you open Failover Cluster Manager in Administrative Tools, you’ll find the Database Availability Group (DAG), cluster networks and so on. Don’t try to manage the DAG
using the Failover Cluster Manager, as this isn’t supported. The Exchange Management Console (EMC) or the Exchange Management Shell (EMS) are the only ways to manage the DAG.
Unless you’re doing a DC switchover and/or being assisted by Microsoft Support services (premier)
Now back to the point:
When we are left with 2 nodes and 1 witness server for Exchange HA. The Dynamic Quorum functionality kind of stops dealing with it. As 2nodes/2+1=2votes this means we need to have atleast 2 votes to have quorum.
So if we assume Dynamic Quorum triggers and removes 2 votes, 1 from Witness and 1 from nodeB.
Then the new formula we have is 1node/2+1=1vote which would mean this would allow us to loose both the witness and the nodeB. And nodeA will be the last man standing as in this
article.
However having this scenario in a two node cluster brings in the split-brain problem. As if there is a full disconnect of nodeA site and nodeB+Witness can talk, they form quorum , nodeB mounts the database. Which is undesirable.
Hence Dynamic Quorum keeps the votes to 3 in a 2nodes+1witness scenario contrary to what is expected and in turn keeps everything running fine till we have 2votes available, just like 2010,Windows2008 days.
Regards,
Satyajit
Please “Vote As Helpful”
if you find my contribution useful or “Mark As Answer” if it does answer your question. That will encourage me - and others - to take time out to help you.

Exchange 2010 DAG Failover does not works

Hi Experts,
I have a Exchange 2010 setup in a DAG environment. We have 2 MBX servers in the main site and 1 MBX server in the DR site , all part of one DAG. We have 2 HUB/CAS servers in the main site and 1 HUB/CAS server in the DR site.
Recently we had to do our BCP test for audit purpose. We had issues in doing failover to the DR site and below is the error faced.
Please advise urgently on the possible causes and resolution steps for it as we need to do this test again on the coming weekend.
"EvictDagClusterNode got exception Microsoft.Exchange.Cluster.Replay.AmClusterEvictWithoutCleanupException: An Active Manager operation failed. Error An error
occurred while attempting a cluster operation. Error: Evict node 'sme-ho-mbx01' returned without the node being fully cleaned up. Please run cluster.exe node <NodeName> /forcecleanup to complete clean up for this node.. ---> System.ComponentModel.Win32Exception:
The wait operation timed out"
So, basically one of the MBX server was not evicting from the Cluster due to which failover did not work.
Would appreciate some urgent thoughts for the possible resolution.
regards
abubakar
Md.Abubakar Noorani IT Systems Engineer Serco Ltd.

Hi,
Yes, you can run the Stop-DatabaseAvailabilityGroup without shutting down the Mailbox server. During the process of DAG failover to DR site, the Stop-DatabaseAvailabilityGroup cmdlet should be run against all servers in the primary datacenter. If the Mailbox
server is unavailable but Active Directory is operating in the primary datacenter, the Stop-DatabaseAvailabilityGroup command with the ConfigurationOnly parameter must be run against all servers in this state in the primary datacenter.
And please note that the Stop-DatabaseAvailabilityGroup cmdlet can be run against a DAG only when the DAG is configured with a DatacenterActivationMode value of DagOnly.
Based on the error message, it seems that you should run the cluster node nodename /forcecleanup cmdlet against the specified node in the main site. Have you tried this to check the result?
Best regards,
Belinda
Belinda Ma
TechNet Community Support

Losing quorum on 3 node dag with the lose of only 1 node.

We have exchange 2010 latest version and rollups as of June 30, 2014. running on Windows 2008R2 also patched on June 30th. Two nodes in Seattle (primary DC) and 1 node in Spokane (backup DC) active mailboxes are in Seattle site only when running normally.
All 3 servers run MB, CAS and HUB. We are about to LB the CAS on an F5 BigIP box, but that is not in place yet.
We had discovered a few months ago that something is amiss when we were doing updates on the 1 server in Spokane. Normally it would be no issue to suspend replication, patch the server, restart it and resume replication. This would not cause any outages.
Now when we restart that server, or any of the 3 server for that matter, the quorum is lost and the cluster service is terminated on the 2 server that were still up and running when the other was rebooted.
I've been researching this problem and did discover that the 2nd server in Seattle had no vote and have resolved that issue, but the problem continues.
Strangely enough, when looking at Failover cluster Manager, the warning is there next to Quorum configuration that the loss of a node will bring the cluster down, YET when running validation testing, the report says the cluster can survive the failure of
a node. It's contradictory. So we are looking to find out what is the issue. Once the server reboots and is back up, I can manually restart the cluster service on the 2 nodes that were not rebooted and the infrastructure comes up just fine.
However, this is not what a 3 node cluster should do. Any ideas where else to look to resolve this problem?

Thanks. I did remove a few lines that I hope you were not looking for.
some background: Originally this was a 2 node DAG, just in Seattle, a third node was added in Spokane about a year ago. At the 2nd member was sea-mail-02, but it was removed (properly) and sea-mail-03 was added (this was about 5 months ago)
It was not until we did our 3rd DR test that we had some problems getting the DAG to mount in Spokane. We got some help from MS pro support and got things working. Lots of cluster commands were issued in PS and since then, we are having this issue.
RunspaceId                             : 2b8c2445-86c4-41d9-9bbb-cf5a4196a586
Name                                   : DAGWPI
Servers                                : {SEA-MAIL-03, SPOK-MAIL-01, SEA-MAIL-01}
WitnessServer                          : vc-sea01.wpi.com
WitnessDirectory                       : c:\dagwpiwitness1
AlternateWitnessServer                 : spok-vc01.wpi.com
AlternateWitnessDirectory              : c:\dagwpiwitness1
NetworkCompression                     : InterSubnetOnly
NetworkEncryption                      : InterSubnetOnly
DatacenterActivationMode               : DagOnly
StoppedMailboxServers                  : {}
StartedMailboxServers                  : {sea-mail-01.wpi.com, sea-mail-03.wpi.com, SPOK-MAIL-01.wpi.com}
DatabaseAvailabilityGroupIpv4Addresses : {10.ww.xx.47, 10.yy.zz.47}
DatabaseAvailabilityGroupIpAddresses   : {10.ww.xx.47, 10.yy.zz.47}
AllowCrossSiteRpcClientAccess          : False
OperationalServers                     :
PrimaryActiveManager                   :
ServersInMaintenance                   :
ServersInDeferredRecovery              :
ThirdPartyReplication                  : Disabled
ReplicationPort                        : 0
NetworkNames                           : {}
WitnessShareInUse                      :
AdminDisplayName                       :
ExchangeVersion                        : 0.10 (14.0.100.0)
DistinguishedName                      : CN=DAGWPI,CN=Database Availability Groups,CN=Exchange
WhenChanged                            : 6/29/2014 1:46:45 PM
WhenCreated                            : 12/27/2012 12:15:49 PM
WhenChangedUTC                         : 6/29/2014 8:46:45 PM
WhenCreatedUTC                         : 12/27/2012 8:15:49 PM
OrganizationId                         :
OriginatingServer                      : sea-dc01.internal.westernpeterbilt.com
IsValid                                : True

Outlook connectivity and mail flow issue on Exchange 2010 3 node DAG

Hi,
I have 3 node DAG, 2 (NODE A, NODE B) in Production site and 1(NODE C) in DR site. but just 1 AD Site.
when I have the active databases on NODE B, everything works fine. no issues to report.
but when I move the databases on NODE A, outlook gets connected with trying to update folders. Furhter, emails will be in outbox and no new emails will be delivered to the users. After an hour or so emails get delivered from outbox.
Even after supplying the credentials OWA doesnt login. just tries to load.
No issues when I move the servcies on NODE C.
ANy idea what could be the issue?
Thanks,
Vinod

Hi Vinod,
According to your description, there is one-hour delay in the process of sending and receiving emails.
To check if it’s caused by Autodiscover service, we can use the tool Test Email AutoConfiguration：
1. Hold CTRL and Click the outlook Icon in the system tray and select “Test Email Auto Configuration”.
2. Deselect the “Use Guessmart” and “Secure Guessmart Authentication” and click Test.
3. Check SCP value returned
For more information, please refer to the following article:
http://blogs.technet.com/b/exchdxb/archive/2012/05/10/troublshooting-autodiscover-exchange-2007-2010.aspx
http://support.microsoft.com/kb/2663435
http://support.microsoft.com/kb/262052
If connectivity is well, the issue is mostly likely caused by the Exchange mail submission service.
We can check if the service work properly. And the issue may be not related to the Hardware Load Balancer. Because the emails are stuck in the client side.
Thanks,
Angela Shi
TechNet Community Support

Exchange 2010 Decommisioning a DAG member from a 3 node Dag

Hi Guys,
What the safest way to remove one server from a 3 Node Dag.
Currently I am planning the following steps:-
Remove Exchange server MAIL03 from DAG
1.Start up Exchange Management Console
2.Go to Organization Configuration > Mailbox > Database Management;
3.if mail boxes are mounted on MAIL03, Select the database where ” Mounted on Server” reads the MAIL03;
3.1.Select Move Active Mailbox Database;
3.2.Select the Mailbox server i.e. MAIL01\MAIL02 to host the mailbox database copy and select Move;
5.When the move has finished, select the database copy hosted on the MAIL03 in the lower pane. There, select Remove.
6.Remove the MAIL03 server from the DAG. Select tab Database Availability Groups;
7.Select the DAG the MAL03 server is a member of and select Manage Database Availability Group Membership;
8.Select the server and click the red cross to remove it from the list. Click Manage to proceed with the actual removal;
9.When finished the mailbox server is no longer member of the DAG.
10.Rebalance Mailbox Databases in DAG.
What do you guys recommend?
Also after the removal of the DAG node, what the best way to reduce the Passive Database/Transaction logs? our current setup is as follows:-
I want to remove the drives L,M,N,O
Thanks
Jitinder

Hello,
Based on your description.
If you want to remove drives only from that server and leave databases working on the rest of nodes:
1. Remove database copy on that server.
2. Remove drives.
It is not required to move mailboxes from these databases. In this scenario you will not be able to re-add copy of these databases because all nodes need to have the same drive
letters.
If you want to remove L, M, N, O from all servers:
1. Move mailboxes from these databases.
2. Remove them (copies and empty dabatabes).
3. Remove drives.
There is no need to remove server from DAG if you just want to remove database copy from one node.
Hope it helps,
Adam
www.codetwo.com
If this post helps resolve your issue, please click the "Mark as Answer" or "Helpful" button at the top of this message. By marking a post as Answered, or Helpful you help others
find the answer faster.

Database DAG Failover

We have a 2 node DAG cluster. The database are evenly split between the two servers. The other day all databases moved to one of the DAGs. Normally when this happens I expect to see a server reboot or some sort of failure. But there
was none. I could not find anything in the event logs to advise as to what happens.
Does anyone know where else I should be looking and if there is any way to alert when databases move from one node to the other?
Thanks
Paul
Paul Glickenhaus

Hi,
If you are checking the event viewer in the following location you can find the associated event entries corresponding to DAG
Event Viewer>Applications and Services logs>Microsoft>Exchange>High Availability>
Regards from ExchangeOnline.in|Windows Administrator Area | Skype:[email protected]

Outlook looses connectivity after DAG Failover

Hi team,
Im running a lab environment with 2xCAS (WNLB) and 2xMBX (DAG) servers. Since it's a lab environment I've set the DC as my FSW. All Virtual Directories are pointing to 'mail.contoso.com' pointing to the two CASs which is WNLB'ed.
To test the DAG Failover I set up a Database(MDB01) on MBX Server 1 which is active, made a passive replica on MBX Server 2. I shut down Server 01, which holds the active databases mounted. meanwhile two clients were running and both were connected. Once
the server got shutdown, both clients(Outlook 2013) went to 'Trying to connect to server' status.
Meanwhile checking the powershell on the Mbx Server 02, i noticed that the database was mounted. Went back to clients and open'ed up OWA, working as normal. However, the outlook client is still stuck in the 'Trying to connect to server' status. And
i opened up the 'Connection Status' and tried to 'Reconnect' but still no luck. I had to close and open up outlook to get back into the 'Connected' mode.
Any idea why this might be happening?
*Since unlike Exchange 2010's CAS Array architecture is no more there I didn't made any changes to any RPCClientAccess.... attribute. Both mail.contoso.com and autodiscover.contoso.com are pointed to the VIP of the CAS WNLB.
Cheers!

Hello,
After you configure a single namespace, you need to restart outlook to connect the newly configured namespace.
Before DAG Failover, I recommend you use netstat -ano | findstr ":80" command to check your outlook connect to CAS server.
In order to check the issue, please do DAG failover again, and then use netstat -ano | findstr ":80" command to check your outlook connect to CAS server.
Additional article for your reference. (Note: Microsoft is providing this information as a convenience to you. The sites are not controlled by Microsoft. Microsoft
cannot make any representations regarding the quality, safety, or suitability of any software or information found there. Please make sure that you completely understand the risk before retrieving any suggestions from the above link.)
http://exchangeserverpro.com/exchange-2013-client-access-server-high-availability/
If you have any feedback on our support, please click
here
Cara Chen
TechNet Community Support
Hi Cara,
Yes, here's the result from the query.
[IMG]http://i57.tinypic.com/im01na.png[/IMG]
But whenever i restart the client it gets connected in no time. I'm confused in why after a manual DAG failover this happens.
Manual failover - I shut the active copy, the passive copy was also down for around 15 mins, booted up the passive server, it said Databases as Disconnected and Active Manager error in getting them mounted up and had a 99234343231 of CQL. So
to mark the server active did a 'net start clussvc /forcequorum'. This got the databases mounted up. But clients, they doesn't get connected automatically :(
UPDATE - The client got connected after like
15 mins. :s

3 Node DAG with DR site

Hi team,
I have a primary site with 2 CAS (WNLB) and 2 MBX servers with DAG configured. I want to extend this to include a DR site with 1 CAS and 1 MBX server. FSW on Primary site is set to CAS server.
If the PRimary site goes down completely, is it correct that the DR DAG member will not come up since there's no majority?
related to Q1, what steps do i need to follow to get the DR DAG node up and running?
If I set up a secondary FSW on the DR Site CAS, can I get majority vote and get the DR DAG node up automatically?
What's DAC's role here?
Thank you.
Chris!

Hi Crypto,
1.If the PRimary site goes down completely, is it correct that the DR DAG member will not come up since there's no majority?For a 3 server DAG it will be offline if two servers
are lost. You must manually intervene at this point. we discourage automatic fail-over of DAG to DR manual fail-over is better. Make sure DAG is in DAC mode.
2. related to Q1, what steps do i need to follow
to get the DR DAG node up and running?
Install a server with all roles and add the MBX server to the DAG install an ADC in DR site and point Exchange server in DR to the ADC in DR using Set-ADServerSettings. Configure alternate witness(FSW in a 3 member DAG check next question's answer).
Please check this to configure alternatewitness server and enable DAC on the DAG. http://technet.microsoft.com/en-us/library/dd297934(v=exchg.150).aspx
3.If I set up a secondary FSW on the DR Site CAS, can I get majority
vote and get the DR DAG node up automatically?
In a 3 member DAG FSW is being ignored. Read this. You have to manually bring the DR Exchange online.
4.What's DAC's role here?
We enable DAC to prevent split-brain when data center goes offliine.
http://technet.microsoft.com/en-us/library/dd979790(v=exchg.150).aspx
Please read this http://social.technet.microsoft.com/wiki/contents/articles/23333.exchange-2013-dag-datacenter-failover-and-disaster-recovery.aspx
Please read this to understand the HA changes in CU1
http://blogs.technet.com/b/scottschnoll/archive/2013/04/02/high-availability-changes-in-exchange-server-2013-cumulative-update-1.aspx
Thanks, MAS
Please mark as helpful if you find my comment helpful or as an answer if it does answer your question. That will encourage me - and others - to take time out to help you.
Hi MAS,
Thank you and sorry for the delay. Can I have the same setup with 3 nodes under a single site and keep one node as a DR location (But not in an AD Site)?
Thank you,
Chris.

Exchange server 2010 DAG failover

Hi team ,
We have configured exchange server 2010 in DAG environment .
We have added 3 mailbox server in DAG . but my active mailbox copy failed the database are failover to another passive copy server & status is mounted . after failover emails service are not working.
Note : We have also configured NLB on CAS server .All exchange servers placed in single AD site
Please suggest

Hi
So are all your exchange servers the same? settings?
have you tried to fail the DB back to its original server? all services started on the other server?

Exchange 2013 DAG - Failover Cluster Warnings

I have two new Exchange servers (PROD1 and DR1) that I recently updated to CU8 in preparation for going into production. After reboots, I noticed that I had a warning on the "Server Manager" window.
PROD1 lists itself, DR1, and DAG1 as servers though it marks DAG1 as not accessible.
DR1 lists only itself and PROD1. It does not list DAG1.
PROD1 is generating a ton of failover cluster errors (1205, 1254, 1069, 1044) while DR1 is clean.
In the Failover Cluster Manager of each server, the object of DAG1 is listed as "Offline". It has two IPs - one for each of the Exchange servers' subnets (this is where I think I screwed up). The IP for PROD1's subnet is 10.2.8.131
and is "Failed" and the IP of DR1 is 10.4.8.131 and is "Offline". The IP of PROD1 is 10.2.8.132 and DR1 is 10.4.8.132.
Attempts to bring the cluster object online fail "due to the failure of one or more provider resources".
The instructions I followed to set up the DAG did not mention the need for multiple NICs. I now think this is incorrect and would be willing to use separate subnets and NICs for the DAG if necessary. (I could use 10.2.12.X and 10.4.12.X instead.)
I am currently still able to fail the database copy between DR1 and PROD1 without any problems.

Hi Jkm,
As you said :PROD1 is generating a ton of failover cluster errors (1205, 1254, 1069, 1044) while DR1 is clean.
I suggest you post failover cluster errors to
[email protected] for our troubleshooting.
If there are any questions regarding this issue, please be free to let me know.
Best Regard,
Jim
Please remember to mark the replies as answers if they help, and unmark the answers if they provide no help. If you have feedback for TechNet Support, contact [email protected]
Jim Xu
TechNet Community Support

DAG Failover Cluster Service Errors

Hi all,
I have six Exchange 2013 SP1 mailbox servers installed on Server 2008 R2 configured in two seperate DAG's (3 and 3).
I'm using a single NIC on all the servers for replication and server connectivity.
I pre-staged the DAG computer object in AD (and assigned permissions), and configured DNS entries for both (confirmed and resolvable). I also configured the DAG network subnet and DAG IP address via EAC.
At the moment I have active DB's on each server and then a passive copy on each other server, and all instances are healthy.
However, when I look in the WFC on each server, there are a ton of errors in the cluster event logs. All of the nodes show up as green, along with the cluster network, but the cluster shows "The cluster network name is not online".
Cluster resource 'Cluster Name' in clustered service or application 'Cluster Group' failed.
Cluster network name resource 'Cluster Name' cannot be brought online. The computer object associated with the resource could not be updated in domain 'contoso.com' for the following reason:
Unable to obtain the Primary Cluster Name Identity token.
The text for the associated error code is: An attempt has been made to operate on an impersonation token by a thread that is not currently impersonating a client.
The cluster identity 'DAG$' may lack permissions required to update the object. Please work with your domain administrator to ensure that the cluster identity can update computer objects in the domain.
I've removed and re-added DNS records, and double checked the permissions on the CNO object. What else can I do?

Hi,
Its not a good idea to add a static DNS-Record for the DAG - Remove it and on the member holding the PAM, run ipconfig /registerdns.
Martina Miskovic
It also turns out that the CNO objects were never re-enabled by Exchange during the DAG creation process. Once I followed your steps above, and manually enabled the object in AD, the DNS entries auto-populated and everything came online.
Thanks for the help!

Exchange 2010 3 node DAG and split datacenters

Hi all,
I have a Exchange 2010 setup with 3 DAG nodes, 2 servers in Primary site & one in DR site, 2 active directory Sites, one domain controller per site. All roles on each server.
Each server has a single NIC, with the following configs:
Primary Site:
MBX1 IP = 192.168.1.11 - Network Name MAPI-Primary
MBX2 IP = 192.168.1.12 - Network Name MAPI-Primary
Cluster IP = 192.168.1.10
DR Site:
DRMBX1 IP = 192.168.100.11 - Network Name MAPI-DR
Cluster IP = 192.168.100.10
Dag name = DAG1
Under the DAG Networks, I see 3 Networks
DAG1_MAP1-Primary with the 192.168.1.0/24 subnet and the 2 MBX Network IP's in it.
DAG1_MAP1-DR with the 192.168.100.0/24 subnet and the 1 DR MBX Network IP in it.
and one called DAG1_Replication, which has a 10.1.1.0/24 subnet and NO NIC's in it. I think this was an original replication subnet used when there was only 2 nodes in the DAG and a cross over cable was used.
Can I delete the DAG_Replication network from the config?
Should I consolidate the 2 DAG1_MAPI-Primary and DAG1-MAPI-DR networks under one simply called DAG1-MAPI?
Secondly,
When I look at cluster manager on each of the 2 Nodes MBX1/2, I can see the 3 Nodes. Trying to run cluster manager on the DR node, and I can't connect to the DAG1 cluster, get an error saying not ready. Am I right in thinking this should not be
happening? I have a managed firewall, but don't have access too logs, any idea what ports could be dropped from DRMBX1?
Finally,
Cluster manager on the 2 working nodes shows an error:
Cluster Network Name is not online, and the Cluster Core resources panel shows cluster name DAG1 Offline, with ONLY 1 IP address, the DAG VIP (192.168.1.10) being online. Should I see the other 192.168.100.10 DAG VIP as well? Is this also a manifestation
of the firewall?
I have inherited this config, so I am not fully sure what a correct working condition should report look like. Any help on a clearer understanding would be greatly appreciated.
Thanks in advance !!

In answer to your first question, yes, if no server is listed with an IP address in that replication network, you can delete it without affecting the DAG operations.
As for your second question, if your servers are in different subnets, you need at least two networks defined in your DAG. And since your servers are in different Windows sites, I have to assume they are in different subnets. I believe that if
you check them in the EMC, you will only see IP addresses for the local servers in each network. So no consolidation is possible.
On your question about Cluster Manager, don't do anything in Cluster Manager - my personal experience has been that the Exchange toolset is sufficient for all management tasks related to an Exchange DAG. What are you trying to determine? Also,
Microsoft doesn't support a DAG installed across a firewall - either you open all ports between these servers (and all domain controllers in your forest), or you will have an unsupported configuration.
For your final question, I assume DAG1 is your DAG's network name, and those two IPs are the ones for the two datacenters. If this is the case, only one IP address will be active at a time.

Content Index State Failed - Exchange 2010 2 Node Dag

Hello,
Last night I needed to restart one of my Exchange 2010 DAG nodes. I moved all Active Database copies of this node to the other node in the DAG. I then restarted the node with no Active Database copies. When the server came back up I moved
the Active Database copies back onto the node that they were initially on. The node that I restarted now shows the ContentIndexState of all Databases as "Failed". Test-replicationhealth gives the following results:
[PS] C:\Windows\system32>Get-MailboxDatabaseCopyStatus -Server S01
Name                                          Status
CopyQueue ReplayQueue LastInspectedLogTime   ContentIndex
Length    Length                             State
DB01-MS01 (A-B)\S01                     Mounted         0         0
Failed
DB02-MS01 (C-D)\S01                     Healthy         0         0
1/26/2011 9:57:40 AM           Failed
DB03-MS01 (E-G)\S01                     Healthy         0         1
1/26/2011 10:02:38 AM         Failed
DB04-MS01 (H-J)\S01                     Mounted         0         0
Failed
DB06-MS01 (M-N)\S01                     Healthy         0         0
1/26/2011 10:02:25 AM         Failed
DB07-MS01 (0-P)\S01                     Healthy         0         0
1/26/2011 9:57:17 AM            Failed
DB08-MS01 (Q-R)\S01                     Mounted         0         0
Failed
DB09-MS01 (S-T)\S01                     Mounted         0         0
Failed
DB05-MS01 (K-L)\S01                     Mounted         0         0
Failed
DB10-MS01 (U-V)\S01                     Healthy         0         0
1/26/2011 10:01:59 AM          Failed
DB11-MS01 (W-Z)\S01                     Healthy         0         0
1/26/2011 9:57:50 AM           Failed
[PS] C:\Windows\system32>Get-MailboxDatabaseCopyStatus -Server S02
Name                                          Status
CopyQueue ReplayQueue LastInspectedLogTime   ContentIndex
Length    Length                             State
DB01-MS01 (A-B)\S02                     Healthy         0         0
1/26/2011 10:00:36 AM Healthy
DB02-MS01 (C-D)\S02                     Mounted         0         0
Healthy
DB03-MS01 (E-G)\S02                     Mounted         0         0
Healthy
DB04-MS01 (H-J)\S02                     Healthy         0         0
1/26/2011 10:02:29 AM Healthy
DB06-MS01 (M-N)\S02                     Mounted         0         0
Healthy
DB07-MS01 (0-P)\S02                     Mounted         0         0
Healthy
DB08-MS01 (Q-R)\S02                     Healthy         0         0
1/26/2011 10:02:32 AM Healthy
DB10-MS01 (U-V)\S02                     Mounted         0         0
Healthy
DB11-MS01 (W-Z)\S02                     Mounted         0         0
Healthy
DB05-MS01 (K-L)\S02                     Healthy         0         0
1/26/2011 10:02:21 AM Healthy
DB09-MS01 (S-T)\S02                     Healthy         0         0
1/26/2011 10:01:33 AM Healthy
[PS] C:\Windows\system32>test-replicationhealth
Server          Check                      Result     Error
S01          ClusterService             Passed
S01          ReplayService              Passed
S01          ActiveManager             Passed
S01          TasksRpcListener         Passed
S01          TcpListener                  Passed
S01          DagMembersUp           Passed
S01          ClusterNetwork            Passed
S01          QuorumGroup               Passed
S01          FileShareQuorum            Passed
S01          DBCopySuspended            Passed
S01          DBCopyFailed               Passed
S01          DBInitializing             Passed
S01          DBDisconnected             Passed
S01          DBLogCopyKeepingUp         Passed
S01          DBLogReplayKeepingUp       Passed
I appreciate any help. Thank you.
R

[PS] C:\Windows\system32>Get-MailboxDatabaseCopyStatus
Name                                          Status
CopyQueue ReplayQueue LastInspectedLogTime   ContentIndex
Length    Length                             State
DB01-MS01 (A-B)\01                                 Mounted
0         0                                  Healthy
DB02-MS01 (C-D)\S01                               Healthy         0
0           1/28/2011 11:05:45 AM Healthy
DB03-MS01 (E-G)\S01                               Mounted         0
0                                  Healthy
DB04-MS01 (H-J)\S01                                Healthy
0         0           1/28/2011 11:07:50 AM Healthy
DB06-MS01 (M-N)\S01                               Healthy         0
0           1/28/2011 11:02:12 AM Healthy
DB07-MS01 (0-P)\S01                                Mounted
0         0                                  Healthy
DB08-MS01 (Q-R)\S01                               Healthy         0
0           1/28/2011 11:09:08 AM Healthy
DB09-MS01 (S-T)\S01                                Mounted
0         0                                  Healthy
DB05-MS01 (K-L)\S01                                Mounted
0         0                                  Healthy
DB10-MS01 (U-V)\S01                               Healthy         0
0           1/28/2011 11:02:53 AM Healthy
DB11-MS01 (W-Z)\S01                              Mounted         0
0                                  Healthy
[PS] C:\Windows\system32>Get-MailboxDatabaseCopyStatus
Name                                          Status
CopyQueue ReplayQueue LastInspectedLogTime   ContentIndex
Length    Length                             State
DB01-MS01 (A-B)\S02                              Healthy         0
0           1/28/2011 11:08:27 AM Healthy
DB02-MS01 (C-D)\S02                    Mounted         0
0                                  Healthy
DB03-MS01 (E-G)\S02                              Healthy         0
0           1/28/2011 11:10:36 AM Healthy
DB04-MS01 (H-J)\S02                               Mounted         0
0                                  Healthy
DB06-MS01 (M-N)\S02                              Mounted         0
0                                  Healthy
DB07-MS01 (0-P)\S02                              Healthy         0
0           1/28/2011 10:58:23 AM Healthy
DB08-MS01 (Q-R)\S02                              Mounted         0
0                                  Healthy
DB10-MS01 (U-V)\S02                               Mounted         0
0                                  Healthy
DB11-MS01 (W-Z)\S02                              Healthy         0
0           1/28/2011 11:09:45 AM Healthy
DB05-MS01 (K-L)\S02                               Healthy         0
0           1/28/2011 11:09:42 AM Healthy
DB09-MS01 (S-T)\S02                              Healthy         0
0           1/28/2011 11:08:34 AM Healthy

Need DAG Failover/ switchover monitoring email alerts

hello
I want to setup email alerts on exchnage 2010 DAG.
Like when a switchover happened
DAG mailbox failure
Can i set these with windows or i have take any third party utility

Hi,
Sometimes we need to know when a database failover to another server automatically, even though there are no problems for end users. To monitor database failover in DAG, you can check the following article.
Monitor Databases in DAGs
http://letsexchange.blogspot.com/2011/08/monitor-databases-in-dags.html
Note: Microsoft is providing this information as a convenience to you. The sites are not controlled by Microsoft. Microsoft cannot make any representations regarding the quality, safety, or suitability of any software or information found there. Please make
sure that you completely understand the risk before retrieving any suggestions from the above link.
Hope this helps.
Best regards,
Belinda
Belinda Ma
TechNet Community Support

3 Node DAG - Failover??

Similar Messages

Maybe you are looking for