Cannot failover cluster

Hi all,
In my environment I have 2 exchange 2013 servers : ex01 & ex02 in 1 DAG01. Recently my ex02 server has problem, some exchange service crash ... I'm still finding reason, maybe because lack of memory ... BTW I have a "Mailbox Database 01":
Get-MailboxDatabaseCopyStatus "Mailbox Database 01"
Name Status ContentIndexState
Mailbox Database 01\EX01 Mounted Healthy
Mailbox Database 01\EX02 Healthy Healthy
When ex02 has problem, something happen with "Mailbox Database 01" copy on both ex01 & ex02, it cannot be mounted on boths. The status of them keep switching : Mounted , Mounting , Initializing , Disconnected... like this:
Get-MailboxDatabaseCopyStatus "Mailbox Database 01"
Name Status ContentIndexState
Mailbox Database 01\EX01 Mounting Failed
Mailbox Database 01\EX02 Initializing Failed
Get-MailboxDatabaseCopyStatus "Mailbox Database 01"
Name Status ContentIndexState
Mailbox Database 01\EX01 Initializing Failed
Mailbox Database 01\EX02 Mounting Failed
Get-MailboxDatabaseCopyStatus "Mailbox Database 01"
Name Status ContentIndexState
Mailbox Database 01\EX01 Disconected Failed
Mailbox Database 01\EX02 Mounting Failed
Until I suspend "Mailbox Database 01" copy on ex02, "Mailbox Database 01" will mount on ex01 successfully, then "Mailbox Database 01" edb file on ex02 has "dirty shutdown" state, and I have to reseed "Mailbox
Database 01" from ex01 to ex02 manually.
Get-ClusterGroup -Cluster EX01
Name OwnerNode State
ClusterGroup ex01 Online
Available Storage ex02 Offline
Get-DatabaseAvailabilityGroup -Status -Identity DAG01 | fl name,primaryActiveManager
Name : DAG01
PrimaryActiveManager : EX01
Please let me know if you need any information.
Thanks for your help.
Jack.

Hi John,
Last time, after re-seed the database status changed to healthy on EX02. I have to note that my DAG can failover before, ex: "Mailbox database 01" can be "mounted" on EX01 , "healthy" on EX02 or vice versa and when one server
is down (I unplug network cable for example), the mailbox database copy on the other server is mounted automatically, when down server back to online , the mailbox database copy on it is resynchronized from the other automatically. Like I said it has just
happened recently.
Recently, even when "Mailbox Database 01" is mounted on EX01 , "healthy" on EX02
Get-MailboxDatabaseCopyStatus "Mailbox Database 01"
Name Status ContentIndexState
Mailbox Database 01\EX01 Mounted Healthy
Mailbox Database 01\EX02 Healthy Healthy
When EX02 has problem, "Mailbox Database 01" copy cannot be mounted on boths, its status keep switching...
During problem I have noticed that there are events appear only on EX02, relating to some exchange programs/services are crashed , ex lastime:
Faulting application name: MSExchangeHMWorker.exe, version: 15.0.712.0, time stamp: 0x5199cd1a
Faulting module name: RPCRT4.dll, version: 6.1.7601.21855, time stamp: 0x4eb4c921
Exception code: 0xc0020043
Fault offset: 0x000000000008aa13
Faulting process id: 0x5cfc
Faulting application start time: 0x01d0067aa7da6ac7
Faulting application path: C:\Program Files\Microsoft\Exchange Server\V15\Bin\MSExchangeHMWorker.exe
Faulting module path: C:\Windows\system32\RPCRT4.dll
Report Id: 432d5040-cdfe-11e4-9bb7-3440b58d323f
Faulting application name: svchost.exe_RpcEptMapper, version: 6.1.7600.16385, time stamp: 0x4a5bc3c1
Faulting module name: ntdll.dll, version: 6.1.7601.17725, time stamp: 0x4ec4aa8e
Exception code: 0xc0000374
Fault offset: 0x00000000000c40f2
Faulting process id: 0x2c0
Faulting application start time: 0x01ce90e16804a9e2
Faulting application path: C:\Windows\system32\svchost.exe
Faulting module path: C:\Windows\SYSTEM32\ntdll.dll
[RpcHttp] An internal server error occurred. The unhandled exception was: System.TypeInitializationException: The type initializer for 'Microsoft.Exchange.Data.Directory.Globals' threw an exception. ---> System.Runtime.InteropServices.COMException: Call was canceled by the message filter. (Exception from HRESULT: 0x80010002 (RPC_E_CALL_CANCELED))
at System.Runtime.InteropServices.Marshal.ThrowExceptionForHRInternal(Int32 errorCode, IntPtr errorInfo)
at System.Management.ManagementScope.InitializeGuts(Object o)
at System.Management.ManagementScope.Initialize()
at System.Management.ManagementObjectSearcher.Initialize()
at System.Management.ManagementObjectSearcher.Get()
at Microsoft.Exchange.Data.Directory.Globals.DetectIfMachineIsVirtualMachine()
at Microsoft.Exchange.Data.Directory.Globals..cctor()
Watson report about to be sent for process id: 19380, with parameters: E12IIS, c-RTL-AMD64, 15.00.0712.024, w3wp#MSExchangeRpcProxyAppPool, M.E.Data.Directory, M.E.D.D.Globals.DetectIfMachineIsVirtualMachine, System.TypeInitializationException, 55de, 15.00.0712.016.
ErrorReportingEnabled: False
[RpcHttp] An internal server error occurred. The unhandled exception was: System.TypeInitializationException: The type initializer for 'Microsoft.Exchange.Data.Directory.Globals' threw an exception. ---> System.Runtime.InteropServices.COMException: Call was canceled by the message filter. (Exception from HRESULT: 0x80010002 (RPC_E_CALL_CANCELED))
at System.Runtime.InteropServices.Marshal.ThrowExceptionForHRInternal(Int32 errorCode, IntPtr errorInfo)
at System.Management.ManagementScope.InitializeGuts(Object o)
at System.Management.ManagementScope.Initialize()
at System.Management.ManagementObjectSearcher.Initialize()
at System.Management.ManagementObjectSearcher.Get()
at Microsoft.Exchange.Data.Directory.Globals.DetectIfMachineIsVirtualMachine()
at Microsoft.Exchange.Data.Directory.Globals..cctor()
and this time, I found one event appear only on EX02 before problem happened :
Log Name: Application
Source: MSExchange Transport Migration
Date: 3/28/2015 10:34:01 AM
Event ID: 2005
Task Category: General
Level: Error
Keywords: Classic
User: N/A
Computer: EX02.mydomain.com
Description:
An unexpected failure has occurred. The problem was ignored but may indicate other problems in the system. Diagnostic information:
at Microsoft.Exchange.Data.Storage.MapiPropertyBag.SaveChanges(Boolean force) at Microsoft.Exchange.Data.Storage.StoreObjectPropertyBag.SaveChanges(Boolean force) at Microsoft.Exchange.Data.Storage.AcrPropertyBag.SaveChanges(Boolean force) at Microsoft.Exchange.Data.Storage.CoreItem.InternalSave(SaveMode saveMode, CallbackContext callbackContext) at Microsoft.Exchange.Data.Storage.Item.SaveInternal(SaveMode saveMode, Boolean commit) at Microsoft.Exchange.Data.Storage.Item.Save(SaveMode saveMode) at Microsoft.Exchange.Migration.MigrationJob.UpdatePoisonCount(IMigrationDataProvider provider, Int32 count) at Microsoft.Exchange.MailboxReplicationService.CommonUtils.ProcessKnownExceptions(Action actionDelegate, FailureDelegate failureDelegate)|Error clearing posion count for job: local move 3:14a96ad4-4506-4ab8-9be7-c0308a72e555:ExchangeLocalMove:Staged:4:[email protected]:Completed:11/26/2013 7:59:11 PM::|Microsoft.Exchange.Data.Storage.ConnectionFailedTransientException|Cannot save changes made to an item to store.|InnerException:MapiExceptionNetworkError:16.55847:3E000000, 18.59943:BE060000BD07000000000000, 0.62184:00000000, 255.16280:BE0600006E2F610000000000, 255.8600:A81D0000, 255.12696:802C8F080869D001000FC882, 255.10648:02000000, 255.14744:BE060000, 255.9624:F2030000, 255.13720:00000000, 255.11672:01000000, 255.12952:00000000010700C000000000, 3.23260:BE060000, 0.43249:000FC882, 4.39153:15010480, 4.32881:15010480, 0.50035:07000000, 4.64625:15010480, 20.52176:000FC88211001010FE000000, 20.50032:000FC8827E17401076040000, 0.50128:00000000, 0.50288:00000000, 4.23354:15010480, 0.25913:76040000, 255.21817:15010480, 0.17361:76040000, 4.19665:15010480, 0.37632:76040000, 4.37888:15010480|Microsoft.Mapi.MapiExceptionNetworkError: MapiExceptionNetworkError: Unable to save changes. (hr=0x80040115, ec=0) Diagnostic context: Lid: 55847 EMSMDBPOOL.EcPoolSessionDoRpc called [length=62] Lid: 59943 EMSMDBPOOL.EcPoolSessionDoRpc exception [rpc_status=0x6BE][latency=1981] Lid: 62184 Lid: 16280 dwParam: 0x0 Msg: EEInfo: ComputerName: n/a Lid: 8600 dwParam: 0x0 Msg: EEInfo: ProcessID: 7592 Lid: 12696 dwParam: 0x0 Msg: EEInfo: Generation Time: 3/28/0415 3:34:01 AM Lid: 10648 dwParam: 0x0 Msg: EEInfo: Generating component: 2 Lid: 14744 dwParam: 0x0 Msg: EEInfo: Status: 1726 Lid: 9624 dwParam: 0x0 Msg: EEInfo: Detection location: 1010 Lid: 13720 dwParam: 0x0 Msg: EEInfo: Flags: 0 Lid: 11672 dwParam: 0x0 Msg: EEInfo: NumberOfParameters: 1 Lid: 12952 dwParam: 0x0 Msg: EEInfo: prm[0]: Long val: 3221227265 Lid: 23260 Win32Error: 0x6BE Lid: 43249 Lid: 39153 StoreEc: 0x80040115 Lid: 32881 StoreEc: 0x80040115 Lid: 50035 Lid: 64625 StoreEc: 0x80040115 Lid: 52176 ClientVersion: 15.0.712.17 Lid: 50032 ServerVersion: 15.0.712.6014 Lid: 50128 Lid: 50288 Lid: 23354 StoreEc: 0x80040115 Lid: 25913 Lid: 21817 ROP Failure: 0x80040115 Lid: 17361 Lid: 19665 StoreEc: 0x80040115 Lid: 37632 Lid: 37888 StoreEc: 0x80040115 at Microsoft.Mapi.MapiExceptionHelper.InternalThrowIfErrorOrWarning(String message, Int32 hresult, Boolean allowWarnings, Int32 ec, DiagnosticContext diagCtx, Exception innerException) at Microsoft.Mapi.MapiExceptionHelper.ThrowIfError(String message, Int32 hresult, IExInterface iUnknown, Exception innerException) at Microsoft.Mapi.MapiProp.SaveChanges(SaveChangesFlags flags) at Microsoft.Exchange.Data.Storage.MapiPropertyBag.SaveChanges(Boolean force)|: ,,
%2
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
<System>
<Provider Name="MSExchange Transport Migration" />
<EventID Qualifiers="49152">2005</EventID>
<Level>2</Level>
<Task>1</Task>
<Keywords>0x80000000000000</Keywords>
<TimeCreated SystemTime="2015-03-28T03:34:01.000000000Z" />
<EventRecordID>10409597</EventRecordID>
<Channel>Application</Channel>
<Computer>IDCEXC002.itl.com</Computer>
<Security />
</System>
<EventData>
<Data> at Microsoft.Exchange.Data.Storage.MapiPropertyBag.SaveChanges(Boolean force) at Microsoft.Exchange.Data.Storage.StoreObjectPropertyBag.SaveChanges(Boolean force) at Microsoft.Exchange.Data.Storage.AcrPropertyBag.SaveChanges(Boolean force) at Microsoft.Exchange.Data.Storage.CoreItem.InternalSave(SaveMode saveMode, CallbackContext callbackContext) at Microsoft.Exchange.Data.Storage.Item.SaveInternal(SaveMode saveMode, Boolean commit) at Microsoft.Exchange.Data.Storage.Item.Save(SaveMode saveMode) at Microsoft.Exchange.Migration.MigrationJob.UpdatePoisonCount(IMigrationDataProvider provider, Int32 count) at Microsoft.Exchange.MailboxReplicationService.CommonUtils.ProcessKnownExceptions(Action actionDelegate, FailureDelegate failureDelegate)|Error clearing posion count for job: local move 3:14a96ad4-4506-4ab8-9be7-c0308a72e555:ExchangeLocalMove:Staged:4:[email protected]:Completed:11/26/2013 7:59:11 PM::|Microsoft.Exchange.Data.Storage.ConnectionFailedTransientException|Cannot save changes made to an item to store.|InnerException:MapiExceptionNetworkError:16.55847:3E000000, 18.59943:BE060000BD07000000000000, 0.62184:00000000, 255.16280:BE0600006E2F610000000000, 255.8600:A81D0000, 255.12696:802C8F080869D001000FC882, 255.10648:02000000, 255.14744:BE060000, 255.9624:F2030000, 255.13720:00000000, 255.11672:01000000, 255.12952:00000000010700C000000000, 3.23260:BE060000, 0.43249:000FC882, 4.39153:15010480, 4.32881:15010480, 0.50035:07000000, 4.64625:15010480, 20.52176:000FC88211001010FE000000, 20.50032:000FC8827E17401076040000, 0.50128:00000000, 0.50288:00000000, 4.23354:15010480, 0.25913:76040000, 255.21817:15010480, 0.17361:76040000, 4.19665:15010480, 0.37632:76040000, 4.37888:15010480|Microsoft.Mapi.MapiExceptionNetworkError: MapiExceptionNetworkError: Unable to save changes. (hr=0x80040115, ec=0) Diagnostic context: Lid: 55847 EMSMDBPOOL.EcPoolSessionDoRpc called [length=62] Lid: 59943 EMSMDBPOOL.EcPoolSessionDoRpc exception [rpc_status=0x6BE][latency=1981] Lid: 62184 Lid: 16280 dwParam: 0x0 Msg: EEInfo: ComputerName: n/a Lid: 8600 dwParam: 0x0 Msg: EEInfo: ProcessID: 7592 Lid: 12696 dwParam: 0x0 Msg: EEInfo: Generation Time: 3/28/0415 3:34:01 AM Lid: 10648 dwParam: 0x0 Msg: EEInfo: Generating component: 2 Lid: 14744 dwParam: 0x0 Msg: EEInfo: Status: 1726 Lid: 9624 dwParam: 0x0 Msg: EEInfo: Detection location: 1010 Lid: 13720 dwParam: 0x0 Msg: EEInfo: Flags: 0 Lid: 11672 dwParam: 0x0 Msg: EEInfo: NumberOfParameters: 1 Lid: 12952 dwParam: 0x0 Msg: EEInfo: prm[0]: Long val: 3221227265 Lid: 23260 Win32Error: 0x6BE Lid: 43249 Lid: 39153 StoreEc: 0x80040115 Lid: 32881 StoreEc: 0x80040115 Lid: 50035 Lid: 64625 StoreEc: 0x80040115 Lid: 52176 ClientVersion: 15.0.712.17 Lid: 50032 ServerVersion: 15.0.712.6014 Lid: 50128 Lid: 50288 Lid: 23354 StoreEc: 0x80040115 Lid: 25913 Lid: 21817 ROP Failure: 0x80040115 Lid: 17361 Lid: 19665 StoreEc: 0x80040115 Lid: 37632 Lid: 37888 StoreEc: 0x80040115 at Microsoft.Mapi.MapiExceptionHelper.InternalThrowIfErrorOrWarning(String message, Int32 hresult, Boolean allowWarnings, Int32 ec, DiagnosticContext diagCtx, Exception innerException) at Microsoft.Mapi.MapiExceptionHelper.ThrowIfError(String message, Int32 hresult, IExInterface iUnknown, Exception innerException) at Microsoft.Mapi.MapiProp.SaveChanges(SaveChangesFlags flags) at Microsoft.Exchange.Data.Storage.MapiPropertyBag.SaveChanges(Boolean force)|: ,,</Data>
</EventData>
</Event>
I also noticed that "Mailbox Database 01" edb file on EX02 has "dirty shutdown" state (yes, last time too)
eseutil /mh “path to Mailbox Database 01 edb file on EX02″ and result :
State: Dirty Shutdown
Log Required: 4656967-4657077 (0x470f47-0x470fb5)
Log Committed: 0-4657078 (0x0-0x470fb6)
I'm going to reseed "Mailbox Database 01" copy on EX02 tonight, should I repair it with eseutil /r to get it into clean shutdown state before ?
How can I check if my DAG configuration is fine ? Is result from Get-Clustergroup before ok ?

Similar Messages

Cannot add multiple members of a failover cluster to a DFSR replication group

Server 2012 RTM. I have two physical servers, in two separate data centers 35 miles apart, with a GbE link over metro fibre between them. Both have a large (10TB+) local RAID storage arrays, but given the physical separation there is no physical shared storage.
The hosts need to be in a Windows failover cluster (WSFC), so that I can run high-availability VMs and SQL Availability Groups across these two hosts for HA and DR. VM and SQL app data storage is using a SOFS (scale out file server) network share on separate
servers.
I need to be able to use DFSR to replicate multi-TB user data file folders between the two local storage arrays on these two hosts for HA and DR. But when I try to add the second server to a DFSR replication group, I get the error:
The specified member is part of a failover cluster that is already a member of the replication group. You cannot add multiple members for the same cluster to a replication group.
I'm not clear why this has to be a restriction. I need to be able to replicate files somehow for HA & DR of the 10TB+ of file storage. I can't use a clustered file server for file storage, as I don't have any shared storage on these two servers. Likewise
I can't run a HA single DFSR target for the same reason (no shared storage) - and in any case, this doesn't solve the problem of replicating files between the two hosts for HA & DR. DFSR is the solution for replicating files storage across servers with
non-shared storage.
Why would there be a restriction against using DFSR between multiple hosts in a cluster, so long as you are not trying to replicate folders in a shared storage target accessible to both hosts (which would obviously be a problem)? So long as you are not replicating
folders in c:\ClusterStorage, there should be no conflict.
Is there a workaround or alternative solution?

Yes, I read that series. But it doesn't address the issue. The article is about making a DFSR target highly available. That won't help me here.
I need to be able to use DFSR to replicate files between two different servers, with those servers being in a WSFC for the purpose of providing other clustered services (Hyper-V, SQL availability groups, etc.). DFSR should not interfere with this, but it
is being blocked between nodes in the same WSFC for a reason that is not clear to me.
This is a valid use case and I can't see an alternative solution in the case where you only have two physical servers. Windows needs to be able to provide HA, DR, and replication of everything - VMs, SQL, and file folders. But it seems that this artificial
barrier is causing us to need to choose either clustered services or DFSR between nodes. But I can't see any rationale to block DFSR between cluster nodes - especially those without shared storage.
Perhaps this blanket block should be changed to a more selective block at the DFSR folder level, not the node level.

Cannot migrate VM in VMM but can in Failover Cluster Manager network adapters network optimization warning

I have a 4 node Server 2012 R2 Hyper-V Cluster and manage it with VMM 2012 R2. I just upgraded the cluster from 2012 RTM to 2012 R2 last week which meant pulling 2 nodes out of the existing cluster, creating the new R2 cluster, running the copy
cluster roles wizard since the VHDs are stored on CSVs, and then added the other 2 nodes after installing R2 on them, back into the cluster. After upgrading the cluster I am unable to migrate some VMs from one node to another. When trying to do
a live migration, I get the following notifications under the Rating Explanation tab:
Warning: There currently are not network adapters with network optimization available on host Node7.
Error: Configuration issues related to the virtual machine VM1 prevent deployment and must be resolved before deployment can continue.
I get this error for 3 out of the 4 nodes in the cluster. I do not get this error for Node10 and I can live migrate to that node in VMM. It has a green check for Network optimization. The others do not. These errors only affect
VMM. In the Failover Cluster Manager, I can live migrate any VM to any node in the cluster without any issues. In the old 2012 RTM cluster I used to get the warning but I could still migrate the VMs anywhere I wanted to. I've checked the network
adapter settings in VMM on VM1 and they are the same as VM2 which can migrate to any host in VMM. I then checked the network adapter settings of the VMs from the Failover Cluster Manager and VM1 under Hardware Acceleration has "Enable virtual machine
queue" and Enable IPsec task offloading" checked. I unchecked those 2 boxes refreshed the VMs, refreshed the cluster, rebooted the VM and refreshed again but I still could not live migrate VM1. Why is this an issue now but it wasn't before
running on the new cluster? How do I resolve the issue? VMM is useless if I can't migrate all my VMs with it.

I checked the settings on the physical nics on each node and here is what I found:
Node7: Virtual machine queue is not listed (Cannot live migrate problem VM's to this node in VMM)
Node8: Virtual machine queue is not listed (Cannot live migrate problem VM's to this node in VMM)
Node9: Virtual machine queue is listed and enabled (Cannot live migrate problem VM's to this node in VMM)Node10: Virtual machine queue is listed and enabled (Live Migration works on all VMs in VMM)
From Hyper-V or the Failover Cluster manager I can see in the network adapter settings of the VMs under Hardware Acceleration that these two settings are checked "Enable virtual machine queue" and Enable IPsec task offloading". I unchecked those
2 boxes, refreshed the VMs, refreshed the cluster, rebooted the VM and refreshed again but I still cannot live migrate the problem VMs.
It seems to me that if I could adjust those VM settings from VMM that it might fix the problem. Why isn't that an option to do so in VMM?
Do I have to rebuild the VMM server with a new DB and then before adding the Hyper-V cluster uncheck those two settings on the VM's from Hyper-V manager? That would be a lot of unnecessary work but I don't know what else to do at this point.

Windows 2008 Failover Cluster - Cannot add a generic service

Trying to add a generic service in a failover cluster.
Select the option Services and Application and it opens the wizard and then displays the error "An error was encountered while loading the list of services. QueryServiceConfig failed. The system cannot find the file specified"
The cluster validation wizard completes successfully. Permissions do not appear to be an issue as this account can seemly do everything else so I am at a loss to understand why this API is failing when it tries to query the server for services information.
Having searched the Internet the only thing I have found was someone posting a similar issue in the Greek language Technet forum(if I recall correctly) and their comment was they rebuild their cluster.
Windows 2008 (SP2) x64 two node cluster running a non-Microsoft database. We need to add a non-Microsoft Enterpirse backup solution and this is their documented method (adding it as a generic service) - both bits of software are from big vendors.
Symantec AV, but have tried with that disabled so don't think it has anything to do with that. Something is stopping the API from reporting back but I can't find what.
Really appreciate some help before we have to log a chargable call with Microsoft support
Thank you

Hi,
Have you tried the suggestion? I want to see if the information provided was helpful. Your feedback is
very useful for the further research. Please feel free to let me know if you have addition questions.
Best regards,
Vincent Hu

Reporting Services as a generic service in a failover cluster group?

There is some confusion on whether or not Microsoft will support a Reporting Services deployment on a failover cluster using scale-out, and adding the Reporting Services service as a generic service in a cluster group to achieve active-passive high
availability.
A deployment like this is described by Lukasz Pawlowski (Program Manager on the Reporting Services team) in this blog article
http://blogs.msdn.com/b/lukaszp/archive/2009/10/28/high-availability-frequently-asked-questions-about-failover-clustering-and-reporting-services.aspx. There it is stated that it can be done, and what needs to be considered when doing such a deployment.
This article (http://technet.microsoft.com/en-us/library/bb630402.aspx) on the other hand states: "Failover clustering is supported only for the report server database; you
cannot run the Report Server service as part of a failover cluster."
This is somewhat confusing to me. Can I expect to receive support from Microsoft for a setup like this?
Best Regards,
Peter Wretmo

Hi Peter,
Thanks for your posting.
As Lukasz said in the
blog, failover clustering with SSRS is possible. However, during the failover there is some time during which users will receive errors when accessing SSRS since the network names will resolve to a computer where the SSRS service is in the process of starting.
Besides, there are several considerations and manual steps involved on your part before configuring the failover clustering with SSRS service:
Impact on other applications that share the SQL Server. One common idea is to put SSRS in the same cluster group as SQL Server. If SQL Server is hosting multiple application databases, other than just the SSRS databases, a failure in SSRS may cause
a significant failover impact to the entire environment.
SSRS fails over independently of SQL Server.
If SSRS is running, it is going to do work on behalf of the overall deployment so it will be Active. To make SSRS Passive is to stop the SSRS service on all passive cluster nodes.
So, SSRS is designed to achieve High Availability through the Scale-Out deployment. Though a failover clustered SSRS deployment is achievable, it is not the best option for achieving High Availability with Reporting Services.
Regards,
Mike Yin
If you have any feedback on our support, please click
here
Mike Yin
TechNet Community Support

Server 2008 Hyper-V Failover Cluster Error on Domain Controller Reboot

I am pretty new to Hyper-V virtual but I have 2 Hyper-V Clusters, each with 2 Nodes and a SAN, 1 Physical Domain Controller for failover cluster management and 1 virtual domain controller as backup. All is running well, no issues. I installed
windows updates on the physical DC and upon reboot, got an error 5120 on cluster 2 that says "Cluster Shared Volume 'Volume1' ('Cluster Disk 1') is no longer available on this node because of 'STATUS_CONNECTION_DISCONNECTED(c000020c)'. All I/O will
temporarily be queued until a path to the volume is reestablished. It pointed to the 2nd node in that cluster as being the issue but when I look at it, it is online and all healthy so I don't understand why the error was triggered and if the DC would
go down for a failure, would that node not be able to access the CSV permanently.
Appreciate any help anyone can provide.

Hi mtnbikediver,
In theory, if you has the correct configuration of cluster the DC restart will not cause the CSV down, does your shared storage installed on your DC? Did you run
the cluster validation before you install the cluster? We strongly recommend you run the cluster validation before you build the cluster, same time please install the recommend update of 2008 cluster first.
Recommended hotfixes for Windows Server 2008-based server clusters
http://support.microsoft.com/kb/957311
I found a similar scenario issue the DC restart will effect the cluster network name resource offline, but it is for 2008R2.
Cluster network name resource cannot be brought online when one of the domain controllers is partly down in Windows Server 2008 R2
http://support2.microsoft.com/?id=2860142
I’m glad to be of help to you!
Please remember to mark the replies as answers if they help and unmark them if they provide no help. If you have feedback for TechNet Support, contact [email protected]

Windows server 2012 failover cluster error: Cluster resource 'Virtual Machine Configuration ... of type 'Virtual machine configuration in clustered role ... failed.

I have two windows 2012 host server that are clustered using windows failover cluster feature. Each server is hosting four VMs. When migrating from Host2 to Host1, the migration failed with the following error:
Cluster resource 'Virtual Machine Configuration SCPCSQLSRV01' of type 'Virtual Machine Configuration' in clustered role 'SCPCSQLSRV01' failed. The error code was '0x569' ('Logon failure: the user has not been granted the requested logon type at this computer.').
When this happens, the VM that I was migrating can no longer be started even on the original host. The only remedy is to restart the host server.
Any suggestion on resolving this problem?
Thanks
Ikad

Thanks. The article referred to above gives the solution to my issue. There is a group policy that is applied to the OU where the host servers were placed. Doing gpupdate /force temporarily removes the problem. Unfortunately the NT Virtual Machine\Virtual
Machines account is a special account that cannot be added like other accounts and granted the log on as a service right. The thread
http://social.technet.microsoft.com/Forums/en-US/winserverhyperv/thread/d56f2eae-726e-409a-8813-670a406593e8 contains how it can be added which is by creating a group and running the command
Net localgroup VMTest “NT Virtual Machine\Virtual Machines” /add
to add it to a local group VMTest. VMTest is then assigned the right to log on as a service.
Ikad

Failover Cluster - GHOST VMS / ROLES

I mean Ghost as in mysterious non-existent machine, not the old Norton program. I've periodically had random cluster crashes, mainly due to my own negligence. 99%
of the time everything comes back up normally. However periodically a machine will have very strange symptoms that i'm unable to resolve. The only resolution I've found is to create a new VM and link to the the old VHD. A description of the
machines with this issue:
Shown in Failover Cluster role as Running but cannot Connect, turn off, shutdown, etc.
Login to Host machine for the VM and open Hyper-V Manager the machine does not exist. The only place this machine seems to exist is in Failover cluster.
No details available on the Summary Tab, machine doesn't actually appear to be running despite what the console says.
Under the resources tab for that Machine is shows the VM as running, but the VM Configuration as failed.
Unable to bring the configuration back online. Error is "The group or resource is not the correct state to perform the requested operation"
I've seen other vague areas about null context pointers or something alone those lines. I've tried researching the users methods to no avail. How can I fix these? Or at least remove them when i've recreated the machine.

Hi,
Unfortunately, the available information is not enough to have a clear view of the occurred behavior.Could you provide more information about your environment. The server version of the problem on, when this problem occurs the system log record information,
screenshots is the best information.
If you are using Server 2012R2 failover cluster please install the following update:
Recommended hotfixes and updates for Windows Server 2012 R2-based failover clusters
http://support.microsoft.com/kb/2920151
More information:
Event Logs
http://technet.microsoft.com/en-us/library/cc722404.aspx
Thanks.
We
are trying to better understand customer views on social support experience, so your participation in this
interview project would be greatly appreciated if you have time.
Thanks for helping make community forums a great place.

Failover cluster not cleanly shutting down service

I've got a two node 2008 R2 failover cluster. I have a single service being managed by it that I configured just as a generic service. The failover works perfectly when the service is stopped, or when one of the machines goes down, and the immediate
failback I have configured works perfectly in both scenarios as well.
However, there's an issue when I take the networking down on the preferred owner of the service. As far as I can tell (this is the first time I've tried failover clustering, so I'm learning), when I take the networking down, the cluster service shuts
down, and in turn shuts down the service I've told it to manage. At this point, when the services aren't running, the service fails over to the secondary as intended. The problem shows up when I turn the networking back on. The service tries
and fails to start on the primary (as many times as I've configured it to try), and then eventually gives up and goes back to the secondary.
The reason for this, examining logs for the service, is that the required port is already in use. I checked some more, and sure enough, when I take the networking offline the service gets shut down, but the executable is still running. This is
repeatable every time. When I just stop the service, though, the executables go away. So it's something to do specifically with how the managed service gets shut down *when it's shut down due to the cluster service stopping*. For some reason
it's not cleaning up that associated executable.
Any ideas as to why this is happening and how to fix/work around it would be extremely welcome. Thank you!

Try to generate cluster log using closter log /g /copy:<path to a local folder>. You might need to bump up log verbosity using cluster /prop ClusterLogLevel=5 (you can check current level using cluster /prop).
You also can look at the SCM diagnostic channel in the event viewer. Start eventvwr. Wait for the clock icon on the Application and Services Logs to go away. Once the clock icon is gone select this entry and in the menu check Show Analytic and Debug Logs.
Now expand to the SCM provider located at
Application and Services Logs\Microsoft\Service Control Manager Performance Diagnostic Provider\Diagnostic.
or Microsoft-Windows-Services/Diagnostic
Enable the log, run repro, disable the log. After that you should see events from the SCM showing you your service state transitions.
The terminate parameters do not seems to be configurable. I can think of two ways fixing the issue
- Writing your own cluster resource DLL where you can implement your own policies. THis would be a place to start http://blogs.msdn.com/b/clustering/archive/2010/08/24/10053405.aspx.
- This option is assuming you cannot change the source code of the service to kill orphaned child processes on startup so you have to clenup using some other means. Create another service and make your service dependent on this new service. This new serice
must be much faster in responding do the SCM commands. On start of this service you using PSAPI enumirate all processes running on the machine and kill the orphaned child processes. You probably should be able to acheve something similar using GenScript resource
+ VB script that does the cleanup.
Regards, Vladimir Petter, Microsoft Corporation

MS- WIN SERVER 2008-R2 FAILOVER CLUSTER --ERROR 1127 ,1583

DEARS ,
I HAVE BEEN FACING PROBLEM FOR MY WINDOWS FAIL OVER CLUSTER SCENARIO.
THE ERROR CODES ARE 1127 AND 1583 WHICH ARE STATING THAT
All nodes IPv4 networks should be configured properly.
Nodes which are using Automatic Private IP Address (APIPA) for adapter Cluster.will not be added to the Windows Failover Cluster. If the adapter is to be used by Windows Failover Cluster, the IPv4 properties of the adapter should be changed to allow assignment
of a valid IP address that is not in the APIPA range.
However , I have checked that all of my NIC adapters have valid Static IP addresses. Also , when one of the nodes is down ,it cannot look for another node.
Please advise for the solution.

Hi,
Could you clarify your described “Nodes which are using Automatic Private IP Address (APIPA) for adapter Cluster.will not be added to the Windows Failover Cluster.” Did you
found one of your cluster node “automatically” have the APIPA address? If so, it seems is the IP conflict issue.
If it is not the IP address conflict issue, please try to disable your all the nodes firewall
or AV soft have the event 1583 typically cause by this function incorrect configured.
More information:
Event ID 1127 — Cluster Network Connectivity
http://technet.microsoft.com/en-us/library/dd353958(v=ws.10).aspx
Event ID 1583 — Network Connectivity and Configuration
http://technet.microsoft.com/en-us/library/ee830320(v=ws.10).aspx
Hope this helps.
We
are trying to better understand customer views on social support experience, so your participation in this
interview project would be greatly appreciated if you have time.
Thanks for helping make community forums a great place.

SQL Server Failover Cluster Questions

Dear All,
            I am building a two-node failover cluster on SQL Server 2012 SP1 (inside Hyper-V as a Guest Cluster) and want clarification on few things that I am facing.
1.  I am receiving MSDTC Warning. I can go ahead and create the cluster, but want to understand whether this MSDTC is to be configured as a role on the cluster or not. I plan to run SCVMM, SCOM, Orchestrator and Windows Azure Pack Databases
and Reports through it so in such a scenario, do I need MSDTC? If yes, how much should be the size of the MSDTC Drive? Is following process correct?
http://www.sqlnotebook.info/configure-msdtc-on-windows-cluster-2012/
2. During First Node configuration, one needs to provide the "SQL CLUSTER RESEOURCE GROUP NAME". Does it have any bearing on how it will be accessed by other servers for databases and logs? or is it just how the cluster resource group
would be named? would it be required for every instance that is created inside the cluster? Just to be clear, so one can name it according to the instance name.
3. During the instance creation, one needs to provide "SQL Server Network Name". As stated above, I plan to run SCVMM, SCOM, Orchestrator and Windows Azure Pack Databases and Reports through it, so would I be required to provide this
for all instances that I create or this is only required once in the cluster:
4. During the instance creation, one needs to provide the features required for installation i.e. instance features and shared features. As stated above, I plan to run SCVMM, SCOM, Orchestrator and Windows Azure Pack Databases and Reports through
it, so which features should be selected? so that there is less workload on the server.
5. All the instances use TempDB for databases that are present inside it. What would be the best practice with respect to TempDB. One TempDB for all instance on the servers on a separate LUN or all instance having their own TempDB LUN?  What
should be the ideal size of the TempDB LUN?
6. Should all the disks required for DBs and Logs be added to Cluster?  Should they be added normal disks or CSV Volumes?
Thanks in advance.

Hello,
1.You can run the Microsoft Distributed Transaction Coordinator service (MSDTC) as a clustered resource on a failover cluster server for increased reliability, based on the failover capabilities of the clustered servers. You can
refer to the MSDTC section of the following blog about determine whether the Microsoft Distributed Transaction Coordinator (MSDTC) cluster resource must be created.
Reference:http://msdn.microsoft.com/en-us/library/ms189910.aspx#MSDTC
2. The Cluster Resource Group is where SQL Server failover cluster resource will be placed. Each clustered SQL Server will belong to a Failover
Cluster Resource Group. For example, if you had configure a two node SQL Server Cluster, each clustered instance on the two node belong to a same Cluster Resource Group.
You can change the Cluster Resource Group name, but notes the following name is reserved and already used as Resource Group names: Available Storage, Cluster Group.
3. Each SQL Server cluster is assigned a virtual Network name and IP address, which client applications use to connect to the clustered SQL Server.
4. Not familiar with SCVMM, SCOM, Orchestrator, but you should install the Database Engine Services and SQL Server Management tools.If you want to use SQL Server Reporting Services, you can install Reporting Servers, but Report Server service cannot participate
in a failover cluster.
5. You can use isolated disk for user database and temp DB of each SQL Server Cluster
6. Yes. You should use Cluster Disks which add to Clustered Shared Volumes to host the data file and log of databases.
http://www.pythian.com/blog/how-to-install-a-clustered-sql-server-2012-instance-step-by-step-part-1/
Regards,
Fanny Liu
Fanny Liu
TechNet Community Support

Can we setup FILESTREAM on Failover Cluster

I saw following point on Technet article about RBS.
The local FILESTREAM provider is supported only when it is used on local hard disk drives or an attached Internet Small Computer System Interface (iSCSI) device. You cannot use the local RBS FILESTREAM provider on remote storage devices such as network attached storage (NAS).
It looks like that we cannot use FILESTREAM on Failover Cluster because to setup Failover Cluster we need to have NAS. But then the NAS is made available locally for Failover Cluster so FILESTREAM should work right?
Found another article which talks about setting up FILESTREAM on Failover Cluster so I am a bit confused.
https://msdn.microsoft.com/en-us/library/cc645886.aspx

Hi Frank,
As other post, we can set up FILESTREAM on a Failover cluster.
However, FILESTREAM can't live on a network addressable storage (NAS) device unless the NAS device is presented as a local NFS volume via iSCSI. With iSCSI , it is supported by Microsoft
FILESTREAM provider.
Reference:
Description of support for network database files in SQL Server
Programming with FileStreams in SQL Server 2008
Thanks,
Lydia Zhang
Lydia Zhang
TechNet Community Support

Powershell: Set Access Based Enumeration on share in Failover Cluster

Hi guys,
I'm facing the following problem. Below you see my script to create a shared folder. (My folder share is visible in failover cluster manager, underneath clustergroup TESTSTO01.)
Now I need to enable Access Based Enumeration on this share. Has anyone a clue how to do that in powershell? (Version 2).
I also need to make sure that the files and programs are not available offline.
Thanks in advance!
$SHARE_READ = 1179817 # 100100000000010101001
$SHARE_CHANGE = 1245631 # 100110000000100010110
$SHARE_FULL = 2032127 # 111110000000111111111
$SHARE_NONE = 1 # 000000000000000000001
$ACETYPE_ACCESS_ALLOWED = 0
$ACETYPE_ACCESS_DENIED = 1
$ACETYPE_SYSTEM_AUDIT = 2
$ACEFLAG_INHERIT_ACE = 2
$ACEFLAG_NO_PROPAGATE_INHERIT_ACE = 4
$ACEFLAG_INHERIT_ONLY_ACE = 8
$ACEFLAG_INHERITED_ACE = 16
$ACEFLAG_VALID_INHERIT_FLAGS = 31
$ACEFLAG_SUCCESSFUL_ACCESS = 64
$ACEFLAG_FAILED_ACCESS = 128
# New Trustee
function New-Trustee($Domain, $User)
$Trustee = ([WMIClass]"\\TESTSTO01\root\cimv2:Win32_Trustee").CreateInstance()
   $Trustee.Domain = $Domain
   $Trustee.Name = $User
   if ($User -eq "Administrators")
{$Trustee.SID = @(1, 2, 0, 0, 0, 0, 0, 5, 32, 0, 0, 0,32,2,0,0)}
else
{$Trustee.SID = @(1, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0)}
   return $Trustee
# New ACE
function New-ACE($Domain, $User, $Access, $Type, $Flags)
$ACE = ([WMIClass]"\\TESTSTO01\root\cimv2:Win32_ACE").CreateInstance()
   $ACE.AccessMask = $Access
   $ACE.AceFlags = $Flags
   $ACE.AceType = $Type
   $ACE.Trustee = New-Trustee $Domain $User
   return $ACE
# Get SD
function Get-SD
$sd = ([WMIClass]"\\TESTSTO01\root\cimv2:Win32_SecurityDescriptor").CreateInstance()
$ACE1 = New-ACE -Domain $null -User "Everyone" -Access $SHARE_CHANGE -Type $ACETYPE_ACCESS_ALLOWED -Flags $ACEFLAG_INHERIT_ACE
$ACE2 = New-ACE -Domain $null -User "Administrators" -Access $SHARE_FULL -Type $ACETYPE_ACCESS_ALLOWED -Flags $ACEFLAG_INHERIT_ACE
[System.Management.ManagementObject[]] $DACL = $ACE1, $ACE2
$sd.DACL =$DACL
return $sd
# Create-Share
function Create-Share($ShareName, $Path, $Comment,$Access)
   $checkShare = (Get-WmiObject Win32_Share -Filter "Name='$ShareName'")
   if ($checkShare -ne $null) {
   # "Share exists and will now be deteted!!!"
   get-WmiObject Win32_Share -Filter "Name='$ShareName'" | foreach-object { $_.Delete() }
   $wmishare = [WMIClass] "\\TESTSTO01\ROOT\CIMV2:Win32_Share"
$Access = Get-SD
   $R = $wmishare.Create($Path,$Sharename,0,$null,$Comment,"", $Access)
   if ($R.ReturnValue -ne 0) {
   Write-Error "Error while creating share: " + $R.ReturnValue
   exit
   # Write-Host "Share has been created."
# Create first share with permissons **********************************
$ShareName = "$Company$"
$Path = "$Driveletter" + ":\$Company"
$Comment = ""
$Domain = $Null
Create-Share $ShareName $Path $Comment $Access

Unable to find type [CmdletBinding(SupportsShouldProcess=$TRUE)]: make sure tha
t the assembly containing this type is loaded.
At C:\Script Nathalie\Everyware2.ps1:294 char:45
+ [CmdletBinding(SupportsShouldProcess=$TRUE)] <<<<
   + CategoryInfo : InvalidOperation: (CmdletBinding(S...dProcess=$T
   RUE):String) [], RuntimeException
   + FullyQualifiedErrorId : TypeNotFound
The term 'param' is not recognized as the name of a cmdlet, function, script fi
le, or operable program. Check the spelling of the name, or if a path was inclu
ded, verify that the path is correct and try again.
At C:\Script Nathalie\Everyware2.ps1:295 char:6
+ param <<<< (
   + CategoryInfo : ObjectNotFound: (param:String) [], CommandNotFou
   ndException
   + FullyQualifiedErrorId : CommandNotFoundException
The term 'begin' is not recognized as the name of a cmdlet, function, script fi
le, or operable program. Check the spelling of the name, or if a path was inclu
ded, verify that the path is correct and try again.
At C:\Script Nathalie\Everyware2.ps1:304 char:6
+ begin <<<< {
   + CategoryInfo : ObjectNotFound: (begin:String) [], CommandNotFou
   ndException
   + FullyQualifiedErrorId : CommandNotFoundException
Get-Process : Cannot evaluate parameter 'Name' because its argument is specifie
d as a script block and there is no input. A script block cannot be evaluated w
ithout input.
At C:\Script Nathalie\Everyware2.ps1:331 char:8
+ process <<<< {
   + CategoryInfo : MetadataError: (:) [Get-Process], ParameterBindi
   ngException
   + FullyQualifiedErrorId : ScriptBlockArgumentNoInput,Microsoft.PowerShell.
   Commands.GetProcessCommand
The term 'end' is not recognized as the name of a cmdlet, function, script file
, or operable program. Check the spelling of the name, or if a path was include
d, verify that the path is correct and try again.
At C:\Script Nathalie\Everyware2.ps1:345 char:4
+ end <<<< {
   + CategoryInfo : ObjectNotFound: (end:String) [], CommandNotFound
   Exception
   + FullyQualifiedErrorId : CommandNotFoundException
The term 'set-shareABE' is not recognized as the name of a cmdlet, function, sc
ript file, or operable program. Check the spelling of the name, or if a path wa
s included, verify that the path is correct and try again.
At C:\Script Nathalie\Everyware2.ps1:348 char:13
+ set-shareABE <<<< TESTSTO01 $Company$ -Enable
   + CategoryInfo : ObjectNotFound: (set-shareABE:String) [], Comman
   dNotFoundException
   + FullyQualifiedErrorId : CommandNotFoundException

2008 R2 SP1 failover cluster, after connect to VM from failover cluster manager console I've get only upper quourter of screen console

Hello :)
I have to ask because this issue is boring me a long time ago :(
From time to time when I connect to virtual machine from failover cluster manager console I've get virtual machine connection screen (console) which is reduced only to upper left quorter of full screen. There is no scroll bar ... If I live migrate
VM to another node (or restart VM on same node) and reconnect, console screen is displayed ok (ie all console screen is visible).
It is happening regardless of OS version installed into guest VM: 2003 R2, 2008 R2, 2012, 2012 R2).
I've check inside VM ( I can see that integration services are installed and running but when I click onto console window with mouse I've get message:
"Virtual Machine Connection
Mouse not captured in Remote Desktop session
The mouse is available in a Remote Desktop session when
integration services are installed in the guest operating
system...."
From SCVMM console I can see that IC version on guest VM is: 6.1.7601.17514
I would like to know what is going on and is there any way how can I detect those situation or even better how to prevent it ?
Thank you for any idea.
Best regards
Nenad

Hi Nenad,
As you mentioned you have tried 2003 R2, 2008 R2, 2012, 2012 R2 guest vm, but all this do not work properly, With Server 2003 platform the 2008r2 Hyper-V the 2003R2 are not
supported, in your case you must update to Windows Server 2003 R2 with Service Pack 2.
Server 2012 as guest vm on 2008r2 you must install the following hot fix:
You cannot run a Windows 8-based or Windows Server 2012-based virtual machine in Windows Server 2008 R2
https://support2.microsoft.com/kb/2744129?wa=wsignin1.0
Server 2012 is the last version of Windows that will be supported as a guest operating system on 2008r2 Hyper-V therefore the 2012r2 guest vm is not supported.
Please use the following command can query all guest vm’s IC version, please compare this problematic vms IC version with functioning vms.
The following powershell command can be used to display the version of integration services installed on all VMs on the Hyper-V host:
PS C:\Users\administrator> get-vm | ft name, IntegrationServicesVersion
Name
IntegrationServicesVersion
TestVM2012
6.2.9200.16433
SQL01
6.2.9200.16433
SQL02
                                  6.2.9200.16433
SCVMM01
6.2.9200.16384
SCVMM02
6.2.9200.16384
I’m glad to be of help to you!
We
are trying to better understand customer views on social support experience, so your participation in this
interview project would be greatly appreciated if you have time.
Thanks for helping make community forums a great place.

Configuring Failover Cluster Server Share as a Distribution Point

Hello and thanks for this helpful forum!
I am trying to configure a server share as a Distribution Point and am running into problems. I am running SCCM R2 on Server 2008. I am trying to use a clustered share coming from 2008 R2 Failover Cluster.
I have been trying to get this working for the better part of 2 days now, and I have successfully created a Distribution Point to another 2008 Server (non-R2) by simply using the site system (not the server share).
When I try to use the server share option, I keep receiving an error with Message ID 4701 stating that the SMS Site System Status Summarizer cannot access storage object
\\servername\sharename$ on site system
\\servername\sharename$
The operating system reported error 2147942467: The network name cannot be found.
The server share is defined as a site system as
\\servername\sharename$
I am trying to create a "Packages" directory under it, so I'm using the option "Share the distribution folder" under the Data Access tab of the Package.
In addition, I have given local admin rights to the site server on both nodes of the cluster. I have given the site server modify rights to the share and NTFS permissions. I have given Authenticated Users Read access to the share and NTFS.
As I mentioned, I can take another server and define a Site System as a DP and everything will work properly. I am still missing something though regarding using a Server Share.
Thanks so much in advance for your assistance.

Hi,
No DP on NLBs:
http://technet.microsoft.com/en-us/library/bb735883.aspx
Yes, but that is NOT a failover file cluster. Is there a workaround? It seems silly to have to stand up an additional server for this. It also seems odd that the forthcoming version of SCCM would not have some support for failover clustering which was a HUGE
improvement over the previous clustering solutions from MS.Jeffrey S. Patton Assistant Director of IT School of Engineering Computing Services University of Kansas 1520 West 15th Street Lawrence, KS. 66045-7621

Cannot failover cluster

Similar Messages

Maybe you are looking for