Cluster nodes loosing communication

Hello,
http://support.microsoft.com/kb/2462468
I have checked the services on both nodes, they are all started: Cluster Service, TCP/IP etc... all automatic services
are started on both nodes.
Log Name: Application
Source: SRMSVC
Date: 3/9/2015 11:56:25 AM
Event ID: 12317
Task Category: None
Level: Warning
Keywords: Classic
User: N/A
Computer: MBFSCL1B.ad.
Description:
File Server Resource Manager failed to enumerate share paths or DFS paths. Mappings from local file paths to share and DFS paths may be incomplete or temporarily unavailable. FSRM will retry the operation at a later time.
Error-specific details:
Error: NetShareEnum, 0x80070005, Access is denied.
this event is only on Node B....
get-clusteraccess
IdentityReference AccessControlType ClusterRights
NT AUTHORITY\SYSTEM Allow Full
NT AUTHORITY\NETWORK SERVICE Allow Full
BUILTIN\Administrators Allow Full
NT SERVICE\MSDTC Allow Full
Where to look for more information about this issue as periodically, once or twice a week the two nodes do not see each other!!!
I restart the services (Cluster Service) and they see each other again!!!
This cluster has been working fine for 3 years and the issue came within the last two weeks!!!
- Changes:
- Tivoli upgrades
- Security tied
etc...
Node A I have also time to time some errors:
Log Name: System
Source: srv
Date: 3/10/2015 8:03:27 AM
Event ID: 2012
Task Category: None
Level: Warning
Keywords: Classic
User: N/A
Computer: MBFSCL1A.ad
Description:
While transmitting or receiving data, the server encountered a network error. Occassional errors are expected, but large amounts of these indicate a possible error in your network configuration. The error status code is contained within the returned data (formatted as Words) and may point you towards the problem.
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
<System>
<Provider Name="srv" />
<EventID Qualifiers="32768">2012</EventID>
<Level>3</Level>
<Task>0</Task>
<Keywords>0x80000000000000</Keywords>
<TimeCreated SystemTime="2015-03-10T15:03:27.958443300Z" />
<EventRecordID>469908</EventRecordID>
<Channel>System</Channel>
<Computer>MBFSCL1A.ad</Computer>
<Security />
</System>
<EventData>
<Data>\Device\LanmanServer</Data>
<Binary>0000040001002C0000000000DC07008000000000840100C0000000000000000000000000000000008F050000</Binary>
</EventData>
</Event>
not sure what could have created this problem, any clue?
Thanks,
Dom
System Center Operations Manager 2007 / System Center Configuration Manager 2007 R2 / Forefront Client Security / Forefront Identity Manager

Hi Felyjos,
There have a article for the detail steps troubleshooting about this known issue, have you try it?
An Event 12317 showing access denied is caused when the NT AUTHORITY\Authenticated Users group is not a member of the BUILTIN\Users group in the domain. (As you might know, the local BUILTIN\Users group on a domain controller is mapped to the domain built-in
group Users. Therefore, the effect of removing NT AUTHORITY\Authenticated Users from the BUILTIN\Users group on a domain controller has a domain-wide effect.) 
The realted article:
Troubleshooting event ID 12317, “File Server Resource Manager failed to enumerate share paths or DFS paths.”
http://blogs.technet.com/b/filecab/archive/2006/09/29/459430.aspx
I’m glad to be of help to you!
Please remember to mark the replies as answers if they help and unmark them if they provide no help. If you have feedback for TechNet Support, contact [email protected]

Similar Messages

  • Cluster node problems in communciation channel

    Hello,
    I often have node problems where all messags get stuck in particular cluster node of communication channel.
    Once it is restarted, all messages are working problem,  With respect to that were the following issues. Error when getting an FTP connection from connection pool: com.sap.aii.af.service.util.concurrent.ResourcePoolException: Unable to create new pooled resource: ConnectException: Connection refused (errno:239)
    Please do let me know for the solution to avoid it in the future.
    Regards,
    Anandh

    Hello,
    Question 43 of file adapter faq,
    Q: J2EE engine hangs with the File/FTP sender channels. How to resolve this ?
    A: The reason for this is, some times due to network level issues, a message is waiting forever for a response from a FTP server which is down for some time. Adapter does not know this and try to poll FTP server again with second message. This goes on and on and eventually we would have J2EE engine hanging. To solve this, following things need to be applied:
        1) Set FTP timeout to appropraite value if channel is FTP sender channel.
            2) In advanced mode table options, add a new parameter 'clusterSyncMode' and its value set to 'lock'. This parameter             is without single quotes and case-sensitive.
            3) Last but not least, make sure that you are in latest patches            of SP19/SP20/SP21/SP22 for SAPXIAFC component of XI 3.0                 release and SP10/SP11/SP12/SP13/SP14 for SAPXIAFC component             of XI 7.0 release. Any patch which is released after 11th of    Febraury 2008 for the above releases is fine.
    We had the same issue and this did the trick.
    Regards,
    Bhavesh

  • Unable to failover the services in active-active cluster node

    Hi,
    i am applying the sp2 patch for sql server 2008 r2 in active-active cluster, we have 3 services in the cluster , node 1 as 2 prefered owner and node 2 as 1 prefered owner, when i try to move the service from node 2 to node1 , i am getting the below errors
    DCOM was unable to communicate with the computer XXXXXXXXX using any of the configured protocols.
    The Kerberos client received a KRB_AP_ERR_MODIFIED error from the server XXXXXXXXX. The target name used was RPCSS/XXXXXX. This indicates that the target server failed to decrypt the ticket provided by the client. This can occur when the target server principal
    name (SPN) is registered on an account other than the account the target service is using. Please ensure that the target SPN is registered on, and only registered on, the account used by the server. This error can also happen when the target service is using
    a different password for the target service account than what the Kerberos Key Distribution Center (KDC) has for the target service account. Please ensure that the service on the server and the KDC are both updated to use the current password. If the server
    name is not fully qualified, and the target domain (XXXXXX) is different from the client domain (XXXXXXX), check if there are identically named server accounts in these two domains, or use the fully-qualified name to identify the server.
    The Cluster service failed to bring clustered service or application 'CHCROCHC045' completely online or offline. One or more resources may be in a failed state. This may impact the availability of the clustered service or application.
    Cluster resource 'SQL Server (CHCROCHC045)' in clustered service or application 'CHCROCHC045' failed.
    any inputs appreciated to resolve this issue as i could not procedd with patching
    BR
    PGR

    Hi PGR,
    As the issue is more related to Windows Server, I would like to recommend you post the issue in the
    Windows Server forums for better support.
    In addition, below are some article about troubleshooting error ” DCOM was unable to communicate with the computer XXXXXXXXX using any of the configured protocols” for your reference.
    Event ID 10009 — COM Remote Service Availability
    How to troubleshoot DCOM 10009 error logged in system event?
    Thanks,
    Lydia Zhang
    Lydia Zhang
    TechNet Community Support

  • CUA: One or more errors occurred while checking the status of Windows Firewall on the cluster nodes

    Cluster with 2 hosts 2012 R2
    Scheduled CAU fails with:
    CAU run {4EFE116C-AB49-456D-8EED-F7EDC764DA49} on cluster Cluster1 failed. Error Message:One or more errors occurred while checking the status of Windows Firewall on the cluster nodes. Review the errors for more information on how to resolve the problems.
    Error Code:-2146233088 Stack:   at MS.Internal.ClusterAwareUpdating.Util.<CheckFirewallsAsync>d__3a.MoveNext()
    --- End of stack trace from previous location where exception was thrown ---
       at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
       at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
       at Microsoft.ClusterAwareUpdating.Commands.InvokeCauRunCommand.<_ProcessCluster>d__78.MoveNext()
    If I run CAU "Analyze Readiness" ALL comes as PASS
    If I run CUA by hand on same hosts with NO change to the system (not even reboot) it finishes OK
    Anybody any ideas?
    Thanks
    Seb

    Hi,
    In some case if you disabled the connection in Windows firewall inbound of
     "Cluster aware updating" service it will can’t use the CAU.
    More information:
    Starting with Cluster-Aware Updating: Self-Updating
    http://blogs.technet.com/b/filecab/archive/2012/05/17/starting-with-cluster-aware-updating-self-updating.aspx
    What is Cluster Aware Updating in Windows Server 2012? (Part 1)
    http://blogs.technet.com/b/mspfe/archive/2013/02/06/what-is-cluster-aware-updating-in-windows-server-2012.aspx
    Cluster-Aware Updating Overview
    http://technet.microsoft.com/en-us/library/hh831694.aspx
    Hope this helps.
    We
    are trying to better understand customer views on social support experience, so your participation in this
    interview project would be greatly appreciated if you have time.
    Thanks for helping make community forums a great place.

  • Processing in  Multiple Cluster Nodes

    Hi All,
    In our PI system we have 2 Java nodes due to some requirement. When the communication channel runs and we check the message log, in one Cluster node we have a successful message. In other Cluster Node we have an error message that says "File not found".
    The file processing is completeing successfully on one Cluster node. But I wanted to know if there is any way to suppress the processing of the same file by same channel on another Node. Some setting in administration or IB where we can get this done.
    Is there any way to get this done by some setting?
    Thanks,
    Rashmi.

    Hello!
    As per note #801926, please set the clusterSyncMode parameter on Advanced tab of the communication channel with LOCK value.
    And also check the entries 4 and 48 of the FAQ note #821267:
    4. FTP Sender File Processing in Cluster Environment
    48. File System(NFS) File Sender Processing in Cluster Environment
    Best regards,
    Lucas

  • Hyper-V Failover Cluster Node Corruption

    Dear All,
                Some of my nodes are showing abnormal behavior.  They are restarting every now and then.  I had updated the cluster nodes, but all updates were OS specific, there was nothing specific
    with respect to hardware update.
    I have analyzed crash dumps and find out that following is causing the crash:
    page_fault_in_nonpaged_area
    anyone has any idea about this?
    Thanks in advance.

    Hi ,
    What is the OS of the cluster node ?
    Did you try to remove the protection client for troubleshooing ?
    If it is a 2008R2 cluster , please refer to this thread :
    http://social.technet.microsoft.com/Forums/en-US/32ab6a85-6002-4c3c-97ea-27cb1091e9b3/windows-cluster-server-is-getting-restarted?forum=winservergen
    Hope it helps
    Best Regards
    Elton Ji
    We
    are trying to better understand customer views on social support experience, so your participation in this
    interview project would be greatly appreciated if you have time.
    Thanks for helping make community forums a great place.

  • Hyper-V Guest Cluster Node Failing Regularly

    Hi,
    We currently have a 4-node Server 2012 R2 Cluster witch hosts among other things, a 3 node Guest Cluster running a single clustered file service.  
    Around once a week, the guest cluster node that is currently hosting the clustered file service will fail.  It's as if the VM is blue screening.  That in itself is fairly anoying and I'll be doing all the updates and checking event log for clues
    as to the cause.  
    The problem then is that whichever physical cluster node that is hosting the VM when it fails,  will not unlock some of the VM's files.  The Virtual machine configuration lists as Online Pending.  This means that the failed VM cannot be restarted
    on any other cluster node.  The only fix is to drain the physical host it failed on, and reboot. 
    Looking for suggestions on how to fix the following.
    1. Crashing guest file cluster node
    2. Failed VM with shared VHDX requiring Phyiscal host reboot.
    Event messages for the physical host that was hosting the failed vm in order that they occured.
    Hyper-V-Worker: Event ID 18590 - 'FS-03' has encountered a fatal error.  The guest operating system reported that it failed with the following error codes: ErrorCode0: 0x9E, ErrorCode1: 0x6C2A17C0, ErrorCode2: 0x3C, ErrorCode3: 0xA, ErrorCode4:
    0x0.  If the problem persists, contact Product Support for the guest operating system.  (Virtual machine ID 36166B47-D003-4E51-AFB5-7B967A3EFD2D)
    FailoverClustering: Event ID 1069 - Cluster resource 'Virtual Machine FS-03' of type 'Virtual Machine' in clustered role 'FS-03' failed.
    Hyper-V-High-Availability: Event ID 21128 - 'Virtual Machine FS-03' failed to shutdown the virtual machine during the resource termination. The virtual machine will be forcefully stopped.
    Hyper-V-High-Availability: Event ID 21110 - 'Virtual Machine FS-03' failed to terminate.
    Hyper-V-VMMS: Event ID 20108 - The Virtual Machine Management Service failed to start the virtual machine '36166B47-D003-4E51-AFB5-7B967A3EFD2D': The group or resource is not in the correct state to perform the requested operation. (0x8007139F).
    Hyper-V-High-Availability: Event ID 21107 - 'Virtual Machine FS-03' failed to start.
    FailoverClustering: Event ID 1205 - The Cluster service failed to bring clustered role 'FS-03' completely online or offline. One or more resources may be in a failed state. This may impact the availability of the clustered role.

    Hi,
    I don’t found the similar issue, Does your cluster can pass the cluster validation? Does all your Hyper-V host compatible with Server 2012r2? Have you try to disable all your
    AV soft and firewall? Please rerun Storage validation on the Cluster in non-production hours, the cluster validation report will quickly locate the issue.
    More information:
    Cluster
    http://technet.microsoft.com/en-us/library/dd581778(v=ws.10).aspx
    Hope this helps.
    We
    are trying to better understand customer views on social support experience, so your participation in this
    interview project would be greatly appreciated if you have time.
    Thanks for helping make community forums a great place.

  • Cluster Shared Volume is no longer accessible from cluster node

    Hello,
    We have a 3 nodes Hyper-v Cluster running Windows Server 2012. Recently we start having error below intermittently on a node, and the VMs running on this host and LUN will power off.
    Alert: Cluster Shared Volume is no longer accessible from cluster node
    Source: Cluster Service
    Path: HV01.itl.local
    Last modified by: System
    Last modified time: 12/1/2013 12:27:18 AM
    Alert description: Cluster Shared Volume 'Volume1' ('Cluster_Vol1_R6') is no longer accessible from this cluster node because of error 'ERROR_TIMEOUT(1460)'. Please troubleshoot this node's connectivity to the storage device and network connectivity.
    The only changes made recently is we installed VEEAM on test basis for DR replication. We switched off the Veeam server and stop the Veeam Services on the Hyper-V Hosts but we are still having same issue.
    We are using an EMC SAN connected via FC as Shared storage and Powerpath as Multi-Pathing. No errors were found on the SAN.
    I don't think the issue is related to the number of IO as we also experienced the issue at midnight during the week-end where no one was working.
    Any help would be very much appreciated.
    Thanks.
    Irfan
    Irfan Goolab SALES ENGINEER (Microsoft UC) MCP, MCSA, MCTS, MCITP, MCT

    Hi,
    Also, try to install the following recommend KBs.
     Recommended hotfixes and updates for Windows Server 2012-based Failover Clusters
    http://support.microsoft.com/kb/2784261
    Also, there please confirm your VSS provider have the correct version.
    The third party article:
    VSS Provider with 2012 HyperV and CSV
    https://community.emc.com/thread/170636
    Thanks.
    We
    are trying to better understand customer views on social support experience, so your participation in this
    interview project would be greatly appreciated if you have time.
    Thanks for helping make community forums a great place.

  • Oracle Cluster Node Reboots Abruptly

    One of our RAC 11gR2 Cluster Node rebooted abruptly. We found the following error in the grid home alter log file and ocssd.log file.
    [cssd(6014)]CRS-1611:Network communication with node mumchora12 (1) missing for 75% of timeout interval.  Removal of this node from cluster in 6.190 secondsWe need to find the Root Cause for this node reboot. Kindly assist.
    OS Version : RHEL 5.8
    GRID : 11.2.0.2
    Database : 11.2.0.2.10

    Hi,
    By looking the logs it seems private interconnect problem. I would suggest you to refer one of nice metalink doc on same issue.
    Node reboot or eviction: How to check if your private interconnect CRS can transmit network heartbeats [ID 1445075.1]
    Hope it will help you to identify the root cause of node eviction.
    Thanks

  • Sql server 2008 r2 setup support rules missing cluster node

    i have error on install sql r2 there are problems in cluster node its failed on windows 8

    Hi ,
    Before you install SQL Server on a computer that is running Windows 8, about Microsoft SQL Server 2008 , you must apply and install Microsoft SQL Server 2008 R2 Service Pack 1 or a later after the initial setup is complete. To install a SQL Server failover
    cluster, you must create and configure a failover cluster instance by running SQL Server setup. For more information, see:
    Installing a SQL Server 2008 R2 Failover Cluster: http://msdn.microsoft.com/en-us/library/ms179410(v=sql.105).aspx
    In addition, as other post, could you please help us to collect the detailed error message and the following error log? It is very useful for our research.
    C:\Program Files\Microsoft SQL Server\100\Setup Bootstrap\LOG\Summary.txt.
    C:\Program Files\Microsoft SQL Server\100\Setup Bootstrap\Log\<YYYYMMDD_HHMM>\Detail.txt
    Regards,
    Sofiya Li
    Sofiya Li
    TechNet Community Support

  • Cluster node addition fails on cleanup

    We have a 2 node cluster setup already
    (2) HP BL460c G8 servers connected to a VNX5300 SAN (Nodes 1 & 2)
    Server 2012 Datacenter installed
    Quarum: Node + Disk
    all failover tests went perfectly and all VMs are healthy
    Verification on the cluster show some warnings but no failures
    We have rebuilt a server (node 3) renamed it and have run a single machine verification test to see if it is suitable for clustering. it succeeded with minor warnings
    We ran verification on all three machines and received the formentioned warnings but no game stoppers, however when trying to add the host to the cluster we get the following error in the logs:
    WARN mscs::ListenerWorker::operator (): ERROR_TIMEOUT(1460)' because of '[FTI][Initiator] Aborting connection because NetFT route to node <machine name> on virtual IP fe80::cdf2:f6ea:5ce:5f9c:~3343~ has failed to come up.'
    This happens after the node is added to the cluster but reports a failure on cleanup processes and reverts everything back. I have done all of this under my domain_admin account.
    before and after the attempt to add the NetFT adapter is in media disconnect, during the attempts it does pull down a 169 address as it is supposed to
    Node 3 Networking breakdown
    The new host uses an Intel/HP NC365T Quard port adaptor
    port 1: Mgmt : Static assignment subnet 1
    port 2: VM net: Static assignment sibmet 2
    port 3: Heartbeat: assigned via DHCP subnet 1 pool (we have attempted the above with this disabled as well)
    NCU is not installed for the adapter and bridging in server 2012 is not enabled.
    I am at a loss, and would appreciate any additional help as i have spent 3 days researching this to try and find the cause.

    Hi,
    The error message mentioned an IPv6 address, have you enable IPv6 network for the cluster?
    Check the IPv6 network configuration in the 3<sup>rd</sup> node server, what’s the status, enabled or disabled?
    When two or more cluster nodes are running IPv6 for heartbeat communications, they will require any additional nodes that join to also running IPv6. If the node server has IPv6 disabled, it will fail to join.
    Also whether these cluster node server has antivirus software installed, you may temporarily disable it and rejoin the new node.
    Check that and give us feedback for further troubleshooting, for more information please refer to following MS articles:
    Failover Cluster Creation Issue
    http://social.technet.microsoft.com/Forums/en-US/winserverClustering/thread/1ed1936d-6283-46cc-951d-9c236329b8be
    Failure to re-add rebuilt cluster node to Windows 2008 R2 Cluster: System error 1460 has occurred (0x000005b4). Timeout.
    http://social.technet.microsoft.com/Forums/en-US/winserverClustering/thread/a21e9a8e-9f68-4d83-a747-204000cda65a
    Hope this helps!
    TechNet Subscriber Support
    If you are
    TechNet Subscription
    user and have any feedback on our support quality, please send your feedback
    here.
    Lawrence
    TechNet Community Support

  • Cluster node has exceeded it's failover threshold

    I am trying to create the Availability group listener for a 2 node cluster and cluster node events show failure due to "Cluster node has exceeded it's failover threshold" after I make one attempt that has failed for a variety of reasons, usually
    permission. How do I set the threshold higher. All the information I get tells me to open processes that are not listed in the failover cluster manager. I haven't seen code that works in powershell. How can I set the failover threshold higher than one?

    Hi,
    Please try to install the recommended hotfixes and updates for Windows Server 2012-based failover cluster then monitor it again.
    The related hotfixes.
    Recommended hotfixes and updates for Windows Server 2012-based failover clusters
    http://support.microsoft.com/kb/2784261/en-us
    Hope this helps.
    We
    are trying to better understand customer views on social support experience, so your participation in this
    interview project would be greatly appreciated if you have time.
    Thanks for helping make community forums a great place.

  • SCVMM losing connection to cluster nodes

    Hey guys'n girls, I hope this is the right forum for this question. I already opened a ticket at MS support as well because it's impacting our production environment indirectly, but even after a week there's been no contact. Losing faith in MS support there
    The problem we're having is that scvmm is that a host enters the 'needs attention' state, with a winrm error 0x80338126. I guess it has something to do with the network or with Kerberos, and I've found some info on it, but I still haven't been able to solve
    it. Do you guys have any ideas?
    Problem summary:
    We are seeing an issue on our new hyper-v platform. The platform should have been in production last week, but this issue is delaying our project as we can't seem to get it stable.
    The problem we are experiencing is that SCVMM loses the connection to some of the Hyper-V nodes. Not one
     specific node. Last week it happened to two nodes, and today it happened to another node. I see issues with WinRM, and I expect something to do with kerberos. See the bottom of this post for background details and software versions.
    The host gets the status 'needs attention', and if you look at the status of the machine, WinRM gives an error. The error is:
    Error (2916)
    VMM is unable to complete the request. The connection to the agent cc1-hyp-10.domaincloud1.local was lost.
    WinRM: URL: [http://cc1-hyp-10.domaincloud1.local:5985], Verb: [ENUMERATE], Resource: [http://schemas.microsoft.com/wbem/wsman/1/wmi/root/cimv2/Win32_Service], Filter: [select * from Win32_Service where Name="WinRM"]
    Unknown error (0x80338126)
    Recommended Action
    Ensure that the Windows Remote Management (WinRM) service and the VMM agent are installed and running and that a firewall is not blocking HTTP/HTTPS traffic. Ensure that VMM server is able to communicate with cc1-hyp-10.domaincloud1.local over WinRM by successfully
    running the following command:
     winrm id –r:cc1-hyp-10.domaincloud1.local
    This
     problem can also be caused by a Windows Management Instrumentation (WMI) service crash. If the server is running Windows Server 2008 R2, ensure that KB 982293 (http://support.microsoft.com/kb/982293)
    is installed on it.
    If the error persists, restart cc1-hyp-10.domaincloud1.local and then try the operation again. /nRefer to
    http://support.microsoft.com/kb/2742275 for more details.
    Doing a simple test from the VMM server to the problematic cluster node shows this error:
    PS C:\> hostname
    CC1-VMM-01
    PS C:\> winrm id -r:cc1-hyp-10.domaincloud1.local
    WSManFault
        Message = WinRM cannot complete the operation. Verify that the specified computer name is valid, that the computer is accessible over the network, and that a firewall exception for the WinRM service is enabled and allows access from this
    computer. By default, the WinRM firewall exception for public profiles limits access to remote computers within the same local subnet.
    Error number:  -2144108250 0x80338126
    WinRM cannot complete the operation. Verify that the specified computer name is valid, that the computer is accessible over the network, and that a firewall exception for the WinRM service is enabled and allows access from this computer. By default, the WinRM
    firewall exception for public profiles limits access to remote computers within the same local subnet.
    I CAN connect from other hosts to this problematic cluster node:
    PS C:\> hostname
    CC1-HYP-16
    PS C:\> winrm id -r:cc1-hyp-10.domaincloud1.local
    IdentifyResponse
        ProtocolVersion =
    http://schemas.dmtf.org/wbem/wsman/1/wsman.xsd
        ProductVendor = Microsoft Corporation
        ProductVersion = OS: 6.3.9600 SP: 0.0 Stack: 3.0
        SecurityProfiles
            SecurityProfileName =
    http://schemas.dmtf.org/wbem/wsman/1/wsman/secprofile/http/spnego-kerberos
    And I can connect from the vmm server to all other cluster nodes:
    PS C:\> hostname
    CC1-VMM-01
    PS C:\> winrm id -r:cc1-hyp-11.domaincloud1.local
    IdentifyResponse
        ProtocolVersion =
    http://schemas.dmtf.org/wbem/wsman/1/wsman.xsd
        ProductVendor = Microsoft Corporation
        ProductVersion = OS: 6.3.9600 SP: 0.0 Stack: 3.0
        SecurityProfiles
            SecurityProfileName =
    http://schemas.dmtf.org/wbem/wsman/1/wsman/secprofile/http/spnego-kerberos
    So at this point only the test from the cc1-vmm-01 to cc1-hyp-10 seems to be problematic.
    I followed the steps in the page
    https://support.microsoft.com/kb/2742275 (which is referred to above). I tried the VMMCA, but it can't really get it working the way I want, or it seems to give outdated recommendations.
    I tried checking for duplicate SPN's by running setspn -x on affected machines. No results (although I do not understand
     what an SPN is or how it works). I rebuilt the performance counters.
    It tried setting 'sc config winrm type= own' as described in [http://blinditandnetworkadmin.blogspot.nl/2012/08/kb-how-to-troubleshoot-needs-attention.html].
    If I reboot this cc1-hyp-10 machine, it will start working perfectly again. However, then I can't troubleshoot the issue, and it will happen again.
    I want this problem to be solved, so vmm never loses connection to the hypervisors it's managing again!
    Background information:
    We've set up a platform with Hyper-V to run a VM workload. The platform consists of the following hardware:
    2 Dell R620's with 32GB of RAM, running hyper-v to virtualize the cloud management layer (DC's, VMM, SQL). These machines are called cc1-hyp-01 and cc1-hyp-02. They run the management vm's like cc1-dc-01/02, cc1-sql-01, cc1-vmm-01, etc. The names are self-explanatory.
    The VMM machine is NOT clustered.
    8 Dell M620 blades with 320GB of RAM, running hyper-v to virtualize the customer workload. The machines are
    called cc1-hyp-10 until cc1-hyp-17. They are in a cluster.
    2 Equallogic units form a SAN (premium storage), and we have a Dell R515 running iscsi target (budget storage).
    We have Dell Force10 switches and Cisco C3750X switches to connect everything together (mostly 10GB links).
    All hosts run Windows Server 2012R2 Datacenter edition. The VMM server runs System Center Virtual Machine Manage 2012 R2.
    All the latest Windows updates are installed on every host. There are no firewalls between any host (vmm and hypervisors) at this level. Windows firewalls are all disabled. No antivirus software is installed, no symantec software is installed.
    The only non-standard software that is installed is the Dell Host Integration Tools 4.7.1, Dell Openmanage Server Administrator, and some small stuff like 7-zip, bginfo, net-snap, etc.
    The SCVMM service is running under the domain account DOMAINCLOUD1\scvmm. This machine is in the local administrators group of each cluster node.
    On top of this cloud layer we're running the tenant layer with a lot of vm's for a specific customer (although they are all off now).

    I think I found the culprit, after an hour of analyzing wireshark dumps I found the vmm had jumbo frames enabled on the management interface to the hosts (and the underlying infrastructure does not).. Now my winrm commands started working again.

  • Error while getting cluster node subtree

    Hi,
      We are on SP15.
    The console logs show the following error
    log generation timestamp : 2006_01_17_at_17_14_05
    java.rmi.RemoteException: Error while getting cluster node subtree of :name=ClusterNodeRepresentative,j2eeType=com.sap.engine.services.adminadapter.impl.ClusterNodeRepresentative,SAP_J2EEClusterNode=2053400,SAP_J2EECluster=""; nested exception is:
         com.sap.engine.services.jmx.exception.MBeanServerClusterException: Exception during invocation of remote MBeanServer method, target node: 2053400
         at com.sap.engine.services.adminadapter.impl.ConvenienceEngineAdministratorImpl.getClusterNodeSubTree(ConvenienceEngineAdministratorImpl.java:242)
         at com.sap.engine.services.adminadapter.impl.ConvenienceEngineAdministratorImplp4_Skel.dispatch(ConvenienceEngineAdministratorImplp4_Skel.java:99)
         at com.sap.engine.services.rmi_p4.DispatchImpl._runInternal(DispatchImpl.java:304)
         at com.sap.engine.services.rmi_p4.DispatchImpl._run(DispatchImpl.java:193)
         at com.sap.engine.services.rmi_p4.server.P4SessionProcessor.request(P4SessionProcessor.java:122)
         at com.sap.engine.core.service630.context.cluster.session.ApplicationSessionMessageListener.process(ApplicationSessionMessageListener.java:33)
         at com.sap.engine.core.cluster.impl6.session.MessageRunner.run(MessageRunner.java:41)
         at com.sap.engine.core.thread.impl3.ActionObject.run(ActionObject.java:37)
         at java.security.AccessController.doPrivileged(Native Method)
         at com.sap.engine.core.thread.impl3.SingleThread.execute(SingleThread.java:100)
         at com.sap.engine.core.thread.impl3.SingleThread.run(SingleThread.java:170)
    Caused by: com.sap.engine.services.jmx.exception.MBeanServerClusterException: Exception during invocation of remote MBeanServer method, target node: 2053400
         at com.sap.engine.services.jmx.ClusterInterceptor.invoke(ClusterInterceptor.java:816)
         at com.sap.pj.jmx.server.interceptor.MBeanServerInterceptorChain.invoke(MBeanServerInterceptorChain.java:330)
         at com.sap.engine.services.adminadapter.impl.ConvenienceEngineAdministratorImpl.getClusterNodeSubTree(ConvenienceEngineAdministratorImpl.java:239)
         ... 10 more
    Caused by: com.sap.engine.services.jmx.exception.JmxConnectorException: Unable to de-serialize request parameters, message [ JMX request (java) v1.0 len: 345 |  src: cluster target-node: 2053400 req: invoke params-number: 4 params-bytes: 0 | :name=ClusterNodeRepresentative,j2eeType=com.sap.engine.services.adminadapter.impl.ClusterNodeRepresentative,SAP_J2EEClusterNode=2053400,SAP_J2EECluster="" null null null ]
         at com.sap.engine.services.jmx.MBeanServerConnectionImpl.invokeMbsInternal(MBeanServerConnectionImpl.java:680)
         at com.sap.engine.services.jmx.MBeanServerConnectionImpl.invoke(MBeanServerConnectionImpl.java:467)
         at com.sap.engine.services.jmx.MBeanServerConnectionSecurityWrapper.invoke(MBeanServerConnectionSecurityWrapper.java:221)
         at com.sap.engine.services.jmx.ClusterInterceptor.invoke(ClusterInterceptor.java:813)
         ... 12 more
    Caused by: javax.management.InstanceNotFoundException: MBean with name com.sap.default:name=ClusterNodeRepresentative,j2eeType=com.sap.engine.services.adminadapter.impl.ClusterNodeRepresentative,SAP_J2EEClusterNode=2053400,SAP_J2EECluster=XD1 not found in repository
         at com.sap.pj.jmx.server.MBeanServerImpl.getClassLoaderFor(MBeanServerImpl.java:1408)
         at com.sap.pj.jmx.server.interceptor.MBeanServerWrapperInterceptor.getClassLoaderFor(MBeanServerWrapperInterceptor.java:455)
         at com.sap.engine.services.jmx.CompletionInterceptor.getClassLoaderFor(CompletionInterceptor.java:567)
         at com.sap.pj.jmx.server.interceptor.BasicMBeanServerInterceptor.getClassLoaderFor(BasicMBeanServerInterceptor.java:438)
         at com.sap.jmx.provider.ProviderInterceptor.getClassLoaderFor(ProviderInterceptor.java:330)
         at com.sap.engine.services.jmx.RedirectInterceptor.getClassLoaderFor(RedirectInterceptor.java:501)
         at com.sap.pj.jmx.server.interceptor.MBeanServerInterceptorChain.getClassLoaderFor(MBeanServerInterceptorChain.java:443)
         at com.sap.engine.services.jmx.RequestMessage.readParams(RequestMessage.java:523)
         at com.sap.engine.services.jmx.RequestMessage.getParams(RequestMessage.java:578)
         at com.sap.engine.services.jmx.MBeanServerInvoker.invokeMbs(MBeanServerInvoker.java:106)
         at com.sap.engine.services.jmx.JmxServiceConnectorServer.receiveWait(JmxServiceConnectorServer.java:173)
         at com.sap.engine.core.service630.context.cluster.message.MessageListenerWrapper.process(MessageListenerWrapper.java:81)
         at com.sap.engine.core.cluster.impl6.ms.MSListenerThread.run(MSListenerThread.java:47)
         at com.sap.engine.frame.core.thread.Task.run(Task.java:64)
         at com.sap.engine.core.thread.impl6.SingleThread.execute(SingleThread.java:78)
         at com.sap.engine.core.thread.impl6.SingleThread.run(SingleThread.java:148)
    java.lang.NullPointerException
         at com.sap.engine.services.adminadapter.gui.ClusterView.addGlobalDispatcherServiceProperties(ClusterView.java:455)
         at com.sap.engine.services.adminadapter.gui.ClusterView.createGlobalTrees(ClusterView.java:508)
         at com.sap.engine.services.adminadapter.gui.ClusterView.access$1200(ClusterView.java:29)
         at com.sap.engine.services.adminadapter.gui.ClusterView$4.run(ClusterView.java:420)
    java.rmi.RemoteException: Error while getting cluster node subtree of :name=ClusterNodeRepresentative,j2eeType=com.sap.engine.services.adminadapter.impl.ClusterNodeRepresentative,SAP_J2EEClusterNode=2053400,SAP_J2EECluster=""; nested exception is:
         com.sap.engine.services.jmx.exception.MBeanServerClusterException: Exception during invocation of remote MBeanServer method, target node: 2053400
         at com.sap.engine.services.adminadapter.impl.ConvenienceEngineAdministratorImpl.getClusterNodeSubTree(ConvenienceEngineAdministratorImpl.java:242)
         at com.sap.engine.services.adminadapter.impl.ConvenienceEngineAdministratorImplp4_Skel.dispatch(ConvenienceEngineAdministratorImplp4_Skel.java:99)
         at com.sap.engine.services.rmi_p4.DispatchImpl._runInternal(DispatchImpl.java:304)
         at com.sap.engine.services.rmi_p4.DispatchImpl._run(DispatchImpl.java:193)
         at com.sap.engine.services.rmi_p4.server.P4SessionProcessor.request(P4SessionProcessor.java:122)
         at com.sap.engine.core.service630.context.cluster.session.ApplicationSessionMessageListener.process(ApplicationSessionMessageListener.java:33)
         at com.sap.engine.core.cluster.impl6.session.MessageRunner.run(MessageRunner.java:41)
         at com.sap.engine.core.thread.impl3.ActionObject.run(ActionObject.java:37)
         at java.security.AccessController.doPrivileged(Native Method)
         at com.sap.engine.core.thread.impl3.SingleThread.execute(SingleThread.java:100)
         at com.sap.engine.core.thread.impl3.SingleThread.run(SingleThread.java:170)
    Caused by: com.sap.engine.services.jmx.exception.MBeanServerClusterException: Exception during invocation of remote MBeanServer method, target node: 2053400
         at com.sap.engine.services.jmx.ClusterInterceptor.invoke(ClusterInterceptor.java:816)
         at com.sap.pj.jmx.server.interceptor.MBeanServerInterceptorChain.invoke(MBeanServerInterceptorChain.java:330)
         at com.sap.engine.services.adminadapter.impl.ConvenienceEngineAdministratorImpl.getClusterNodeSubTree(ConvenienceEngineAdministratorImpl.java:239)
         ... 10 more
    Caused by: com.sap.engine.services.jmx.exception.JmxConnectorException: Unable to de-serialize request parameters, message [ JMX request (java) v1.0 len: 345 |  src: cluster target-node: 2053400 req: invoke params-number: 4 params-bytes: 0 | :name=ClusterNodeRepresentative,j2eeType=com.sap.engine.services.adminadapter.impl.ClusterNodeRepresentative,SAP_J2EEClusterNode=2053400,SAP_J2EECluster="" null null null ]
         at com.sap.engine.services.jmx.MBeanServerConnectionImpl.invokeMbsInternal(MBeanServerConnectionImpl.java:680)
         at com.sap.engine.services.jmx.MBeanServerConnectionImpl.invoke(MBeanServerConnectionImpl.java:467)
         at com.sap.engine.services.jmx.MBeanServerConnectionSecurityWrapper.invoke(MBeanServerConnectionSecurityWrapper.java:221)
         at com.sap.engine.services.jmx.ClusterInterceptor.invoke(ClusterInterceptor.java:813)
         ... 12 more
    Caused by: javax.management.InstanceNotFoundException: MBean with name com.sap.default:name=ClusterNodeRepresentative,j2eeType=com.sap.engine.services.adminadapter.impl.ClusterNodeRepresentative,SAP_J2EEClusterNode=2053400,SAP_J2EECluster=XD1 not found in repository
         at com.sap.pj.jmx.server.MBeanServerImpl.getClassLoaderFor(MBeanServerImpl.java:1408)
         at com.sap.pj.jmx.server.interceptor.MBeanServerWrapperInterceptor.getClassLoaderFor(MBeanServerWrapperInterceptor.java:455)
         at com.sap.engine.services.jmx.CompletionInterceptor.getClassLoaderFor(CompletionInterceptor.java:567)
         at com.sap.pj.jmx.server.interceptor.BasicMBeanServerInterceptor.getClassLoaderFor(BasicMBeanServerInterceptor.java:438)
         at com.sap.jmx.provider.ProviderInterceptor.getClassLoaderFor(ProviderInterceptor.java:330)
         at com.sap.engine.services.jmx.RedirectInterceptor.getClassLoaderFor(RedirectInterceptor.java:501)
         at com.sap.pj.jmx.server.interceptor.MBeanServerInterceptorChain.getClassLoaderFor(MBeanServerInterceptorChain.java:443)
         at com.sap.engine.services.jmx.RequestMessage.readParams(RequestMessage.java:523)
         at com.sap.engine.services.jmx.RequestMessage.getParams(RequestMessage.java:578)
         at com.sap.engine.services.jmx.MBeanServerInvoker.invokeMbs(MBeanServerInvoker.java:106)
         at com.sap.engine.services.jmx.JmxServiceConnectorServer.receiveWait(JmxServiceConnectorServer.java:173)
         at com.sap.engine.core.service630.context.cluster.message.MessageListenerWrapper.process(MessageListenerWrapper.java:81)
         at com.sap.engine.core.cluster.impl6.ms.MSListenerThread.run(MSListenerThread.java:47)
         at com.sap.engine.frame.core.thread.Task.run(Task.java:64)
         at com.sap.engine.core.thread.impl6.SingleThread.execute(SingleThread.java:78)
         at com.sap.engine.core.thread.impl6.SingleThread.run(SingleThread.java:148)
    java.lang.NullPointerException
         at com.sap.engine.services.adminadapter.gui.ClusterView.addGlobalDispatcherServiceProperties(ClusterView.java:455)
         at com.sap.engine.services.adminadapter.gui.ClusterView.createGlobalTrees(ClusterView.java:508)
         at com.sap.engine.services.adminadapter.gui.ClusterView.access$1200(ClusterView.java:29)
         at com.sap.engine.services.adminadapter.gui.ClusterView$4.run(ClusterView.java:420)
    Any clue whats it?
    rgds

    Go the same error
    + /usr/java14_64/bin/java -showversion -Duser.language=en -DP4ClassLoad=P4Connection -Dp4Cache=clean -jar go.jar
    java version "1.4.2"
    Java(TM) 2 Runtime Environment, Standard Edition (build 1.4.2)
    Classic VM (build 1.4.2, J2RE 1.4.2 IBM AIX 5L for PowerPC (64 bit JVM) build caix64142ifx-20061222 (ifix 113727: SR7 + 112603) (JIT enabled: jitc))
    java.lang.NullPointerException
            at com.sap.engine.services.adminadapter.gui.ClusterView$4.run(ClusterView.java:405)
    Need some help!
    Bernard

  • INS-40925 - One or more nodes have interfaces not configured with a subnet that is common across all cluster nodes.

    Hi All,
    I am facing the below error while installing Oracle RAC in Silent Mode.
    SEVERE: There are no common subnets represented by network interfaces across all cluster nodes.
    SEVERE: [FATAL] [INS-40925] One or more nodes have interfaces not configured with a subnet that is common across all cluster nodes.
       CAUSE: Not all nodes have network interfaces that are configured on subnets that are common to all nodes in the cluster.
       ACTION: Ensure all cluster nodes have a public interface defined with the same subnet accessible by all nodes in the cluster.
    My /etc/hosts is given below.
    127.0.0.1        localhost    localhost.localdomain
    #Public
    192.168.1.101      rac1        rac1.localdomain
    192.168.1.102    rac2        rac2.localdomain
    #Private
    192.168.2.101    rac1-priv    rac1-priv.localdomain
    192.168.2.102    rac2-priv    rac2-priv.localdomain
    #Virtual
    192.168.1.103      rac1-vip    rac1-vip.localdomain
    192.168.1.104    rac2-vip    rac2-vip.localdomain
    #SCAN
    192.168.1.105    rac-scan    rac-scan.localdomain
    Could you please help me to get rid of the error INS-40925....Any Idea...???

    Hi Ramesh,
    Please find the result of ifconfig -a from both nodes RAC1 & RAC2.
    ifconfig -a in RAC1
    [oracle@rac1 Desktop]$ ifconfig -a
    eth0      Link encap:Ethernet  HWaddr 08:00:27:17:7A:D5
              inet addr:192.168.1.101  Bcast:192.168.1.255  Mask:255.255.255.0
              inet6 addr: fe80::a00:27ff:fe17:7ad5/64 Scope:Link
              UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
              RX packets:102 errors:0 dropped:0 overruns:0 frame:0
              TX packets:48 errors:0 dropped:0 overruns:0 carrier:0
              collisions:0 txqueuelen:1000
              RX bytes:25472 (24.8 KiB)  TX bytes:3322 (3.2 KiB)
              Interrupt:19 Base address:0xd020
    eth1      Link encap:Ethernet  HWaddr 08:00:27:C0:AC:DB
              inet addr:192.168.2.101  Bcast:192.168.2.255  Mask:255.255.255.0
              inet6 addr: fe80::a00:27ff:fec0:acdb/64 Scope:Link
              UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
              RX packets:4 errors:0 dropped:0 overruns:0 frame:0
              TX packets:12 errors:0 dropped:0 overruns:0 carrier:0
              collisions:0 txqueuelen:1000
              RX bytes:240 (240.0 b)  TX bytes:816 (816.0 b)
              Interrupt:16 Base address:0xd240
    lo        Link encap:Local Loopback
              inet addr:127.0.0.1  Mask:255.0.0.0
              inet6 addr: ::1/128 Scope:Host
              UP LOOPBACK RUNNING  MTU:16436  Metric:1
              RX packets:56 errors:0 dropped:0 overruns:0 frame:0
              TX packets:56 errors:0 dropped:0 overruns:0 carrier:0
              collisions:0 txqueuelen:0
              RX bytes:6394 (6.2 KiB)  TX bytes:6394 (6.2 KiB)
    virbr0    Link encap:Ethernet  HWaddr 52:54:00:CC:BD:FB
              inet addr:192.168.122.1  Bcast:192.168.122.255  Mask:255.255.255.0
              UP BROADCAST MULTICAST  MTU:1500  Metric:1
              RX packets:0 errors:0 dropped:0 overruns:0 frame:0
              TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
              collisions:0 txqueuelen:0
              RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)
    virbr0-nic Link encap:Ethernet  HWaddr 52:54:00:CC:BD:FB
              BROADCAST MULTICAST  MTU:1500  Metric:1
              RX packets:0 errors:0 dropped:0 overruns:0 frame:0
              TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
              collisions:0 txqueuelen:500
              RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)
    ifconfig -a in RAC2
    [oracle@rac2 Desktop]$ ifconfig -a
    eth0      Link encap:Ethernet  HWaddr 08:00:27:C9:38:82
              inet addr:192.168.1.102  Bcast:192.168.1.255  Mask:255.255.255.0
              inet6 addr: fe80::a00:27ff:fec9:3882/64 Scope:Link
              UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
              RX packets:122 errors:0 dropped:0 overruns:0 frame:0
              TX packets:59 errors:0 dropped:0 overruns:0 carrier:0
              collisions:0 txqueuelen:1000
              RX bytes:32617 (31.8 KiB)  TX bytes:5157 (5.0 KiB)
              Interrupt:19 Base address:0xd020
    eth1      Link encap:Ethernet  HWaddr 08:00:27:90:B5:A0
              inet addr:192.168.2.102  Bcast:192.168.2.255  Mask:255.255.255.0
              inet6 addr: fe80::a00:27ff:fe90:b5a0/64 Scope:Link
              UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
              RX packets:4 errors:0 dropped:0 overruns:0 frame:0
              TX packets:11 errors:0 dropped:0 overruns:0 carrier:0
              collisions:0 txqueuelen:1000
              RX bytes:240 (240.0 b)  TX bytes:746 (746.0 b)
              Interrupt:16 Base address:0xd240
    lo        Link encap:Local Loopback
              inet addr:127.0.0.1  Mask:255.0.0.0
              inet6 addr: ::1/128 Scope:Host
              UP LOOPBACK RUNNING  MTU:16436  Metric:1
              RX packets:56 errors:0 dropped:0 overruns:0 frame:0
              TX packets:56 errors:0 dropped:0 overruns:0 carrier:0
              collisions:0 txqueuelen:0
              RX bytes:6390 (6.2 KiB)  TX bytes:6390 (6.2 KiB)
    virbr0    Link encap:Ethernet  HWaddr 52:54:00:CC:BD:FB
              inet addr:192.168.122.1  Bcast:192.168.122.255  Mask:255.255.255.0
              UP BROADCAST MULTICAST  MTU:1500  Metric:1
              RX packets:0 errors:0 dropped:0 overruns:0 frame:0
              TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
              collisions:0 txqueuelen:0
              RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)
    virbr0-nic Link encap:Ethernet  HWaddr 52:54:00:CC:BD:FB
              BROADCAST MULTICAST  MTU:1500  Metric:1
              RX packets:0 errors:0 dropped:0 overruns:0 frame:0
              TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
              collisions:0 txqueuelen:500
              RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)

Maybe you are looking for