Ticket evaluation error on other cluster server node
Hello, All
Could you help me to solve such problem?
I created cluster on one server instance, for example J2E with two server nodes (0 and 1).
The some application working on server node 0 generates sso ticket. When I send this ticket to application on server node 1 the authorization process is not performed. But when I send this ticket to server node 0 all works fine. And and vice versa, i.e. the sso ticket generated on node 1 is right for the application running on node 1 but not for node 0.
In case of different nodes the EvaluationTicketLoginModule throws exception : Own SID not available.
Why does it happen thus? I work within the same J2E server instance.
Thanks in advance.
Sergey
Hi,
In our environment we have installed OBIEE 10.1.3.4 on two windows servers(primary and secondary) and these servers not clustered using Microsoft or any other cluster software .
We are planning to use SAN drive to hold the shared cache, now we are facing a challenge in configuring the SAN to see from both OBIEE servers, is it possible to use the SAN with out installing any cluster between these servers.
I understood from above post that you guys implemented/installed OBIEE on Windows environment with SAN.
Is it possible to have a quick call. Your help is greatly appreciated.
Thanks
Similar Messages
-
Cluster Server node port Details
Hi,
I have created server node with server1. How do i find the port number for server1. my server0 port number is 50000 and there is no change in IP address for server1.
Please help me.
Thanks,
Naveen kumarKishore
There will be only one port for each instance and its internal to the dispatcher of that instance
from the above statement there should be a port number for a instance and that is diff instance to instance, how do we find the port number for a particular instance
reason why i am asking i want to access my application with particular instance means direct access to instance thr iv-headers
i hope you understand my question
Thanks,
Naveen -
SCSI reset error on a cluster 2 nodes connected to a jbod 3310 storage
Dear Gents,
I will detail the environment which which the customer face the problem in. then i will till you what the problem he face:
1- he has 2 clustered servers which are connected to a 3310 JBOD storage (no controller), the 2 connections come through 2 different SCSI Cards Types as following (V480 has the SCSI Card P/N 375-3057 which is a PCI Dual Ultra3 SCSI Adapter) but (V440 has the SCSI Card P/N 375-3365 which is PCI Dual Ultra3 SCSI Adapter 320)
2- Add to the above point that both SCSI Cards use a different drivers to work as the V480 SCSI card use the QUS driver but the V440 SCSI Card use the MPT driver
3- 3310 storage support both SCSI Cards (Both Cards are compatible with the Storage but an issues are published and alerted from sun related to the V440 SCSI 320 card which we need to start by avoiding it)
---------------> The problem:
the problem that an errors come frequently on the both nodes as following:
Oct 11 14:22:19 tracssa01 scsi: [ID 107833 kern.notice] Sense Key: Unit Attention
Oct 11 14:22:19 tracssa01 scsi: [ID 107833 kern.notice] ASC: 0x29 (scsi bus reset occurred), ASCQ: 0x2, FRU: 0x0
Oct 11 14:25:48 tracssa01 scsi: [ID 107833 kern.warning] WARNING: /pci@8,600000/pci@1/scsi@4 (qus0):
Oct 11 14:25:48 tracssa01 Received unexpected SCSI Reset
Oct 11 14:25:50 tracssa01 scsi: [ID 107833 kern.warning] WARNING: /pci@8,600000/pci@1/scsi@4 (qus0):
Oct 11 14:25:50 tracssa01 Received unexpected SCSI Reset
Oct 11 14:25:50 tracssa01 scsi: [ID 107833 kern.warning] WARNING: /pci@8,600000/pci@1/scsi@4 (qus0):
Oct 11 14:25:50 tracssa01 Received unexpected SCSI Reset
Oct 11 14:25:53 tracssa01 scsi: [ID 107833 kern.warning] WARNING: /pci@8,600000/pci@1/scsi@4/sd@c,0 (sd13):
Oct 11 14:25:53 tracssa01 Error for Command: write(10) Error Level: Retryable
Oct 11 14:25:53 tracssa01 scsi: [ID 107833 kern.notice] Requested Block: 77279977 Error Block: 77279977
Oct 11 14:25:53 tracssa01 scsi: [ID 107833 kern.notice] Vendor: FUJITSU Serial Number: 0522B05YDB
Oct 11 14:25:53 tracssa01 scsi: [ID 107833 kern.notice] Sense Key: Unit Attention
Oct 11 14:25:53 tracssa01 scsi: [ID 107833 kern.notice] ASC: 0x29 (scsi bus reset occurred), ASCQ: 0x2, FRU: 0x0
Oct 11 14:25:53 tracssa01 scsi: [ID 107833 kern.warning] WARNING: /pci@8,600000/pci@1/scsi@4/sd@8,0 (sd9):
Oct 11 14:25:53 tracssa01 Error for Command: write Error Level: Retryable
Oct 11 14:25:53 tracssa01 scsi: [ID 107833 kern.notice] Requested Block: 475456 Error Block: 475456
i would highly appreciate if any one could fix it
Could you help me to fix this problem
Best Regardsthank you very much for your answer. i will replace the Card to make it Ultra3 instead of Ultra 320.
could you provide a procedure in how to do it as the cluster nodes will dettect a new path for the external storage and might gice an error or problems? -
Error Message on cluster server start
Hi all,
I'm trying to install a cluster with two Windows 2000 machines, and every
time I try to start my
server I get the same message from the output of the server:
It's about the weblogic.properties located in the installation directory and
the cluster directory.
java.lang.IlegalArgumentException : Malformed \uxxxx encoding
Any hint?
The only place I've seen that exception is from the JSP compiler
with -encoding utf-8 is set and it comes across some invalid utf-8 characters.
If it is really referring to the weblogic.properties file, you
may wish to look for strange characters in it.
It's always nice to post the stack-trace.
Mike
"Miquel Taberner" <[email protected]> wrote:
> Hi all,
>
>I'm trying to install a cluster with two Windows 2000 machines, and every
>time I try to start my
>server I get the same message from the output of the server:
>
>It's about the weblogic.properties located in the installation directory and
>the cluster directory.
>
>java.lang.IlegalArgumentException : Malformed \uxxxx encoding
>
>
>Any hint?
>
>
>
>
-
Hi,
New to 2012 and implementing a clustered environment for our File Services role. Have got to a point where I have successfully configured the Shadow copy settings.
Have a large (15tb) disk. S:
Have a VSS drive (volume shadow copy drive) V:
Have successfully configured through Windows Explorer the Shadow copy settings.
Created dependencies in Failcover Cluster Server console whereby S: depends on V:
However, when I failover the resource and browse the Client Access Point share there are no entries under the "Previous Versions" tab.
When I visit the S: drive in windows explorer and open the Shadow copy dialogue box, there are entries showing the times and dates of the shadow copies ran when on the original node. So the disk knows about the shadow copies that were ran on the
original node but the "previous versions" tab has no entries to display.
This is in a 2012 server (NOT R2 version).
Can anyone explain what might be the reason? Do I have an "issue" or is this by design?
All help apprecieated!
Kathy
Kathleen Hayhurst Senior IT Support AnalystHi,
Please first check the requirements in following article:
Using Shadow Copies of Shared Folders in a server cluster
http://technet.microsoft.com/en-us/library/cc779378(v=ws.10).aspx
Cluster-managed shadow copies can only be created in a single quorum device cluster on a disk with a Physical Disk resource. In a single node cluster or majority node set cluster without a shared cluster disk, shadow copies can only be created and managed
locally.
You cannot enable Shadow Copies of Shared Folders for the quorum resource, although you can enable Shadow Copies of Shared Folders for a File Share resource.
The recurring scheduled task that generates volume shadow copies must run on the same node that currently owns the storage volume.
The cluster resource that manages the scheduled task must be able to fail over with the Physical Disk resource that manages the storage volume.
If you have any feedback on our support, please send to [email protected] -
Error while getting cluster node subtree
Hi,
We are on SP15.
The console logs show the following error
log generation timestamp : 2006_01_17_at_17_14_05
java.rmi.RemoteException: Error while getting cluster node subtree of :name=ClusterNodeRepresentative,j2eeType=com.sap.engine.services.adminadapter.impl.ClusterNodeRepresentative,SAP_J2EEClusterNode=2053400,SAP_J2EECluster=""; nested exception is:
com.sap.engine.services.jmx.exception.MBeanServerClusterException: Exception during invocation of remote MBeanServer method, target node: 2053400
at com.sap.engine.services.adminadapter.impl.ConvenienceEngineAdministratorImpl.getClusterNodeSubTree(ConvenienceEngineAdministratorImpl.java:242)
at com.sap.engine.services.adminadapter.impl.ConvenienceEngineAdministratorImplp4_Skel.dispatch(ConvenienceEngineAdministratorImplp4_Skel.java:99)
at com.sap.engine.services.rmi_p4.DispatchImpl._runInternal(DispatchImpl.java:304)
at com.sap.engine.services.rmi_p4.DispatchImpl._run(DispatchImpl.java:193)
at com.sap.engine.services.rmi_p4.server.P4SessionProcessor.request(P4SessionProcessor.java:122)
at com.sap.engine.core.service630.context.cluster.session.ApplicationSessionMessageListener.process(ApplicationSessionMessageListener.java:33)
at com.sap.engine.core.cluster.impl6.session.MessageRunner.run(MessageRunner.java:41)
at com.sap.engine.core.thread.impl3.ActionObject.run(ActionObject.java:37)
at java.security.AccessController.doPrivileged(Native Method)
at com.sap.engine.core.thread.impl3.SingleThread.execute(SingleThread.java:100)
at com.sap.engine.core.thread.impl3.SingleThread.run(SingleThread.java:170)
Caused by: com.sap.engine.services.jmx.exception.MBeanServerClusterException: Exception during invocation of remote MBeanServer method, target node: 2053400
at com.sap.engine.services.jmx.ClusterInterceptor.invoke(ClusterInterceptor.java:816)
at com.sap.pj.jmx.server.interceptor.MBeanServerInterceptorChain.invoke(MBeanServerInterceptorChain.java:330)
at com.sap.engine.services.adminadapter.impl.ConvenienceEngineAdministratorImpl.getClusterNodeSubTree(ConvenienceEngineAdministratorImpl.java:239)
... 10 more
Caused by: com.sap.engine.services.jmx.exception.JmxConnectorException: Unable to de-serialize request parameters, message [ JMX request (java) v1.0 len: 345 | src: cluster target-node: 2053400 req: invoke params-number: 4 params-bytes: 0 | :name=ClusterNodeRepresentative,j2eeType=com.sap.engine.services.adminadapter.impl.ClusterNodeRepresentative,SAP_J2EEClusterNode=2053400,SAP_J2EECluster="" null null null ]
at com.sap.engine.services.jmx.MBeanServerConnectionImpl.invokeMbsInternal(MBeanServerConnectionImpl.java:680)
at com.sap.engine.services.jmx.MBeanServerConnectionImpl.invoke(MBeanServerConnectionImpl.java:467)
at com.sap.engine.services.jmx.MBeanServerConnectionSecurityWrapper.invoke(MBeanServerConnectionSecurityWrapper.java:221)
at com.sap.engine.services.jmx.ClusterInterceptor.invoke(ClusterInterceptor.java:813)
... 12 more
Caused by: javax.management.InstanceNotFoundException: MBean with name com.sap.default:name=ClusterNodeRepresentative,j2eeType=com.sap.engine.services.adminadapter.impl.ClusterNodeRepresentative,SAP_J2EEClusterNode=2053400,SAP_J2EECluster=XD1 not found in repository
at com.sap.pj.jmx.server.MBeanServerImpl.getClassLoaderFor(MBeanServerImpl.java:1408)
at com.sap.pj.jmx.server.interceptor.MBeanServerWrapperInterceptor.getClassLoaderFor(MBeanServerWrapperInterceptor.java:455)
at com.sap.engine.services.jmx.CompletionInterceptor.getClassLoaderFor(CompletionInterceptor.java:567)
at com.sap.pj.jmx.server.interceptor.BasicMBeanServerInterceptor.getClassLoaderFor(BasicMBeanServerInterceptor.java:438)
at com.sap.jmx.provider.ProviderInterceptor.getClassLoaderFor(ProviderInterceptor.java:330)
at com.sap.engine.services.jmx.RedirectInterceptor.getClassLoaderFor(RedirectInterceptor.java:501)
at com.sap.pj.jmx.server.interceptor.MBeanServerInterceptorChain.getClassLoaderFor(MBeanServerInterceptorChain.java:443)
at com.sap.engine.services.jmx.RequestMessage.readParams(RequestMessage.java:523)
at com.sap.engine.services.jmx.RequestMessage.getParams(RequestMessage.java:578)
at com.sap.engine.services.jmx.MBeanServerInvoker.invokeMbs(MBeanServerInvoker.java:106)
at com.sap.engine.services.jmx.JmxServiceConnectorServer.receiveWait(JmxServiceConnectorServer.java:173)
at com.sap.engine.core.service630.context.cluster.message.MessageListenerWrapper.process(MessageListenerWrapper.java:81)
at com.sap.engine.core.cluster.impl6.ms.MSListenerThread.run(MSListenerThread.java:47)
at com.sap.engine.frame.core.thread.Task.run(Task.java:64)
at com.sap.engine.core.thread.impl6.SingleThread.execute(SingleThread.java:78)
at com.sap.engine.core.thread.impl6.SingleThread.run(SingleThread.java:148)
java.lang.NullPointerException
at com.sap.engine.services.adminadapter.gui.ClusterView.addGlobalDispatcherServiceProperties(ClusterView.java:455)
at com.sap.engine.services.adminadapter.gui.ClusterView.createGlobalTrees(ClusterView.java:508)
at com.sap.engine.services.adminadapter.gui.ClusterView.access$1200(ClusterView.java:29)
at com.sap.engine.services.adminadapter.gui.ClusterView$4.run(ClusterView.java:420)
java.rmi.RemoteException: Error while getting cluster node subtree of :name=ClusterNodeRepresentative,j2eeType=com.sap.engine.services.adminadapter.impl.ClusterNodeRepresentative,SAP_J2EEClusterNode=2053400,SAP_J2EECluster=""; nested exception is:
com.sap.engine.services.jmx.exception.MBeanServerClusterException: Exception during invocation of remote MBeanServer method, target node: 2053400
at com.sap.engine.services.adminadapter.impl.ConvenienceEngineAdministratorImpl.getClusterNodeSubTree(ConvenienceEngineAdministratorImpl.java:242)
at com.sap.engine.services.adminadapter.impl.ConvenienceEngineAdministratorImplp4_Skel.dispatch(ConvenienceEngineAdministratorImplp4_Skel.java:99)
at com.sap.engine.services.rmi_p4.DispatchImpl._runInternal(DispatchImpl.java:304)
at com.sap.engine.services.rmi_p4.DispatchImpl._run(DispatchImpl.java:193)
at com.sap.engine.services.rmi_p4.server.P4SessionProcessor.request(P4SessionProcessor.java:122)
at com.sap.engine.core.service630.context.cluster.session.ApplicationSessionMessageListener.process(ApplicationSessionMessageListener.java:33)
at com.sap.engine.core.cluster.impl6.session.MessageRunner.run(MessageRunner.java:41)
at com.sap.engine.core.thread.impl3.ActionObject.run(ActionObject.java:37)
at java.security.AccessController.doPrivileged(Native Method)
at com.sap.engine.core.thread.impl3.SingleThread.execute(SingleThread.java:100)
at com.sap.engine.core.thread.impl3.SingleThread.run(SingleThread.java:170)
Caused by: com.sap.engine.services.jmx.exception.MBeanServerClusterException: Exception during invocation of remote MBeanServer method, target node: 2053400
at com.sap.engine.services.jmx.ClusterInterceptor.invoke(ClusterInterceptor.java:816)
at com.sap.pj.jmx.server.interceptor.MBeanServerInterceptorChain.invoke(MBeanServerInterceptorChain.java:330)
at com.sap.engine.services.adminadapter.impl.ConvenienceEngineAdministratorImpl.getClusterNodeSubTree(ConvenienceEngineAdministratorImpl.java:239)
... 10 more
Caused by: com.sap.engine.services.jmx.exception.JmxConnectorException: Unable to de-serialize request parameters, message [ JMX request (java) v1.0 len: 345 | src: cluster target-node: 2053400 req: invoke params-number: 4 params-bytes: 0 | :name=ClusterNodeRepresentative,j2eeType=com.sap.engine.services.adminadapter.impl.ClusterNodeRepresentative,SAP_J2EEClusterNode=2053400,SAP_J2EECluster="" null null null ]
at com.sap.engine.services.jmx.MBeanServerConnectionImpl.invokeMbsInternal(MBeanServerConnectionImpl.java:680)
at com.sap.engine.services.jmx.MBeanServerConnectionImpl.invoke(MBeanServerConnectionImpl.java:467)
at com.sap.engine.services.jmx.MBeanServerConnectionSecurityWrapper.invoke(MBeanServerConnectionSecurityWrapper.java:221)
at com.sap.engine.services.jmx.ClusterInterceptor.invoke(ClusterInterceptor.java:813)
... 12 more
Caused by: javax.management.InstanceNotFoundException: MBean with name com.sap.default:name=ClusterNodeRepresentative,j2eeType=com.sap.engine.services.adminadapter.impl.ClusterNodeRepresentative,SAP_J2EEClusterNode=2053400,SAP_J2EECluster=XD1 not found in repository
at com.sap.pj.jmx.server.MBeanServerImpl.getClassLoaderFor(MBeanServerImpl.java:1408)
at com.sap.pj.jmx.server.interceptor.MBeanServerWrapperInterceptor.getClassLoaderFor(MBeanServerWrapperInterceptor.java:455)
at com.sap.engine.services.jmx.CompletionInterceptor.getClassLoaderFor(CompletionInterceptor.java:567)
at com.sap.pj.jmx.server.interceptor.BasicMBeanServerInterceptor.getClassLoaderFor(BasicMBeanServerInterceptor.java:438)
at com.sap.jmx.provider.ProviderInterceptor.getClassLoaderFor(ProviderInterceptor.java:330)
at com.sap.engine.services.jmx.RedirectInterceptor.getClassLoaderFor(RedirectInterceptor.java:501)
at com.sap.pj.jmx.server.interceptor.MBeanServerInterceptorChain.getClassLoaderFor(MBeanServerInterceptorChain.java:443)
at com.sap.engine.services.jmx.RequestMessage.readParams(RequestMessage.java:523)
at com.sap.engine.services.jmx.RequestMessage.getParams(RequestMessage.java:578)
at com.sap.engine.services.jmx.MBeanServerInvoker.invokeMbs(MBeanServerInvoker.java:106)
at com.sap.engine.services.jmx.JmxServiceConnectorServer.receiveWait(JmxServiceConnectorServer.java:173)
at com.sap.engine.core.service630.context.cluster.message.MessageListenerWrapper.process(MessageListenerWrapper.java:81)
at com.sap.engine.core.cluster.impl6.ms.MSListenerThread.run(MSListenerThread.java:47)
at com.sap.engine.frame.core.thread.Task.run(Task.java:64)
at com.sap.engine.core.thread.impl6.SingleThread.execute(SingleThread.java:78)
at com.sap.engine.core.thread.impl6.SingleThread.run(SingleThread.java:148)
java.lang.NullPointerException
at com.sap.engine.services.adminadapter.gui.ClusterView.addGlobalDispatcherServiceProperties(ClusterView.java:455)
at com.sap.engine.services.adminadapter.gui.ClusterView.createGlobalTrees(ClusterView.java:508)
at com.sap.engine.services.adminadapter.gui.ClusterView.access$1200(ClusterView.java:29)
at com.sap.engine.services.adminadapter.gui.ClusterView$4.run(ClusterView.java:420)
Any clue whats it?
rgdsGo the same error
+ /usr/java14_64/bin/java -showversion -Duser.language=en -DP4ClassLoad=P4Connection -Dp4Cache=clean -jar go.jar
java version "1.4.2"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.4.2)
Classic VM (build 1.4.2, J2RE 1.4.2 IBM AIX 5L for PowerPC (64 bit JVM) build caix64142ifx-20061222 (ifix 113727: SR7 + 112603) (JIT enabled: jitc))
java.lang.NullPointerException
at com.sap.engine.services.adminadapter.gui.ClusterView$4.run(ClusterView.java:405)
Need some help!
Bernard -
Team,
Unable to start FE service on one of the FE server, 2 Enterprise lync 2013 pool one FE in each pool, only following error in event vrw,
Log Name: Lync Server
Source: LS Server
Date: 12/30/2013 12:10:55 PM
Event ID: 12290
Task Category: (1000)
Level: Error
Keywords: Classic
User: N/A
Computer: ACS465-BH102.me.ykgw.net
Description:
The evaluation period has expired.
The evaluation period for Microsoft Lync Server 2013 has expired. Please upgrade from the evaluation version to the fully licensed version of the product. Look at help for Setup.exe to learn how to upgrade from evaluation version to the licensed version.
Cause: The evaluation period for Microsoft Lync Server 2013 has expired.
Resolution:
Please upgrade from the evaluation version to the licensed version of the product. Look at help for Setup.exe to learn how to upgrade from evaluation version to the licensed version.
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
<System>
<Provider Name="LS Server" />
<EventID Qualifiers="50152">12290</EventID>
<Level>2</Level>
<Task>1000</Task>
<Keywords>0x80000000000000</Keywords>
<TimeCreated SystemTime="2013-12-30T09:10:55.000000000Z" />
<EventRecordID>73953</EventRecordID>
<Channel>Lync Server</Channel>
<Computer>ACS465-BH102.me.ykgw.net</Computer>
<Security />
</System>
<EventData>
</EventData>
</Event>
Log Name: System
Source: Schannel
Date: 12/30/2013 12:13:36 PM
Event ID: 36888
Task Category: None
Level: Error
Keywords:
User: SYSTEM
Computer: ACS465-BH102.me.ykgw.net
Description:
The following fatal alert was generated: 10. The internal error state is 1203.
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
<System>
<Provider Name="Schannel" Guid="{1F678132-5938-4686-9FDC-C8FF68F15C85}" />
<EventID>36888</EventID>
<Version>0</Version>
<Level>2</Level>
<Task>0</Task>
<Opcode>0</Opcode>
<Keywords>0x8000000000000000</Keywords>
<TimeCreated SystemTime="2013-12-30T09:13:36.779033200Z" />
<EventRecordID>83985</EventRecordID>
<Correlation />
<Execution ProcessID="556" ThreadID="3668" />
<Channel>System</Channel>
<Computer>ACS465-BH102.me.ykgw.net</Computer>
<Security UserID="S-1-5-18" />
</System>
<EventData>
<Data Name="AlertDesc">10</Data>
<Data Name="ErrorState">1203</Data>
</EventData>
</Event>
Server is already on Full version, ran Reset-CsPoolRegistrarState
for fillrest as wel as serviceret
Followed below as well,
http://social.technet.microsoft.com/Forums/lync/en-US/2a7e27ce-2dea-4e37-91ea-1ed42e110198/issue-updating-from-eval-to-volume-licenses?forum=ocsplanningdeployment
Currently done failover to another pool and users can login,
however i cant get the front end service on this server.Any pointers would be appreciated.
Praveen | MCSE Messaging 2003rit, the command is not doing the trick since its already full version,
PS C:\Users\lyncadmin> Get-CsServerVersion
Microsoft Lync Server 2013 (5.0.8308.0): Volume license key installed.
only one server in one pool, total two pools.
Lync 2013 is on Win 2008 R2, and the event in system im inclining to since i have tried all,
Log Name: System
Source: Schannel
Date: 12/30/2013 9:26:34 AM
Event ID: 36888
Task Category: None
Level: Error
Keywords:
User: SYSTEM
Computer: ACS465-BH102.me.ykgw.net
Description:
The following fatal alert was generated: 10. The internal error state is 1203.
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
<System>
<Provider Name="Schannel" Guid="{1F678132-5938-4686-9FDC-C8FF68F15C85}" />
<EventID>36888</EventID>
<Version>0</Version>
<Level>2</Level>
<Task>0</Task>
<Opcode>0</Opcode>
<Keywords>0x8000000000000000</Keywords>
<TimeCreated SystemTime="2013-12-30T06:26:34.877077500Z" />
<EventRecordID>64911</EventRecordID>
<Correlation />
<Execution ProcessID="556" ThreadID="620" />
<Channel>System</Channel>
<Computer>ACS465-BH102.me.ykgw.net</Computer>
<Security UserID="S-1-5-18" />
</System>
<EventData>
<Data Name="AlertDesc">10</Data>
<Data Name="ErrorState">1203</Data>
</EventData>
</Event>
Praveen | MCSE Messaging 2003 -
Error: Halting this cluster node due to unrecoverable service failure
Our cluster has experienced some sort of fault that has only become apparent today. The origin appears to have been nearly a month ago yet the symptoms have only just manifested.
The node in question is a standalone instance running a DistributedCache service with local storage. It output the following to stdout on Jan-22:
Coherence <Error>: Halting this cluster node due to unrecoverable service failure
It finally failed today with OutOfMemoryError: Java heap space.
We're running coherence-3.5.2.jar.
Q1: It looks like this node failed on Jan-22 yet we did not notice. What is the best way to monitor node health?
Q2: What might the root cause be for such a fault?
I found the following in the logs:
2011-01-22 01:18:58,296 Coherence Logger@9216774 3.5.2/463 ERROR 2011-01-22 01:18:58.296/9910749.462 Oracle Coherence EE 3.5.2/463 <Error> (thread=Cluster, member=33): Attempting recovery (due to soft timeout) of Guard{Daemon=DistributedCache}
2011-01-22 01:18:58,296 Coherence Logger@9216774 3.5.2/463 ERROR 2011-01-22 01:18:58.296/9910749.462 Oracle Coherence EE 3.5.2/463 <Error> (thread=Cluster, member=33): Attempting recovery (due to soft timeout) of Guard{Daemon=DistributedCache}
2011-01-22 01:19:04,772 Coherence Logger@9216774 3.5.2/463 ERROR 2011-01-22 01:19:04.772/9910755.938 Oracle Coherence EE 3.5.2/463 <Error> (thread=Cluster, member=33): Terminating guarded execution (due to hard timeout) of Guard{Daemon=DistributedCache}
2011-01-22 01:19:04,772 Coherence Logger@9216774 3.5.2/463 ERROR 2011-01-22 01:19:04.772/9910755.938 Oracle Coherence EE 3.5.2/463 <Error> (thread=Cluster, member=33): Terminating guarded execution (due to hard timeout) of Guard{Daemon=DistributedCache}
2011-01-22 01:19:05,785 Coherence Logger@9216774 3.5.2/463 ERROR 2011-01-22 01:19:05.785/9910756.951 Oracle Coherence EE 3.5.2/463 <Error> (thread=Termination Thread, member=33): Full Thread Dump
Thread[Reference Handler,10,system]
java.lang.Object.wait(Native Method)
java.lang.Object.wait(Object.java:485)
java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116)
Thread[DistributedCache,5,Cluster]
java.nio.Bits.copyToByteArray(Native Method)
java.nio.DirectByteBuffer.get(DirectByteBuffer.java:224)
com.tangosol.io.nio.ByteBufferInputStream.read(ByteBufferInputStream.java:123)
java.io.DataInputStream.readFully(DataInputStream.java:178)
java.io.DataInputStream.readFully(DataInputStream.java:152)
com.tangosol.util.Binary.readExternal(Binary.java:1066)
com.tangosol.util.Binary.<init>(Binary.java:183)
com.tangosol.io.nio.BinaryMap$Block.readValue(BinaryMap.java:4304)
com.tangosol.io.nio.BinaryMap$Block.getValue(BinaryMap.java:4130)
com.tangosol.io.nio.BinaryMap.get(BinaryMap.java:377)
com.tangosol.io.nio.BinaryMapStore.load(BinaryMapStore.java:64)
com.tangosol.net.cache.SerializationPagedCache$WrapperBinaryStore.load(SerializationPagedCache.java:1547)
com.tangosol.net.cache.SerializationPagedCache$PagedBinaryStore.load(SerializationPagedCache.java:1097)
com.tangosol.net.cache.SerializationMap.get(SerializationMap.java:121)
com.tangosol.net.cache.SerializationPagedCache.get(SerializationPagedCache.java:247)
com.tangosol.net.cache.AbstractSerializationCache$1.getOldValue(AbstractSerializationCache.java:315)
com.tangosol.net.cache.OverflowMap$Status.registerBackEvent(OverflowMap.java:4210)
com.tangosol.net.cache.OverflowMap.onBackEvent(OverflowMap.java:2316)
com.tangosol.net.cache.OverflowMap$BackMapListener.onMapEvent(OverflowMap.java:4544)
com.tangosol.util.MultiplexingMapListener.entryDeleted(MultiplexingMapListener.java:49)
com.tangosol.util.MapEvent.dispatch(MapEvent.java:214)
com.tangosol.util.MapEvent.dispatch(MapEvent.java:166)
com.tangosol.util.MapListenerSupport.fireEvent(MapListenerSupport.java:556)
com.tangosol.net.cache.AbstractSerializationCache.dispatchEvent(AbstractSerializationCache.java:338)
com.tangosol.net.cache.AbstractSerializationCache.dispatchPendingEvent(AbstractSerializationCache.java:321)
com.tangosol.net.cache.AbstractSerializationCache.removeBlind(AbstractSerializationCache.java:155)
com.tangosol.net.cache.SerializationPagedCache.removeBlind(SerializationPagedCache.java:348)
com.tangosol.util.AbstractKeyBasedMap$KeySet.remove(AbstractKeyBasedMap.java:556)
com.tangosol.net.cache.OverflowMap.removeInternal(OverflowMap.java:1299)
com.tangosol.net.cache.OverflowMap.remove(OverflowMap.java:380)
com.tangosol.coherence.component.util.daemon.queueProcessor.service.grid.DistributedCache$Storage.clear(DistributedCache.CDB:24)
com.tangosol.coherence.component.util.daemon.queueProcessor.service.grid.DistributedCache.onClearRequest(DistributedCache.CDB:32)
com.tangosol.coherence.component.util.daemon.queueProcessor.service.grid.DistributedCache$ClearRequest.run(DistributedCache.CDB:1)
com.tangosol.coherence.component.net.message.requestMessage.DistributedCacheRequest.onReceived(DistributedCacheRequest.CDB:12)
com.tangosol.coherence.component.util.daemon.queueProcessor.service.Grid.onMessage(Grid.CDB:9)
com.tangosol.coherence.component.util.daemon.queueProcessor.service.Grid.onNotify(Grid.CDB:136)
com.tangosol.coherence.component.util.daemon.queueProcessor.service.grid.DistributedCache.onNotify(DistributedCache.CDB:3)
com.tangosol.coherence.component.util.Daemon.run(Daemon.CDB:42)
java.lang.Thread.run(Thread.java:619)
Thread[Finalizer,8,system]
java.lang.Object.wait(Native Method)
java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:118)
java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:134)
java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159)
Thread[PacketReceiver,7,Cluster]
java.lang.Object.wait(Native Method)
com.tangosol.coherence.component.util.Daemon.onWait(Daemon.CDB:18)
com.tangosol.coherence.component.util.daemon.queueProcessor.packetProcessor.PacketReceiver.onWait(PacketReceiver.CDB:2)
com.tangosol.coherence.component.util.Daemon.run(Daemon.CDB:39)
java.lang.Thread.run(Thread.java:619)
Thread[RMI TCP Accept-0,5,system]
java.net.PlainSocketImpl.socketAccept(Native Method)
java.net.PlainSocketImpl.accept(PlainSocketImpl.java:390)
java.net.ServerSocket.implAccept(ServerSocket.java:453)
java.net.ServerSocket.accept(ServerSocket.java:421)
sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:369)
sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:341)
java.lang.Thread.run(Thread.java:619)
Thread[PacketSpeaker,8,Cluster]
java.lang.Object.wait(Native Method)
com.tangosol.coherence.component.util.queue.ConcurrentQueue.waitForEntry(ConcurrentQueue.CDB:16)
com.tangosol.coherence.component.util.queue.ConcurrentQueue.remove(ConcurrentQueue.CDB:7)
com.tangosol.coherence.component.util.Queue.remove(Queue.CDB:1)
com.tangosol.coherence.component.util.daemon.queueProcessor.packetProcessor.PacketSpeaker.onNotify(PacketSpeaker.CDB:62)
com.tangosol.coherence.component.util.Daemon.run(Daemon.CDB:42)
java.lang.Thread.run(Thread.java:619)
Thread[Logger@9216774 3.5.2/463,3,main]
java.lang.Object.wait(Native Method)
com.tangosol.coherence.component.util.Daemon.onWait(Daemon.CDB:18)
com.tangosol.coherence.component.util.Daemon.run(Daemon.CDB:39)
java.lang.Thread.run(Thread.java:619)
Thread[PacketListener1,8,Cluster]
java.net.PlainDatagramSocketImpl.receive0(Native Method)
java.net.PlainDatagramSocketImpl.receive(PlainDatagramSocketImpl.java:136)
java.net.DatagramSocket.receive(DatagramSocket.java:712)
com.tangosol.coherence.component.net.socket.UdpSocket.receive(UdpSocket.CDB:20)
com.tangosol.coherence.component.net.UdpPacket.receive(UdpPacket.CDB:4)
com.tangosol.coherence.component.util.daemon.queueProcessor.packetProcessor.PacketListener.onNotify(PacketListener.CDB:19)
com.tangosol.coherence.component.util.Daemon.run(Daemon.CDB:42)
java.lang.Thread.run(Thread.java:619)
Thread[main,5,main]
java.lang.Object.wait(Native Method)
com.tangosol.net.DefaultCacheServer.main(DefaultCacheServer.java:79)
com.networkfleet.cacheserver.Launcher.main(Launcher.java:122)
Thread[Signal Dispatcher,9,system]
Thread[RMI TCP Accept-41006,5,system]
java.net.PlainSocketImpl.socketAccept(Native Method)
java.net.PlainSocketImpl.accept(PlainSocketImpl.java:390)
java.net.ServerSocket.implAccept(ServerSocket.java:453)
java.net.ServerSocket.accept(ServerSocket.java:421)
sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:369)
sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:341)
java.lang.Thread.run(Thread.java:619)
ThreadCluster
java.lang.Object.wait(Native Method)
com.tangosol.coherence.component.util.Daemon.onWait(Daemon.CDB:18)
com.tangosol.coherence.component.util.daemon.queueProcessor.service.Grid.onWait(Grid.CDB:9)
com.tangosol.coherence.component.util.Daemon.run(Daemon.CDB:39)
java.lang.Thread.run(Thread.java:619)
Thread[TcpRingListener,6,Cluster]
java.net.PlainSocketImpl.socketAccept(Native Method)
java.net.PlainSocketImpl.accept(PlainSocketImpl.java:390)
java.net.ServerSocket.implAccept(ServerSocket.java:453)
java.net.ServerSocket.accept(ServerSocket.java:421)
com.tangosol.coherence.component.net.socket.TcpSocketAccepter.accept(TcpSocketAccepter.CDB:18)
com.tangosol.coherence.component.util.daemon.TcpRingListener.acceptConnection(TcpRingListener.CDB:10)
com.tangosol.coherence.component.util.daemon.TcpRingListener.onNotify(TcpRingListener.CDB:9)
com.tangosol.coherence.component.util.Daemon.run(Daemon.CDB:42)
java.lang.Thread.run(Thread.java:619)
Thread[PacketPublisher,6,Cluster]
java.lang.Object.wait(Native Method)
com.tangosol.coherence.component.util.Daemon.onWait(Daemon.CDB:18)
com.tangosol.coherence.component.util.daemon.queueProcessor.packetProcessor.PacketPublisher.onWait(PacketPublisher.CDB:2)
com.tangosol.coherence.component.util.Daemon.run(Daemon.CDB:39)
java.lang.Thread.run(Thread.java:619)
Thread[RMI TCP Accept-0,5,system]
java.net.PlainSocketImpl.socketAccept(Native Method)
java.net.PlainSocketImpl.accept(PlainSocketImpl.java:390)
java.net.ServerSocket.implAccept(ServerSocket.java:453)
java.net.ServerSocket.accept(ServerSocket.java:421)
sun.management.jmxremote.LocalRMIServerSocketFactory$1.accept(LocalRMIServerSocketFactory.java:34)
sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:369)
sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:341)
java.lang.Thread.run(Thread.java:619)
Thread[PacketListenerN,8,Cluster]
java.net.PlainDatagramSocketImpl.receive0(Native Method)
java.net.PlainDatagramSocketImpl.receive(PlainDatagramSocketImpl.java:136)
java.net.DatagramSocket.receive(DatagramSocket.java:712)
com.tangosol.coherence.component.net.socket.UdpSocket.receive(UdpSocket.CDB:20)
com.tangosol.coherence.component.net.UdpPacket.receive(UdpPacket.CDB:4)
com.tangosol.coherence.component.util.daemon.queueProcessor.packetProcessor.PacketListener.onNotify(PacketListener.CDB:19)
com.tangosol.coherence.component.util.Daemon.run(Daemon.CDB:42)
java.lang.Thread.run(Thread.java:619)
Thread[Invocation:Management,5,Cluster]
java.lang.Object.wait(Native Method)
com.tangosol.coherence.component.util.Daemon.onWait(Daemon.CDB:18)
com.tangosol.coherence.component.util.daemon.queueProcessor.service.Grid.onWait(Grid.CDB:9)
com.tangosol.coherence.component.util.Daemon.run(Daemon.CDB:39)
java.lang.Thread.run(Thread.java:619)
Thread[DistributedCache:PofDistributedCache,5,Cluster]
java.lang.Object.wait(Native Method)
com.tangosol.coherence.component.util.Daemon.onWait(Daemon.CDB:18)
com.tangosol.coherence.component.util.daemon.queueProcessor.service.Grid.onWait(Grid.CDB:9)
com.tangosol.coherence.component.util.Daemon.run(Daemon.CDB:39)
java.lang.Thread.run(Thread.java:619)
Thread[Invocation:Management:EventDispatcher,5,Cluster]
java.lang.Object.wait(Native Method)
com.tangosol.coherence.component.util.Daemon.onWait(Daemon.CDB:18)
com.tangosol.coherence.component.util.daemon.queueProcessor.Service$EventDispatcher.onWait(Service.CDB:7)
com.tangosol.coherence.component.util.Daemon.run(Daemon.CDB:39)
java.lang.Thread.run(Thread.java:619)
Thread[Termination Thread,5,Cluster]
java.lang.Thread.dumpThreads(Native Method)
java.lang.Thread.getAllStackTraces(Thread.java:1487)
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
java.lang.reflect.Method.invoke(Method.java:597)
com.tangosol.net.GuardSupport.logStackTraces(GuardSupport.java:791)
com.tangosol.coherence.component.net.Cluster.onServiceFailed(Cluster.CDB:5)
com.tangosol.coherence.component.util.daemon.queueProcessor.service.Grid$Guard.terminate(Grid.CDB:17)
com.tangosol.net.GuardSupport$2.run(GuardSupport.java:652)
java.lang.Thread.run(Thread.java:619)
2011-01-22 01:19:05,785 Coherence Logger@9216774 3.5.2/463 ERROR 2011-01-22 01:19:05.785/9910756.951 Oracle Coherence EE 3.5.2/463 <Error> (thread=Termination Thread, member=33): Full Thread Dump
Thread[Reference Handler,10,system]
java.lang.Object.wait(Native Method)
java.lang.Object.wait(Object.java:485)
java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116)
Thread[DistributedCache,5,Cluster]
java.nio.Bits.copyToByteArray(Native Method)
java.nio.DirectByteBuffer.get(DirectByteBuffer.java:224)
com.tangosol.io.nio.ByteBufferInputStream.read(ByteBufferInputStream.java:123)
java.io.DataInputStream.readFully(DataInputStream.java:178)
java.io.DataInputStream.readFully(DataInputStream.java:152)
com.tangosol.util.Binary.readExternal(Binary.java:1066)
com.tangosol.util.Binary.<init>(Binary.java:183)
com.tangosol.io.nio.BinaryMap$Block.readValue(BinaryMap.java:4304)
com.tangosol.io.nio.BinaryMap$Block.getValue(BinaryMap.java:4130)
com.tangosol.io.nio.BinaryMap.get(BinaryMap.java:377)
com.tangosol.io.nio.BinaryMapStore.load(BinaryMapStore.java:64)
com.tangosol.net.cache.SerializationPagedCache$WrapperBinaryStore.load(SerializationPagedCache.java:1547)
com.tangosol.net.cache.SerializationPagedCache$PagedBinaryStore.load(SerializationPagedCache.java:1097)
com.tangosol.net.cache.SerializationMap.get(SerializationMap.java:121)
com.tangosol.net.cache.SerializationPagedCache.get(SerializationPagedCache.java:247)
com.tangosol.net.cache.AbstractSerializationCache$1.getOldValue(AbstractSerializationCache.java:315)
com.tangosol.net.cache.OverflowMap$Status.registerBackEvent(OverflowMap.java:4210)
com.tangosol.net.cache.OverflowMap.onBackEvent(OverflowMap.java:2316)
com.tangosol.net.cache.OverflowMap$BackMapListener.onMapEvent(OverflowMap.java:4544)
com.tangosol.util.MultiplexingMapListener.entryDeleted(MultiplexingMapListener.java:49)
com.tangosol.util.MapEvent.dispatch(MapEvent.java:214)
com.tangosol.util.MapEvent.dispatch(MapEvent.java:166)
com.tangosol.util.MapListenerSupport.fireEvent(MapListenerSupport.java:556)
com.tangosol.net.cache.AbstractSerializationCache.dispatchEvent(AbstractSerializationCache.java:338)
com.tangosol.net.cache.AbstractSerializationCache.dispatchPendingEvent(AbstractSerializationCache.java:321)
com.tangosol.net.cache.AbstractSerializationCache.removeBlind(AbstractSerializationCache.java:155)
com.tangosol.net.cache.SerializationPagedCache.removeBlind(SerializationPagedCache.java:348)
com.tangosol.util.AbstractKeyBasedMap$KeySet.remove(AbstractKeyBasedMap.java:556)
com.tangosol.net.cache.OverflowMap.removeInternal(OverflowMap.java:1299)
com.tangosol.net.cache.OverflowMap.remove(OverflowMap.java:380)
com.tangosol.coherence.component.util.daemon.queueProcessor.service.grid.DistributedCache$Storage.clear(DistributedCache.CDB:24)
com.tangosol.coherence.component.util.daemon.queueProcessor.service.grid.DistributedCache.onClearRequest(DistributedCache.CDB:32)
com.tangosol.coherence.component.util.daemon.queueProcessor.service.grid.DistributedCache$ClearRequest.run(DistributedCache.CDB:1)
com.tangosol.coherence.component.net.message.requestMessage.DistributedCacheRequest.onReceived(DistributedCacheRequest.CDB:12)
com.tangosol.coherence.component.util.daemon.queueProcessor.service.Grid.onMessage(Grid.CDB:9)
com.tangosol.coherence.component.util.daemon.queueProcessor.service.Grid.onNotify(Grid.CDB:136)
com.tangosol.coherence.component.util.daemon.queueProcessor.service.grid.DistributedCache.onNotify(DistributedCache.CDB:3)
com.tangosol.coherence.component.util.Daemon.run(Daemon.CDB:42)
java.lang.Thread.run(Thread.java:619)
Thread[Finalizer,8,system]
java.lang.Object.wait(Native Method)
java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:118)
java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:134)
java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159)
Thread[PacketReceiver,7,Cluster]
java.lang.Object.wait(Native Method)
com.tangosol.coherence.component.util.Daemon.onWait(Daemon.CDB:18)
com.tangosol.coherence.component.util.daemon.queueProcessor.packetProcessor.PacketReceiver.onWait(PacketReceiver.CDB:2)
com.tangosol.coherence.component.util.Daemon.run(Daemon.CDB:39)
java.lang.Thread.run(Thread.java:619)
Thread[RMI TCP Accept-0,5,system]
java.net.PlainSocketImpl.socketAccept(Native Method)
java.net.PlainSocketImpl.accept(PlainSocketImpl.java:390)
java.net.ServerSocket.implAccept(ServerSocket.java:453)
java.net.ServerSocket.accept(ServerSocket.java:421)
sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:369)
sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:341)
java.lang.Thread.run(Thread.java:619)
Thread[PacketSpeaker,8,Cluster]
java.lang.Object.wait(Native Method)
com.tangosol.coherence.component.util.queue.ConcurrentQueue.waitForEntry(ConcurrentQueue.CDB:16)
com.tangosol.coherence.component.util.queue.ConcurrentQueue.remove(ConcurrentQueue.CDB:7)
com.tangosol.coherence.component.util.Queue.remove(Queue.CDB:1)
com.tangosol.coherence.component.util.daemon.queueProcessor.packetProcessor.PacketSpeaker.onNotify(PacketSpeaker.CDB:62)
com.tangosol.coherence.component.util.Daemon.run(Daemon.CDB:42)
java.lang.Thread.run(Thread.java:619)
Thread[Logger@9216774 3.5.2/463,3,main]
java.lang.Object.wait(Native Method)
com.tangosol.coherence.component.util.Daemon.onWait(Daemon.CDB:18)
com.tangosol.coherence.component.util.Daemon.run(Daemon.CDB:39)
java.lang.Thread.run(Thread.java:619)
Thread[PacketListener1,8,Cluster]
java.net.PlainDatagramSocketImpl.receive0(Native Method)
java.net.PlainDatagramSocketImpl.receive(PlainDatagramSocketImpl.java:136)
java.net.DatagramSocket.receive(DatagramSocket.java:712)
com.tangosol.coherence.component.net.socket.UdpSocket.receive(UdpSocket.CDB:20)
com.tangosol.coherence.component.net.UdpPacket.receive(UdpPacket.CDB:4)
com.tangosol.coherence.component.util.daemon.queueProcessor.packetProcessor.PacketListener.onNotify(PacketListener.CDB:19)
com.tangosol.coherence.component.util.Daemon.run(Daemon.CDB:42)
java.lang.Thread.run(Thread.java:619)
Thread[main,5,main]
java.lang.Object.wait(Native Method)
com.tangosol.net.DefaultCacheServer.main(DefaultCacheServer.java:79)
com.networkfleet.cacheserver.Launcher.main(Launcher.java:122)
Thread[Signal Dispatcher,9,system]
Thread[RMI TCP Accept-41006,5,system]
java.net.PlainSocketImpl.socketAccept(Native Method)
java.net.PlainSocketImpl.accept(PlainSocketImpl.java:390)
java.net.ServerSocket.implAccept(ServerSocket.java:453)
java.net.ServerSocket.accept(ServerSocket.java:421)
sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:369)
sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:341)
java.lang.Thread.run(Thread.java:619)
ThreadCluster
java.lang.Object.wait(Native Method)
com.tangosol.coherence.component.util.Daemon.onWait(Daemon.CDB:18)
com.tangosol.coherence.component.util.daemon.queueProcessor.service.Grid.onWait(Grid.CDB:9)
com.tangosol.coherence.component.util.Daemon.run(Daemon.CDB:39)
java.lang.Thread.run(Thread.java:619)
Thread[TcpRingListener,6,Cluster]
java.net.PlainSocketImpl.socketAccept(Native Method)
java.net.PlainSocketImpl.accept(PlainSocketImpl.java:390)
java.net.ServerSocket.implAccept(ServerSocket.java:453)
java.net.ServerSocket.accept(ServerSocket.java:421)
com.tangosol.coherence.component.net.socket.TcpSocketAccepter.accept(TcpSocketAccepter.CDB:18)
com.tangosol.coherence.component.util.daemon.TcpRingListener.acceptConnection(TcpRingListener.CDB:10)
com.tangosol.coherence.component.util.daemon.TcpRingListener.onNotify(TcpRingListener.CDB:9)
com.tangosol.coherence.component.util.Daemon.run(Daemon.CDB:42)
java.lang.Thread.run(Thread.java:619)
Thread[PacketPublisher,6,Cluster]
java.lang.Object.wait(Native Method)
com.tangosol.coherence.component.util.Daemon.onWait(Daemon.CDB:18)
com.tangosol.coherence.component.util.daemon.queueProcessor.packetProcessor.PacketPublisher.onWait(PacketPublisher.CDB:2)
com.tangosol.coherence.component.util.Daemon.run(Daemon.CDB:39)
java.lang.Thread.run(Thread.java:619)
Thread[RMI TCP Accept-0,5,system]
java.net.PlainSocketImpl.socketAccept(Native Method)
java.net.PlainSocketImpl.accept(PlainSocketImpl.java:390)
java.net.ServerSocket.implAccept(ServerSocket.java:453)
java.net.ServerSocket.accept(ServerSocket.java:421)
sun.management.jmxremote.LocalRMIServerSocketFactory$1.accept(LocalRMIServerSocketFactory.java:34)
sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:369)
sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:341)
java.lang.Thread.run(Thread.java:619)
Thread[PacketListenerN,8,Cluster]
java.net.PlainDatagramSocketImpl.receive0(Native Method)
java.net.PlainDatagramSocketImpl.receive(PlainDatagramSocketImpl.java:136)
java.net.DatagramSocket.receive(DatagramSocket.java:712)
com.tangosol.coherence.component.net.socket.UdpSocket.receive(UdpSocket.CDB:20)
com.tangosol.coherence.component.net.UdpPacket.receive(UdpPacket.CDB:4)
com.tangosol.coherence.component.util.daemon.queueProcessor.packetProcessor.PacketListener.onNotify(PacketListener.CDB:19)
com.tangosol.coherence.component.util.Daemon.run(Daemon.CDB:42)
java.lang.Thread.run(Thread.java:619)
Thread[Invocation:Management,5,Cluster]
java.lang.Object.wait(Native Method)
com.tangosol.coherence.component.util.Daemon.onWait(Daemon.CDB:18)
com.tangosol.coherence.component.util.daemon.queueProcessor.service.Grid.onWait(Grid.CDB:9)
com.tangosol.coherence.component.util.Daemon.run(Daemon.CDB:39)
java.lang.Thread.run(Thread.java:619)
Thread[DistributedCache:PofDistributedCache,5,Cluster]
java.lang.Object.wait(Native Method)
com.tangosol.coherence.component.util.Daemon.onWait(Daemon.CDB:18)
com.tangosol.coherence.component.util.daemon.queueProcessor.service.Grid.onWait(Grid.CDB:9)
com.tangosol.coherence.component.util.Daemon.run(Daemon.CDB:39)
java.lang.Thread.run(Thread.java:619)
Thread[Invocation:Management:EventDispatcher,5,Cluster]
java.lang.Object.wait(Native Method)
com.tangosol.coherence.component.util.Daemon.onWait(Daemon.CDB:18)
com.tangosol.coherence.component.util.daemon.queueProcessor.Service$EventDispatcher.onWait(Service.CDB:7)
com.tangosol.coherence.component.util.Daemon.run(Daemon.CDB:39)
java.lang.Thread.run(Thread.java:619)
Thread[Termination Thread,5,Cluster]
java.lang.Thread.dumpThreads(Native Method)
java.lang.Thread.getAllStackTraces(Thread.java:1487)
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
java.lang.reflect.Method.invoke(Method.java:597)
com.tangosol.net.GuardSupport.logStackTraces(GuardSupport.java:791)
com.tangosol.coherence.component.net.Cluster.onServiceFailed(Cluster.CDB:5)
com.tangosol.coherence.component.util.daemon.queueProcessor.service.Grid$Guard.terminate(Grid.CDB:17)
com.tangosol.net.GuardSupport$2.run(GuardSupport.java:652)
java.lang.Thread.run(Thread.java:619)
2011-01-22 01:19:06,738 Coherence Logger@9216774 3.5.2/463 INFO 2011-01-22 01:19:06.738/9910757.904 Oracle Coherence EE 3.5.2/463 <Info> (thread=main, member=33): Restarting Service: DistributedCache
2011-01-22 01:19:06,738 Coherence Logger@9216774 3.5.2/463 INFO 2011-01-22 01:19:06.738/9910757.904 Oracle Coherence EE 3.5.2/463 <Info> (thread=main, member=33): Restarting Service: DistributedCache
2011-01-22 01:19:06,738 Coherence Logger@9216774 3.5.2/463 ERROR 2011-01-22 01:19:06.738/9910757.904 Oracle Coherence EE 3.5.2/463 <Error> (thread=main, member=33): Failed to restart services: java.lang.IllegalStateException: Failed to unregister: Distr
butedCache{Name=DistributedCache, State=(SERVICE_STARTED), LocalStorage=enabled, PartitionCount=257, BackupCount=1, AssignedPartitions=16, BackupPartitions=16}
2011-01-22 01:19:06,738 Coherence Logger@9216774 3.5.2/463 ERROR 2011-01-22 01:19:06.738/9910757.904 Oracle Coherence EE 3.5.2/463 <Error> (thread=main, member=33): Failed to restart services: java.lang.IllegalStateException: Failed to unregister: Distr
butedCache{Name=DistributedCache, State=(SERVICE_STARTED), LocalStorage=enabled, PartitionCount=257, BackupCount=1, AssignedPartitions=16, BackupPartitions=16}Hi
It seems like the problem in this case is the call to clear() which will try to load all entries stored in the overflow scheme to emit potential cache events to listeners. This probably requires much more memory than there is Java heap available, hence the OOM.
Our recommendation in this case is to call destroy() since this will bypass the even firing.
/Charlie -
OrainstRoot.sh: Failure to promote local gpnp setup to other cluster nodes
I'm trying to build a 2 node cluster and everything appeared to be going swimmingly until the end of the 1st nodes running of the orainstRoot.sh script.
The following is the end of the output:
Disk Group OCR_VOTE created successfully.
clscfg: -install mode specified
Successfully accumulated necessary OCR keys.
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
CRS-4256: Updating the profile
Successful addition of voting disk 4e3f692529584f8bbf7f16146bd90346.
Successful addition of voting disk 728bed918cf54f6cbf904d37638c674b.
Successful addition of voting disk 8ac20793405d4fdcbfcafc7e311f877d.
Successfully replaced voting disk group with +OCR_VOTE.
CRS-4256: Updating the profile
CRS-4266: Voting file(s) successfully replaced
## STATE File Universal Id File Name Disk group
1. ONLINE 4e3f692529584f8bbf7f16146bd90346 (ORCL:VOTE01) [OCR_VOTE]
2. ONLINE 728bed918cf54f6cbf904d37638c674b (ORCL:VOTE02) [OCR_VOTE]
3. ONLINE 8ac20793405d4fdcbfcafc7e311f877d (ORCL:VOTE03) [OCR_VOTE]
Located 3 voting disk(s).
Failed to rmtcopy "/tmp/fileLgKPGV" to "/u01/app/11.2.0/grid/gpnp/manifest.txt" for nodes {ilprevzedb01,ilprevzedb02}, rc=256
Failed to rmtcopy "/u01/app/11.2.0/grid/gpnp/ilprevzedb01/profiles/peer/profile.xml" to "/u01/app/11.2.0/grid/gpnp/profiles/peer/profile.xml" for nodes {ilprevzedb01,ilprevzedb02}, rc=256
rmtcopy aborted
Failed to promote local gpnp setup to other cluster nodes at /u01/app/11.2.0/grid/crs/install/crsconfig_lib.pm line 6504.
/u01/app/11.2.0/grid/perl/bin/perl -I/u01/app/11.2.0/grid/perl/lib -I/u01/app/11.2.0/grid/crs/install /u01/app/11.2.0/grid/crs/install/rootcrs.pl execution failed
Has anyone run into this problem and found a solution?
Thanks in advance!Ok, for everyone out there, I resolved the issue. Hopefully this will help others encountering the same problem.
It turns out that when the OS was installed, iptables firewall was enabled. This will cause havoc with the installer scripts.
My first inkling should have been when the installer stalled at 65% trying to copy home directories between nodes, the first time I ran through the installer.
At that time, Googling around found that iptables might be the problem and indeed it was running, so I just did a 'service iptables stop' WITHOUT REBOOTING THE NODES and re-ran the installer.
Well, it looks as though NOT REBOOTING THE NODES doesn't quite cut it. I then did a 'chkconfig iptables off' and REBOOTED BOTH NODES.
Oracle support simply provided me with: How to Proceed from Failed 11gR2 Grid Infrastructure (CRS) Installation (Doc ID 942166.1), which didn't really work all that well, lots of failures, errors, etc. So I just deleted the 11.2.0 directory and tried running the installer again.
This time the install went through without problems.
Thanks! -
1. I want to put this Cluster server into our SCOM 2007R2 and I want to monitor the disk space usage stats.
2. If any disk space is below a threshold , I want to receive an alert.
however, the setup for cluster is a bit strange. Sometimes, I could see disk space showing up but sometimes it just disappear.
Any one could help explain?
thanks
hui
--Currently using Reporting Service 2000; Visual Studio .NET 2003; Visual Source Safe SSIS 2008 SSAS 2008, SVN --Hi,
Please also review the link below:
OpsMgr: MP Update: New Base OS MP 6.0.6972.0 Adds new cluster disks, changes free space monitoring, other fixes
http://blogs.technet.com/b/kevinholman/archive/2012/06/14/opsmgr-mpupdate-new-base-os-mp-6-0-6972-0-adds-new-cluster-disks-changes-free-space-monitoring-other-fixes.aspx
We
are trying to better understand customer views on social support experience, so your participation in this
interview project would be greatly appreciated if you have time.
Thanks for helping make community forums a great place. -
Just trying out my new mac but email won't load, error says "The mail server denied access to the account because an administrator or other mail client was using it when Mail tried to log in. Try again later." A little lock is beside the email inbox account, no password prompted and account is online and enabled... Thoughts?
Have you tried clicking on the lock to see it it will then ask for a passwored. Otherwise, try reblooting.
-
Cluster Node Joining other cluster
Because of a network problem, two of our clusters shared their interconnects. This of course lead to duplicated IPs on the interconnects and reboots of the nodes. Now one of the cluster nodes of cluster a tried to join the ClusterB:
cluster.name
cluster.state enabled
cluster.properties.cluster_id 0x48FDxxxx [ Different from Node A ]
cluster.properties.installmode disabled
cluster.properties.auth_joinlist_type sys
cluster.properties.auth_joinlist_hostslist ,
cluster.properties.cmm_version 1
cluster.nodes.1.name I am really surprised this node tried to join the other cluster, seems it got the ccr from there during one of the reboots.
The real question i have now is jow to get out of this as soon as we have fixed the network problem, how can we bring back this node to the ClusterA, must we reinstall it.
FritzI'm surprised that an invalid node picked up a CCR update from a cluster that it wasn't part of. I would have expected the cluster ids to be different and thus prevent this, but to be honest I haven't checked to see how much prevention there is against this.
Anyway, to get out of it you could hack the CCRs on the other clusters and try and put them onto a different subnet with different private addresses. You'll need to use ccradm. Messy though.
Tim
--- -
NQSError: 47051 error on Windows cluster
I’m getting nQSError: 47051 while installing OBIEE cluster on windows.
Here’s the situation. We are installing OBIEE (10.1.3.4) on a two node (Windows 2003, SP2) cluster. The shared file system is built on a SAN using windows clustering. This windows SAN cluster is also attached to the primary OBIEE server as a local drive. All the shared files (Repository Publishing Directory, Cache directory etc.) are mapped to this cluster.
After the configuration the OBI Server doesn’t start and throws error
[nQSError: 47024] The Oracle BI Server participates in a cluster but errors were found in the cluster configuration file.
[nQSError: 47051] The Oracle BI Server is specified as participating in a cluster, but global cache file storage location, <My shared file location, is invalid.
I am perplexed since I (or the user owning OBIEE) can see this disk using ‘Windows explorer’ and read and right to it. Then what can be the issue with OBIEE server?
Funny thing is when I specify my local directory (the SAN cluster attached locally) as the cache location it works!
Interestingly I found out that there is a bug (6165192) on OBIEE 10.1.3.3 where simillar problem is listed. Not sure if that is still applicable to 10.1.3.4.
I noticed that we have all kinds of firewall (internally) between the servers. Can that block the OBIPS communication with the server to which the SAN disk is attached? If yes, then what Protocol is used for such a communication and on what port?
Did any of you face similar situation? If yes, then what was your resolution?
Any help on this will be greatly appreciated.Hi,
In our environment we have installed OBIEE 10.1.3.4 on two windows servers(primary and secondary) and these servers not clustered using Microsoft or any other cluster software .
We are planning to use SAN drive to hold the shared cache, now we are facing a challenge in configuring the SAN to see from both OBIEE servers, is it possible to use the SAN with out installing any cluster between these servers.
I understood from above post that you guys implemented/installed OBIEE on Windows environment with SAN.
Is it possible to have a quick call. Your help is greatly appreciated.
Thanks -
ORA-29702: error occurred in Cluster Group Service operation
Hi,
After shutting down one node I'm trying but without success to start this node up:
SQL> startup
ORA-29702: error occurred in Cluster Group Service operation
I got this message.
Here is a log from file: orcl2_lmon_13886.trc
/oracle/admin/orcl/bdump/orcl2_lmon_13886.trc
Oracle Database 10g Enterprise Edition Release 10.2.0.3.0 - 64bit Production
With the Partitioning, Real Application Clusters and Data Mining options
ORACLE_HOME = /oracle/DB
System name: Linux
Node name: DB2
Release: 2.6.18-53.el5
Version: #1 SMP Wed Oct 10 16:34:19 EDT 2007
Machine: x86_64
Instance name: orcl2
Redo thread mounted by this instance: 0 <none>
Oracle process number: 5
Unix process pid: 13886, image: oracle@DB2 (LMON)
*** SERVICE NAME:() 2008-12-19 10:08:26.214
*** SESSION ID:(1102.1) 2008-12-19 10:08:26.214
GES resources 21522 pool 8
GES enqueues 33158
GES IPC: Receivers 3 Senders 3
GES IPC: Buffers Receive 1000 Send (i:2750 b:1934) Reserve 1000
GES IPC: Msg Size Regular 408 Batch 8192
Batching factor: enqueue replay 201, ack 224
Batching factor: cache replay 126 size per lock 64
kjxggin: receive buffer size = 32768
clsc_connect: (0x1d19a030) no listener at (ADDRESS=(PROTOCOL=ipc)(KEY=OCSSD_LL_db2_crs))
2008-12-19 10:08:29.819: [ CSSCLNT]clsssInitNative: connect failed, rc 9
kgxgncin: CLSS init failed with status 3
kjxgmin: kgxgncin fails - (2)
kjxggin: generic group layer init fails
*** 2008-12-19 10:08:29.819
Global Enqueue Service Shutdown
*** 2008-12-19 10:08:29.820
global enqueue service detaching from CM:pid=13886
I did check listener, it did not work but I start it up:
LSNRCTL> status
Connecting to (ADDRESS=(PROTOCOL=tcp)(HOST=)(PORT=1521))
STATUS of the LISTENER
Alias LISTENER
Version TNSLSNR for Linux: Version 10.2.0.3.0 - Production
Start Date 19-DEC-2008 10:49:42
Uptime 0 days 0 hr. 3 min. 55 sec
Trace Level off
Security ON: Local OS Authentication
SNMP ON
Listener Parameter File /oracle/DB/network/admin/listener.ora
Listener Log File /oracle/DB/network/log/listener.log
Listening Endpoints Summary...
(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=localhost.localdomain)(PORT=1521)))
The listener supports no services
The command completed successfully
LSNRCTL>
Could you give me a guide how to start up this node? or where is a problem?A quick search of the Oracle documentation on the error number comes up with this
http://download-west.oracle.com/docs/cd/B10501_01/server.920/a96525/e29250.htm#1000969
ORA-29702 error occurred in Cluster Group Service operation
Cause: An unexpected error occurred while performing a CGS operation.
Action: Verify that the LMON process is still active. Also, check the Oracle LMON trace files for errors.
Justin
Distributed Database Consulting, Inc.
http://www.ddbcinc.com/askDDBC -
Failed to start Oracle Weblogic Server node in 11g
Hello friends,
I have a problem starting the Weblogic Server node Oracle 11g (10.3.5).
Any suggestions to solve the problem
Logs----------------------------------------------------------------------------------
<Nov 3, 2011 12:33:03 PM> <INFO> <NodeManager> <Server output log file is '/opt/middleware/user_projects/domains/DomainOWL/servers/Server01/logs/Server01.out'>
<Nov 3, 2011 12:33:07 PM PET> <Info> <Security> <BEA-090905> <Disabling CryptoJ JCE Provider self-integrity check for better startup performance. To enable this check, specify -Dweblogic.security.allowCryptoJDefaultJCEVerification=true>
<Nov 3, 2011 12:33:07 PM PET> <Info> <Security> <BEA-090906> <Changing the default Random Number Generator in RSA CryptoJ from ECDRBG to FIPS186PRNG. To disable this change, specify -Dweblogic.security.allowCryptoJDefaultPRNG=true>
<Nov 3, 2011 12:33:11 PM PET> <Info> <WebLogicServer> <BEA-000377> <Starting WebLogic Server with Java HotSpot(TM) Server VM Version 19.1-b02 from Sun Microsystems Inc.>
<Nov 3, 2011 12:33:18 PM PET> <Info> <Management> <BEA-141107> <Version: WebLogic Server 10.3.5.0 Fri Apr 1 20:20:06 PDT 2011 1398638 >
<Nov 3, 2011 12:33:24 PM PET> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to STARTING>
<Nov 3, 2011 12:33:24 PM PET> <Info> <WorkManager> <BEA-002900> <Initializing self-tuning thread pool>
<Nov 3, 2011 12:33:25 PM PET> <Notice> <Log Management> <BEA-170019> <The server log file /opt/middleware/user_projects/domains/DomainOWL/servers/Server01/logs/Server01.log is opened. All server side log events will be written to this file.>
<Nov 3, 2011 12:33:25 PM PET> <Warning> <NodeManager> <BEA-300043> <Node manager native library not found - server process id not saved.>
<Nov 3, 2011 12:33:26 PM PET> <Error> <Socket> <BEA-000438> <Unable to load performance pack. Using Java I/O instead. Please ensure that a native performance library is in: '"/opt/jdk1.6.0/jre/lib/sparcv9/server:/opt/jdk1.6.0/jre/lib/sparcv9:/opt/jdk1.6.0/jre/../lib/sparcv9:/opt/middleware/patch_wls1035/profiles/default/native:/opt/middleware/patch_ocp360/profiles/default/native:/opt/middleware/wlserver_10.3/server/native/solaris/sparc64:/opt/middleware/wlserver_10.3/server/native/solaris/sparc64:/opt/middleware/wlserver_10.3/server/native/solaris/sparc64/oci920_8:/usr/jdk/packages/lib/sparcv9:/lib/64:/usr/lib/64"'
>
<Nov 3, 2011 12:33:37 PM PET> <Notice> <Security> <BEA-090082> <Security initializing using security realm myrealm.>
<Nov 3, 2011 12:33:40 PM PET> <Warning> <Store> <BEA-280101> <The persistent file store "_WLS_Server01" is forced to use buffered I/O and so may have significantly degraded performance. Either the OS/hardware environment does not support the chosen write policy or the native wlfileio library is missing. See store open log messages for the requested and final write policies. See the documentation on store synchronous write policy configuration for advice.>
<Nov 3, 2011 12:33:49 PM PET> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to STANDBY>
<Nov 3, 2011 12:33:49 PM PET> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to STARTING>
<Nov 3, 2011 12:33:56 PM PET> <Notice> <Log Management> <BEA-170027> <The Server has established connection with the Domain level Diagnostic Service successfully.>
<Nov 3, 2011 12:33:56 PM PET> <Notice> <Cluster> <BEA-000197> <Listening for announcements from cluster using unicast cluster messaging>
<Nov 3, 2011 12:33:56 PM PET> <Notice> <Cluster> <BEA-000133> <Waiting to synchronize with other running members of oimClusterT.>
<Nov 3, 2011 12:34:26 PM PET> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to ADMIN>
<Nov 3, 2011 12:34:27 PM PET> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to RESUMING>
<Nov 3, 2011 12:34:27 PM PET> <Notice> <Cluster> <BEA-000162> <Starting "async" replication service with remote cluster address "null">
<Nov 3, 2011 12:34:31 PM PET> <Notice> <Security> <BEA-090171> <Loading the identity certificate and private key stored under the alias DemoIdentity from the jks keystore file /opt/middleware/wlserver_10.3/server/lib/DemoIdentity.jks.>
<Nov 3, 2011 12:34:31 PM PET> <Notice> <Security> <BEA-090169> <Loading trusted certificates from the jks keystore file /opt/middleware/wlserver_10.3/server/lib/DemoTrust.jks.>
<Nov 3, 2011 12:34:31 PM PET> <Notice> <Security> <BEA-090169> <Loading trusted certificates from the jks keystore file /opt/jdk1.6.0/jre/lib/security/cacerts.>
<Nov 3, 2011 12:34:33 PM PET> <Alert> <Security> <BEA-090152> <Demo trusted CA certificate is being used in production mode: [
Version: V3
Subject: CN=CACERT, OU=FOR TESTING ONLY, O=MyOrganization, L=MyTown, ST=MyState, C=US
Signature Algorithm: MD5withRSA, OID = 1.2.840.113549.1.1.4
Key: SunPKCS11-Solaris RSA public key, 512 bits (id 58638024, session object)
modulus: 9550192877869244258838480703390456015046425375252278279190673063544122510925482179963329236052146047356415957587628011282484772458983977898996276815440753
public exponent: 65537
Validity: [From: Thu Mar 21 15:12:27 PET 2002,
To: Tue Mar 22 15:12:27 PET 2022]
Issuer: CN=CACERT, OU=FOR TESTING ONLY, O=MyOrganization, L=MyTown, ST=MyState, C=US
SerialNumber: [ 33f10648 fcde0deb 4199921f d64537f4]
Certificate Extensions: 1
[1]: ObjectId: 2.5.29.15 Criticality=true
KeyUsage [
Key_CertSign
Algorithm: [MD5withRSA]
Signature:
0000: 9D 26 4C 29 C8 91 C3 A7 06 C3 24 6F AE B4 F8 82 .&L)......$o....
0010: 80 4D AA CB 7C 79 46 84 81 C4 66 95 F4 1E D8 C4 .M...yF...f.....
0020: E9 B7 D9 7C E2 23 33 A4 B7 21 E0 AA 54 2B 4A FF .....#3..!..T+J.
0030: CB 21 20 88 81 21 DB AC 90 54 D8 7D 79 63 23 3C .! ..!...T..yc#<
] The system is vulnerable to security attacks, since it trusts certificates signed by the demo trusted CA.>
<Nov 3, 2011 12:34:33 PM PET> <Notice> <Security> <BEA-090898> <Ignoring the trusted CA certificate "CN=Entrust Root Certification Authority - G2,OU=(c) 2009 Entrust\, Inc. - for authorized use only,OU=See www.entrust.net/legal-terms,O=Entrust\, Inc.,C=US". The loading of the trusted certificate list raised a certificate parsing exception PKIX: Unsupported OID in the AlgorithmIdentifier object: 1.2.840.113549.1.1.11.>
<Nov 3, 2011 12:34:33 PM PET> <Notice> <Security> <BEA-090898> <Ignoring the trusted CA certificate "CN=thawte Primary Root CA - G3,OU=(c) 2008 thawte\, Inc. - For authorized use only,OU=Certification Services Division,O=thawte\, Inc.,C=US". The loading of the trusted certificate list raised a certificate parsing exception PKIX: Unsupported OID in the AlgorithmIdentifier object: 1.2.840.113549.1.1.11.>
<Nov 3, 2011 12:34:34 PM PET> <Notice> <Security> <BEA-090898> <Ignoring the trusted CA certificate "CN=T-TeleSec GlobalRoot Class 3,OU=T-Systems Trust Center,O=T-Systems Enterprise Services GmbH,C=DE". The loading of the trusted certificate list raised a certificate parsing exception PKIX: Unsupported OID in the AlgorithmIdentifier object: 1.2.840.113549.1.1.11.>
<Nov 3, 2011 12:34:34 PM PET> <Notice> <Security> <BEA-090898> <Ignoring the trusted CA certificate "CN=T-TeleSec GlobalRoot Class 2,OU=T-Systems Trust Center,O=T-Systems Enterprise Services GmbH,C=DE". The loading of the trusted certificate list raised a certificate parsing exception PKIX: Unsupported OID in the AlgorithmIdentifier object: 1.2.840.113549.1.1.11.>
<Nov 3, 2011 12:34:34 PM PET> <Notice> <Security> <BEA-090898> <Ignoring the trusted CA certificate "CN=GlobalSign,O=GlobalSign,OU=GlobalSign Root CA - R3". The loading of the trusted certificate list raised a certificate parsing exception PKIX: Unsupported OID in the AlgorithmIdentifier object: 1.2.840.113549.1.1.11.>
<Nov 3, 2011 12:34:34 PM PET> <Notice> <Security> <BEA-090898> <Ignoring the trusted CA certificate "OU=Security Communication RootCA2,O=SECOM Trust Systems CO.\,LTD.,C=JP". The loading of the trusted certificate list raised a certificate parsing exception PKIX: Unsupported OID in the AlgorithmIdentifier object: 1.2.840.113549.1.1.11.>
<Nov 3, 2011 12:34:34 PM PET> <Notice> <Security> <BEA-090898> <Ignoring the trusted CA certificate "CN=VeriSign Universal Root Certification Authority,OU=(c) 2008 VeriSign\, Inc. - For authorized use only,OU=VeriSign Trust Network,O=VeriSign\, Inc.,C=US". The loading of the trusted certificate list raised a certificate parsing exception PKIX: Unsupported OID in the AlgorithmIdentifier object: 1.2.840.113549.1.1.11.>
<Nov 3, 2011 12:34:34 PM PET> <Notice> <Security> <BEA-090898> <Ignoring the trusted CA certificate "CN=KEYNECTIS ROOT CA,OU=ROOT,O=KEYNECTIS,C=FR". The loading of the trusted certificate list raised a certificate parsing exception PKIX: Unsupported OID in the AlgorithmIdentifier object: 1.2.840.113549.1.1.11.>
<Nov 3, 2011 12:34:34 PM PET> <Notice> <Security> <BEA-090898> <Ignoring the trusted CA certificate "CN=GeoTrust Primary Certification Authority - G3,OU=(c) 2008 GeoTrust Inc. - For authorized use only,O=GeoTrust Inc.,C=US". The loading of the trusted certificate list raised a certificate parsing exception PKIX: Unsupported OID in the AlgorithmIdentifier object: 1.2.840.113549.1.1.11.>
<Nov 3, 2011 12:34:34 PM PET> <Notice> <Server> <BEA-002613> <Channel "DefaultSecure" is now listening on 192.168.1.10:7006 for protocols iiops, t3s, CLUSTER-BROADCAST-SECURE, ldaps, https.>
<Nov 3, 2011 12:34:34 PM PET> <Notice> <Server> <BEA-002613> <Channel "Default" is now listening on 192.168.1.10:7005 for protocols iiop, t3, CLUSTER-BROADCAST, ldap, snmp, http.>
<Nov 3, 2011 12:34:34 PM PET> <Notice> <WebLogicServer> <BEA-000330> <Started WebLogic Managed Server "Server01" for domain "DomainOWL" running in Production Mode>
Exception in thread "[STANDBY] ExecuteThread: '7' for queue: 'weblogic.kernel.Default (self-tuning)'" java.lang.OutOfMemoryError: PermGen space
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClassCond(ClassLoader.java:632)
at java.lang.ClassLoader.defineClass(ClassLoader.java:616)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:283)
at java.net.URLClassLoader.access$000(URLClassLoader.java:58)
at java.net.URLClassLoader$1.run(URLClassLoader.java:197)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
at weblogic.work.ExecuteThread.execute(ExecuteThread.java:217)
at weblogic.work.ExecuteThread.run(ExecuteThread.java:178)
Exception in thread "[ACTIVE] ExecuteThread: '8' for queue: 'weblogic.kernel.Default (self-tuning)'" java.lang.OutOfMemoryError: PermGen space
Exception in thread "[ACTIVE] ExecuteThread: '2' for queue: 'weblogic.kernel.Default (self-tuning)'" java.lang.OutOfMemoryError: PermGen space
Exception in thread "[ACTIVE] ExecuteThread: '14' for queue: 'weblogic.kernel.Default (self-tuning)'" java.lang.OutOfMemoryError: PermGen space
Exception in thread "[ACTIVE] ExecuteThread: '16' for queue: 'weblogic.kernel.Default (self-tuning)'" java.lang.OutOfMemoryError: PermGen space
<Nov 3, 2011 12:36:27 PM PET> <Error> <HTTP> <BEA-101020> <[ServletContext@15375304[app:bea_wls_deployment_internal module:bea_wls_deployment_internal.war path:/bea_wls_deployment_internal spec-version:null]] Servlet failed with Exception
java.lang.OutOfMemoryError: PermGen space
>
Exception in thread "[ACTIVE] ExecuteThread: '13' for queue: 'weblogic.kernel.Default (self-tuning)'" java.lang.OutOfMemoryError: PermGen space
Exception in thread "Timer-1" java.lang.OutOfMemoryError: PermGen space
Exception in thread "[ACTIVE] ExecuteThread: '23' for queue: 'weblogic.kernel.Default (self-tuning)'" java.lang.OutOfMemoryError: PermGen space
<Nov 3, 2011 12:36:53 PM PET> <Error> <HTTP> <BEA-101020> <[ServletContext@15375304[app:bea_wls_deployment_internal module:bea_wls_deployment_internal.war path:/bea_wls_deployment_internal spec-version:null]] Servlet failed with Exception
java.lang.OutOfMemoryError: PermGen space
>
<Nov 3, 2011 12:36:24 PM PET> <Error> <Socket> <BEA-000405> <Uncaught Throwable in processSockets
java.lang.OutOfMemoryError: PermGen space.
java.lang.OutOfMemoryError: PermGen space
>
<Nov 3, 2011 12:36:33 PM PET> <Error> <JMX> <BEA-149501> <An exception occurred while registering the MBean com.bea:Name=idmAdminServerT,Type=Server at property WebService.
java.lang.OutOfMemoryError: PermGen space
>
Exception in thread "[ACTIVE] ExecuteThread: '21' for queue: 'weblogic.kernel.Default (self-tuning)'" java.lang.OutOfMemoryError: PermGen space
Exception in thread "[ACTIVE] ExecuteThread: '9' for queue: 'weblogic.kernel.Default (self-tuning)'" java.lang.OutOfMemoryError: PermGen space
Exception in thread "[ACTIVE] ExecuteThread: '24' for queue: 'weblogic.kernel.Default (self-tuning)'" java.lang.OutOfMemoryError: PermGen space
java.lang.OutOfMemoryError: PermGen space
Exception in thread "[ACTIVE] ExecuteThread: '17' for queue: 'weblogic.kernel.Default (self-tuning)'" java.lang.OutOfMemoryError: PermGen space
Exception in thread "[STANDBY] ExecuteThread: '4' for queue: 'weblogic.kernel.Default (self-tuning)'" java.lang.OutOfMemoryError: PermGen space
Exception in thread "[ACTIVE] ExecuteThread: '6' for queue: 'weblogic.kernel.Default (self-tuning)'" java.lang.OutOfMemoryError: PermGen space
<Nov 3, 2011 12:37:16 PM PET> <Warning> <Socket> <BEA-000402> <There are: 5 active sockets, but the maximum number of socket reader threads allowed by the configuration is: 4. You may want to alter your configuration.>
Exception in thread "[ACTIVE] ExecuteThread: '20' for queue: 'weblogic.kernel.Default (self-tuning)'" java.lang.OutOfMemoryError: PermGen space
Exception in thread "[ACTIVE] ExecuteThread: '15' for queue: 'weblogic.kernel.Default (self-tuning)'" java.lang.OutOfMemoryError: PermGen space
Exception in thread "[ACTIVE] ExecuteThread: '12' for queue: 'weblogic.kernel.Default (self-tuning)'" java.lang.OutOfMemoryError: PermGen space
Exception in thread "[ACTIVE] ExecuteThread: '25' for queue: 'weblogic.kernel.Default (self-tuning)'" java.lang.OutOfMemoryError: PermGen space
<Nov 3, 2011 12:39:41 PM PET> <Error> <HTTP> <BEA-101216> <Servlet: "CoordinatorPortTypeServlethttp" failed to preload on startup in Web application: "wls-wsat.war".
java.lang.OutOfMemoryError: PermGen space
>
<Nov 3, 2011 12:42:32 PM PET> <Error> <HTTP> <BEA-101017> <[ServletContext@15375304[app:bea_wls_deployment_internal module:bea_wls_deployment_internal.war path:/bea_wls_deployment_internal spec-version:null]] Root cause of ServletException.
java.lang.OutOfMemoryError: PermGen space
>
<Nov 3, 2011 12:43:15 PM PET> <Error> <HTTP> <BEA-101017> <[ServletContext@15375304[app:bea_wls_deployment_internal module:bea_wls_deployment_internal.war path:/bea_wls_deployment_internal spec-version:null]] Root cause of ServletException.
java.lang.OutOfMemoryError: PermGen space
>
<Nov 3, 2011 12:43:26 PM PET> <Error> <Socket> <BEA-000405> <Uncaught Throwable in processSockets
java.lang.OutOfMemoryError: PermGen space.
java.lang.OutOfMemoryError: PermGen space
>
Exception in thread "[ACTIVE] ExecuteThread: '18' for queue: 'weblogic.kernel.Default (self-tuning)'" java.lang.OutOfMemoryError: PermGen space
java.lang.OutOfMemoryError: PermGen space
Exception in thread "[ACTIVE] ExecuteThread: '5' for queue: 'weblogic.kernel.Default (self-tuning)'" java.lang.OutOfMemoryError: PermGen space
java.lang.OutOfMemoryError: PermGen space
Exception in thread "[STANDBY] ExecuteThread: '19' for queue: 'weblogic.kernel.Default (self-tuning)'" java.lang.OutOfMemoryError: PermGen space
<Nov 3, 2011 12:45:53 PM PET> <Error> <HTTP> <BEA-101017> <[ServletContext@15375304[app:bea_wls_deployment_internal module:bea_wls_deployment_internal.war path:/bea_wls_deployment_internal spec-version:null]] Root cause of ServletException.
java.lang.OutOfMemoryError: PermGen space
>
Exception in thread "[ACTIVE] ExecuteThread: '22' for queue: 'weblogic.kernel.Default (self-tuning)'" java.lang.OutOfMemoryError: PermGen space
Exception in thread "[STANDBY] ExecuteThread: '3' for queue: 'weblogic.kernel.Default (self-tuning)'" java.lang.OutOfMemoryError: PermGen space
Exception in thread "[ACTIVE] ExecuteThread: '11' for queue: 'weblogic.kernel.Default (self-tuning)'" java.lang.OutOfMemoryError: PermGen space
java.lang.OutOfMemoryError: PermGen space
Exception in thread "[ACTIVE] ExecuteThread: '10' for queue: 'weblogic.kernel.Default (self-tuning)'" java.lang.OutOfMemoryError: PermGen space
Exception in thread "[STANDBY] ExecuteThread: '1' for queue: 'weblogic.kernel.Default (self-tuning)'" java.lang.OutOfMemoryError: PermGen space
Exception in thread "weblogic.timers.TimerThread" java.lang.OutOfMemoryError: PermGen space
Thanks<Nov 4, 2011 11:42:08 AM PET> <Error> <Socket> <BEA-000405> <Uncaught Throwable in processSockets
java.lang.OutOfMemoryError: PermGen space.
java.lang.OutOfMemoryError: PermGen space
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClassCond(ClassLoader.java:632)
at java.lang.ClassLoader.defineClass(ClassLoader.java:616)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:283)
Truncated. see log file for complete stacktrace
>
<Nov 4, 2011 11:42:34 AM PET> <Critical> <WebLogicServer> <BEA-000386> <Server subsystem failed. Reason: java.lang.OutOfMemoryError: PermGen space
java.lang.OutOfMemoryError: PermGen space
at java.lang.Class.getDeclaredMethods0(Native Method)
at java.lang.Class.privateGetDeclaredMethods(Class.java:2427)
at java.lang.Class.getDeclaredMethod(Class.java:1935)
at java.io.ObjectStreamClass.getInheritableMethod(ObjectStreamClass.java:1349)
at java.io.ObjectStreamClass.access$2200(ObjectStreamClass.java:52)
Truncated. see log file for complete stacktrace
>
The WebLogic Server encountered a critical failure
Reason: PermGen space
Exception in thread "[ACTIVE] ExecuteThread: '12' for queue: 'weblogic.kernel.Default (self-tuning)'" java.lang.OutOfMemoryError: PermGen space
Exception in thread "main" java.lang.OutOfMemoryError: PermGen space
Exception in thread "[ACTIVE] ExecuteThread: '14' for queue: 'weblogic.kernel.Default (self-tuning)'" java.lang.OutOfMemoryError: PermGen space
Exception in thread "[ACTIVE] ExecuteThread: '15' for queue: 'weblogic.kernel.Default (self-tuning)'" java.lang.OutOfMemoryError: PermGen space
Exception in thread "[ACTIVE] ExecuteThread: '25' for queue: 'weblogic.kernel.Default (self-tuning)'" java.lang.OutOfMemoryError: PermGen space
Exception in thread "[ACTIVE] ExecuteThread: '22' for queue: 'weblogic.kernel.Default (self-tuning)'" java.lang.OutOfMemoryError: PermGen space
Exception in thread "[ACTIVE] ExecuteThread: '2' for queue: 'weblogic.kernel.Default (self-tuning)'" java.lang.OutOfMemoryError: PermGen space
<Nov 4, 2011 11:44:08 AM PET> <Warning> <Socket> <BEA-000402> <There are: 5 active sockets, but the maximum number of socket reader threads allowed by the configuration is: 4. You may want to alter your configuration.>
Exception in thread "[ACTIVE] ExecuteThread: '27' for queue: 'weblogic.kernel.Default (self-tuning)'" java.lang.OutOfMemoryError: PermGen space
Exception in thread "[ACTIVE] ExecuteThread: '21' for queue: 'weblogic.kernel.Default (self-tuning)'" java.lang.OutOfMemoryError: PermGen space
Exception in thread "[ACTIVE] ExecuteThread: '6' for queue: 'weblogic.kernel.Default (self-tuning)'" java.lang.OutOfMemoryError: PermGen space
Exception in thread "[ACTIVE] ExecuteThread: '7' for queue: 'weblogic.kernel.Default (self-tuning)'" java.lang.OutOfMemoryError: PermGen space
Exception in thread "[ACTIVE] ExecuteThread: '23' for queue: 'weblogic.kernel.Default (self-tuning)'" java.lang.OutOfMemoryError: PermGen space
<Nov 4, 2011 11:44:43 AM PET> <Error> <Socket> <BEA-000405> <Uncaught Throwable in processSockets
java.lang.OutOfMemoryError: PermGen space.
java.lang.OutOfMemoryError: PermGen space
I made the change and continue with the same problem, you can tell me which file should make this change, very grateful
Maybe you are looking for
-
Need help to drag a rectangle to define a zoom level
Can some help me figure this out. I am trying to draw some graphics on a JPanel, then drag a rectangle to define a new zoom extent. I am having problems understanding how to do this with the AffineTransform. If you run the following code, you will se
-
i've got a ton of users, whose primary group is "current student". they all belong to the other group "year 13" they've left school, so i thought it would be easy to do a search of users with GID equal to or containing the GID for "year 13" (in this
-
In XI no adapters are visible RWB and the error "AE_DETAILS_GET_ERROR"
In XI no adapters are visible in the RWB and I get the error "AE_DETAILS_GET_ERROR"<BR> It happens after a database copy from PRD to QAS.<BR> I tried restarting the system, and checked the SLD content. Also the cash is refreshed.<BR> The standard af
-
File-to-Idoc, work flow trigger
Hi All, I need on help in taking decision of the following scenario. 1) I have a file contains multiple records 2) If record starts with 00-02 i need to send the information to idoc processing 3) If others (03-04 etc) I need to send the record inform
-
Hi, The directory oracle is missing from the location ]$ pwd /u04/Rel11i/devora/8.0.6/forms60/java $ cd oracle -bash: cd: oracle: No such file or directory How to resolve this.Anyone Pls help... Thanks, Praveen