Weblogic sp4 on linux out of memory expections - Thread dump
Any body has idea why out of memory expections?
Here is the thread dump.
Full thread dump Java HotSpot(TM) Server VM (1.4.2_05-b04 mixed mode):
"Thread-33" daemon prio=1 tid=0x086175f0 nid=0x4f8e runnable [c8d57000..c8d5723c]
at java.net.PlainSocketImpl.socketAccept(Native Method)
at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:353)
- locked <0xd48f7820> (a java.net.PlainSocketImpl)
at java.net.ServerSocket.implAccept(ServerSocket.java:448)
at java.net.ServerSocket.accept(ServerSocket.java:419)
at org.apache.axis.monitor.SOAPMonitorService$ServerSocketThread.run(SOAPMonitorService.java:189)
at java.lang.Thread.run(Thread.java:534)
"CustomerClientDispatcherThread" daemon prio=1 tid=0x08877100 nid=0x4f8e in Object.wait() [c77ff000..c77ff23c]
at java.lang.Object.wait(Native Method)
- waiting on <0xd45b7330> (a com.amgen.ea.orx.queue.impl.PersistenceQueue)
at java.lang.Object.wait(Object.java:429)
at com.amgen.ea.orx.queue.impl.PersistenceQueue.peek(PersistenceQueue.java:58)
- locked <0xd45b7330> (a com.amgen.ea.orx.queue.impl.PersistenceQueue)
at com.amgen.ea.orx.custclient.impl.CustomerClientDispatcher.run(CustomerClientDispatcher.java:50)
at java.lang.Thread.run(Thread.java:534)
"CustomerClientDispatcherThread" daemon prio=1 tid=0xcb625c28 nid=0x4f8e in Object.wait() [c7b8c000..c7b8c23c]
at java.lang.Object.wait(Native Method)
- waiting on <0xd3ce7958> (a com.amgen.ea.orx.queue.impl.PersistenceQueue)
at java.lang.Object.wait(Object.java:429)
at com.amgen.ea.orx.queue.impl.PersistenceQueue.peek(PersistenceQueue.java:58)
- locked <0xd3ce7958> (a com.amgen.ea.orx.queue.impl.PersistenceQueue)
at com.amgen.ea.orx.custclient.impl.CustomerClientDispatcher.run(CustomerClientDispatcher.java:52)
at java.lang.Thread.run(Thread.java:534)
"Thread-8" daemon prio=1 tid=0x08710e80 nid=0x4f8e in Object.wait() [c7cc4000..c7cc423c]
at java.lang.Object.wait(Native Method)
- waiting on <0xd38b44c8> (a com.amgen.ea.orx.queue.impl.PersistenceQueue)
at java.lang.Object.wait(Object.java:429)
at com.amgen.ea.orx.queue.impl.PersistenceQueue.peek(PersistenceQueue.java:58)
- locked <0xd38b44c8> (a com.amgen.ea.orx.queue.impl.PersistenceQueue)
at com.amgen.ea.orx.custclient.impl.CustomerClientDispatcher.run(CustomerClientDispatcher.java:52)
at java.lang.Thread.run(Thread.java:534)
"ListenThread.Default" prio=1 tid=0xcc21b658 nid=0x4f8e runnable [c7c0c000..c7c0c23c]
at java.net.PlainSocketImpl.socketAccept(Native Method)
at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:353)
- locked <0xd3811cf8> (a java.net.PlainSocketImpl)
at java.net.ServerSocket.implAccept(ServerSocket.java:448)
at java.net.ServerSocket.accept(ServerSocket.java:419)
at weblogic.socket.WeblogicServerSocket.accept(WeblogicServerSocket.java:26)
at weblogic.t3.srvr.ListenThread.accept(ListenThread.java:735)
at weblogic.t3.srvr.ListenThread.run(ListenThread.java:301)
"Thread-6" prio=1 tid=0xcb3ef610 nid=0x4f8e in Object.wait() [c92a0000..c92a023c]
at java.lang.Object.wait(Native Method)
at java.util.TimerThread.mainLoop(Timer.java:429)
- locked <0xd348db48> (a java.util.TaskQueue)
at java.util.TimerThread.run(Timer.java:382)
"Thread-5" prio=1 tid=0x08c46fd8 nid=0x4f8e in Object.wait() [c9500000..c950023c]
at java.lang.Object.wait(Native Method)
at java.util.TimerThread.mainLoop(Timer.java:429)
- locked <0xd34901c8> (a java.util.TaskQueue)
at java.util.TimerThread.run(Timer.java:382)
"LDAPConnThread-0 ldap://0.0.0.0:7013" daemon prio=1 tid=0xcb67b150 nid=0x4f8e runnable [c9580000..c958023c]
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:183)
at java.io.BufferedInputStream.read(BufferedInputStream.java:201)
- locked <0xd3490a98> (a java.io.BufferedInputStream)
at netscape.ldap.ber.stream.BERElement.getElement(BERElement.java:101)
at netscape.ldap.LDAPConnThread.run(LDAPConnThread.java:538)
at java.lang.Thread.run(Thread.java:534)
"VDE Transaction Processor Thread" prio=1 tid=0xcb2fe110 nid=0x4f8e in Object.wait() [c9600000..c960023c]
at java.lang.Object.wait(Native Method)
- waiting on <0xd3490b30> (a com.octetstring.vde.backend.standard.TransactionProcessor)
at java.lang.Object.wait(Object.java:429)
at com.octetstring.vde.backend.standard.TransactionProcessor.waitChange(TransactionProcessor.java:365)
- locked <0xd3490b30> (a com.octetstring.vde.backend.standard.TransactionProcessor)
at com.octetstring.vde.backend.standard.TransactionProcessor.run(TransactionProcessor.java:212)
"ExecuteThread: '2' for queue: 'weblogic.admin.RMI'" daemon prio=1 tid=0x08665ba8 nid=0x4f8e in Object.wait() [c9680000..c968023c]
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:429)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
- locked <0xd3490ba8> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
"ExecuteThread: '1' for queue: 'weblogic.admin.RMI'" daemon prio=1 tid=0x0830a7e0 nid=0x4f8e in Object.wait() [c9700000..c970023c]
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:429)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
- locked <0xd3490c28> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
"ExecuteThread: '0' for queue: 'weblogic.admin.RMI'" daemon prio=1 tid=0x080a0c00 nid=0x4f8e in Object.wait() [c9780000..c978023c]
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:429)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
- locked <0xd3490ca8> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
"ExecuteThread: '2' for queue: 'weblogic.socket.Muxer'" daemon prio=1 tid=0xcafb3be8 nid=0x4f8e runnable [c98ff000..c98ff23c]
at weblogic.socket.PosixSocketMuxer.poll(Native Method)
at weblogic.socket.PosixSocketMuxer.processSockets(PosixSocketMuxer.java:100)
- locked <0xd31100e0> (a java.lang.String)
at weblogic.socket.SocketReaderRequest.execute(SocketReaderRequest.java:32)
at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:219)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:178)
"ExecuteThread: '1' for queue: 'weblogic.socket.Muxer'" daemon prio=1 tid=0xcaf65228 nid=0x4f8e waiting for monitor entry [c997f000..c997f23c]
at weblogic.socket.PosixSocketMuxer.processSockets(PosixSocketMuxer.java:91)
- waiting to lock <0xd31100e0> (a java.lang.String)
at weblogic.socket.SocketReaderRequest.execute(SocketReaderRequest.java:32)
at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:219)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:178)
"ExecuteThread: '0' for queue: 'weblogic.socket.Muxer'" daemon prio=1 tid=0xcafb0c20 nid=0x4f8e waiting for monitor entry [c99ff000..c99ff23c]
at weblogic.socket.PosixSocketMuxer.processSockets(PosixSocketMuxer.java:91)
- waiting to lock <0xd31100e0> (a java.lang.String)
at weblogic.socket.SocketReaderRequest.execute(SocketReaderRequest.java:32)
at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:219)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:178)
"weblogic.security.SpinnerRandomSource" daemon prio=1 tid=0xcaf64f98 nid=0x4f8e in Object.wait() [c9a7f000..c9a7f23c]
at java.lang.Object.wait(Native Method)
- waiting on <0xd31182d8> (a java.lang.Object)
at java.lang.Object.wait(Object.java:429)
at weblogic.security.SpinnerRandomBitsSource.run(SpinnerRandomBitsSource.java:60)
- locked <0xd31182d8> (a java.lang.Object)
at java.lang.Thread.run(Thread.java:534)
"weblogic.time.TimeEventGenerator" daemon prio=1 tid=0xcb3e1400 nid=0x4f8e in Object.wait() [c9aff000..c9aff23c]
at java.lang.Object.wait(Native Method)
- waiting on <0xd3118348> (a weblogic.time.common.internal.TimeTable)
at weblogic.time.common.internal.TimeTable.snooze(TimeTable.java:272)
- locked <0xd3118348> (a weblogic.time.common.internal.TimeTable)
at weblogic.time.common.internal.TimeEventGenerator.run(TimeEventGenerator.java:118)
at java.lang.Thread.run(Thread.java:534)
"ExecuteThread: '4' for queue: 'weblogic.kernel.System'" daemon prio=1 tid=0xcb3df3a8 nid=0x4f8e in Object.wait() [c9b7f000..c9b7f23c]
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:429)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
- locked <0xd31183b8> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
"ExecuteThread: '3' for queue: 'weblogic.kernel.System'" daemon prio=1 tid=0xcb3de7a0 nid=0x4f8e in Object.wait() [c9bff000..c9bff23c]
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:429)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
- locked <0xd3118438> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
"ExecuteThread: '2' for queue: 'weblogic.kernel.System'" daemon prio=1 tid=0xcb3ddcc0 nid=0x4f8e in Object.wait() [c9c7f000..c9c7f23c]
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:429)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
- locked <0xd31184b8> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
"ExecuteThread: '1' for queue: 'weblogic.kernel.System'" daemon prio=1 tid=0xcb1c66f0 nid=0x4f8e in Object.wait() [c9cff000..c9cff23c]
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:429)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
- locked <0xd3118538> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
"ExecuteThread: '0' for queue: 'weblogic.kernel.System'" daemon prio=1 tid=0xcb1c5b58 nid=0x4f8e in Object.wait() [c9d7f000..c9d7f23c]
at java.lang.Object.wait(Native Method)
- waiting on <0xd31185b8> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:429)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
- locked <0xd31185b8> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
"ExecuteThread: '39' for queue: 'weblogic.kernel.Default'" daemon prio=1 tid=0xcb1c4f50 nid=0x4f8e in Object.wait() [c9dff000..c9dff23c]
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:429)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
- locked <0xd3118638> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
"ExecuteThread: '38' for queue: 'weblogic.kernel.Default'" daemon prio=1 tid=0xcb1c4348 nid=0x4f8e in Object.wait() [c9e7f000..c9e7f23c]
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:429)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
- locked <0xd31186b8> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
"ExecuteThread: '37' for queue: 'weblogic.kernel.Default'" daemon prio=1 tid=0xcb1c3740 nid=0x4f8e in Object.wait() [c9eff000..c9eff23c]
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:429)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
- locked <0xd3118738> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
"ExecuteThread: '36' for queue: 'weblogic.kernel.Default'" daemon prio=1 tid=0xcb1c2ba8 nid=0x4f8e in Object.wait() [c9f7f000..c9f7f23c]
at java.lang.Object.wait(Native Method)
- waiting on <0xd31187b8> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:429)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
- locked <0xd31187b8> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
"ExecuteThread: '35' for queue: 'weblogic.kernel.Default'" daemon prio=1 tid=0xcb674078 nid=0x4f8e in Object.wait() [c9fff000..c9fff23c]
at java.lang.Object.wait(Native Method)
- waiting on <0xd3118838> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:429)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
- locked <0xd3118838> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
"ExecuteThread: '34' for queue: 'weblogic.kernel.Default'" daemon prio=1 tid=0xcb673470 nid=0x4f8e in Object.wait() [ca07f000..ca07f23c]
at java.lang.Object.wait(Native Method)
- waiting on <0xd31188b8> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:429)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
- locked <0xd31188b8> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
"ExecuteThread: '33' for queue: 'weblogic.kernel.Default'" daemon prio=1 tid=0xcb6728d8 nid=0x4f8e in Object.wait() [ca0ff000..ca0ff23c]
at java.lang.Object.wait(Native Method)
- waiting on <0xd3118938> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:429)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
- locked <0xd3118938> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
"ExecuteThread: '32' for queue: 'weblogic.kernel.Default'" daemon prio=1 tid=0xcb671cd0 nid=0x4f8e in Object.wait() [ca17f000..ca17f23c]
at java.lang.Object.wait(Native Method)
- waiting on <0xd31189b8> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:429)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
- locked <0xd31189b8> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
"ExecuteThread: '31' for queue: 'weblogic.kernel.Default'" daemon prio=1 tid=0xcb6710e0 nid=0x4f8e in Object.wait() [ca1ff000..ca1ff23c]
at java.lang.Object.wait(Native Method)
- waiting on <0xd3118a38> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:429)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
- locked <0xd3118a38> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
"ExecuteThread: '30' for queue: 'weblogic.kernel.Default'" daemon prio=1 tid=0xcbfdb548 nid=0x4f8e in Object.wait() [ca27f000..ca27f23c]
at java.lang.Object.wait(Native Method)
- waiting on <0xd3118ab8> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:429)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
- locked <0xd3118ab8> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
"ExecuteThread: '29' for queue: 'weblogic.kernel.Default'" daemon prio=1 tid=0xcbfda940 nid=0x4f8e in Object.wait() [ca2ff000..ca2ff23c]
at java.lang.Object.wait(Native Method)
- waiting on <0xd3118b38> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:429)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
- locked <0xd3118b38> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
"ExecuteThread: '28' for queue: 'weblogic.kernel.Default'" daemon prio=1 tid=0xcbfd9d58 nid=0x4f8e in Object.wait() [ca37f000..ca37f23c]
at java.lang.Object.wait(Native Method)
- waiting on <0xd3118bb8> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:429)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
- locked <0xd3118bb8> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
"ExecuteThread: '27' for queue: 'weblogic.kernel.Default'" daemon prio=1 tid=0xcbfd91c0 nid=0x4f8e in Object.wait() [ca3ff000..ca3ff23c]
at java.lang.Object.wait(Native Method)
- waiting on <0xd3118c38> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:429)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
- locked <0xd3118c38> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
"ExecuteThread: '26' for queue: 'weblogic.kernel.Default'" daemon prio=1 tid=0xcb119a40 nid=0x4f8e in Object.wait() [ca47f000..ca47f23c]
at java.lang.Object.wait(Native Method)
- waiting on <0xd3118cb8> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:429)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
- locked <0xd3118cb8> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
"ExecuteThread: '25' for queue: 'weblogic.kernel.Default'" daemon prio=1 tid=0xcb118e38 nid=0x4f8e in Object.wait() [ca4ff000..ca4ff23c]
at java.lang.Object.wait(Native Method)
- waiting on <0xd3118d38> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:429)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
- locked <0xd3118d38> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
"ExecuteThread: '24' for queue: 'weblogic.kernel.Default'" daemon prio=1 tid=0xcb118230 nid=0x4f8e in Object.wait() [ca57f000..ca57f23c]
at java.lang.Object.wait(Native Method)
- waiting on <0xd3118db8> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:429)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
- locked <0xd3118db8> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
"ExecuteThread: '23' for queue: 'weblogic.kernel.Default'" daemon prio=1 tid=0xcc207638 nid=0x4f8e in Object.wait() [ca5ff000..ca5ff23c]
at java.lang.Object.wait(Native Method)
- waiting on <0xd3118e38> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:429)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
- locked <0xd3118e38> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
"ExecuteThread: '22' for queue: 'weblogic.kernel.Default'" daemon prio=1 tid=0xcc206a30 nid=0x4f8e in Object.wait() [ca67f000..ca67f23c]
at java.lang.Object.wait(Native Method)
- waiting on <0xd3118eb8> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:429)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
- locked <0xd3118eb8> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
"ExecuteThread: '21' for queue: 'weblogic.kernel.Default'" daemon prio=1 tid=0xcc205e28 nid=0x4f8e in Object.wait() [ca6ff000..ca6ff23c]
at java.lang.Object.wait(Native Method)
- waiting on <0xd3118f38> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:429)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
- locked <0xd3118f38> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
"ExecuteThread: '20' for queue: 'weblogic.kernel.Default'" daemon prio=1 tid=0xcbfbf1e0 nid=0x4f8e in Object.wait() [ca77f000..ca77f23c]
at java.lang.Object.wait(Native Method)
- waiting on <0xd3118fb8> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:429)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
- locked <0xd3118fb8> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
"ExecuteThread: '19' for queue: 'weblogic.kernel.Default'" daemon prio=1 tid=0xcbfbe648 nid=0x4f8e in Object.wait() [ca7ff000..ca7ff23c]
at java.lang.Object.wait(Native Method)
- waiting on <0xd3119038> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:429)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
- locked <0xd3119038> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
"ExecuteThread: '18' for queue: 'weblogic.kernel.Default'" daemon prio=1 tid=0xcbfbda40 nid=0x4f8e in Object.wait() [ca87f000..ca87f23c]
at java.lang.Object.wait(Native Method)
- waiting on <0xd31190b8> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:429)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
- locked <0xd31190b8> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
"ExecuteThread: '17' for queue: 'weblogic.kernel.Default'" daemon prio=1 tid=0xcb62c500 nid=0x4f8e in Object.wait() [ca8ff000..ca8ff23c]
at java.lang.Object.wait(Native Method)
- waiting on <0xd3119138> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:429)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
- locked <0xd3119138> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
"ExecuteThread: '16' for queue: 'weblogic.kernel.Default'" daemon prio=1 tid=0xcb62b8f8 nid=0x4f8e in Object.wait() [ca97f000..ca97f23c]
at java.lang.Object.wait(Native Method)
- waiting on <0xd31191b8> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:429)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
- locked <0xd31191b8> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
"ExecuteThread: '15' for queue: 'weblogic.kernel.Default'" daemon prio=1 tid=0xcb62acf0 nid=0x4f8e in Object.wait() [ca9ff000..ca9ff23c]
at java.lang.Object.wait(Native Method)
- waiting on <0xd3119238> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:429)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
- locked <0xd3119238> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
"ExecuteThread: '14' for queue: 'weblogic.kernel.Default'" daemon prio=1 tid=0xcb61f5a8 nid=0x4f8e in Object.wait() [caa7f000..caa7f23c]
at java.lang.Object.wait(Native Method)
- waiting on <0xd31192b8> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:429)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
- locked <0xd31192b8> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
"ExecuteThread: '13' for queue: 'weblogic.kernel.Default'" daemon prio=1 tid=0xcb61e9e0 nid=0x4f8e in Object.wait() [caaff000..caaff23c]
at java.lang.Object.wait(Native Method)
- waiting on <0xd3119338> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:429)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
- locked <0xd3119338> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
"ExecuteThread: '12' for queue: 'weblogic.kernel.Default'" daemon prio=1 tid=0xcaf13af0 nid=0x4f8e in Object.wait() [cab7f000..cab7f23c]
at java.lang.Object.wait(Native Method)
- waiting on <0xd31193b8> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:429)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
- locked <0xd31193b8> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
"ExecuteThread: '11' for queue: 'weblogic.kernel.Default'" daemon prio=1 tid=0xcaf12ee8 nid=0x4f8e in Object.wait() [cabff000..cabff23c]
at java.lang.Object.wait(Native Method)
- waiting on <0xd3119438> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:429)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
- locked <0xd3119438> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
"ExecuteThread: '10' for queue: 'weblogic.kernel.Default'" daemon prio=1 tid=0xcaf12350 nid=0x4f8e in Object.wait() [cac7f000..cac7f23c]
at java.lang.Object.wait(Native Method)
- waiting on <0xd31194b8> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:429)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
- locked <0xd31194b8> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
"ExecuteThread: '9' for queue: 'weblogic.kernel.Default'" daemon prio=1 tid=0xcb309598 nid=0x4f8e in Object.wait() [cacff000..cacff23c]
at java.lang.Object.wait(Native Method)
- waiting on <0xd3119538> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:429)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
- locked <0xd3119538> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
"ExecuteThread: '8' for queue: 'weblogic.kernel.Default'" daemon prio=1 tid=0xcb3089c8 nid=0x4f8e in Object.wait() [cad7f000..cad7f23c]
at java.lang.Object.wait(Native Method)
- waiting on <0xd31195b8> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:429)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
- locked <0xd31195b8> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
"ExecuteThread: '7' for queue: 'weblogic.kernel.Default'" daemon prio=1 tid=0xcaf56250 nid=0x4f8e in Object.wait() [cadff000..cadff23c]
at java.lang.Object.wait(Native Method)
- waiting on <0xd3119638> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:429)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
- locked <0xd3119638> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
"ExecuteThread: '6' for queue: 'weblogic.kernel.Default'" daemon prio=1 tid=0xcaf55cb0 nid=0x4f8e in Object.wait() [cae7f000..cae7f23c]
at java.lang.Object.wait(Native Method)
- waiting on <0xd31196b8> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:429)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
- locked <0xd31196b8> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
"ExecuteThread: '5' for queue: 'weblogic.kernel.Default'" daemon prio=1 tid=0xcaf55918 nid=0x4f8e in Object.wait() [caeff000..caeff23c]
at java.lang.Object.wait(Native Method)
- waiting on <0xd3119738> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:429)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
- locked <0xd3119738> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
"ExecuteThread: '4' for queue: 'weblogic.kernel.Default'" daemon prio=1 tid=0xcaf55510 nid=0x4f8e in Object.wait() [cb47f000..cb47f23c]
at java.lang.Object.wait(Native Method)
- waiting on <0xd31197b8> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:429)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
- locked <0xd31197b8> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
"ExecuteThread: '3' for queue: 'weblogic.kernel.Default'" daemon prio=1 tid=0xcaf55150 nid=0x4f8e in Object.wait() [cb4ff000..cb4ff23c]
at java.lang.Object.wait(Native Method)
- waiting on <0xd3119838> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:429)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
- locked <0xd3119838> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
"ExecuteThread: '2' for queue: 'weblogic.kernel.Default'" daemon prio=1 tid=0xcbfc7bd8 nid=0x4f8e in Object.wait() [cb57f000..cb57f23c]
at java.lang.Object.wait(Native Method)
- waiting on <0xd31198b8> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:429)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
- locked <0xd31198b8> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
"ExecuteThread: '1' for queue: 'weblogic.kernel.Default'" daemon prio=1 tid=0xcb3cd250 nid=0x4f8e in Object.wait() [cb5ff000..cb5ff23c]
at java.lang.Object.wait(Native Method)
- waiting on <0xd3119938> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:429)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
- locked <0xd3119938> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
"ExecuteThread: '0' for queue: 'weblogic.kernel.Default'" daemon prio=1 tid=0xcb3cd0b8 nid=0x4f8e in Object.wait() [cc1fe000..cc1fe23c]
at java.lang.Object.wait(Native Method)
- waiting on <0xd31199b8> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:429)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
- locked <0xd31199b8> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)
"Thread-1" daemon prio=1 tid=0x086d6478 nid=0x4f8e in Object.wait() [cc3e1000..cc3e123c]
at java.lang.Object.wait(Native Method)
- waiting on <0xd3119a88> (a java.util.TaskQueue)
at java.lang.Object.wait(Object.java:429)
at java.util.TimerThread.mainLoop(Timer.java:403)
- locked <0xd3119a88> (a java.util.TaskQueue)
at java.util.TimerThread.run(Timer.java:382)
"Signal Dispatcher" daemon prio=1 tid=0x080c4170 nid=0x4f8e waiting on condition [0..0]
"Finalizer" daemon prio=1 tid=0x080bfac0 nid=0x4f8e in Object.wait() [cf509000..cf50923c]
at java.lang.Object.wait(Native Method)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:111)
- locked <0xd3129c78> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:127)
at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159)
"Refe
<p>Can you post the stack trace. I assume these thread dumps were taken after the error occured. Can you take a few dumps just before you run out of memory.</p>
<p>
Hussein Badakhchani</br>
</p>
Similar Messages
-
Weblogic 6.1 deployment out of memory error - please help
I am getting an out of memory error when deploying an ear to a weblogic 6.1 server. I tried increasing heap size with the startup param. I also tried upgrading the jdk to 1.3.1_02. Here is how I am starting weblogic -
"%JAVA_HOME%\bin\java" -hotspot -XX:MaxPermSize=256m -ms64m -mx64m -classpath "%CLASSPATH%" -Dweblogic.Domain=mydomain -Dweblogic.management.discover=false -Dweblogic.Name=myserver "-Dbea.home=C:\bea" -Dweblogic.management.password=%WLS_PW% -Dweblogic.ProductionModeEnabled=%STARTMODE% "-Djava.security.policy==C:\bea\wlserver6.1/lib/weblogic.policy" weblogic.Server
I tried 64, 128, and 256 for the MaxPermSize param and the larger I make this number, the longer the deploy takes. I also tried lowering the session timeout to 1 min from 30. Now, when I deploy to the weblogic connection, jdev writes the war, then hangs. The rest of my desktop is slow but responsive, but I let jdev go for up to 1hr when running this deploy, and it never even starts the weblogic.deploy call. I end up killing it. This ear is not very big and is a simple servlet.
Any thoughts?repost
-
Out Of Memory Error - Weblogic server
Hi,
We are facing following errors in our production environment frequently...
Please guide us how can we identify root cause.. because every time this error come in different functionalities so we can not blame one functionality (Code)...
Error # 1
======
<Jun 18, 2009 4:24:41 PM IST> <Error> <Socket> <BEA-000405> <Uncaught Throwable in processSockets
java.lang.OutOfMemoryError: .
java.lang.OutOfMemoryError:
>
<Jun 18, 2009 4:24:45 PM IST> <Error> <Server> <BEA-002608> <The ListenThread because of an error: java.lang.OutOfMemoryError: memsystemAllocObject - Object size: 24
java.lang.OutOfMemoryError: memsystemAllocObject - Object size: 24
at java.net.PlainSocketImpl.socketAccept(Native Method)
at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
at java.net.ServerSocket.implAccept(ServerSocket.java:450)
at java.net.ServerSocket.accept(ServerSocket.java:421)
at weblogic.socket.WeblogicServerSocket.accept(WeblogicServerSocket.java:34)
Truncated. see log file for complete stacktrace
>
Error # 2
======
<Jun 18, 2009 7:32:27 PM IST> <Error> <Server> <BEA-002608> <The ListenThread because of an error: java.lang.OutOfMemoryError:
java.lang.OutOfMemoryError:
>
<Jun 18, 2009 7:32:27 PM IST> <Error> <Server> <BEA-002608> <The ListenThread because of an error: java.lang.OutOfMemoryError: memsystemAllocObject - Object size: 24
java.lang.OutOfMemoryError: memsystemAllocObject - Object size: 24
at java.net.PlainSocketImpl.socketAccept(Native Method)
at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
at java.net.ServerSocket.implAccept(ServerSocket.java:450)
at java.net.ServerSocket.accept(ServerSocket.java:421)
at weblogic.socket.WeblogicServerSocket.accept(WeblogicServerSocket.java:34)
Truncated. see log file for complete stacktrace
>
Thanks in Advance.
Edited by: Rushang.Kansara on Jul 1, 2009 6:46 AMOnce you get to the state where you're out of memory, you'll get any number of possible variations on the error message. These variations will give you absolutely zero help for why you've run out of memory. Without more information, it's impossible to tell whether you have a memory leak, or whether you simply have too small a value for "maxmem" in the JVM.
Typically, I would assume that if the server has been running for a "while" (whatever that means) without error, then it probably isn't a simple issue with setting the "maxmem" value high enough. If it does fail pretty quickly after startup (and processing of initial messages), then it very well could be a maxmem problem. This is done by adding or changing the "-Xmxnnn" command-line parameter to the JVM. For instance, if you wanted to give it 2g of max heap, you would put "-Xmx2048m" on the command line. That value won't work on Windows (not sure about Vista or 64-bit), but it will work perfectly well on Unix/Linux.
If you have a memory leak, it's good to first verify this by sitting on the "Server"->"Performance" page in the WebLogic console. If you watch the "memory free" (or "memory inuse") percentage, you'll see that it varies by a wide margin over time. What you want to do is to be able to force a GC (there should still be a button there to do that) and then note the percent free immediately after that. You should do this every few minutes while the server is actively working. If the percent free value (after the GC) is consistently going down, even slowly, then you likely have a memory leak. This is where it gets a little more difficult. You could look at the free VisualVM tool which provides some memory debugging features, but they are somewhat limited (I only know this from reading some literature, I haven't tried it), or you could look at the commercial tools JProfiler, JProbe, or YourKit (in decreasing order of my personal preference). -
Stand alone weblogic keeps running out of memory ! Cause of Memory Leakage?
Hi all,
I have a machine with 16gb ram and 500gb hard disk with linux os. I have installed weblogic server in it with an adf application deployed on it. So now the problem every two weeks the machine keeps running out of memory . Could it be because of connection leakage ?
There are only a couple of people accessing the application time and again.
When i checked for the data source connection count in weblogic server under data sources , it mostly shows around 10 - 20 connections.
What are various reasons for leakage of connection in ADF application ?
I have not made any explicit jdbc calls in our application .
There is a couple of PL/SQL calls to an external api(this is done in the application module) through db links and rest all is just plain accessing of application modules and view objects .
So what could be the reason if any memory leakage is occuring ?
My application module configuration is
<AppModuleConfigBag ApplicationName="model.am.AuthenticationAM">
<AppModuleConfig jbo.project="model.Model" DeployPlatform="LOCAL" name="AuthenticationAMLocal" ApplicationName="model.am.AuthenticationAM">
<AM-Pooling jbo.ampool.maxinactiveage="300000" jbo.ampool.initpoolsize="44" jbo.ampool.maxpoolsize="60" jbo.recyclethreshold="40" jbo.ampool.minavailablesize="0" jbo.ampool.monitorsleepinterval="120000"/>
<Database jbo.locking.mode="optimistic" jbo.TypeMapEntries="OracleApps"/>
<Security AppModuleJndiName="model.am.AuthenticationAM"/>
<Custom JDBCDataSource="java:comp/env/jdbc/ConnectionDBDS" jbo.rowid_am_datasource_name="java:comp/env/jdbc/ROWIDAM_DBDS"/>Thanks in advance
Sam
Edited by: Sam47 on 19-Oct-2012 01:46Hi Suspito ,
Thanks for the link ..
Is that the standard configuration for weblogic server?
If at all there are connection leaks in the application in the weblogic server under datasource will it show the count of all the database connection open ?
-Sam -
While creating DB using DBCA getting ORA-27102: out of memory in Linux
Hi All,
I am working on 11.2.0.3 oracle Redhat linux. I am getting error "ORA-27102: out of memory" while creating a new database using dbca
Below are the DB ans OS details. Please check it and let me know what i need to do to overcome this issue.
SQL> select * from v$version;
BANNER
Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production
PL/SQL Release 11.2.0.3.0 - Production
CORE 11.2.0.3.0 Production
TNS for Linux: Version 11.2.0.3.0 - Production
NLSRTL Version 11.2.0.3.0 - Production
$uname -a
Linux greenlantern1a 2.6.18-92.1.17.0.1.el5 #1 SMP Tue Nov 4 17:10:53 EST 2008 x86_64 x86_64 x86_64 GNU/Linux
$cat /etc/sysctl.conf
# Controls the maximum shared segment size, in bytes
kernel.shmmax = 68719476736
# Controls the maximum number of shared memory segments, in pages
kernel.shmall = 4294967296
kernel.shmall = 2097152
kernel.shmmax = 4294967295
kernel.shmmni = 4096
kernel.sem = 250 32000 100 128
net.core.rmem_default = 4194304
net.core.wmem_default = 262144
net.core.rmem_max = 4194304
net.core.wmem_max = 1048576
fs.file-max = 6815744
fs.aio-max-nr = 1048576
net.ipv4.ip_local_port_range = 9000 65500
$free -g
total used free shared buffers cached
Mem: 94 44 49 0 0 31
-/+ buffers/cache: 12 81
Swap: 140 6 133
$ulimit -l
32
$ipcs -lm
Shared Memory Limits
max number of segments = 4096
max seg size (kbytes) = 4194303
max total shared memory (kbytes) = 8388608
min seg size (bytes) = 1
Also created a trace file under trace loction and it suggesting to changes shm parameter value. but i am not sure which parameter (shmmax or shmall) and value i need to modify.
below are trace file info
Trace file /u02/app/oracle/diag/rdbms/beaconpt/beaconpt/trace/beaconpt_ora_9324.trc
Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options
ORACLE_HOME = /u02/app/oracle/product/11.2.0.3
System name: Linux
Node name: greenlantern1a
Release: 2.6.18-92.1.17.0.1.el5
Version: #1 SMP Tue Nov 4 17:10:53 EST 2008
Machine: x86_64
Instance name: beaconpt
Redo thread mounted by this instance: 0 <none>
Oracle process number: 0
Unix process pid: 9324, image: oracle@greenlantern1a
*** 2012-02-02 11:09:53.539
Switching to regular size pages for segment size 33554432
Switching to regular size pages for segment size 4261412864
skgm warning: ENOSPC creating segment of size 00000000fe000000
fix shm parameters in /etc/system or equivalent
Please let me what are the kernel parameter values i need to chage to work this.
Thanks in advance.Yes it is same question, but i didn't have any solution there and still looking for some help. the solution it was provided in the last post is not working and getting the same error even with less thn 20% of memory. Please let me know how to overcome this issue.
Thanks -
ORA-27102: out of memory, when I try to increase the SGA in 10gR2 linux 64
Hi, I´m trying to increase sga_max_size parameter, but when I startup the DB the following error is appears:
ORA-27102: out of memory...The maximun value is 3.6 Gb Approximately, if I increase this value, the DB not start and is appears this error.
My DB version is 10.2.0.4.0
My OS: Enterprise Linux Enterprise Linux AS release 4 (October Update 6)
The linux Kernel: 2.6.9-67.0.0.0.1.ELsmp x86_64 GNU/Linux
Physical memory: 14 GB
My init.ora:
orcl10.__db_cache_size=2113929216
orcl10.__java_pool_size=33554432
orcl10.__large_pool_size=16777216
orcl10.__shared_pool_size=301989888
orcl10.__streams_pool_size=16777216
*.audit_file_dest='/home/oracle/product/bbdd/admin/orcl10/adump'
*.background_dump_dest='/home/oracle/product/bbdd/admin/orcl10/bdump'
*.compatible='10.2.0.1.0'
*.control_files='/home/oracle/product/oradata/orcl10/control01.ctl','/home/oracle/product/oradata/orcl10/control02.ctl','/home/oracle/product/oradata/orcl10/
control03.ctl'
*.core_dump_dest='/home/oracle/product/bbdd/admin/orcl10/cdump'
*.db_block_size=8192
*.db_domain=''
*.db_file_multiblock_read_count=16
*.db_name='orcl10'
*.db_recovery_file_dest='/home/oracle/product/bbdd/flash_recovery_area'
*.db_recovery_file_dest_size=96636764160
*.dispatchers='(PROTOCOL=TCP) (SERVICE=orcl10XDB)'
*.job_queue_processes=10
*.open_cursors=300
*.pga_aggregate_target=830472192
*.processes=150
*.remote_login_passwordfile='EXCLUSIVE'
*.session_cached_cursors=50
*.sga_max_size=3670016000
*.sga_target=3670016000
*.shared_pool_size=267386880
*.undo_management='AUTO'
*.undo_tablespace='UNDOTBS1'
*.user_dump_dest='/home/oracle/product/bbdd/admin/orcl10/udump'
My sysctl.conf
# Kernel sysctl configuration file for Enterprise Linux
# For binary values, 0 is disabled, 1 is enabled. See sysctl(8) and
# sysctl.conf(5) for more details.
# Controls IP packet forwarding
net.ipv4.ip_forward = 0
# Controls source route verification
net.ipv4.conf.default.rp_filter = 1
# Do not accept source routing
net.ipv4.conf.default.accept_source_route = 0
# Controls the System Request debugging functionality of the kernel
kernel.sysrq = 0
# Controls whether core dumps will append the PID to the core filename.
# Useful for debugging multi-threaded applications.
kernel.core_uses_pid = 1
#Parametros del kernel para rac-oracle
kernel.shmall = 2097152
kernel.shmmax = 10737418240
kernel.shmmni = 4096
# semaforos: senmsl semmns semopm semmni
kernel.sem = 256 32000 100 128
#kernel.msgmnb = 65535
#kernel.msgmni = 2878
#kernel.msgmax = 8192
fs.file-max = 65536
net.ipv4.ip_local_port_range = 1024 65000
net.core.rmem_default = 262144
net.core.rmem_max = 262144
net.core.wmem_default = 262144
net.core.wmem_max = 262144
I need any idea for how to resolve this problem, thank you
Edited by: cehache on 10-mar-2011 11:50sorry, the previously init.ora is old, this is the actual init.ora file and the shared_pool_size=0:
orcl10.__db_cache_size=3271557120
orcl10.__java_pool_size=33554432
orcl10.__large_pool_size=16777216
orcl10.__shared_pool_size=335544320
orcl10.__streams_pool_size=0
*.audit_file_dest='/home/oracle/product/bbdd/admin/orcl10/adump'
*.background_dump_dest='/home/oracle/product/bbdd/admin/orcl10/bdump'
*.compatible='10.2.0.1.0'
*.control_files='/home/oracle/product/oradata/orcl10/control01.ctl','/home/oracle/product/oradata/orcl10/control02.ctl','/home/oracle/product/oradata/orcl10/
control03.ctl'
*.core_dump_dest='/home/oracle/product/bbdd/admin/orcl10/cdump'
*.db_block_size=8192
*.db_domain=''
*.db_file_multiblock_read_count=16
*.db_name='orcl10'
*.db_recovery_file_dest='/home/oracle/product/bbdd/flash_recovery_area'
*.db_recovery_file_dest_size=96636764160
*.dispatchers='(PROTOCOL=TCP) (SERVICE=orcl10XDB)'
*.job_queue_processes=10
*.open_cursors=300
*.pga_aggregate_target=2147483648
*.processes=150
*.remote_login_passwordfile='EXCLUSIVE'
*.session_cached_cursors=50
*.sga_max_size=3670016000
*.sga_target=3670016000
*.shared_pool_size=0
*.undo_management='AUTO'
*.undo_tablespace='UNDOTBS1'
*.user_dump_dest='/home/oracle/product/bbdd/admin/orcl10/udump'
this is the result of free command:
[oracle@edubi dbs]$ free
total used free shared buffers cached
Mem: 14366428 13844616 521812 0 65468 11599248
-/+ buffers/cache: 2179900 12186528
Swap: 10482404 44200 10438204
the result of cat /proc/meminfo:
[oracle@edubi dbs]$ cat /proc/meminfo
MemTotal: 14366428 kB
MemFree: 123396 kB
Buffers: 67076 kB
Cached: 11823056 kB
SwapCached: 1504 kB
Active: 9671280 kB
Inactive: 3965444 kB
HighTotal: 0 kB
HighFree: 0 kB
LowTotal: 14366428 kB
LowFree: 123396 kB
SwapTotal: 10482404 kB
SwapFree: 10438432 kB
Dirty: 868 kB
Writeback: 56 kB
Mapped: 5582756 kB
Slab: 350380 kB
CommitLimit: 17665616 kB
Committed_AS: 13194688 kB
PageTables: 212384 kB
VmallocTotal: 536870911 kB
VmallocUsed: 267732 kB
VmallocChunk: 536603127 kB
HugePages_Total: 0
HugePages_Free: 0
Hugepagesize: 2048 kB
the result of vmstat command;
[oracle@edubi dbs]$ vmstat
procs -----------memory---------- ---swap-- -----io---- system ----cpu----
r b swpd free buff cache si so bi bo in cs us sy id wa
1 1 43896 58332 67220 11686176 0 0 166 195 15 10 2 0 94 3
thank you -
WebLogic 8.1 SP4 on Linux - thread dump does not work
I have setup WebLogic 8.1 SP4 on RedHat Linux AS 3.3 with Sun JVM (142_05). When I try to perform a thread dump using weblogic.Admin, I receive the following error in the stderr file. I have also tried the kill -3 but still no thread dump. Is there some trick to this ?
cannot load libary 'stackdump': java.lang.UnsatisfiedLinkError: no stackdump in java.library.pathI am getting the same error as well
cannot load libary 'stackdump': java.lang.UnsatisfiedLinkError: no stackdump in java.library.path
I am creating a simple domain using the domain config wizard and I hope all the PATH and LD_LIBRARY_PATH are set by the default scripts. But I am still getting this error.
But Kill -3 <PID> works though!
Is is a know problem OR am I missing something?!
Bala -
Weblogic generating out of memory error when commandline run doesn't
Hello,
I am just beginning to use weblogic. I have a jar file which runs fine if run from command line with the options
"C:\Program Files (x86)\Java\jre6\bin\java" -jar -Xms1024m -Xmx1024m Report.jarIt connects to oracle and selects some data around (500k records) and writes it to a csv.
When I run the same jar from within a web application (I mean obvoiusly a servlet calling the jar's main method) the webapp generates
out of memory error after 80k records itself.
I tried changing the configuration the server Startup arguments from the console (Server Start and then Arguments) and then restart.
I just wrote the same thing there -Xms1024m -Xmx1024m
I guess I am missing something. Please share your answers.
Environment :
Win2k8, weblogic 10gR3, jdk 1.5. The application is installed as a service.
Thanks,
Neetesh
Edited by: user13312817 on 5 Dec, 2011 12:15 AMIf you are not using NodeManager, then I don't think those server settings actually control anything. If you are just exploring WLS, then I suspect you are simply starting an AdminServer in a basic domain and don't have a cluster/nodemanager based environment. I could be wrong, please correct as needed!
If you are simply starting your WebLogic Server instance using the startWebLogic.sh|cnd script, you can set an environment variable on the command shell that will be picked up and used when the server starts.
set USER_MEM_ARGS=-Xmx1024 -Xms1024m
* apply appropriate *nix changes as appropriate.
Then use the startWebLogic.sh|cmd script to start the server and test your application.
It may very well be the case that your particular application consumes > 1GB heap so you may need more. Remember that you now have a server environment running over your "main" class, so there is bound to be more memory used that could be just sneaking your heap use over 1GB. For example.
-steve- -
A few days ago, some processes could not log in and got error message
ORA-01034: ORACLE not available
ORA-27102: out of memory
Linux-x86_64 Error: 12: Cannot allocate memory
After some time, the problem was suddenly gone. We had such issues already in the past.
Is there a way to get/monitor actual memory usage ( a)what Oracle thinks it uses and b)what OS thinks Oracle uses)?
Tools like ps and top seam to show shared memory incorrectly and RAM used may be completely wrong!? We currently only use 30G (memory_target) on a server having 72G. We intended to increase memory_target some time ago but did not implement the change due to ORA-27102 coming up from time.
OS: SUSE SLES 64bit
Oracle 11.2.0.2 64bit
BANNER
Oracle Database 11g Enterprise Edition Release 11.2.0.2.0 - 64bit Production
PL/SQL Release 11.2.0.2.0 - Production
CORE 11.2.0.2.0 Production
TNS for Linux: Version 11.2.0.2.0 - Production
NLSRTL Version 11.2.0.2.0 - Production
cat /proc/meminfo
MemTotal: 74306292 kB
MemFree: 498240 kB
Buffers: 5461300 kB
Cached: 60442304 kB
SwapCached: 185256 kB
Active: 33642300 kB
Inactive: 35066764 kB
Active(anon): 20735148 kB
Inactive(anon): 1302912 kB
Active(file): 12907152 kB
Inactive(file): 33763852 kB
Unevictable: 5932 kB
Mlocked: 5932 kB
SwapTotal: 8388600 kB
SwapFree: 7142712 kB
Dirty: 2700 kB
Writeback: 0 kB
AnonPages: 2628540 kB
Mapped: 19062944 kB
Shmem: 19229668 kB
Slab: 1968204 kB
SReclaimable: 1781108 kB
SUnreclaim: 187096 kB
KernelStack: 5744 kB
PageTables: 2071144 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
WritebackTmp: 0 kB
CommitLimit: 45541744 kB
Committed_AS: 24223804 kB
VmallocTotal: 34359738367 kB
VmallocUsed: 290600 kB
VmallocChunk: 34321279204 kB
HardwareCorrupted: 0 kB
HugePages_Total: 0
HugePages_Free: 0
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 2048 kB
DirectMap4k: 6384 kB
DirectMap2M: 2080768 kB
DirectMap1G: 73400320 kBTop shows constantly values of high RAM usage.
Mem: 72564M total, 65263M used, 7301M free, 3797M buffers
There are no other processes. It's a pure DB server.Top shows the following process list
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
8067 oracle 20 0 30.3g 17g 17g S 0 24.8 4:11.50 oracle
8075 oracle 20 0 30.3g 16g 16g S 0 22.8 40:26.79 oracle
8071 oracle 20 0 30.3g 16g 16g S 0 22.8 42:03.30 oracle
8087 oracle 20 0 30.3g 9.7g 9.7g S 0 13.6 10:35.84 oracle
16561 oracle 20 0 30.3g 3.0g 3.0g S 0 4.3 0:42.38 oracle
8095 oracle 20 0 30.3g 2.7g 2.7g S 0 3.8 6:45.29 oracle
1229 oracle 20 0 30.3g 1.1g 1.1g S 0 1.6 0:32.12 oracle
16570 oracle 20 0 30.3g 996m 992m S 0 1.4 0:05.02 oracle
26773 oracle 20 0 30.3g 966m 962m S 0 1.3 0:14.90 oracle
8099 oracle 20 0 30.3g 907m 906m S 0 1.3 27:40.60 oracle
624 oracle 20 0 30.4g 737m 647m S 0 1.0 0:04.16 oracle
8083 oracle 20 0 30.3g 733m 729m D 0 1.0 11:10.35 oracle
2705 oracle 20 0 30.3g 695m 685m S 0 1.0 0:31.26 oracle
13357 oracle 20 0 30.3g 692m 687m S 0 1.0 0:22.71 oracle
SQL> show parameter memory
NAME TYPE VALUE
hi_shared_memory_address integer 0
memory_max_target big integer 30G
memory_target big integer 30G
shared_memory_address integer 0
SQL> show parameter sga
NAME TYPE VALUE
lock_sga boolean FALSE
pre_page_sga boolean FALSE
sga_max_size big integer 30G
sga_target big integer 15G
SQL> show parameter pga
NAME TYPE VALUE
pga_aggregate_target big integer 2500M
ulimit -a
core file size (blocks, -c) 1
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 580443
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) 63160352
open files (-n) 1024
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) 580443
virtual memory (kbytes, -v) 66155920
file locks (-x) unlimitedEdited by: TimWong765 on Aug 29, 2012 9:26 PMThanks for you reply. We may consider lock_sga=true (events like "latch: shared pool" do actually come up). The referenced thread had the problem during startup though. The problem occurred in our DB while the system was running - but maybe swapping/paging occurred exactly during that time!?
Edited by: TimWong765 on Aug 29, 2012 9:59 PM -
10g XE: ORA-27102: out of memory, Linux Error: 28: No space left on device
Hi,
I just installed oracle-xe-10.2.0.1-1.0.i386.rpm on a virtual CentOS box. After I run "/etc/init.d/oracle-xe configure", only listener is running, not the instance itself and $ORACLE_HOME/config/log/CloneRmanRestore.log says
ORA-27102: out of memory
Linux Error: 28: No space left on device
According to installation requirements (http://download.oracle.com/docs/cd/B25329_01/doc/install.102/b25144/toc.htm) I checked
RAM and swap:
[root@56 ~]# free -m
total used free shared buffers cached
Mem: 7961 7453 507 0 39 923
-/+ buffers/cache: 6491 1470
Swap: 16378 7012 9366
Available disk space:
[root@56 ~]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/vzfs 2.9G 969M 1.9G 34% /
Kernel params:
[root@56 ~]# sysctl -a | egrep 'sem|shm'
error: "Operation not permitted" reading key "kernel.cap-bound"
kernel.sem = 250 32000 100 128
kernel.shmmni = 4096
kernel.shmall = 2097152
kernel.shmmax = 536870912
So it all looks to be OK.
Any ideas what can be wrong?
Thanks,
Radekrskokan wrote:
The distro version is "CentOS release 5.4 (Final)".
Unfortunately the wmem/rmem params can't be changed, as I learned it's a limitation of the virtual hosting, Virtuozzo. Anyway they are "just" for network setting, not for RAM.Yes - but worthy of noting once you run into your next problem. :-)
>
Couldn't be the problem that I really don't have enough disk space? Only 1.9 GB free.It's unlikely.
I suspect that the SHMMAX setting needs to be increased to 1/2 physical RAM, as described in the doc I referenced.
I suspect the installer is looking at the physical RAM and guessing the target SGA size based on that. It would therefore assume, for DB creation, that it should crank the SGA to around 700MB (1GB max, leave some for PGA, but use some SGA because of SHared Server for APEX) -
One of the prod clusters is running on below crashed both nodes with an out of memory exception which looks like below.
How do we get to know if the crash is due to a memory leak in the code or elsewhere?
Any inputs which helps to identify on the issue and submit data to application team supporting the need to update the systems is appreciated.
RHEL 4.6 on a 32 bit OS
BEA JRockit(R) R27.5.0
Below is the dump file generated when the JVM was crashed.
===== BEGIN DUMP =============================================================
JRockit dump produced after 21 days, 21:54:48 on Thu Aug 29 23:22:36 2013
* If you see this dump, please go to *
* http://edocs.bea.com/jrockit/go2troubleshooting.html *
* for troubleshooting information. *
Additional information is available in:
/wls_domains/prod_slimseu_wli_domain/jrockit.20717.dump
No snapshot file (core dump) will be created because core dumps have been
disabled. To enable core dumping, try "ulimit -c unlimited"
before starting JRockit again.
Error Message: Out of memory [68]
Signal info : si_signo=11, si_code=2 si_addr=(nil)
Fatal Error : Lock Iteration lockTokenIterInit src/jvm/code/runtime/refiter.c:233
Version : BEA JRockit(R) R27.5.0-110_o-99226-1.5.0_14-20080528-1505-linux-ia32
GC Strategy : Mode: throughput. Currently using strategy: genparpar
GC Status : OC is not running. Last finished OC was OC#6379.
: YC is not running. Last finished YC was YC#46347.
OC History : Strategy genparpar was used for OC#5680 to OC#5848.
: Strategy singleparpar was used for OC#5849.
: Strategy genparpar was used for OC#5850 to OC#6034.
: Strategy singleparpar was used for OC#6035.
: Strategy genparpar was used for OC#6036 to OC#6379.
YC History : Ran 7 YCs before OC#6375.
: Ran 6 YCs before OC#6376.
: Ran 4 YCs before OC#6377.
: Ran 7 YCs before OC#6378.
: Ran 5 YCs before OC#6379.
: Ran 0 YCs since last OC.
YC Promotion : Last YC successfully promoted all objects
Heap : 0x8100000 - 0x88100000 (Size: 2048 MB)
Compaction : 0x80100000 - 0x88100000 (Current compaction type: external)
NurseryList : 0x6d634138 - 0x798afad0
KeepArea : 0x79218a20 - 0x7c8122f0
NurseryMarker: [ 0x79218a20, 0x7c8122f0 ]
CompRefs : References are 32-bit.
CPU : Intel Core 2 SSE SSE2 SSE3 SSSE3 SSE4.1 EM64T
Number CPUs : 4
Tot Phys Mem : 21292417024 (20306 MB)
OS version : Red Hat Enterprise Linux ES release 4 (Nahant Update 6)
Linux version 2.6.9-67.ELsmp ([email protected]) (gcc version 3.4.6 20060404 (Red Hat 3.4.6-8)) #1 SMP Wed Nov 7 13:58:04 EST 2007 (i686)
Thread System: NPTL
Java locking : Normal
State : JVM is running
Command Line : -Xms2048m -Xmx2048m -Xss=256k -da -Dplatform.home=/opt/bea922/aldsp301/weblogic92 -Dwls.home=/opt/bea922/aldsp301/weblogic92/server -Dwli.home=/opt/bea922/aldsp301/weblogic92/integration -Denvironment=prod -Dlog4j.configuration=file:/wls_domains/prod_slimseu_wli_domain/apacheLog4jCfg.xml -Dweblogic.management.discover=false -Dweblogic.management.server=http://meermin.european.idexxi.com:30050 -Dwlw.iterativeDev=false -Dwlw.testConsole=false -Dwlw.logErrorsToConsole=true -Dweblogic.ext.dirs=/opt/bea922/aldsp301/patch_weblogic922/profiles/default/sysext_manifest_classpath -Xverbose:gcreport,memdbg,gcpause -Xverboselog:/wls_logs/prod_slimseu_wli_domain/prod_slimseu_wli.gc -Duser.timezone=GMT -Dweblogic.Domain=prod_slimseu_wli_domain -Dweblogic.Name=prod_slimseu_wli_02 -Djava.security.policy=/opt/bea922/aldsp301/weblogic92/server/lib/weblogic.policy -Dsun.java.launcher=SUN_STANDARD weblogic.Server
java.home : /opt/bea922/aldsp301/jrockit-R27.5.0/jre
j.class.path : /opt/bea922/aldsp301/aldsp_3.0/lib/binxml.jar:/opt/bea922/aldsp301/patch_weblogic922/profiles/default/sys_manifest_classpath/weblogic_patch.jar:/opt/bea922/aldsp301/jrockit-R27.5.0/lib/tools.jar:/opt/bea922/aldsp301/weblogic92/server/lib/weblogic_sp.jar:/opt/bea922/aldsp301/weblogic92/server/lib/weblogic.jar:/opt/bea922/aldsp301/weblogic92/server/lib/webservices.jar::/opt/bea922/aldsp301/weblogic92/integration/lib/worklist-system-required.jar:/opt/bea922/aldsp301/weblogic92/server/lib/jpd.jar:/opt/bea922/aldsp301/weblogic92/integration/L10N:/opt/bea922/aldsp301/weblogic92/common/eval/pointbase/lib/pbclient51.jar:/opt/bea922/aldsp301/weblogic92/server/lib/xqrl.jar::
j.lib.path : /opt/bea922/aldsp301/jrockit-R27.5.0/jre/lib/i386/jrockit:/opt/bea922/aldsp301/jrockit-R27.5.0/jre/lib/i386:/opt/bea922/aldsp301/jrockit-R27.5.0/jre/../lib/i386:/opt/bea922/aldsp301/patch_weblogic922/profiles/default/native:/opt/bea922/aldsp301/weblogic92/server/native/linux/i686:/opt/bea922/aldsp301/weblogic92/server/native/linux/i686/oci920_8
JAVA_HOME : /opt/bea922/aldsp301/jrockit-R27.5.0
_JAVA_OPTIONS: <not set>
LD_LIBRARY_PATH: /opt/bea922/aldsp301/jrockit-R27.5.0/jre/lib/i386/jrockit:/opt/bea922/aldsp301/jrockit-R27.5.0/jre/lib/i386:/opt/bea922/aldsp301/jrockit-R27.5.0/jre/../lib/i386:/opt/bea922/aldsp301/patch_weblogic922/profiles/default/native:/opt/bea922/aldsp301/weblogic92/server/native/linux/i686:/opt/bea922/aldsp301/weblogic92/server/native/linux/i686/oci920_8
LD_ASSUME_KERNEL: <not set>
C Heap : 1 memory allocations have failed
: First failure was a mmMalloc of 20 bytes
: Last failure was a mmMalloc of 20 bytes
StackOverFlow: 0 StackOverFlowErrors have occured
OutOfMemory : 3 OutOfMemoryErrors have occured
Registers (from ThreadContext: 0xa96d4e94 / OS context: 0xa96d4f90):
eax = 00001267 ecx = a96d520c edx = 00000043 ebx = b7f5d74c
esp = a96d5280 ebp = a96d5298 esi = 00000044 edi = a96d52e8
es = 0000007b cs = 00000073 ss = 0000007b ds = 0000007b
fs = 00000000 gs = 00000033
eip = b7dabd55 eflags = 00000282
Stack:
(* marks the word pointed to by the stack pointer)
a96d5280: b7f72020* 00000200 b7f5d74c a96d52e8 a96d52e8 a96d52a4
a96d5298: a96d52b8 b7f258c4 00000044 b7f5d74c a96d52e8 a96d5310
a96d52b0: a96d5358 a96d5360 a96d52d8 b7f258ef 00000044 b7f5d74c
a96d52c8: a96d52e8 a96d5360 a96d5310 a96d5358 a96d52f8 b7ecee0a
Code:
(* marks the word pointed to by the instruction pointer)
b7dabd24: 4c892074 458b0824 2404c710 b7f72020 0c244489 000200b8
b7dabd3c: 24448900 92fae804 01b80017 a3000000 b7f72004 001267b8
b7dabd54: 0000a300* 04c70000 00003f24 9aeae800 768d0017 27bc8d00
b7dabd6c: 00000000 e589fc55 53c03157 b9e87d8d 00000004 00c0ec81
Loaded modules:
(* denotes the module causing the exception)
08048000-08056ff3 /opt/bea922/aldsp301/jrockit-R27.5.0/bin/java
0093f000-0094c85b /lib/tls/libpthread.so.0
00833000-00853c8f /lib/tls/libm.so.6
00858000-00859967 /lib/libdl.so.2
00705000-0082a578 /lib/tls/libc.so.6
006eb000-0070039b /lib/ld-linux.so.2
b7d16000-b7f6c6a7 */opt/bea922/aldsp301/jrockit-R27.5.0/jre/lib/i386/jrockit/libjvm.so
00db2000-00db9a76 /lib/tls/librt.so.1
b7cdb000-b7cf9103 /lib/libnss_centrifydc.so.2
0099a000-009a8fef /lib/libresolv.so.2
00d76000-00d8866f /lib/libnsl.so.1
b7cd0000-b7cd8a57 /lib/libnss_files.so.2
b7be5000-b7bef81b /opt/bea922/aldsp301/jrockit-R27.5.0/jre/lib/i386/libverify.so
b7bc2000-b7be2317 /opt/bea922/aldsp301/jrockit-R27.5.0/jre/lib/i386/libjava.so
b6b1b000-b6b20f13 /opt/bea922/aldsp301/jrockit-R27.5.0/jre/lib/i386/native_threads/libhpi.so
b65ae000-b65bc4c4 /opt/bea922/aldsp301/jrockit-R27.5.0/jre/lib/i386/libzip.so
b4e71000-b4e82f03 /opt/bea922/aldsp301/jrockit-R27.5.0/jre/lib/i386/libnet.so
b4e69000-b4e6f207 /opt/bea922/aldsp301/jrockit-R27.5.0/jre/lib/i386/libnio.so
b5f03000-b5f06313 /lib/libnss_dns.so.2
b3a6e000-b3a73246 /opt/bea922/aldsp301/jrockit-R27.5.0/jre/lib/i386/libmanagement.so
b39e1000-b39ea357 /opt/bea922/aldsp301/jrockit-R27.5.0/jre/lib/i386/libjmapi.so
b36d8000-b36d9de4 /opt/bea922/aldsp301/weblogic92/server/native/linux/i686/libwlfileio2.so
a59ed000-a5a4c026 /opt/bea922/aldsp301/jrockit-R27.5.0/jre/lib/i386/libawt.so
a5926000-a59eb21f /opt/bea922/aldsp301/jrockit-R27.5.0/jre/lib/i386/libmlib_image.so
a5920000-a5924f1c /opt/bea922/aldsp301/jrockit-R27.5.0/jre/lib/i386/headless/libmawt.so
a53f8000-a54710a7 /opt/bea922/aldsp301/jrockit-R27.5.0/jre/lib/i386/libfontmanager.so
"[ACTIVE] ExecuteThread: '25' fo" id=75 idx=0x128 tid=20894 lastJavaFrame=0xa96d54cc
Stack 0: start=0xa9698000, end=0xa96da000, guards=0xa969d000 (ok), forbidden=0xa969b000
Thread Stack Trace:
at dumpForceDump+117()@0xb7dabd55
at vmFatalErrorMsgV+84()@0xb7f258c4
at vmFatalErrorMsg+31()@0xb7f258ef
at fatalError+42()@0xb7ecee0a
at lockTokenIterInit+151()@0xb7ecf317
at trProcessLocksForThread+102()@0xb7edda26
at get_all_locks+106()@0xb7df61ca
at javalockConvertTokenForObject+74()@0xb7df83aa
-- Java stack --
at jrockit/vm/Locks.convertThinLockedToFatLocked(Ljava/lang/Object;)V(Native Method)
at jrockit/vm/Locks.createMonitorAndConvert(Locks.java:1521)[inlined]
at jrockit/vm/Locks.wait(Locks.java:2184)[inlined]
at weblogic/rjvm/ResponseImpl.waitForData(ResponseImpl.java:81)[optimized]
^-- Holding lock: weblogic/rjvm/ResponseImpl@0x6daebaa8[thin lock]
at weblogic/rjvm/ResponseImpl.getTxContext(ResponseImpl.java:108)[inlined]
at weblogic/rjvm/BasicOutboundRequest.sendReceive(BasicOutboundRequest.java:109)[optimized]
at weblogic/rmi/cluster/ClusterableRemoteRef.invoke(ClusterableRemoteRef.java:335)[inlined]
at weblogic/rmi/cluster/ClusterableRemoteRef.invoke(ClusterableRemoteRef.java:252)[optimized]
at com/bea/ld/Server_ydm4ie_EOImpl_1030_WLStub.executeOperationStreaming(Lcom/bea/dsp/ejb/QualName;[Ljava/lang/Object;Lcom/bea/ld/QueryAttributes;Lcom/bea/ld/filter/FilterXQuery;)Lcom/bea/dsp/ejb/StreamingQueryResult;(Unknown Source)[optimized]
at com/bea/dsp/das/ejb/EJBClient.invokeOperation(EJBClient.java:148)[optimized]
at com/bea/dsp/das/DataAccessServiceImpl.invokeOperation(DataAccessServiceImpl.java:171)[inlined]
at com/bea/dsp/das/DataAccessServiceImpl.invoke(DataAccessServiceImpl.java:122)[inlined]
at com/bea/dsp/control/core/delegates/impl/DSPControlInvocationDelegate.executeFunction(DSPControlInvocationDelegate.java:219)[inlined]
at com/bea/dsp/control/core/delegates/impl/DSPControlInvocationDelegate.invokeOnDataService(DSPControlInvocationDelegate.java:64)[optimized]
at com/bea/dsp/control/core/DSPControlImpl.invoke(DSPControlImpl.java:81)[optimized]
at com/idexx/lims/controls/dsp/ResultsReportDSPControlFileBean.getPathEventTimeStampForDepartment(ResultsReportDSPControlFileBean.java:267)[optimized]
at com/idexx/lims/orderresults/dao/OrderResultsDAOImpl.getPathEventTimeStampForDepartment(OrderResultsDAOImpl.java:109)[optimized]
at com/idexx/lims/controls/OrderResultsCustomControlImpl.releaseResultsAndSaveReport(OrderResultsCustomControlImpl.java:360)[optimized]
at com/idexx/lims/controls/OrderResultsCustomControlBean.releaseResultsAndSaveReport(OrderResultsCustomControlBean.java:440)[optimized]
at sun/reflect/GeneratedMethodAccessor831.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object;(Unknown Source)[optimized]
at sun/reflect/DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)[optimized]
at java/lang/reflect/Method.invoke(Method.java:585)[optimized]
at com/bea/wli/knex/runtime/jcs/container/JcsProxy.invokeBeehiveControl(JcsProxy.java:666)[optimized]
at com/bea/wli/knex/runtime/jcs/container/JcsProxy.invoke(JcsProxy.java:433)[optimized]
at $Proxy163.releaseResultsAndSaveReport(LorderResults/logical/orderResultsWs/ReleaseResultsAndSaveReportDocumentXmlBean;)LorderResults/logical/orderResultsWs/ReleaseResultsAndSaveReportResponseDocumentXmlBean;(Unknown Source)
at com/idexx/lims/controls/process/OrderResultsProcess.orderResultsCustomControlReleaseResultsAndSaveReport(OrderResultsProcess.java:649)
at sun/reflect/GeneratedMethodAccessor830.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object;(Unknown Source)
at sun/reflect/DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)[optimized]
at java/lang/reflect/Method.invoke(Method.java:585)[optimized]
at com/bea/wli/bpm/runtime/Perform.invoke(Perform.java:39)[inlined]
at com/bea/wli/bpm/runtime/Perform.execute(Perform.java:50)[optimized]
at com/bea/wli/bpm/runtime/SyncReceive.messageDelivery(SyncReceive.java:63)[optimized]
at com/bea/wli/bpm/runtime/ProcessState.processMessage(ProcessState.java:217)[optimized]
at com/bea/wli/bpm/runtime/ProcessState.dispatchRequest(ProcessState.java:241)[inlined]
at com/bea/wli/bpm/runtime/JpdContainer.dispatchProcessRequest(JpdContainer.java:1077)[optimized]
at com/bea/wli/bpm/runtime/JpdContainer.preInvoke(JpdContainer.java:1041)[optimized]
at com/bea/wli/knex/runtime/core/container/Invocable.invoke(Invocable.java:248)[optimized]
at com/bea/wli/bpm/runtime/JpdContainer.invoke(JpdContainer.java:814)[optimized]
at com/bea/wli/knex/runtime/core/bean/BaseContainerBean.invokeBase(BaseContainerBean.java:224)[optimized]
at com/bea/wli/knex/runtime/core/bean/SLSBContainerBean.invoke(SLSBContainerBean.java:136)[optimized]
at com/bea/wlwgen/StatelessContainer_2hozgx_ELOImpl.invoke(StatelessContainer_2hozgx_ELOImpl.java:137)[optimized]
at com/bea/wlwgen/SLSBContAdpt.invokeOnBean(SLSBContAdpt.java:29)[optimized]
at com/bea/wli/knex/runtime/core/bean/BaseDispatcherBean.runAsInvoke(BaseDispatcherBean.java:185)[optimized]
at com/bea/wli/knex/runtime/core/bean/BaseDispatcherBean.invoke(BaseDispatcherBean.java:54)[optimized]
at com/bea/wli/knex/runtime/core/bean/SyncDispatcherBean.invoke(SyncDispatcherBean.java:168)[optimized]
at com/bea/wli/knex/runtime/core/bean/SyncDispatcher_k1mrl8_EOImpl.invoke(SyncDispatcher_k1mrl8_EOImpl.java:133)[optimized]
at com/bea/wli/knex/runtime/core/dispatcher/Dispatcher.remoteDispatch(Dispatcher.java:165)[inlined]
at com/bea/wli/knex/runtime/core/dispatcher/ServiceHandleImpl.invoke(ServiceHandleImpl.java:460)[optimized]
at com/bea/wli/knex/runtime/core/call/JavaCall.invoke(JavaCall.java:56)[optimized]
at com/bea/wli/bpm/runtime/SubFlowCall.invoke(SubFlowCall.java:112)[optimized]
at com/bea/wli/knex/runtime/core/control/ServiceControlImpl.invoke(ServiceControlImpl.java:1307)[inlined]
at com/bea/control/ProcessControlImpl.invoke(ProcessControlImpl.java:825)[optimized]
at com/bea/wli/knex/runtime/core/control/ServiceControlImpl.invoke(ServiceControlImpl.java:1174)[optimized]
at com/bea/control/ProcessControlImpl.invoke(ProcessControlImpl.java:760)[optimized]
at com/idexx/lims/controls/process/OrderResultsProcessControlBean.releaseResultsClientRequestwithReturn(OrderResultsProcessControlBean.java:394)
at sun/reflect/GeneratedMethodAccessor828.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object;(Unknown Source)
at sun/reflect/DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)[optimized]
at java/lang/reflect/Method.invoke(Method.java:585)[optimized]
at com/bea/wli/knex/runtime/jcs/container/JcsProxy.invokeBeehiveControl(JcsProxy.java:666)[optimized]
at com/bea/wli/knex/runtime/jcs/container/JcsProxy.invoke(JcsProxy.java:446)[optimized]
at $Proxy171.releaseResultsClientRequestwithReturn(LorderResults/logical/orderResultsWs/ReleaseResultsAndSaveReportDocumentXmlBean;)LorderResults/logical/orderResultsWs/ReleaseResultsAndSaveReportResponseDocumentXmlBean;(Unknown Source)
at com/idexx/lims/controls/process/OrderResultsProcess.orderResultsProcessControlReleaseResultsClientRequestwithReturn111(OrderResultsProcess.java:1686)[optimized]
at sun/reflect/GeneratedMethodAccessor827.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object;(Unknown Source)
at sun/reflect/DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)[optimized]
at java/lang/reflect/Method.invoke(Method.java:585)[optimized]
at com/bea/wli/bpm/runtime/Perform.invoke(Perform.java:39)[inlined]
at com/bea/wli/bpm/runtime/Perform.execute(Perform.java:50)[optimized]
at com/bea/wli/bpm/runtime/SyncReceive.messageDelivery(SyncReceive.java:63)[optimized]
at com/bea/wli/bpm/runtime/ProcessState.processMessage(ProcessState.java:217)[optimized]
at com/bea/wli/bpm/runtime/ProcessState.dispatchRequest(ProcessState.java:241)[inlined]
at com/bea/wli/bpm/runtime/JpdContainer.dispatchProcessRequest(JpdContainer.java:1077)[optimized]
at com/bea/wli/bpm/runtime/JpdContainer.preInvoke(JpdContainer.java:1041)[optimized]
at com/bea/wli/knex/runtime/core/container/Invocable.invoke(Invocable.java:248)[optimized]
at com/bea/wli/bpm/runtime/JpdContainer.invoke(JpdContainer.java:814)[optimized]
at com/bea/wli/knex/runtime/core/bean/BaseContainerBean.invokeBase(BaseContainerBean.java:224)[optimized]
at com/bea/wli/knex/runtime/core/bean/SLSBContainerBean.invoke(SLSBContainerBean.java:136)[optimized]
at com/bea/wlwgen/StatelessContainer_2hozgx_ELOImpl.invoke(StatelessContainer_2hozgx_ELOImpl.java:137)[optimized]
at com/bea/wlwgen/SLSBContAdpt.invokeOnBean(SLSBContAdpt.java:29)[optimized]
at com/bea/wli/knex/runtime/core/bean/BaseDispatcherBean.runAsInvoke(BaseDispatcherBean.java:185)[optimized]
at com/bea/wli/knex/runtime/core/bean/BaseDispatcherBean.invoke(BaseDispatcherBean.java:54)[optimized]
at com/bea/wli/knex/runtime/core/bean/SyncDispatcherBean.invoke(SyncDispatcherBean.java:168)[optimized]
at com/bea/wli/knex/runtime/core/bean/SyncDispatcher_k1mrl8_EOImpl.invoke(SyncDispatcher_k1mrl8_EOImpl.java:133)[optimized]
at com/bea/wli/knex/runtime/core/dispatcher/Dispatcher.remoteDispatch(Dispatcher.java:165)[inlined]
at com/bea/wli/knex/runtime/core/dispatcher/ServiceHandleImpl.invoke(ServiceHandleImpl.java:460)[optimized]
at com/bea/wli/knex/runtime/core/call/JavaCall.invoke(JavaCall.java:56)[optimized]
at com/bea/wli/bpm/runtime/SubFlowCall.invoke(SubFlowCall.java:112)[optimized]
at com/bea/wli/knex/runtime/core/control/ServiceControlImpl.invoke(ServiceControlImpl.java:1307)[inlined]
at com/bea/control/ProcessControlImpl.invoke(ProcessControlImpl.java:825)[optimized]
at com/bea/wli/knex/runtime/core/control/ServiceControlImpl.invoke(ServiceControlImpl.java:1174)[optimized]
at com/bea/control/ProcessControlImpl.invoke(ProcessControlImpl.java:760)[optimized]
at com/idexx/lims/controls/process/OrderResultsProcessControlBean.releaseResultsToCustomer(OrderResultsProcessControlBean.java:677)[optimized]
at sun/reflect/GeneratedMethodAccessor46336.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object;(Unknown Source)
at sun/reflect/DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)[optimized]
at java/lang/reflect/Method.invoke(Method.java:585)[optimized]
at com/bea/wli/knex/runtime/jcs/container/JcsProxy.invokeBeehiveControl(JcsProxy.java:666)[optimized]
at com/bea/wli/knex/runtime/jcs/container/JcsProxy.invoke(JcsProxy.java:446)[optimized]
at $Proxy12611.releaseResultsToCustomer(LorderResults/logical/orderResultsWs/ReleaseResultsToCustomerDocumentXmlBean;)LorderResults/logical/orderResultsWs/ReleaseResultsToCustomerResponseDocumentXmlBean;(Unknown Source)
at com/idexx/lims/processes/ReleaseResultsToCustomerProcess.sendReleaseRequest(ReleaseResultsToCustomerProcess.java:101)
at sun/reflect/GeneratedMethodAccessor46335.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object;(Unknown Source)
at sun/reflect/DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)[optimized]
at java/lang/reflect/Method.invoke(Method.java:585)[optimized]
at com/bea/wli/bpm/runtime/Perform.invoke(Perform.java:39)[inlined]
at com/bea/wli/bpm/runtime/Perform.execute(Perform.java:50)[optimized]
at com/bea/wli/bpm/runtime/Receive.messageDelivery(Receive.java:93)[optimized]
at com/bea/wli/bpm/runtime/ProcessState.processMessage(ProcessState.java:217)[optimized]
at com/bea/wli/bpm/runtime/ProcessState.dispatchRequest(ProcessState.java:241)[inlined]
at com/bea/wli/bpm/runtime/JpdContainer.dispatchProcessRequest(JpdContainer.java:1077)[optimized]
at com/bea/wli/bpm/runtime/JpdContainer.preInvoke(JpdContainer.java:1041)[optimized]
at com/bea/wli/knex/runtime/core/container/Invocable.invoke(Invocable.java:248)[optimized]
at com/bea/wli/bpm/runtime/JpdContainer.invoke(JpdContainer.java:814)[optimized]
at com/bea/wli/knex/runtime/core/bean/BaseContainerBean.invokeBase(BaseContainerBean.java:224)[optimized]
at com/bea/wli/knex/runtime/core/bean/SLSBContainerBean.invoke(SLSBContainerBean.java:136)[optimized]
at com/bea/wlwgen/StatelessContainer_2hozgx_ELOImpl.invoke(StatelessContainer_2hozgx_ELOImpl.java:137)[optimized]
at com/bea/wlwgen/SLSBContAdpt.invokeOnBean(SLSBContAdpt.java:29)[optimized]
at com/bea/wli/knex/runtime/core/bean/BaseDispatcherBean.runAsInvoke(BaseDispatcherBean.java:185)[optimized]
at com/bea/wli/knex/runtime/core/bean/BaseDispatcherBean.invoke(BaseDispatcherBean.java:54)[optimized]
at com/bea/wli/knex/runtime/core/bean/AsyncDispatcherBean.onMessage(AsyncDispatcherBean.java:259)[optimized]
at weblogic/ejb/container/internal/MDListener.execute(MDListener.java:429)[inlined]
at weblogic/ejb/container/internal/MDListener.transactionalOnMessage(MDListener.java:335)[inlined]
at weblogic/ejb/container/internal/MDListener.onMessage(MDListener.java:291)[optimized]
at weblogic/jms/client/JMSSession.onMessage(JMSSession.java:4072)[optimized]
at weblogic/jms/client/JMSSession.execute(JMSSession.java:3962)[optimized]
at weblogic/jms/client/JMSSession$UseForRunnable.run(JMSSession.java:4490)[optimized]
at weblogic/work/ServerWorkManagerImpl$WorkAdapterImpl.run(ServerWorkManagerImpl.java:518)[optimized]
at weblogic/work/ExecuteThread.execute(ExecuteThread.java:209)[optimized]
at weblogic/work/ExecuteThread.run(ExecuteThread.java:181)
at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
-- end of trace
Extended, platform specific info:
libc release: 2.3.4-stable
Elf headers:
libc ehdrs: EI: 7f454c46010101000000000000000000 ET: 3 EM: 3 V: 1 ENTRY: 00719ed0 PHOFF: 00000034 SHOFF: 00174ab8 EF: 0x0 HS: 52 PS: 32 PHN; 10 SS: 40 SHN: 67 STIDX: 66
libpthread ehdrs: EI: 7f454c46010101000000000000000000 ET: 3 EM: 3 V: 1 ENTRY: 00943850 PHOFF: 00000034 SHOFF: 00019f00 EF: 0x0 HS: 52 PS: 32 PHN; 9 SS: 40 SHN: 39 STIDX: 38
libjvm ehdrs: EI: 7f454c46010101000000000000000000 ET: 3 EM: 3 V: 1 ENTRY: 0004bbe0 PHOFF: 00000034 SHOFF: 00297e24 EF: 0x0 HS: 52 PS: 32 PHN; 4 SS: 40 SHN: 19 STIDX: 16
* If you see this dump, please go to *
* http://edocs.bea.com/jrockit/go2troubleshooting.html *
* for troubleshooting information. *
===== END DUMP ===============================================================Vicky77 wrote:
We are seeing memory leaks in our application, this is causing Out of Memory exception after every 5-7 days in prod environment. We tried to analyse our application with JPROBE and these are our findings...
1) Once these session is expiring, all application objects are getting garbage collected.
2) We are seeing huge increase in String and Char[] with each submitted applicationWhere are these coming from?
Are you using any libraries like JAXB? Are you binding XML to objects?
We are using Weblogic 10.0 with jdk version 1.5.0_06. Are there any known String related memory leak issues.
My understanding is that unless we are using any static variable or application scope objects we should have any memory leakage.Apparently not.
Can someone please validate if this understanding is correct? The JVM is telling you that it's incorrect. Believe the evidence you have before you.
Your JVM settings might help. What are the settings for the GC? How big is your perm space, and how is that changing with time?
What could be other possible causes of memory leaks? We are also using private static ThreadLocal variablesSounds like the perm space is exploding in size. It fills up and you're done.
I'd recommend that you download JVMStat from Sun and watch what memory is doing as you run your test. It's not a nice graphic view of eden, old & new gen, and perm spaces that will give you an idea of what's going on. It won't tell you why. Keep drilling down with JProbe.
% -
Oracle BPM installed Java out-of-memory exception
Hi,
I have installed the oracle BPM software in the server have the Ram capacity 8 GB,Could you please suggest some of the do's and don't's to avoid the Java out-of-memory exception which normally appearing on the console of weblogic server.
My Server details,
Windows 2008 R2 64 bit, RAM 8 GB,
The required BPM s/w I installed from the link below,
http://www.oracle.com/technetwork/middleware/bpm/downloads/index.html?ssSourceSiteId=ocomen
Thanks in advance for your help.
Regards,
Shyam
Edited by: user13821489 on 06-Feb-2011 22:45I had to increase JVM xmx/maxperm in the "set*Env.sh" scripts in the directory where you start WLS. In my dev environment, deploying BAM & BPM together with the Admin server, I finally allocated 4GB max. Jdev.conf is also better if you allocate > 1GB - mine is 1.4GB. I also watch process memory. Multiple re-deployments in my development environment seem to increase the memory, even if I remove process instances and undeploy first. I don't understand the internals very well so perhaps it is behaving correctly, but restarting WLS frees the unused memory that I expected gc to reclaim.
-
We are getting out of memory error with jrockit1.4.2_08. We used memory debugger tools to find the leak. Apparently there do not seem to be a leak from application perspective. Here is the core dump. Please help me resolve this.
===== BEGIN DUMP =============================================================
JRockit dump produced after 0 days, 05:37:55 on Thu Jun 5 20:39:11 2008
Additional information is available in:
/opt/obs/obs_app/obs31.0/user_projects/admin/jrockit.17430.dump
No core file will be created because core dumps have been
disabled. To enable core dumping, try "ulimit -c unlimited"
before starting JRockit again.
If you see this dump, please open a support case with BEA and
supply as much information as you can on your system setup and
the program you were running. You can also search for solutions
to your problem at http://forums.bea.com in
the forum jrockit.developer.interest.general.
Error code: 52
Error Message: Null pointer exception in native code
Signal info : si_signo=11, si_code=2
Version : BEA WebLogic JRockit(TM) 1.4.2_08 JVM R24.5.0-61 ari-49095-20050826-1856-linux-ia32
Threads / GC : Native Threads, GC strategy: parallel
: mmHeap->data = 0x40416000, mmHeap->top = 0x7ec16000
: mmStartCompaction = 0x45236000, mmEndCompaction = 0x4a056000
CPU : Intel Pentium 4 (HT)
Number CPUs : 4
Tot Phys Mem : 4021997568
OS version : Red Hat Enterprise Linux ES release 2.1 (Panama)
Linux version 2.4.9-e.72smp ([email protected]) (gcc version 2.96 20000731 (Red Hat Linux 7.2 2.96-129.7.2)) #1 SMP Tue Jul 3 22:04:51 EDT 2007
State : JVM is running
Command Line : -Djava.class.path=<my own jars list>
java.library.path=/opt/obs/wls/jrockit-j2sdk1.4.2_08/jre/lib/i386/jrockit:/opt/obs/wls/jrockit-j2sdk1.4.2_08/jre/lib/i386:/opt/obs/wls/jrockit-j2sdk1.4.2_08/jre/../lib/i386:/opt/mqm/java/lib:/opt/tib/rv7.1.2/lib:/opt/obs/obs_app/obs31.0/thirdparty:/opt/obs/wls/wls8.1sp3/weblogic81/server/lib/linux/i686:/opt/obs/wls/wls8.1sp3/weblogic81/server/lib/linux/i686/oci920_8
C Heap : Good; no memory allocations have failed
Registers (from context struct at 0x8d5b724/0x8d5b80c):
EAX = 000005f4 EBX = 0000007f
ECX = 7ff8bd90 EDX = 24a18f94
ESI = 7ff46af0 EDI = 00000100
ESP = 7f262bc0 EIP = 4026d58f
EBP = 7f262bd8 EFL = 00010203
CS = 0023 DS = 002b ES = 002b
SS = 002b FS = 00df GS = 00df
Stack:
7f262bc0 :00000000 7ff46af0 00000f94 00000000 00028000 00000000
7f262bd8 :7f262c08 401ffcae 7ff8bd90 24a18f94 00028000 a0000000
7f262bf0 :0a5f1800 7f262c74 08d137a0 00000050 00000000 7f262c34
7f262c08 :7f262c48 4028a7fb 24a18f94 00000000 08d137a0 08d139e4
7f262c20 :581a664c 00000003 08d137a0 00000001 7f262c74 00000000
7f262c38 :7f262ca8 402c28bd 7f262c74 20040144 7f262ca8 402c292f
7f262c50 :7f262c74 00000000 00000000 7f262c84 581a6640 08d137a0
7f262c68 :08d1382c 414d2db8 210d41e8 0a5f1800 08d137a0 7f262d9c
7f262c80 :24a18f95 00000002 00000000 7f262cf8 08d1382c 581a6640
7f262c98 :00000000 00000000 00000000 00000000 7f262ce8 402e0861
7f262cb0 :08d137a0 581a6640 00000020 7f262cf8 210cc499 00000001
7f262cc8 :00000000 7f262ce4 08d139e8 7f262cf0 581a66c0 53b8d780
7f262ce0 :00000000 00000020 08d137a0 210cc43e 08d1382c 08d139e4
7f262cf8 :08116840 08d139e4 00000000 581a6400 210b41ff 00000001
7f262d10 :581a6618 581a6560 210cc499 581a6565 581a6618 581a6560
7f262d28 :210d2744 581a62f0 581a6560 210d2725 2112a995 2112a985
7f262d40 :2112a975 23a67584 581a6578 00000001 00000001 220e35a9
7f262d58 :23a53381 581a62f0 00000000 00000000 581a6548 00000046
7f262d70 :00000000 00000002 00000000 581a62f0 00000001 00000000
7f262d88 :00000000 00000000 581a6530 581a62f0 23a56e15 24a18f95
7f262da0 :24a18ba3 00000000 581a6130 210bead9 581a6038 581a6038
7f262db8 :581a6038 53b88f50 40fa9ca8 581a6308 55b54b00 40fa9c38
7f262dd0 :80000000 55a06030 581a6020 581a6038 581a6020 24a189bb
7f262de8 :40fa9c38 581a6020 40fa9ca8 581a60c0 55a06030 53942ff8
7f262e00 :55b54b00 40fa9ca8 40fa9c38 08d137a0 7f262e24 210b34d8
7f262e18 :40fa9c38 40fa9c68 40fa9ca8 24a18800 402e3a4d 7f262fb3
7f262e30 :0000002c 00000003 7f262ec4 08d1382c 24b196e0 249c4f84
7f262e48 :581a5390 00000000 00000058 7f262e84 402b861a 249c4f84
7f262e60 :7f262e80 08d1382c 00000001 00000001 00000008 00000013
7f262e78 :00000005 581a5fd8 08d139dc 7f262e18 00000003 00000005
7f262e90 :00000005 00000000 00000000 55a06030 55b54b00 7f262e24
7f262ea8 :7f262e24 08d137a0 7f262ee0 24a18800 210b34a0 0000002c
7f262ec0 :7f262fb3 7f262f04 402c8931 08d1382c 1aea85c0 24b196e0
7f262ed8 :00000000 7f262f5c 402c8b74 7f262f54 401aaab0 00000000
7f262ef0 :7f262f5c 7f262f54 401aaab0 401aa6a0 1aea85c0 7f262f64
7f262f08 :402c8c41 08d1382c 249c4f84 00000000 7f262f5c 402c8b74
7f262f20 :00000000 7f262f54 08d13dd0 08d1382c 7f262f9c 00000000
7f262f38 :00000000 7f262f7c 402f2b2d 08d137a0 00000001 7f262f7c
7f262f50 :402f2b24 08d13dd0 402f2b24 7f262fd4 08d1382c 7f262f94
7f262f68 :402dbd4c 08d1382c 08d139dc 249c4f84 7f262fd4 08d139e4
7f262f80 :581a5ee0 00000000 476bd4c0 210b67f6 00000000 08d137a0
7f262f98 :210e7b1b 08d1382c 08d139e0 249c4f84 08d139dc 7f262fd4
7f262fb0 :083e6898 08d139dc 00000000 210eb30d 40fa9ca8 210eb184
7f262fc8 :405bff10 00000005 210e7c48 581a5fb8 581a5fb8 581a5fe0
7f262fe0 :00000000 00000000 581a5690 210e7253 581a5fb8 00000009
7f262ff8 :73a44aa8 73a44aa8 00000005 581a5fb8 00000009 581a5690
7f263010 :246e7cd0 581a5fb8 581a53e8 581a5f48 40fa9b90 40fa9be8
7f263028 :55a06030 55b54b00 40fa9be8 40fa9bd0 246e7b13 55b54b00
7f263040 :55a06030 40fa9b60 55b54b00 55a06030 581a5378 246e79b6
7f263058 :55a06030 55b54b00 57712968 405c6858 40fa9b40 00000001
7f263070 :40fa9b40 2426bab4 55a06030 57712968 57712928 57712968
7f263088 :55b54b00 55b54b00 4b345448 2426ba1b 2426b717 53b88f50
7f2630a0 :405c6858 5393c038 53b88f50 00000002 53b88f40 24266f1f
7f2630b8 :53b88f50 53b88f50 53b88f40 5393c038 53b849c0 00000000
7f2630d0 :53912b90 5393acd0 5393c038 53912b90 242613c8 5393c038
7f2630e8 :00000000 5393acd0 00000000 5393aa60 5393c038 24658a31
7f263100 :5393acd0 4a0dbf40 00000000 4a0dbf40 539125c0 23a74a88
7f263118 :539126e8 00000001 6553be00 539125c0 246587dc 53ebb610
7f263130 :4a0dbf40 00000000 4a124210 00000000 00000000 5393aa60
7f263148 :218aff64 09ccec00 221aa582 5393a440 4a1507e8 4a1507e8
7f263160 :536f07d8 533cdee0 5393a440 533cdee0 000378b6 536f07d8
7f263178 :5393a430 00000004 210b48f0 00000050 0000000f 211080dd
7f263190 :40576ec8 21108145 5393a480 5393aa60 5393aa60 21108074
7f2631a8 :00000003 536f07d8 4a124210 7d1c9048 5393a430 654d5040
7f2631c0 :4a090428 232a5328 7d1c9048 4a124210 229f2aeb 7d1c9048
7f2631d8 :4a124210 4a127090 538f80a8 654d5040 536f07d8 4a127090
7f2631f0 :536f07d8 242438f0 536f07d8 7d1c9048 7d1c9048 538f80a8
7f263208 :5393a418 7d1c9048 229f2a12 536f07d8 7d1c9048 22147208
7f263220 :40528fa8 4056b138 00000000 00000000 405d4858 7d1c9048
7f263238 :2214718a 5393a418 7d1c9048 229f239d 5393a418 536f07d8
7f263250 :7d1c9048 4056b138 41356f20 00000000 536f07d8 40528fa8
7f263268 :4a127090 7d1c9048 5393a418 538f80a8 42562b68 538f81e0
7f263280 :229f2230 219161d1 41356f20 218e2089 42562b6d 41356f20
7f263298 :7f2632b4 210b8c73 402e37ef 7f2633e8 08d137a0 7f2632b4
7f2632b0 :210b3478 210b8c60 402e3a4d ffffffff 00000010 00000000
7f2632c8 :7f263354 08d1382c 200920e0 200282cc 00000000 00000000
7f2632e0 :00000000 7f263314 402b861a 200282cc 7f263310 08d1382c
7f2632f8 :00000001 00000001 00000006 00000006 00000001 08d137a0
7f263310 :00000000 7f2632b4 00000000 00000001 00000001 00000000
7f263328 :00000000 00000000 42562b68 7f2632b4 7f2632b4 08d137a0
7f263340 :00000000 210b8c60 210b3440 00000010 ffffffff 7f263394
7f263358 :402c8931 08d1382c 080d97e0 200920e0 00000000 7f2633e8
7f263370 :402e3550 7f2633cc 00000000 08d1382c 08d1382c 7f263aac
7f263388 :00000000 00000000 080d97e0 7f2633d4 402c9e9f 08d1382c
7f2633a0 :200282cc 00000000 7f2633e8 402e3550 00000001 7f2633cc
7f2633b8 :00000000 00000000 00000000 00000000 00000000 00000000
Code:
4026d48f :83403d9a 008bf4c4 166be850 c4830009 f4c48310 a817e853
4026d4a7 :b70f0009 c0010643 06438966 0ff8c483 048dc0b7 02e0c140
4026d4bf :08438b50 57b7e850 43890003 20c48308 53f4c483 09a87ce8
4026d4d7 :9a38a100 8b64403d 30050300 83403d9a c48310c4 50008bf4
4026d4ef :0915f4e8 084b8b00 4574c985 0f0c558b 8d0443b7 14894004
4026d507 :10558b81 0443b70f c140048d 430302e0 04508908 0f14558b
4026d51f :8d0443b7 e0c14004 08430302 66085089 b80443ff 00000001
4026d537 :b48d09eb 00000026 8bc03100 ec89e85d 768dc35d e5895500
4026d54f :570cec83 458b5356 08558b08 00fc45c7 83000000 b70ff4c4
4026d567 :478d0478 8df8d101 e852ff58 0009a7ed 8310c483 4974fffb
4026d57f :084d8b90 8b5b148d 048d0841 0c558b90 77045039 0850391c
4026d597 :188b0f72 51f4c483 09a7fce8 ebd88900 fc5d892e 768d05eb
4026d5af :8bdf8900 de89fc4d 8939148d 1fe8c1d0 d1101c8d 75f339fb
4026d5c7 :08458bb8 50f4c483 09a7cce8 8dc03100 5e5be865 5dec895f
4026d5df :e58955c3 571cec83 458b5356 fc45c70c 00000000 04788366
4026d5f7 :c7627400 0000f845 558b0000 f8458b0c 8b084203 8b118b08
4026d60f :388b0442 8b08428b 8b028b30 f8c48300 8b04598b 08418b10
4026d627 :5040d829 145d2b52 52575653 10458b53 2c406850 558b4036
4026d63f :bae85208 8bc7ddb3 c4830c55 f8458330 fc45ff0c 0442b70f
4026d657 :7cfc4539 d8658da5 895f5e5b 89c35dec e58955f6 572cec83
4026d66f :38a15356 64403d9a 0503008b 403d9a30 8bf4c483 5ee85000
4026d687 :83000914 c48310c4 e45d8df4 19bbe853 b2e80009 e8000918
Loaded modules:
(* denotes the module causing the exception)
0x08048000-0x0804ce46 /opt/obs/wls/jrockit-j2sdk1.4.2_08/bin/java
0x4001d000-0x40029fab /lib/i686/libpthread.so.0
0x4004e000-0x4006fb42 /lib/i686/libm.so.6
0x40071000-0x4007300c /lib/libdl.so.2
0x40075000-0x401a76e5 /lib/i686/libc.so.6
0x401b2000-0x40388eef* /opt/obs/wls/jrockit-j2sdk1.4.2_08/jre/lib/i386/jrockit/libjvm.so
0x7ec57000-0x7ec66fa5 /opt/obs/wls/jrockit-j2sdk1.4.2_08/jre/lib/i386/libverify.so
0x7ec75000-0x7ec94a0f /opt/obs/wls/jrockit-j2sdk1.4.2_08/jre/lib/i386/libjava.so
0x7ec9c000-0x7ecae3ba /lib/libnsl.so.1
0x7ec97000-0x7ec98705 /opt/obs/wls/wls8.1sp3/weblogic81/server/lib/linux/i686/libweblogicunix1.so
0x7f5f2000-0x7f5f3eff /opt/obs/wls/wls8.1sp3/weblogic81/server/lib/linux/i686/libmuxer.so
0x7f5f6000-0x7f5f95c1 /opt/obs/wls/jrockit-j2sdk1.4.2_08/jre/lib/i386/libioser12.so
0x80318000-0x8031f8bf /opt/obs/obs_app/obs31.0/thirdparty/libXMLC.so
0x80344000-0x8034c66f /opt/tib/rv7.1.2/lib/libtibrvj.so
0x8034e000-0x803563e0 /opt/tib/rv7.1.2/lib/libtibrvcmq.so
0x803d2000-0x803e66bb /opt/tib/rv7.1.2/lib/libtibrvcm.so
0x7fef9000-0x7fefdcaf /opt/tib/rv7.1.2/lib/libtibrvft.so
0x803e8000-0x804341a4 /opt/tib/rv7.1.2/lib/libtibrv.so
Thread Stack Trace:
WARNING: Memory exhausted
at java/lang/Throwable.fillInStackTrace0(Native Method)@0x210cc3f0
at java/lang/Throwable.fillInStackTrace(Unknown Source)@0x210cc499
at java/lang/Throwable.<init>(Unknown Source)@0x210d2744
at java/lang/Exception.<init>(Exception.java:41)@0x210d2725
at java/lang/RuntimeException.<init>(RuntimeException.java:43)@0x2112a995
at java/lang/IllegalArgumentException.<init>(IllegalArgumentException.java:36)@0x2112a985
at java/lang/NumberFormatException.<init>(NumberFormatException.java:38)@0x2112a975
at java/lang/NumberFormatException.forInputString(NumberFormatException.java:48)@0x23a67584
at java/lang/FloatingDecimal.readJavaFormatString(FloatingDecimal.java:1207)@0x220e35a9
at java/lang/Double.parseDouble(Double.java:220)@0x23a56e15I can't tell from the crash dump what the exact problem is - maybe you are running out of native memory?
You are running a very old JRockit version. Since you are a Oracle/BEA customer, you can get a newer one from BEA Support, or through edelivery.oracle.com.
-- Henrik -
System hanging when it runs out of memory
Hello,
my system has a finite amount of RAM and swap (it doesn't matter, to my purpose, if it's 16GB or 128MB, I'm not interested in increasing it anyway).
Sometimes my apps completely use all the available memory. I would expect that in these cases the kernel kills some apps to keep working correctly. The OOM Killer exists just for this, doesn't it?
What happens, instead, is that the system hangs. Even the mouse stops working. Sometimes it manages to get back to life in a few seconds/minutes, other times hours pass by and nothing changes.
I don't want to add more memory, I just want that the kernel kills some application when it's running out of memory.
Why isn't this happening?
Right now I'm writing a bash script that will kill the most memory-hungry process when available memory gets below 10%, because I'm sick of freezing my machine.
But why do I have to do this? Why do I need an user space tool polling memory usage and sentencing applications according to a cheap policy? What the hell is wrong with my OOM Killer, why isn't it doing is job?!Alright, you won, now quit pointing out my ignorance
Your awkish oom killer is a lot cooler than mine, switching to it, thanks!
I did some testing (initially just to test the oom killing script) and found out that if a program tries to allocate all the memory it can, it gets eventually killed by linux's oom killer. If instead it stops allocating new memory when there are less than 4MB of free memory (or similar values) the OOM won't do anything, and the system will get stuck, like if a forkbomb was running.
That's it, this program with MINSIZE=1 will be killed, while with MINSIZE=4MB will force me to hard reboot:
#include <string.h>
#include <stdlib.h>
#define MINSIZE (1024*1024*4) // 4MB
int main( )
int block = 1024*1024*1024; // 1GB
void *p;
while( 1 ) {
p = malloc( block );
if( p ) {
memset( p, 85, block );
else if( block > MINSIZE ) {
block /= 2;
return 0;
Guess I'd need to go deeper to understand why Linux' oom killer works like that, but I won't (assuming the oom killing script will behave). -
Out of memory error. Throughput zero. Server crashes
Hi,
We have this issue about 'java.lang.outofmemory' error, which brings down the
nodes one at a time. The logs don't help at all.
At the time of crash follows:
- Memory shows fine on weblogic console. It cruises flat at approx 160 megs. We
have ms
and mx heap of 512megs.
- jdbc connections on the console show 15 and we have set initial of 15 and max
of 95.
- throughput shows zero on the console
- queue length just spikes and goes like 50+ till the server finally crashes.
- weglogic.log has an entry of jdbc connection leak, which doesn't make sense.
- then a large number of out of memory statements are printed out.
It is our existing application, which worked fine. We put in small additional
features which don't really do any big thing. We have already checked our connection.close()
statements. We are closing all the connections to return them back to the pool.
Our environment is as follows:
OS: Win2k
Weblogic: 6.1 sp4
Clustered with 2 nodes
Any help would be highly appreciated.
Abbas"siauwea" <[email protected]> wrote in message news:[email protected]...
>
HI,
here i only want to inform that i'm using JRockit java version "1.4.1_03" rather
than jdk.
I wonder is it "-XX:MaxPermSize" cannot be set in JRockit??This option is not available for JRockit - may be it uses different heap structure.
When your server runs out of memory - how much heap is available.
Regards,
Slava Imeshev
"siauwea" <[email protected]> wrote:
Hi,
When i added "-XX:MaxPermSize=128m" in startWeblogic.cmd, why ther server
will
come out "Unknown option: -XX:MaxPermSize=128m"??
Here is what i put in startWeblogic.cmd:
%JAVA_HOME%\bin\java -XX:MaxPermSize=128m %JAVA_VM% %MEM_ARGS% %JAVA_OPTIONS%
-Dweblogic.Name=%SERVER_NAME% -Dweblogic.ProductionModeEnabled=%PRODUCTION_MODE%
-Djava.security.policy="%WL_HOME%\server\lib\weblogic.policy" -DEA_HOME=C:\bea\user_projects\domains\TrekEasy1.0
-DCONF_MODE=SERVER weblogic.Server
Cheers,
Siau Wea
"siauwea" <[email protected]> wrote:
Hi Abbas,
I meet the same problem as you, get the java.lang.OutOfMemory,
you said that u added -XX:MaxPermSize=128m then problem resolved,
but i dont know where should i put this info.
Some say put it in startup parameter, but i dont know where is it.
Can you please tell me clearly which file i should add this line??
Is it in startWeblogic.cmd or other file??
Can you show me all of the setting you did in this jvm option??
Urgent!!
Thanks!!
Siau Wea
"Abbas" <[email protected]> wrote:
Slava,
Thanks for the info. We added -XX:MaxPermSize=128m and it worked. Bingo.
As you said, the parameter MaxPermSize specifies memory for long lived
objects.
Also, what I heard from someone in BEA, that this memory allocationis
something
aside from the heap assigned with -ms and -mx parameters. What I would
really
appreciate is if anyone can talk about any benchmark for this parameter.
The jvm
tuning recommendations for wls 6.1 on BEA's website (http://e-docs.bea.com/wls/docs61/perform/JVMTuning.html#1104200)
doesn't mention this.
Thanks in advance.
Abbas
"Slava Imeshev" <[email protected]> wrote:
Abbas,
Try to add -XMaxPermSize=64M to JVM startup parameters. That will
increase heap size allocaed for long-living objects. If you don't
have
an
application-caused memory leak, this should fix the problem.
Regards,
Slava Imeshev
"Abbas" <[email protected]> wrote in message news:[email protected]...
Hi,
We have this issue about 'java.lang.outofmemory' error, which bringsdown the
nodes one at a time. The logs don't help at all.
At the time of crash follows:
- Memory shows fine on weblogic console. It cruises flat at approx160 megs. We
have ms
and mx heap of 512megs.
- jdbc connections on the console show 15 and we have set initial
of
15 and max
of 95.
- throughput shows zero on the console
- queue length just spikes and goes like 50+ till the server finallycrashes.
- weglogic.log has an entry of jdbc connection leak, which doesn'tmake sense.
- then a large number of out of memory statements are printed out.
It is our existing application, which worked fine. We put in smalladditional
features which don't really do any big thing. We have already checkedour connection.close()
statements. We are closing all the connections to return them backto the pool.
Our environment is as follows:
OS: Win2k
Weblogic: 6.1 sp4
Clustered with 2 nodes
Any help would be highly appreciated.
Abbas
Maybe you are looking for
-
Another itunes failing to start post - no error message
I've read a fair few posts like this & so far the one started by theazzi seems to be about the closest. A few weeks ago Itunes mysteriously stopped loading. The symptom is that itunes looks like it's about to load but nothing happens. The app comes u
-
Xmlparserv2.jar cause IBM WebSphere v5.1 down.
Anyone knows: I have a j2ee project based on IBM WebSphere v5.1 + Oracle DB 10g(XMLDB included).There's a big trouble bother me. My project used Oracle xml implemention,such as DOM parser included in xmlparserv2.jar.and when i deploy my project to WA
-
OK, this just started yesterday: I can enter ONE (1) addressee for a new email message. If I want to add another name (Boba...), this is what I get: BBoBobBobbBobbaBobbaBobba If I touch the delete key, I just get another "Bobba." If I highlight the w
-
Is there a file size limit or other limit
Hi, I imported a movie into imovie as a new event. It's an MOV file. It's pretty long, 3 hours and 55 minutes. However, the idvd option in the Share menu is grayed out. I even tried doing it without any chapter markers or anything. I then just took a
-
I'm sure this has been discussed here many times, but I haven't yet found the answer. I'm running iPhoto on two iMacs, one at home and one at work, both with Mountain Lion, and one Macbook Air, also with Mountain Lion. I've imported photos from ca