Stuck Thread - JMS Business proxy Service (ALSB 2.5)

Hi All,
We are having a jms proxy service which uses a JMS Business Service to send messages to a jms queue on weblogic 8.1 sp6. The alsb domain runs on weblogic 9.2.The jms proxy service is made single threaded by configuring a suitable work manager.
Now and then, we get a stuck thread issue in production and the proxy service stops consuming the messaging off the jms queue in wls 9.2. These are the errors in the logs:
<[ACTIVE] ExecuteThread: '18' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1235437298516> <BEA-000337> <[STUCK] ExecuteThread: '8' for queue: 'weblogic.kernel.Default (self-tuning)' has been busy for "695" seconds working on the request "weblogic.work.ServerWorkManagerImpl$WorkAdapterImpl@5321caf", which is more than the configured time (StuckThreadMaxTime) of "600" seconds. Stack trace:
Thread-70 "[STUCK] ExecuteThread: '8' for queue: 'weblogic.kernel.Default (self-tuning)'" <alive, in native, waiting, priority=1, DAEMON> {
-- Waiting for notification on: weblogic.rjvm.ResponseImpl@51961a5[fat lock]
jrockit.vm.Threads.waitForSignal(Native Method)
java.lang.Object.wait(Native Method)
java.lang.Object.wait(Native Method)
weblogic.rjvm.ResponseImpl.waitForData(ResponseImpl.java:73)
^-- Lock released while waiting: weblogic.rjvm.ResponseImpl@51961a5[fat lock]
weblogic.rjvm.ResponseImpl.getTxContext(ResponseImpl.java:100)
weblogic.rjvm.BasicOutboundRequest.sendReceive(BasicOutboundRequest.java:109)
weblogic.rmi.internal.BasicRemoteRef.invoke(BasicRemoteRef.java:223)
weblogic.jms.dispatcher.DispatcherImpl_816_WLStub.dispatchSyncTranFuture(Unknown Source)
weblogic.jms.dispatcher.DispatcherInteropAdapter.dispatchSyncTranFuture(DispatcherInteropAdapter.java:80)
weblogic.messaging.dispatcher.DispatcherWrapperState.dispatchSyncTran(DispatcherWrapperState.java:230)
weblogic.jms.dispatcher.DispatcherAdapter.dispatchSyncTran(DispatcherAdapter.java:51)
weblogic.jms.client.JMSProducer.toFEProducer(JMSProducer.java:1047)
weblogic.jms.client.JMSProducer.deliveryInternal(JMSProducer.java:670)
weblogic.jms.client.JMSProducer.sendInternal(JMSProducer.java:455)
weblogic.jms.client.JMSProducer.send(JMSProducer.java:352)
weblogic.jms.client.WLProducerImpl.send(WLProducerImpl.java:1009)
weblogic.deployment.jms.WrappedMessageProducer.send(WrappedMessageProducer.java:157)
com.bea.wli.sb.transports.jms.JmsOutboundMessageContext$SendAction.run(JmsOutboundMessageContext.java:795)
com.bea.wli.sb.transports.jms.JmsOutboundMessageContext.send(JmsOutboundMessageContext.java:500)
com.bea.wli.sb.transports.jms.JmsTransportProvider.sendMessageAsync(JmsTransportProvider.java:612)
jrockit.reflect.CompiledMethodInvoker.invoke0(Unknown Source)
jrockit.reflect.CompiledMethodInvoker.invoke(Unknown Source)
java.lang.reflect.Method.invoke(Unknown Source)
com.bea.wli.sb.transports.Util$1.invoke(Util.java:241)
$Proxy39.sendMessageAsync(Unknown Source)
com.bea.wli.sb.transports.LoadBalanceFailoverListener.sendMessageAsync(LoadBalanceFailoverListener.java:137)
com.bea.wli.sb.transports.TransportManagerImpl.sendMessageToService(TransportManagerImpl.java:437)
com.bea.wli.sb.transports.TransportManagerImpl.sendMessageAsync(TransportManagerImpl.java:328)
com.bea.wli.sb.pipeline.PipelineContextImpl.doDispatch(PipelineContextImpl.java:503)
com.bea.wli.sb.pipeline.PipelineContextImpl.dispatch(PipelineContextImpl.java:415)
stages.publish.runtime.PublishRuntimeStep.processMessage(PublishRuntimeStep.java:106)
com.bea.wli.sb.stages.StageMetadataImpl$WrapperRuntimeStep.processMessage(StageMetadataImpl.java:327)
com.bea.wli.sb.stages.impl.SequenceRuntimeStep.processMessage(SequenceRuntimeStep.java:33)
com.bea.wli.sb.pipeline.PipelineStage.processMessage(PipelineStage.java:100)
com.bea.wli.sb.pipeline.PipelineContextImpl.execute(PipelineContextImpl.java:771)
com.bea.wli.sb.pipeline.Pipeline.processMessage(Pipeline.java:73)
com.bea.wli.sb.pipeline.PipelineContextImpl.execute(PipelineContextImpl.java:771)
com.bea.wli.sb.pipeline.PipelineNode.doRequest(PipelineNode.java:47)
com.bea.wli.sb.pipeline.Node.processMessage(Node.java:57)
com.bea.wli.sb.pipeline.PipelineContextImpl.execute(PipelineContextImpl.java:771)
com.bea.wli.sb.pipeline.Router.processMessage(Router.java:141)
com.bea.wli.sb.pipeline.MessageProcessor.processRequest(MessageProcessor.java:75)
com.bea.wli.sb.pipeline.RouterManager$1.run(RouterManager.java:646)
com.bea.wli.sb.pipeline.RouterManager$1.run(RouterManager.java:645)
weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:363)
weblogic.security.service.SecurityManager.runAs(SecurityManager.java:147)
com.bea.wli.sb.pipeline.RouterManager.processMessage(RouterManager.java:627)
com.bea.wli.sb.transports.TransportManagerImpl.receiveMessage(TransportManagerImpl.java:264)
com.bea.wli.sb.transports.jms.JmsInboundMDB.onMessage(JmsInboundMDB.java:108)
weblogic.ejb.container.internal.MDListener.execute(MDListener.java:429)
weblogic.ejb.container.internal.MDListener.transactionalOnMessage(MDListener.java:335)
weblogic.ejb.container.internal.MDListener.onMessage(MDListener.java:292)
weblogic.jms.client.JMSSession.onMessage(JMSSession.java:4060)
weblogic.jms.client.JMSSession.execute(JMSSession.java:4012)
weblogic.jms.client.JMSSession$UseForRunnable.run(JMSSession.java:4467)
weblogic.work.ServerWorkManagerImpl$WorkAdapterImpl.run(ServerWorkManagerImpl.java:518)
weblogic.work.ExecuteThread.execute(ExecuteThread.java:209)
weblogic.work.ExecuteThread.run(ExecuteThread.java:181)
jrockit.vm.RNI.c2java(Native Method)
Any one know the root cause of this issue or ways to prevent this?
Also Is there a way to make the proxy service to reconsume messages off the queue without a server restart?
Thanks,
Atheek

Hi atheek, did you find the solution??
I already got the same/identic issue, do you mind giving me an aid?
Really thanks
Regards,
Mr.K

Similar Messages

Increases number of JMS consumers in an OSB jms-polling proxy service

Hi All,
I'm looking for an option of increasing the number of JMS consumers in a JMS OSB proxy service. I know that this is possible by modifying the ejb, but I'm looking for any other option by using weblogic/proxy configuration. When I monitor the queue I have a number of 16 message consumers, but I would need to increse it to a higher number in order to improve the performance.
Please send me any suggestion o advice to solve it.
Thanks in advance
Luis

Sorry I sent it to the wrong place. I have moved it to Weblogic

JMS Biz & Proxy Services Using OSB & SSL

We have a potential requirement being enforced upon us which states that our communication to another system using a JMS proxy service should be using SSL.
Has anyone done this before using OSB?
There is a "Use SSL" tick box on the proxy and business services, but I think this is purely for authentication and not transport - but I could be wrong as its not an area I know that much about.
We're pushing for an IPSEC VPN link as that makes much more sense, but in the meantime I thought I'd ask if anyone had done any JMS + SSL work using OSB.
Thanks,
Pete

Manoj, Thanks for your reply.
Actually the other system is OSB!!
It's a government system we're talking to but in the interests of open-ness, we cannot use OSB->OSB out of the box interfaces such as SAF etc, as that isn't an open enough interface and users of the interface who don't have OSB may be disadvantaged! Something like that anyway! SAF would be a perfect option for this interface, but .....
What we've been so far is that all communication via JMS must be SSL based. But - t3s is not supported on this interface, so the SSL negotiation must be handled by a hardware device or something else.
We think we've found BigIP load balancers do the job, but that's a large cost as we don't have those in our current infrastructure.
We don't have much SSL communication apart from https, so I don't have a whole lot of experience with it and I just wondered if there was any OSB feature I'm missing, or something you can do in plan java code to get JMS communicating via SSL.
Pete
ps:A standard IPSEC VPN would do the job too, which we're going to try to push for, then, you just use the jms:// url to an address and the security is via the VPN.

In OSB 10,3, a Java Callout runs in the same thread of the Proxy Service?

Hello,
a I have a Java Callout within the Proxy Service, request pipeline.
Does it runs in the same thread of the Proxy?
If NOT,
which work manager is the Java Callout pinned to?
can I set the work manager I want?
Thank you,
regards
F.Costa

Thanks a lot.
Pointer to great info.
However doesn't mention Java Callout threading, nor the documentation does.
Anybody with a YES or NO, please...
F.Costa

OSB Business -Proxy Service

Hi,
I need to call WSDL which has 2 operations. I need to pass default xml payload to operation 1 and the response I get, has to be passed to invoke operation 2. I have created 2 business services and after I create a proxy service :
How to pass default xml to business service 1 ?
How to pass response from business service 1 to business service 2 ?
Thanks

somebol, Prabhu
If I use Service Callout , I see Request Document Variable where I can pass in the request ($body/<request>), as you mentioned.But, the default XML payload to business service does not come as input to Proxy Service, I need to pass that XML payload explicitly to business service op1.
I was thinking if I have to use assign or replace to pass the XML payload to business service op1, but I am not sure how to do in my Proxy.
My Input to Proxy Service is date , which at this point is not used anywhere.
Thanks

Call 2 or More Business/proxy services in parallel

Hi ,
As per our requirement we have to create a common proxy which has to route 2 or more services(proxy/business) in parallel.Please provide us link or sample /inf(apart from split join concept) to acquire the solution for this since we are new to OSB.(like flow or flow N in BPEL)
Thanks.................

Sorry, the last post was incorrect.
What it happens is that the .jsp of the BIB calls it self when you navigate it. This is the code from the client side:
<a href="#" accesskey="I" onclick="document.BIForm.target='';submitForm('BIForm',1,{source:'FDEResumen_Presentation1',event:'bi_page'});return false"><img src="/MetalsaBIWar/cabo/images/cache/es/b-Ir-i.gif" alt="Ir a cambiar página: Clave de acceso = i" border="0" align="absmiddle" width="28" height="22"></a>
The problem when you put it in the Portal is that it's the same code but embedded in a new html (that is the portal page). When you try to navigate it references at the Portal Page, not at the .jsp page.
This is the question. How can I program the .jsp (of a BI Bean) in JDeveloper so that the reference to navigate must be the .jsp itself and not a dinamic reference?
Thanks in advance,
IM

When we should use JMS based proxy services and when JMS Adapter in OSB

Hi,
Can somebody explain me when we could go for JMS Adapter and when we should go for JMS based service in OSB?
The one scenerio i can think of is when my provider/consumer is using jms provider other than WLS jms provider we should use jms adapter.
Edited by: user12883209 on Jan 5, 2012 5:29 AM

I don't think there are clear guidelines available, however here is my view.
1) OSB 11gR1 is not certified with Oracle JMS Adapter. It is certified with Oracle FTP, Database, AQ and application adapters.
http://docs.oracle.com/cd/E21764_01/doc.1111/e15867/interop.htm#OSBAG1403
2) In addition of the Weblogic JMS, OSB can also integrate with following provider without using adapter.
- WebSphere MQ
- Tibco EMS
3) if you need to integrate JMS with BPEL Process or Oracle SOA suite composite, it has built in support for generic Oracle JMS adapter. Oracle generic JMS adapter can integrate with AQ JMS (JMS providers OJMS 8.1.7, 9.0.1.4, and 9.2), TIBCO JMS, IBM Websphere MQSeries (IBM MQSeries JMS 6.0), Weblogic JMS, Apache, and Active MQ
http://docs.oracle.com/cd/E14571_01/integration.1111/e10231/adptr_jms.htm
4) If you need to integrate OSB with Weblogic JMS, WebSphere MQ, Tbco EMS it does not need JMS adapter.
Hope this helps !!!
Jayesh Patel
http://jayesh-patel.blogspot.com/
http://www.yagnasys.com/

Mapping Stuck Threads to Proxy Services

Hello OSB Experts:
Is there a way I can map the information of a thread dump showing stuck threads to the proxy service that those threads are executing?
I am facing a situation that seems to be usual: one of several backends is performing poorly, producing stuck threads in the OSB app. servers. After a while, the performance of the whole OSB is affected because of the number of threads in stuck state.
I would like to be able to pinpoint the backend with problems in order to solve the problem before the health of the whole OSB is affected.
Can this be done without 3rd party tools?
Thanks
Sebastián

No, it is not possible to get the the proxy service name from the thread dump. The OSB stuff is handled under a weblogic wrapper thread.
One of the main reason for stuck threads is some of the proxy service calls taking a lot of time through the business services and there is no appropriate timeout set for the calls on the business service.
You have couple of options to find out the culprit. You can enable tracing on the proxy services and the business services and capture the metrics that should help you find a pattern for the performance. The numbers will help you find which backend is taking more time.
You can get nmore details on OSB Monitoring @ http://docs.oracle.com/cd/E23943_01/admin.1111/e15867/monitoring_ops.htm#OSBAG472
Secondly, always set some realistic timeout value (like 30 seconds/60 seconds depending on the sla of the back-end) on the business service. Leaving it to default (that is 0) is like configuring a indefinite timeout (in which case it uses the default weblogic connection timeout setting of 10/15 minutes).
This way the threads are not stuck waiting for the response and dont cause issues.
Additionally you use OSB alerts to notify you for service behavior discrepancies w.r.t response times, number of errors etc .. Refer http://docs.oracle.com/cd/E23943_01/admin.1111/e15867/monitoring_console.htm#CACDCDAH for more details.
Hope this information helps.
Thanks,
Patrick
It is considered good etiquette to reward answerers with points (as "helpful" - 5 pts - or "correct" - 10pts).+
https://forums.oracle.com/forums/ann.jspa?annID=893

OSB -11g - Calling JMS Proxy Service in a stage

Hi,
When we createa JMS based proxy service, the proxy service keeps on polling the JMS Queue for any new messages arriving in the queue. But I do not want the proxy service to keep on polling the Queue. Instead I want to call the JMS Proxy Service in any stage (of any other Proxy Service) using a service callout. It means that I do not want the Queue to be polled always, instead the queue should be polled only when this JMS Proxy Service is invoked.
Is it possible to do? If so, how?
Thanks,
Sanjay

Hi Anuj,
Following is my requirement:
The producer wants to interact with us using Web Services. The input payload has some more values (username and password) apart from the HL7 data (base64 encoded) itself. I have to receive the input payload, authenticate the call and convert the HL7 encoded message into an XML and send it to the consumer.
What have I done till now:
1) I have created two Proxy Services PS1 and PS2
2) Proxy Service (PS1) will be invoked by the producer.
3) Then I get the username, password and data from the payload
4) Populate the username and password in the SOAP Header and HL7 data in SOAP Body
5) Call PS2 from PS1
6) In PS2 I have configured Message Level Security to authenticate with the username and password
7) After authentication I make a Java Callout to decode the base64 encoded HL7 data
8) I now have written my own HL7 sender (Java code) which wraps the decoded HL7 data with MLLP requirements and then send it to the B2B server using a Socket (TCP connection). I do a Java Callout to this Java class.
9) In B2B I have configured a RTP (Remote Trading Partner) which has a MLLP channel.
10) I created a ADT message document and a ACK document
11) I created two agreements between HTP and RTP (one agreement receives the HL7 data from the TRP using MLLP and another which sends the ACK to the RTP using MLLP channel)
12) In the receiveing agreement I have configured a JMS channel (on the HTP side) becuase of which B2B receives the HL7 Pipe delimited data and translates it and then pushes the XML data into the JMS queue (which needs to be transformed to producer XML)
13) I have also configured a JMS channel for the RTP which I am using in second agreement to get the acknowledgement. Because of this the ACK is placed in a JMS queue.
I need to poll this JMS queue in order to send the ACK back to the producer (which I have to do in the response pipeline of PS2). Hence I wanted to pop the message from the JMS queue in the response pipeline and then send that message.
Please advice if my approach is correct.
Thanks,
Sanjay
Edited by: Sanjay Bharatiya on 17-Mar-2011 12:06 PM

Local transport proxy service vs http proxy service

Hi,
Can anybody tell me why it is better to use Local transport proxy service in OSB than http for modularity?

Hi Atheek,
Thanks for you reply.
My doubt is where local transport proxy service is deployed?
Like if we invoke a http based proxy service which is deployed at certain Endpoint and jms based proxy service is deployed as MDB which kepps listening to a queue.
Similarly, when i invoke local transport proxy service where and how OSB runtime looks for it?
Edited by: user12883209 on Jan 10, 2012 6:58 AM

ALSB Proxy Service error calling Business service

I have an external webservice on Weblogic 8.1 SP5, and I configured Business Service in ALSB 2.6, that is able to invoke the external webservice. But When I create Proxy Service based on the existing Business service and try to invoke the proxy service I get "no SOAP Action Header" error in WebLogic 8.1 SP5 External webservice.
AxisFault
faultCode: {http://xml.apache.org/axis/}Client.NoSOAPAction
faultSubcode:
faultString: no SOAPAction header!
faultActor:
faultNode:
faultDetail:
{http://xml.apache.org/axis/}stackTrace:no SOAPAction header!
at org.apache.axis.transport.http.AxisServlet.getSoapAction(AxisServlet.java:1004)
at org.apache.axis.transport.http.AxisServlet.doPost(AxisServlet.java:678)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:760)
at org.apache.axis.transport.http.AxisServletBase.service(AxisServletBase.java:327)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:853)
at weblogic.servlet.internal.ServletStubImpl$ServletInvocationAction.run(ServletStubImpl.java:1072)
at weblogic.servlet.internal.ServletStubImpl.invokeServlet(ServletStubImpl.java:465)
at weblogic.servlet.internal.ServletStubImpl.invokeServlet(ServletStubImpl.java:348)
at weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.run(WebAppServletContext.java:6981)
at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:321)
at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:121)
at weblogic.servlet.internal.WebAppServletContext.invokeServlet(WebAppServletContext.java:3892)
at weblogic.servlet.internal.ServletRequestImpl.execute(ServletRequestImpl.java:2766)
at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:224)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:183)
{http://xml.apache.org/axis/}hostname:ossinv1d
no SOAPAction header!
at org.apache.axis.transport.http.AxisServlet.getSoapAction(AxisServlet.java:1004)
at org.apache.axis.transport.http.AxisServlet.doPost(AxisServlet.java:678)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:760)
at org.apache.axis.transport.http.AxisServletBase.service(AxisServletBase.java:327)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:853)
at weblogic.servlet.internal.ServletStubImpl$ServletInvocationAction.run(ServletStubImpl.java:1072)
at weblogic.servlet.internal.ServletStubImpl.invokeServlet(ServletStubImpl.java:465)
at weblogic.servlet.internal.ServletStubImpl.invokeServlet(ServletStubImpl.java:348)
at weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.run(WebAppServletContext.java:6981)
at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:321)
at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:121)
at weblogic.servlet.internal.WebAppServletContext.invokeServlet(WebAppServletContext.java:3892)
at weblogic.servlet.internal.ServletRequestImpl.execute(ServletRequestImpl.java:2766)
at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:224)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:183)
Also I get
com.bea.wli.sb.pipeline.PipelineException in the ALSB.
Is it a known issue ?

The back end service basically complains the SOAPAction http header was not included in the request. Typically this header is added automatically by ALSB if the proxy is of type SOAP and the wsdl contain the SOAPAction value for the operation invoked.
Alternatively you can force the SOAPAction header to be set to the correct value by adding a transport header action in the request part of the routing node.
Gregory Haardt
ALSB Prg. Manager
[email protected]

Soap proxy to jms business service response not available.

Hi
Please can anyone help me with this really urgent, help is greatly appreciated.
We have one Proxy Service which takes in SOAP Request, and I have a route node in it which calls the biz service.
Business service is a Messaging service built using jms queue and response URI is configured with jms out queue.
If I run business service standalone it gets the message and response comes to out queue.
if I run it through proxy service which routes to biz service, the message goes to biz service, and response goes to out queue, but I do not get response in http proxy.
Please can someone help.
thanks
K.

Basically the logic to consume msg requires full SOAP message as text, now when I use BS as Messaging Service it works fine because in test I put the full soap message as text. If I run it through Proxy it strips the header and sends only Body or if I base both my BS and PS using WSDL, then it attaches full soap message in multiple soap header and that causes the problem, so I have to remove extra header from message to have it processed correctly.
this is my actual request:
<SOAP-ENV:Envelope      xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/">
<SOAP-ENV:Body>
</SOAP-ENV:Body>
</SOAP-ENV:Envelope>
the trace shows that request to BS is:
<soapenv:Body      xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
<SOAP-ENV:Envelope      xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/">
<SOAP-ENV:Body>
/SOAP-ENV:Body>
</SOAP-ENV:Envelope>
</soapenv:Body>
so my message consumer has problem with extra soapenv:Body wrapper... any ideas will help.
thanks
K.

Report and Alert don't executed in the error handler with a JMS proxy service

Hi,
I'm working with OSB 11.1.1.4.0 and I'm facing a problem with the error handler of my JMS proxy service.
My error handler contains two main elements :
- a Report action
- an Alert with a JMS destination defined
When the business service failed the message remain in the queue. It's the expected behavior.
The problem is :
- the report is missing
- the alert is raised but missing in the JMS destination defined in the Alert setting.
I've noticed if I add a Reply with Failure or Success, Report and Alert are successfully executed.
Unfortunately the message don't remain in the JMS queue.
What can I do in order to have my report/alert done and the JMS transaction rollback in order to keep the message in the queue ?
I've already met this problem in the past with alsb v3 and my solution was to add Service Callout in the error handler in order to externalize these actions.
This solution is not convenient and I hope another solution is possible.

Inventorying workgroups can be difficult, especially when it comes to remote access and network security. Because workgroups are not centrally managed, some of the items discussed in this
wiki article on preparing your workgroup environment may require you to visit each machine individually.
For non-domain credentials, you do not use the <systemname>\<user> format, you simply enter the user name. Regarding how to enter the credentials, if you have an account that uses the same username and password on all machines and is an administrator
on all of those machines, then you can enter that in the All computers credentials page of the wizard. You can also do this if they are different user names. However, if some machines have an account with the same user name, such as Administrator,
but different passwords on each machine, you will need to use the Manually enter computer names discovery method, and then enter the information for each group or each machine.
As you can tell, workgroup environments can quickly negate any benefit that the agentless inventory nature of MAP provides.
Please remember to click "Mark as Answer" on the post that helps you, and to click
"Unmark as Answer" if a marked post does not actually answer your question. Please
VOTE as HELPFUL if the post helps you. This can be beneficial to other community members reading the thread.

Oracle Service Bus "ThreadPool has stuck threads"

Hi all,
I have JMS Proxy service that listen to the queue, with small quantity of message every second, it works great.
But after put message with bigger quantity and faster insertion on queue, the proxy service cannot
pull the message anymore, the error message is "STUCK THREAD", it makes the weblogic console give warning status.
This is the Dump Thread Stacks :
===== FULL THREAD DUMP ===============
Tue Jun 8 11:30:24 2010
BEA JRockit(R) R27.6.1-20-104573-1.6.0_05-20081016-1236-linux-ia32
"Main Thread" id=1 idx=0x4 tid=7168 prio=5 alive, in native, waiting
-- Waiting for notification on: weblogic/t3/srvr/T3Srvr@0x1cd20688[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)
at java/lang/Object.wait(Object.java:485)
at weblogic/t3/srvr/T3Srvr.waitForDeath(T3Srvr.java:811)
^-- Lock released while waiting: weblogic/t3/srvr/T3Srvr@0x1cd20688[fat lock]
at weblogic/t3/srvr/T3Srvr.run(T3Srvr.java:459)
at weblogic/Server.main(Server.java:67)
at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
-- end of trace
"(Signal Handler)" id=2 idx=0x8 tid=7169 prio=5 alive, in native, daemon
"(GC Main Thread)" id=3 idx=0xc tid=7170 prio=5 alive, in native, native_waiting, daemon
"(GC Worker Thread 1)" id=? idx=0x10 tid=7171 prio=5 alive, in native, daemon
"(GC Worker Thread 2)" id=? idx=0x14 tid=7172 prio=5 alive, in native, daemon
"(GC Worker Thread 3)" id=? idx=0x18 tid=7173 prio=5 alive, in native, daemon
"(GC Worker Thread 4)" id=? idx=0x1c tid=7174 prio=5 alive, in native, daemon
"(GC Worker Thread 5)" id=? idx=0x20 tid=7175 prio=5 alive, in native, daemon
"(GC Worker Thread 6)" id=? idx=0x24 tid=7176 prio=5 alive, in native, daemon
"(GC Worker Thread 7)" id=? idx=0x28 tid=7177 prio=5 alive, in native, daemon
"(GC Worker Thread 8)" id=? idx=0x2c tid=7178 prio=5 alive, in native, daemon
"(Code Generation Thread 1)" id=4 idx=0x30 tid=7179 prio=5 alive, in native, native_waiting, daemon
"(Code Optimization Thread 1)" id=5 idx=0x34 tid=7180 prio=5 alive, in native, native_waiting, daemon
"(VM Periodic Task)" id=6 idx=0x38 tid=7181 prio=10 alive, in native, daemon
Is there any tuning that I need to do on Jrockit?
Really thanks in advance.
Regards,
Mr.K

Hi Eric,
Thank you for the link, I'll try for increasing it.
Hi Manoj,
hope it'll give you some additional information for helping me.
####<Jun 8, 2010 10:43:44 AM WIT> <Error> <WebLogicServer> <cspproduction> <AdminServerOSB> <[ACTIVE] ExecuteThread: '18' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1275968624958> <BEA-000337> <[STUCK] ExecuteThread: '7' for queue: 'weblogic.kernel.Default (self-tuning)' has been busy for "653" seconds working on the request "weblogic.work.SelfTuningWorkManagerImpl$WorkAdapterImpl@39c79ac", which is more than the configured time (StuckThreadMaxTime) of "600" seconds. Stack trace:
Thread-50 "[STUCK] ExecuteThread: '7' for queue: 'weblogic.kernel.Default (self-tuning)'" <alive, in native, suspended, waiting, priority=1, DAEMON> {
-- Waiting for notification on: weblogic.rjvm.ResponseImpl@39cf5b1[fat lock]
java.lang.Object.wait(Object.java:???)
weblogic.rjvm.ResponseImpl.waitForData(ResponseImpl.java:80)
weblogic.rjvm.ResponseImpl.getTxContext(ResponseImpl.java:127)
weblogic.messaging.dispatcher.DispatcherProxy.unmarshalResponse(DispatcherProxy.java:236)
weblogic.messaging.dispatcher.DispatcherProxy.dispatchSyncTranFuture(DispatcherProxy.java:133)
weblogic.messaging.dispatcher.DispatcherWrapperState.dispatchSyncTran(DispatcherWrapperState.java:333)
weblogic.messaging.dispatcher.DispatcherWrapperState.dispatchSyncNoTran(DispatcherWrapperState.java:384)
weblogic.jms.dispatcher.DispatcherAdapter.dispatchSyncNoTran(DispatcherAdapter.java:59)
weblogic.jms.client.JMSProducer.toFEProducer(JMSProducer.java:1151)
weblogic.jms.client.JMSProducer.deliveryInternal(JMSProducer.java:679)
weblogic.jms.client.JMSProducer.sendInternal(JMSProducer.java:541)
weblogic.jms.client.JMSProducer.sendWithListener(JMSProducer.java:366)
weblogic.jms.client.JMSProducer.send(JMSProducer.java:353)
weblogic.jms.client.WLProducerImpl.send(WLProducerImpl.java:1050)
weblogic.jms.client.WLProducerImpl.send(WLProducerImpl.java:107)
weblogic.deployment.jms.WrappedMessageProducer.send(WrappedMessageProducer.java:235)
com.bea.wli.sb.transports.jms.JmsOutboundMessageContext$SendAction.sendMessage(JmsOutboundMessageContext.java:823)
com.bea.wli.sb.transports.jms.JmsOutboundMessageContext$SendAction.run(JmsOutboundMessageContext.java:745)
weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:350)
weblogic.security.service.SecurityManager.runAs(Unknown Source)
weblogic.security.Security.runAs(Security.java:60)
com.bea.wli.sb.transports.jms.JmsOutboundMessageContext.send(JmsOutboundMessageContext.java:502)
com.bea.wli.sb.transports.jms.JmsTransportProvider.sendMessageAsync(JmsTransportProvider.java:680)
sun.reflect.GeneratedMethodAccessor302.invoke(Unknown Source)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
java.lang.reflect.Method.invoke(Method.java:575)
com.bea.wli.sb.transports.Util$1.invoke(Util.java:56)
$Proxy61.sendMessageAsync(Unknown Source)
com.bea.wli.sb.transports.LoadBalanceFailoverListener.sendMessageAsync(LoadBalanceFailoverListener.java:136)
com.bea.wli.sb.transports.LoadBalanceFailoverListener.sendMessageToServiceAsync(LoadBalanceFailoverListener.java:510)
com.bea.wli.sb.transports.LoadBalanceFailoverListener.sendMessageToService(LoadBalanceFailoverListener.java:477)
com.bea.wli.sb.transports.TransportManagerImpl.sendMessageToService(TransportManagerImpl.java:491)
com.bea.wli.sb.transports.TransportManagerImpl.sendMessageAsync(TransportManagerImpl.java:405)
com.bea.wli.sb.pipeline.PipelineContextImpl.doDispatch(PipelineContextImpl.java:508)
com.bea.wli.sb.pipeline.PipelineContextImpl.dispatch(PipelineContextImpl.java:486)
stages.publish.runtime.PublishRuntimeStep.processMessage(PublishRuntimeStep.java:83)
com.bea.wli.sb.stages.StageMetadataImpl$WrapperRuntimeStep.processMessage(StageMetadataImpl.java:343)
com.bea.wli.sb.stages.impl.SequenceRuntimeStep.processMessage(SequenceRuntimeStep.java:31)
com.bea.wli.sb.pipeline.PipelineStage.processMessage(PipelineStage.java:84)
com.bea.wli.sb.pipeline.PipelineContextImpl.execute(PipelineContextImpl.java:840)
com.bea.wli.sb.pipeline.Pipeline.processMessage(Pipeline.java:140)
com.bea.wli.sb.pipeline.PipelineContextImpl.execute(PipelineContextImpl.java:840)
com.bea.wli.sb.pipeline.PipelineNode.doRequest(PipelineNode.java:51)
com.bea.wli.sb.pipeline.Node.processMessage(Node.java:65)
com.bea.wli.sb.pipeline.PipelineContextImpl.execute(PipelineContextImpl.java:840)
com.bea.wli.sb.pipeline.Router.processMessage(Router.java:178)
com.bea.wli.sb.pipeline.MessageProcessor.processRequest(MessageProcessor.java:39)
com.bea.wli.sb.pipeline.RouterManager$1.run(RouterManager.java:508)
com.bea.wli.sb.pipeline.RouterManager$1.run(RouterManager.java:506)
weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:350)
weblogic.security.service.SecurityManager.runAs(Unknown Source)
com.bea.wli.sb.security.WLSSecurityContextService.runAs(WLSSecurityContextService.java:55)
com.bea.wli.sb.pipeline.RouterManager.processMessage(RouterManager.java:487)
com.bea.wli.sb.transports.TransportManagerImpl.receiveMessage(TransportManagerImpl.java:248)
com.bea.wli.sb.transports.jms.JmsInboundMDB.onMessage(JmsInboundMDB.java:79)
sun.reflect.NativeMethodAccessorImpl.invoke0(NativeMethodAccessorImpl.java:???)
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:27)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
java.lang.reflect.Method.invoke(Method.java:575)
com.bea.core.repackaged.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:277)
com.bea.core.repackaged.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:187)
com.bea.core.repackaged.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:153)
com.bea.core.repackaged.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:86)
com.bea.core.repackaged.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:153)
com.bea.core.repackaged.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:126)
com.bea.core.repackaged.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:100)
com.bea.core.repackaged.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:153)
com.bea.core.repackaged.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:148)
$Proxy79.onMessage(Unknown Source)
weblogic.ejb.container.internal.MDListener.execute(MDListener.java:403)
weblogic.ejb.container.internal.MDListener.transactionalOnMessage(MDListener.java:352)
weblogic.ejb.container.internal.MDListener.onMessage(MDListener.java:264)
weblogic.jms.client.JMSSession.onMessage(JMSSession.java:4526)
weblogic.jms.client.JMSSession.execute(JMSSession.java:4022)
weblogic.jms.client.JMSSession.executeMessage(JMSSession.java:3708)
weblogic.jms.client.JMSSession.access$000(JMSSession.java:114)
weblogic.jms.client.JMSSession$UseForRunnable.run(JMSSession.java:5058)
weblogic.work.SelfTuningWorkManagerImpl$WorkAdapterImpl.run(SelfTuningWorkManagerImpl.java:516)
weblogic.work.ExecuteThread.execute(ExecuteThread.java:198)
weblogic.work.ExecuteThread.run(ExecuteThread.java:165)
That is error message from Oracle service bus console, it stated there is JMS Proxy service that got stuck.
On previous post, I already attached the dump trace, it explained something about jrockit. But I still don't get the way for tuning it properly.
Regards,
Mr.K

ALSB 3.0: Problem with Nested Policies in Proxy Service WSDL

Hello,
I'm trying to implement client applications that consume Web Services.
In ALSB, i imported the WSDLs of the services, created business and proxy Services.
Creating web references in .NET and Netbeans is working well, as long as there is no security policy attached to the WSDLs of the proxy services.
When attaching a policy, ALSB nests it inside the WSDL:
<s0:Policy s1:Id="signMessage">
<wssp:Integrity SignToken="false">
<wssp:SignatureAlgorithm URI="http://www.w3.org/2000/09/xmldsig#rsa-sha1"/>
<wssp:CanonicalizationAlgorithm URI="http://www.w3.org/2001/10/xml-exc-c14n#"/>
<wssp:Target>
and references it, where i want it to have. in this case it's in the request of the service:
<s2:input>
<s4:body use="literal"/>
<wsp:Policy>
<wsp:PolicyReference URI="#signMessage"/>
</wsp:Policy>
</s2:input>
The problem is, that neither .Net (C#) nor Netbeans (Java) correctly import and handle these policies. .Net even mentions in the generated code that the "Policy" element was not handled. So when running my applications, i logically get exceptions that no credentials/tokens or else were delivered from the client side. In general the mentioned IDEs that i'm using support the configuration of those security issues. But when not correctly parsing the WSDL from the Service Bus, they don't work.
I tried many ways to dodge this problem, but without success.
Manually writing policy Statements into the WSDL files is not what i'm searching for as a solution. I want it dynamically.
I hope i could describe my problem clear enough and to get a possible solution.
Thanks

Sorry, found the answer but forgot to update this thread.
The ?WSDL mechanism fails if Enforce WS-I Interoperability is true for the Proxy Service.
While BEA support (and the docs relating to this flag) say this is by design (Only POST not GET is supported by WS-I) personally it seems to me that this rule should apply to operation invocation not a metadata inquiry but there you go...

Stuck Thread - JMS Business proxy Service (ALSB 2.5)

Similar Messages

Maybe you are looking for