Managed server in WARNING state in Obiee 11.1.1.7.0 prod env

Question

Hi All,
We have OBIEE 11.1.1.7.0 running in our company. Last couple of days we are experiencing that, the managed server "bi_server1" always changes to "warning" state in every 20 to 30 minutes or so. Reports are too slow and behaves funny after warning. When I restart opmn, then it shows health is "OK". But again changes to "WARNING" after sometimes. It says: Thread pool has stuck threads.
We have got a RAC environment of database and our OBIEE is pointing to one of the node and i can tnsping to it. I can increase the StuckThread time to 1200, 1200 or so instead of 600, 60 for bi_server1 in the console as found somewhere as a tuning process but not sure if it will help much!! I can't even cancel the long running queries in the "managed session" of obiee analytics.
Kindly advise what exactly is causing this problem and what must I do to resolve this??
The bi_server1 log file displays the following error:
####<Oct 15, 2013 1:31:59 PM SAST> <Error> <WebLogicServer> <SWAOBIEE> <bi_server1> <[ACTIVE] ExecuteThread: '5' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <705beb8dd8c1b40f:-c2b96b4:141a7012d14:-8000-000000000004373a> <1381836719713> <BEA-000337> <[STUCK] ExecuteThread: '2' for queue: 'weblogic.kernel.Default (self-tuning)' has been busy for "709" seconds working on the request "weblogic.servlet.internal.ServletRequestImpl@46631799[
GET /analytics/saw.dll?Sessions HTTP/1.1
Connection: keep-alive
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.35 (KHTML, like Gecko) Chrome/27.0.1448.0 Safari/537.35
Referer: http://10.20.16.72:9704/analytics/saw.dll?Admin
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8
Cookie: ORA_BIPS_LBINFO=141bbd5ed78; ORA_BIPS_NQID=v1b3rdkjen27ul12vejs054ra8v5mgmsk25pu7a; JSESSIONID=5xfZSdkK6N0HN49H2hKFT9SmQ0ngVp6vLJvw81QfTychYjpwpmFZ!-2050741508; ADMINCONSOLESESSION=Q2TLSdkdv1GT79BFgzPyw5njCtPDXmmyWbhb523hgV5DRqtv7csD!1865413782
]", which is more than the configured time (StuckThreadMaxTime) of "600" seconds. Stack trace:
        java.net.SocketInputStream.socketRead0(Native Method)
        java.net.SocketInputStream.read(SocketInputStream.java:129)
        com.siebel.analytics.web.sawconnect.SAWConnection$NotifyInputStream.read(SAWConnection.java:165)
        java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
        java.io.BufferedInputStream.read(BufferedInputStream.java:237)
        com.siebel.analytics.web.sawconnect.sawprotocol.SAWProtocol.readInt(SAWProtocol.java:188)
        com.siebel.analytics.web.sawconnect.sawprotocol.SAWProtocolInputStreamImpl.readChunkHeader(SAWProtocolInputStreamImpl.java:282)
        com.siebel.analytics.web.sawconnect.sawprotocol.SAWProtocolInputStreamImpl.startReadingNewMessage(SAWProtocolInputStreamImpl.java:49)
        com.siebel.analytics.web.sawconnect.SAWServletHttpBinding.forwardResponse(SAWServletHttpBinding.java:201)
        com.siebel.analytics.web.SAWBridge.processRequest(SAWBridge.java:189)
        com.siebel.analytics.web.SAWBridge.doGet(SAWBridge.java:224)
        javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
Thanks,
BK.

Answer 1

I had similar issue in windows 7 [64 bit].
It is found that the nodemanger windows service was not present and I had to create it by executing installNodeMgrSvc.cmd present in wls server location say, 'C:\Oracle\Middleware\wlserver_10.3\server\bin'  in my case and then come to bi wls console and 'Resume' the bi_server1 managed server. You would see progress in the installer once the managed server bi_server1 is 'Running' state.
Hope this helps.