Restarting Error Workflows in Production Environment
Hi Experts,
I m having an issue in the prod enviroment there are some error workflows because of the error issued by the function in a method which i used in a workflow step.Know i have trapped that error .And i tried to restart the workflows by using SWPR.
Still the workflow is going to error status.
Please advice.
Thanks,
Rajesh.
Hello,
Is it showing the same error? Try SWU_OBUF.
regards
Rick Bakker
Hanabi Technology
Similar Messages
-
Error #2032 in Production Environment
We're leveraging BOE to automate the refresh of Xcelsius dashboards. Connecting to Webi via Live Office, which plugs into the Xcelsius template through its data manager. The configuration worked fine in integration server but when it was migrated to the Production Server, we received the famous general error #2032.
We installed the crossdomain xml file correctly, but it did not resolve the issue. We tried to debug using 'Fiddler' but it did not provide any pertinent info, aside from the fact that it wasn't connecting to Business Objects. Has anyone come across this issue in the Production Server?
Any insight would be helpful.
thanks.Hy,
you can find a solution on this blog:
/people/andrea.maraviglia/blog/2010/03/22/tips-on-xcelsius-and-sap-netweaver-bw-connection-2032-errorSDNWeblogs_BusinessIntelligencebi%2528SAPNetworkWeblogs%253ABusinessIntelligence+%2528BI%2529%2529 -
[Help] - Deploy the Discussion generates error in a production environment
Folow the error:
AxisFault
faultCode: {http://schemas.xmlsoap.org/soap/envelope/}Client
faultSubcode:
faultString: Fault: java.lang.NegativeArraySizeException
faultActor:
faultNode:
faultDetail:
{http://xml.apache.org/axis/}stackTrace:Fault: java.lang.NegativeArraySizeException
at org.apache.axis.message.SOAPFaultBuilder.createFault(SOAPFaultBuilder.java:222)
at org.apache.axis.message.SOAPFaultBuilder.endElement(SOAPFaultBuilder.java:129)
at org.apache.axis.encoding.DeserializationContext.endElement(DeserializationContext.java:1087)
at oracle.xml.parser.v2.NonValidatingParser.parseElement(NonValidatingParser.java:1554)
at oracle.xml.parser.v2.NonValidatingParser.parseRootElement(NonValidatingParser.java:409)
at oracle.xml.parser.v2.NonValidatingParser.parseDocument(NonValidatingParser.java:355)
at oracle.xml.parser.v2.XMLParser.parse(XMLParser.java:226)
at weblogic.xml.jaxp.RegistryXMLReader.parse(RegistryXMLReader.java:173)
at javax.xml.parsers.SAXParser.parse(SAXParser.java:395)
at org.apache.axis.encoding.DeserializationContext.parse(DeserializationContext.java:227)
at org.apache.axis.SOAPPart.getAsSOAPEnvelope(SOAPPart.java:696)
at org.apache.axis.Message.getSOAPEnvelope(Message.java:435)
at org.apache.axis.handlers.soap.MustUnderstandChecker.invoke(MustUnderstandChecker.java:62)
at org.apache.axis.client.AxisClient.invoke(AxisClient.java:206)
at org.apache.axis.client.Call.invokeEngine(Call.java:2784)
at org.apache.axis.client.Call.invoke(Call.java:2767)
at org.apache.axis.client.Call.invoke(Call.java:2443)
at org.apache.axis.client.Call.invoke(Call.java:2366)
at org.apache.axis.client.Call.invoke(Call.java:1812)
at com.jivesoftware.webservices.ForumServiceHttpBindingStub.createForumUnderCategory(ForumServiceHttpBindingStub.java:1050)
at br.com.embraer.discussions.DiscussionServices.createForumForDocument(DiscussionServices.java:107)
Any idea?You can also catch such timeout exceptions via the portlet error page reference.
http://docs.oracle.com/cd/E15919_01/wlp.1032/e14235/chap_config_remote.htm#i1022393 -
How do perform error handling in production environment?
Hi,
I am using CKM and flow control true, tp trace the invalid records while moving data from File to DB. For all invalid records it make n entry in E$_ Table.
But when I correct the record in file and if its success full, those error gets deleted... I want to keep those error as a history?? How can i do that?
Secondly, If there is an error records in E$ table..and when i run next time.. Will it insert only the error records or entire file again???????????????
Thanks:)To your first question,
Yes this can be achieved by amending the CKM. You can remove the step to 'delete previous errors' and make sure that the 'DROP_ERROR_TABLE' option is not set to 'true', therefore all errors will remain in the E$ table even if they have been loaded.
Your second question is dependent on what you are trying to achieve and how.
Are you using IKM incremental update/control Append to load your data?
You may want to have the RECYCLE_ERRORS' set to false on your IKM, then the previously error'ed records will not be re-inserted to your I$ table.
Your third point is basically due to to incorrect model definition in ODI and probably your target database. This is not an error in ODI terms but incorrect data modelling.
Edited by: Bos on Apr 13, 2011 11:38 AM -
How to restart an workflow if ended in error at SEND Mail step.
Hello All
I am facing some problem in restarting the error workflow.
Issue is:
Workflow ended in error at the mail notification step, since the mail id is not filled into the container so there is no address (mail id) and ended in error.
How to restart these kinds of workflow in the system?, we have tried to change the container value by assigning mail id and restarted the workflow but still went into error.
Is it possible to restart an workflow which ended in error other than rule resolution.
Regards
Anand KSI think you went in the write approach. I think you have to change the container properly by populating the ADDRESSTRING container element. Also check whether type of email recipient in your container is et to Email.
Thanks
Arghadip -
Production environment, error in BPELConsole
Hi there,
I had installed, following step by step the installation guides, a production environment on three linux boxes RHE3; one for the DB v10g and in the others two an AS whit j2ee and web cache and obviously BpelPM. I do not installed WebCache AS component because an Oracle guy told me that bpel go crazy if webcache runs.
I can connect BPELConsole, but after the login I get this error message:
==========================================================
ORABPEL-04001
Cannot find instances.
An attempt to fetch the instances using the where condition " ( ci_state = 1 OR ci_state = 2 OR ci_state = 3 ) order by ci_modify_date desc " from the datastore has failed. The exception reported is: ORA-00942: table or view does not exist
Please check that the machine hosting the datasource is physically connected to the network. Otherwise, check that the datasource connection parameters (user/password) is currently valid.
sql statement: Select * from ( select a.*, ROWNUM rnum From ( SELECT * FROM admin_list_ci WHERE ci_domain_ref = 0 AND ( ci_state = 1 OR ci_state = 2 OR ci_state = 3 ) order by ci_modify_date desc ) a Where ROWNUM <= 20 ) Where rnum >= 0
at com.collaxa.cube.engine.data.AdminFinder.getXInstanceCount(AdminFinder.java:583)
at com.collaxa.cube.engine.data.AdminFinder.getOpenInstanceCount(AdminFinder.java:529)
at com.collaxa.cube.ejb.impl.BPELProcessManagerBean.getOpenInstanceCount(BPELProcessManagerBean.java:117)
at IBPELProcessManagerBean_StatelessSessionBeanWrapper46.getOpenInstanceCount(IBPELProcessManagerBean_StatelessSessionBeanWrapper46.java:955)
at com.oracle.bpel.client.BPELProcessHandle.getOpenInstanceCount(BPELProcessHandle.java:519)
at displayProcess.jspService(_displayProcess.java:508)
at com.orionserver.http.OrionHttpJspPage.service(OrionHttpJspPage.java:56)
at oracle.jsp.runtimev2.JspPageTable.service(JspPageTable.java:347)
at oracle.jsp.runtimev2.JspServlet.internalService(JspServlet.java:509)
at oracle.jsp.runtimev2.JspServlet.service(JspServlet.java:413)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:853)
at com.evermind.server.http.ServletRequestDispatcher.invoke(ServletRequestDispatcher.java:810)
at com.evermind.server.http.ServletRequestDispatcher.forwardInternal(ServletRequestDispatcher.java:322)
at com.evermind.server.http.ServletRequestDispatcher.forward(ServletRequestDispatcher.java:220)
at com.collaxa.cube.fe.DomainFilter.doFilter(DomainFilter.java:152)
at com.evermind.server.http.ServletRequestDispatcher.invoke(ServletRequestDispatcher.java:649)
at com.evermind.server.http.ServletRequestDispatcher.forwardInternal(ServletRequestDispatcher.java:322)
at com.evermind.server.http.HttpRequestHandler.processRequest(HttpRequestHandler.java:798)
at com.evermind.server.http.AJPRequestHandler.run(AJPRequestHandler.java:211)
at com.evermind.server.http.AJPRequestHandler.run(AJPRequestHandler.java:127)
at com.evermind.util.ReleasableResourcePooledExecutor$MyWorker.run(ReleasableResourcePooledExecutor.java:192)
at java.lang.Thread.run(Thread.java:534)
==========================================================
Can Anyone help me?
Thanks in advance
Stefano
nullHi,
as you know, your help was very useful.
Maybe is better to add the whole path of file that have to be changed
(only for beginners like me)
IRCA/integration/bpelpm/admin/sql/bpmuser.sql
This is true for DB version 10g release 2 or upper. For the previous versions the original string is enough.
Thanks again,
Stefano. -
Hi all,
After restarting the workflow through SWPR we are getting an error message as per mentioned below.
Error coming out of Workflow restart - Error:OBJECTS_OBJREF_NOT_ASSIGNED_NO.
Kindly suggest what can be the possibility of the error. I scehduled the SWPR program in the background and passed some workflows and getting this error message in the background job. Kindly suggest.
Thanks,
Neslin.I think you need to analyse the Workflow log and then try to restart. I think you might have to change the container element value in order to remove the error. Please check and act accordingly.
Thanks
Arghadip -
Workflow Restart Error from transaction SWPR
Hi workflow Gurus,
I am facing problem restarting workflow from transaction SWPR. Instead of restarting workflow I am getting an error message as unable to start subflow (ID: XYZ, node: abc).
How can I restart the workflow?
Regards,
Abhishek.The problem wad caused by mistake in workflow template WS75100021. After task 'Create test log' there is a loop. The end condition of the loop is 'test log is Submitted or Validated'. But the test log can have also status 'Closed' (The tester created the test log and close it immediately). It cause endless loop.
You can proceed following manual steps.
1. Go to transaction SWDD (Workflow Builder).
2. Open workflow template WS75100021.
3. Open loop number 000177
4. Change the condition this way
&FOPC_CASEL.STATUS& = IFO09 or
&FOPC_CASEL.STATUS& = IFO10 or
&FOPC_CASEL.STATUS& = IFO07
5. Activate the workflow template
if it helpful rewards points are appreciated -
Hello All,
We have few workflow items with status error due some missing agents. we have assigned the agents.
Now user asking us to restart or retrigge the same workflow item .
But when I tried to restart using SWIA getting message like ' Workflow cannot be restarted in status ERROR'
Is ther any possibility to restart ? If it's not how to get the same material name to reuse ....
Regards,
Lavanya.Hi,
1. You can restart the same workflow by using the Transcation SWPR
2. If you the problem still exists then please re-trigger the new workflow and delete the error workflow.
a. Please use transaction SWUS_WITH_REFERENCE to trigger the new workflow where you need to pass the workflow
instance number
b. You can also use transaction SWUS to trigger the workflow where you need to pass all the required workflow container
element values
Note : while using SWUS_WITH_REFERENCE all the required values will be fetched from the workflow instance number. No need to pass manualy.
Thanks and regards,
SNJY
Edited by: Sanju.pal on Jun 29, 2011 3:28 PM -
Getting error while migrating single mpr from pilot to production environment
Hi,
I am migrating single MPR form pilot to production and I am facing the following issue
Join-FIMConfig : Two objects with same AnchorAttributeValue were detected. A requirement for using this migration tool is that AnchorAttributeValue is unique with objectType
AnchorAttributeNames = DisplayName
AnchorAttributeValues = DisplayName
ObjectType = AttributeTypeDescription
ObjectID 1 = urn:uuid:acbcad3b-8394-4af3-a1d4-149cabae05d1
ObjectID 2 = urn:uuid:9f35c912-18bb-4b73-b128-281be0cae32f
Any help to resolve this issue. The DisplayName is built-in attribute.
Thanks
PavaniSearch these object ID in your production environnement it corresponds to something
ObjectID 1 = urn:uuid:acbcad3b-8394-4af3-a1d4-149cabae05d1
ObjectID 2 = urn:uuid:9f35c912-18bb-4b73-b128-281be0cae32f -
Hello Friends,
I have a problem. Let me explain my workflow design before explaining the problem. This is related to Service Entry Sheet workflow.
<b>Design.</b>
In this workflow, I have added a method for identifying agents. if it doesnt get any agent, it will divert the workflow to another path called as missing agents and workflow got stuck here. Like this i have 30 secnario in PRD.
Idear behind this desing is whenever we get missing agents, we will correct our data and restart the workflow again.
Problem: I want to take ur advice before starting this operation. Two best options are aviable for starting the workflow again.
1. Delete the exsisting workitem logically and create an event through SWUE
2. Set the workitem in error state and restart it through SWPR.
Please give me ur valuable solution. If you have some other option, please share it with me.
Thanks & Regards,
ManojUse everything the same workflow. Make use of fork. This workflow will get completed unless the required result has been reached. This you can make sure by putting the whole step in a Loop Ultil step. It is not good to always create new Workflow instance. You should use the same Workflow instance.
As a workaround you should use those SWPR and logical deletion transaction but no client will want that. There is aflaw in design which should have a bad reputation on the Wflow expert who developed it. Think on a broader aspect and do not try to teach me about production environment I think I know better than you about production environment.
Thanks
Arghadip
Message was edited by:
Arghadip Kar -
Adf security misbehaving in production environment
Hi all,
I am using jdev 11.1.2.2 and weblogic 10.3.6
I have implemented adf security from based authentication in my web application and i have used sql authenticator for authentication.
In my integrated WLS everything works fine . but in the production WLS what is happening is when the user access a Protected Page without login it navigates to the protected page instead of navigating him to the login page. In the integrated WLS this happens normally .
Has anyone faced this issue before ? What can be wrong ?
I have added my web.xml
<?xml version = '1.0' encoding = 'windows-1252'?>
<web-app xmlns="http://java.sun.com/xml/ns/javaee" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd"
version="2.5">
<context-param>
<param-name>javax.faces.STATE_SAVING_METHOD</param-name>
<param-value>client</param-value>
</context-param>
<context-param>
<param-name>javax.faces.PARTIAL_STATE_SAVING</param-name>
<param-value>false</param-value>
</context-param>
<session-config>
<session-timeout>5</session-timeout>
</session-config>
<context-param>
<description>If this parameter is true, there will be an automatic check of the modification date of your JSPs, and saved state will be discarded when JSP's change. It will also automatically check if your skinning css files have changed without you having to restart the server. This makes development easier, but adds overhead. For this reason this parameter should be set to false when your application is deployed.</description>
<param-name>org.apache.myfaces.trinidad.CHECK_FILE_MODIFICATION</param-name>
<param-value>false</param-value>
</context-param>
<context-param>
<param-name>oracle.adf.view.rich.SUPPRESS_IDS</param-name>
<param-value>auto</param-value>
</context-param>
<context-param>
<description>Whether the 'Generated by...' comment at the bottom of ADF Faces HTML pages should contain version number information.</description>
<param-name>oracle.adf.view.rich.versionString.HIDDEN</param-name>
<param-value>false</param-value>
</context-param>
<context-param>
<description>Security precaution to prevent clickjacking: bust frames if the ancestor window domain(protocol, host, and port) and the frame domain are different. Another options for this parameter are always and never.</description>
<param-name>org.apache.myfaces.trinidad.security.FRAME_BUSTING</param-name>
<param-value>differentOrigin</param-value>
</context-param>
<context-param>
<param-name>javax.faces.FACELETS_VIEW_MAPPINGS</param-name>
<param-value>*.jsf;*.xhtml</param-value>
</context-param>
<context-param>
<param-name>javax.faces.FACELETS_SKIP_XML_INSTRUCTIONS</param-name>
<param-value>true</param-value>
</context-param>
<context-param>
<param-name>javax.faces.FACELETS_SKIP_COMMENTS</param-name>
<param-value>true</param-value>
</context-param>
<context-param>
<param-name>javax.faces.FACELETS_DECORATORS</param-name>
<param-value>oracle.adfinternal.view.faces.facelets.rich.AdfTagDecorator</param-value>
</context-param>
<context-param>
<param-name>javax.faces.FACELETS_RESOURCE_RESOLVER</param-name>
<param-value>oracle.adfinternal.view.faces.facelets.rich.AdfFaceletsResourceResolver</param-value>
</context-param>
<filter>
<filter-name>JpsFilter</filter-name>
<filter-class>oracle.security.jps.ee.http.JpsFilter</filter-class>
<init-param>
<param-name>enable.anonymous</param-name>
<param-value>true</param-value>
</init-param>
<init-param>
<param-name>remove.anonymous.role</param-name>
<param-value>false</param-value>
</init-param>
</filter>
<filter>
<filter-name>trinidad</filter-name>
<filter-class>org.apache.myfaces.trinidad.webapp.TrinidadFilter</filter-class>
</filter>
<filter>
<filter-name>ADFLibraryFilter</filter-name>
<filter-class>oracle.adf.library.webapp.LibraryFilter</filter-class>
</filter>
<filter>
<filter-name>adfBindings</filter-name>
<filter-class>oracle.adf.model.servlet.ADFBindingFilter</filter-class>
</filter>
<filter-mapping>
<filter-name>JpsFilter</filter-name>
<url-pattern>/*</url-pattern>
<dispatcher>FORWARD</dispatcher>
<dispatcher>REQUEST</dispatcher>
<dispatcher>INCLUDE</dispatcher>
</filter-mapping>
<filter-mapping>
<filter-name>trinidad</filter-name>
<servlet-name>Faces Servlet</servlet-name>
<dispatcher>FORWARD</dispatcher>
<dispatcher>REQUEST</dispatcher>
<dispatcher>ERROR</dispatcher>
</filter-mapping>
<filter-mapping>
<filter-name>ADFLibraryFilter</filter-name>
<url-pattern>/*</url-pattern>
<dispatcher>FORWARD</dispatcher>
<dispatcher>REQUEST</dispatcher>
</filter-mapping>
<filter-mapping>
<filter-name>adfBindings</filter-name>
<servlet-name>Faces Servlet</servlet-name>
<dispatcher>FORWARD</dispatcher>
<dispatcher>REQUEST</dispatcher>
</filter-mapping>
<filter-mapping>
<filter-name>adfBindings</filter-name>
<servlet-name>adfAuthentication</servlet-name>
<dispatcher>FORWARD</dispatcher>
<dispatcher>REQUEST</dispatcher>
</filter-mapping>
<listener>
<listener-class>oracle.adf.mbean.share.connection.ADFConnectionLifeCycleCallBack</listener-class>
</listener>
<listener>
<listener-class>oracle.adf.mbean.share.config.ADFConfigLifeCycleCallBack</listener-class>
</listener>
<servlet>
<servlet-name>Faces Servlet</servlet-name>
<servlet-class>javax.faces.webapp.FacesServlet</servlet-class>
<load-on-startup>1</load-on-startup>
</servlet>
<servlet>
<servlet-name>resources</servlet-name>
<servlet-class>org.apache.myfaces.trinidad.webapp.ResourceServlet</servlet-class>
</servlet>
<servlet>
<servlet-name>BIGRAPHSERVLET</servlet-name>
<servlet-class>oracle.adf.view.faces.bi.webapp.GraphServlet</servlet-class>
</servlet>
<servlet>
<servlet-name>BIGAUGESERVLET</servlet-name>
<servlet-class>oracle.adf.view.faces.bi.webapp.GaugeServlet</servlet-class>
</servlet>
<servlet>
<servlet-name>MapProxyServlet</servlet-name>
<servlet-class>oracle.adf.view.faces.bi.webapp.MapProxyServlet</servlet-class>
</servlet>
<servlet>
<servlet-name>adflibResources</servlet-name>
<servlet-class>oracle.adf.library.webapp.ResourceServlet</servlet-class>
</servlet>
<servlet>
<servlet-name>adfAuthentication</servlet-name>
<servlet-class>oracle.adf.share.security.authentication.AuthenticationServlet</servlet-class>
<load-on-startup>1</load-on-startup>
</servlet>
<servlet-mapping>
<servlet-name>Faces Servlet</servlet-name>
<url-pattern>/faces/*</url-pattern>
</servlet-mapping>
<servlet-mapping>
<servlet-name>resources</servlet-name>
<url-pattern>/adf/*</url-pattern>
</servlet-mapping>
<servlet-mapping>
<servlet-name>resources</servlet-name>
<url-pattern>/afr/*</url-pattern>
</servlet-mapping>
<servlet-mapping>
<servlet-name>BIGRAPHSERVLET</servlet-name>
<url-pattern>/servlet/GraphServlet/*</url-pattern>
</servlet-mapping>
<servlet-mapping>
<servlet-name>BIGAUGESERVLET</servlet-name>
<url-pattern>/servlet/GaugeServlet/*</url-pattern>
</servlet-mapping>
<servlet-mapping>
<servlet-name>MapProxyServlet</servlet-name>
<url-pattern>/mapproxy/*</url-pattern>
</servlet-mapping>
<servlet-mapping>
<servlet-name>resources</servlet-name>
<url-pattern>/bi/*</url-pattern>
</servlet-mapping>
<servlet-mapping>
<servlet-name>adflibResources</servlet-name>
<url-pattern>/adflib/*</url-pattern>
</servlet-mapping>
<servlet-mapping>
<servlet-name>adfAuthentication</servlet-name>
<url-pattern>/adfAuthentication</url-pattern>
</servlet-mapping>
<mime-mapping>
<extension>swf</extension>
<mime-type>application/x-shockwave-flash</mime-type>
</mime-mapping>
<mime-mapping>
<extension>amf</extension>
<mime-type>application/x-amf</mime-type>
</mime-mapping>
<security-constraint>
<web-resource-collection>
<web-resource-name>Allowed ADF Resources</web-resource-name>
<url-pattern>/adf/*</url-pattern>
<url-pattern>/afr/*</url-pattern>
<url-pattern>/bi/*</url-pattern>
</web-resource-collection>
</security-constraint>
<security-constraint>
<web-resource-collection>
<web-resource-name>adfAuthentication</web-resource-name>
<url-pattern>/adfAuthentication</url-pattern>
</web-resource-collection>
<auth-constraint>
<role-name>valid-users</role-name>
</auth-constraint>
</security-constraint>
<login-config>
<auth-method>FORM</auth-method>
<form-login-config>
<form-login-page>/faces/login</form-login-page>
<form-error-page>/faces/login</form-error-page>
</form-login-config>
</login-config>
<security-role>
<role-name>valid-users</role-name>
</security-role>
</web-app>Thanks,
RakeshHi Rakesh,
Make sure you have migrated the policy store to the production server. When Weblogic Server is running in production mode, automatic credential overwrite is not allowed. From the developer's guide:
When the target server is configured for production mode, you typically handle the migration task outside of JDeveloper using tools like Oracle Enterprise Manager. For details about using tools outside of JDeveloper to migrate the policy store to the domain-level in a production environment, see the Oracle Containers for J2EE Security Guide. Note that Oracle WebLogic Server running in production mode does not support the overwriting of system credentials under any circumstances.http://docs.oracle.com/cd/E26098_01/web.1112/e16182/adding_security.htm#CDDGFDFH
HTH,
Joonas -
Functions (back & stop ) are not working in production environment
Hi All ...
Some functions (back and stop) are not working in production environment, where as those functions are working in QA testing environement..please answer.. what could be the reasons for not working some functions
in production environement.
Regards,,
KrishnaHi,
Based on the error message, it is related to the item in your production environment, then the workflow will occur exception.
As your workflow worked perfectly in your other environments, I suggest you can check if some item has is valid in the document library.
You can try to use Fiddler to track the web request and find in which item the workflow occurs exception.
Here are some detailed articles for your reference:
https://msdn.microsoft.com/en-us/library/vstudio/ee231573(v=vs.100).aspx
http://www.andrewconnell.com/blog/SP2013-Workflow-Advanced-Workflow-Debugging-with-Fiddler
Thanks
Best Regards
Jerry Guo
TechNet Community Support -
Urgent - Crash/hang in WL impacting production environment
Sunil,
I think your problems could be hardware related. Make sure there is enough memory on the box on which the server is running. Do you ever restart the server. Clean up JSP class files in the myserver directory. Make sure you are not leaving any open connections to the database. These are just a few things you shuould verify .
Vibhu S
"Sunil Potti" <[email protected]> wrote:
Hi,
This is regarding some serious WL crashes that we are seeing in our
production environment. Before I detail out the problem, here is our
operation environment -
WL 4.5.1 SP 13 on Windows NT 4.0 SP 5
Execute thread count = 30
JDBC connection pool = 25 min, 100 max.
JDBC driver = oracle thin drivers.
JDK 1.1.7B
Oracle 8i database
Of late, we see that the server just randomly stops serving out requests,
for e.g. the server will not serve out JSP files, will not let any users
connect, etc. We found absolutely nothing in the weblogic.log files for most
of the crashes. The server just randomly stops at a point. Earlier we had a
problem with the ExecuteThreads being used up and the server hanging because
of lack of excecute threads and that problem was fixed and we haven't seen
that sort of problem ever since.
We just saw a crash where we had the following error sometime before the
crash -
weblogic.common.ResourceException: Unable to shrink pool oracle from 26 to
25
at java.lang.Throwable.<init>(Compiled Code)
at java.lang.Exception.<init>(Compiled Code)
at weblogic.common.ResourceException.<init>(ResourceException.java:18)
at weblogic.common.internal.ResourceAllocator.shrink_internal(Compiled
Code)
at weblogic.common.internal.ResourceAllocator.trigger(Compiled Code)
at weblogic.time.common.internal.ScheduledTrigger.executeLocally(Compiled
Code)
at weblogic.time.common.internal.ScheduledTrigger.execute(Compiled Code)
at weblogic.time.server.ScheduledTrigger.execute(Compiled Code)
at weblogic.t3.srvr.ExecuteThread.run(Compiled Code)
After that we took a thread dump (Ctrl-Break) which is attached with this
posting.
We have also seen that the server shuts itself down abruptly, i.e. the JVM
exits. Has anybody seen this kind of behaviour with WL 4.5.1/SP 13?
To give a brief background of our startup process -
We have quite a few startup classes (10) that start off when WL starts up.
Some of these startups are periodically running processes. Some of them
create a pool of threads (no. being between 40 -50) that are used when
required. Has anybody seen any problem with this kind of scenario.
This is an urgent production issue. Any help would be gratefully
appreciated.
Thanks in advance.
- SG
begin 666 dump.txt
M1G5L;"!T:')E860@9'5M<#H-"B @(" B06-T:79E475E=654:')E860W.30P
M,C$B("A4240Z,'@Q.3(V9CED."P@<WES7W1H<F5A9%]T.C!X8C<R.&,P+"!7
M:6XS,DE$.C!X,39D+"!S=&%T93I#5RD@<')I;STU#0H):F%V82YL86YG+D]B
M:F5C="YW86ET*$-O;7!I;&5D($-O9&4I#0H)<WEN8RYS97)V97(N;6]N:71O
M<BY!8W1I=F51=65U94EN9F\N9V5T*$-O;7!I;&5D($-O9&4I#0H)<WEN8RYS
M97)V97(N;6]N:71O<BY!8W1I=F51=65U951H<F5A9"YR=6XH0V]M<&EL960@
M0V]D92D-"B @(" B06-T:79E475E=654:')E860W.3,V,C$B("A4240Z,'@Q
M,V%E,S-E."P@<WES7W1H<F5A9%]T.C!X8C<Q-SDP+"!7:6XS,DE$.C!X,3(Q
M+"!S=&%T93I#5RD@<')I;STU#0H):F%V82YL86YG+D]B:F5C="YW86ET*$-O
M;7!I;&5D($-O9&4I#0H)<WEN8RYS97)V97(N;6]N:71O<BY!8W1I=F51=65U
M94EN9F\N9V5T*$-O;7!I;&5D($-O9&4I#0H)<WEN8RYS97)V97(N;6]N:71O
M<BY!8W1I=F51=65U951H<F5A9"YR=6XH0V]M<&EL960@0V]D92D-"B @(" B
M06-T:79E475E=654:')E860W.3,R,C$B("A4240Z,'@Q-60R.3@P,"P@<WES
M7W1H<F5A9%]T.C!X8C<Q9&4P+"!7:6XS,DE$.C!X,C$Q+"!S=&%T93I#5RD@
M<')I;STU#0H):F%V82YL86YG+D]B:F5C="YW86ET*$-O;7!I;&5D($-O9&4I
M#0H)<WEN8RYS97)V97(N;6]N:71O<BY!8W1I=F51=65U94EN9F\N9V5T*$-O
M;7!I;&5D($-O9&4I#0H)<WEN8RYS97)V97(N;6]N:71O<BY!8W1I=F51=65U
M951H<F5A9"YR=6XH0V]M<&EL960@0V]D92D-"B @(" B06-T:79E475E=654
M:')E860W.3(X,C$B("A4240Z,'@Q.#4W,F1D."P@<WES7W1H<F5A9%]T.C!X
M8C<Q,C P+"!7:6XS,DE$.C!X,69C+"!S=&%T93I#5RD@<')I;STU#0H):F%V
M82YL86YG+D]B:F5C="YW86ET*$-O;7!I;&5D($-O9&4I#0H)<WEN8RYS97)V
M97(N;6]N:71O<BY!8W1I=F51=65U94EN9F\N9V5T*$-O;7!I;&5D($-O9&4I
M#0H)<WEN8RYS97)V97(N;6]N:71O<BY!8W1I=F51=65U951H<F5A9"YR=6XH
M0V]M<&EL960@0V]D92D-"B @(" B06-T:79E475E=654:')E860W.3(T,C$B
M("A4240Z,'@Q-#%D.&%C."P@<WES7W1H<F5A9%]T.C!X8C<Q,F0P+"!7:6XS
M,DE$.C!X,60V+"!S=&%T93I#5RD@<')I;STU#0H):F%V82YL86YG+D]B:F5C
M="YW86ET*$-O;7!I;&5D($-O9&4I#0H)<WEN8RYS97)V97(N;6]N:71O<BY!
M8W1I=F51=65U94EN9F\N9V5T*$-O;7!I;&5D($-O9&4I#0H)<WEN8RYS97)V
M97(N;6]N:71O<BY!8W1I=F51=65U951H<F5A9"YR=6XH0V]M<&EL960@0V]D
M92D-"B @(" B06-T:79E475E=654:')E860W.3(P-#$B("A4240Z,'@Q-CDR
M-#,W."P@<WES7W1H<F5A9%]T.C!X8C<Q.6(P+"!7:6XS,DE$.C!X,60W+"!S
M=&%T93I#5RD@<')I;STU#0H):F%V82YL86YG+D]B:F5C="YW86ET*$-O;7!I
M;&5D($-O9&4I#0H)<WEN8RYS97)V97(N;6]N:71O<BY!8W1I=F51=65U94EN
M9F\N9V5T*$-O;7!I;&5D($-O9&4I#0H)<WEN8RYS97)V97(N;6]N:71O<BY!
M8W1I=F51=65U951H<F5A9"YR=6XH0V]M<&EL960@0V]D92D-"B @(" B06-T
M:79E475E=654:')E860W.3(P,C$B("A4240Z,'@Q-F0Q8F(X,"P@<WES7W1H
M<F5A9%]T.C!X8C<Q-S,P+"!7:6XS,DE$.C!X,38Q+"!S=&%T93I#5RD@<')I
M;STU#0H):F%V82YL86YG+D]B:F5C="YW86ET*$-O;7!I;&5D($-O9&4I#0H)
M<WEN8RYS97)V97(N;6]N:71O<BY!8W1I=F51=65U94EN9F\N9V5T*$-O;7!I
M;&5D($-O9&4I#0H)<WEN8RYS97)V97(N;6]N:71O<BY!8W1I=F51=65U951H
M<F5A9"YR=6XH0V]M<&EL960@0V]D92D-"B @(" B06-T:79E475E=654:')E
M860W.3$V,C$B("A4240Z,'@Q-V(Q-C(V."P@<WES7W1H<F5A9%]T.C!X8C<Q
M,S@P+"!7:6XS,DE$.C!X,C-E+"!S=&%T93I#5RD@<')I;STU#0H):F%V82YL
M86YG+D]B:F5C="YW86ET*$-O;7!I;&5D($-O9&4I#0H)<WEN8RYS97)V97(N
M;6]N:71O<BY!8W1I=F51=65U94EN9F\N9V5T*$-O;7!I;&5D($-O9&4I#0H)
M<WEN8RYS97)V97(N;6]N:71O<BY!8W1I=F51=65U951H<F5A9"YR=6XH0V]M
M<&EL960@0V]D92D-"B @(" B06-T:79E475E=654:')E860W.3$R,C$B("A4
M240Z,'@Q-3=B-S4Q,"P@<WES7W1H<F5A9%]T.C!X8C<Q968P+"!7:6XS,DE$
M.C!X,6-D+"!S=&%T93I#5RD@<')I;STU#0H):F%V82YL86YG+D]B:F5C="YW
M86ET*$-O;7!I;&5D($-O9&4I#0H)<WEN8RYS97)V97(N;6]N:71O<BY!8W1I
M=F51=65U94EN9F\N9V5T*$-O;7!I;&5D($-O9&4I#0H)<WEN8RYS97)V97(N
M;6]N:71O<BY!8W1I=F51=65U951H<F5A9"YR=6XH0V]M<&EL960@0V]D92D-
M"B @(" B06-T:79E475E=654:')E860W.3 X,C$B("A4240Z,'@Q,3%F-C$W
M."P@<WES7W1H<F5A9%]T.C!X8C<Q,3(P+"!7:6XS,DE$.C!X,6)B+"!S=&%T
M93I#5RD@<')I;STU#0H):F%V82YL86YG+D]B:F5C="YW86ET*$-O;7!I;&5D
M($-O9&4I#0H)<WEN8RYS97)V97(N;6]N:71O<BY!8W1I=F51=65U94EN9F\N
M9V5T*$-O;7!I;&5D($-O9&4I#0H)<WEN8RYS97)V97(N;6]N:71O<BY!8W1I
M=F51=65U951H<F5A9"YR=6XH0V]M<&EL960@0V]D92D-"B @(" B06-T:79E
M475E=654:')E860W.3 T,C$B("A4240Z,'@Q,S9D-SEA,"P@<WES7W1H<F5A
M9%]T.C!X8C<P,S$P+"!7:6XS,DE$.C!X,34Q+"!S=&%T93I#5RD@<')I;STU
M#0H):F%V82YL86YG+D]B:F5C="YW86ET*$-O;7!I;&5D($-O9&4I#0H)<WEN
M8RYS97)V97(N;6]N:71O<BY!8W1I=F51=65U94EN9F\N9V5T*$-O;7!I;&5D
M($-O9&4I#0H)<WEN8RYS97)V97(N;6]N:71O<BY!8W1I=F51=65U951H<F5A
M9"YR=6XH0V]M<&EL960@0V]D92D-"B @(" B06-T:79E475E=654:')E860W
M.3 P,C$B("A4240Z,'@Q-C=E8S$X."P@<WES7W1H<F5A9%]T.C!X8C9C.3$P
M+"!7:6XS,DE$.C!X,C W+"!S=&%T93I#5RD@<')I;STU#0H):F%V82YL86YG
M+D]B:F5C="YW86ET*$-O;7!I;&5D($-O9&4I#0H)<WEN8RYS97)V97(N;6]N
M:71O<BY!8W1I=F51=65U94EN9F\N9V5T*$-O;7!I;&5D($-O9&4I#0H)<WEN
M8RYS97)V97(N;6]N:71O<BY!8W1I=F51=65U951H<F5A9"YR=6XH0V]M<&EL
M960@0V]D92D-"B @(" B06-T:79E475E=654:')E860W.#DV,C$B("A4240Z
M,'@Q.3@U,C)B,"P@<WES7W1H<F5A9%]T.C!X8C9C.3<P+"!7:6XS,DE$.C!X
M,3=A+"!S=&%T93I#5RD@<')I;STU#0H):F%V82YL86YG+D]B:F5C="YW86ET
M*$-O;7!I;&5D($-O9&4I#0H)<WEN8RYS97)V97(N;6]N:71O<BY!8W1I=F51
M=65U94EN9F\N9V5T*$-O;7!I;&5D($-O9&4I#0H)<WEN8RYS97)V97(N;6]N
M:71O<BY!8W1I=F51=65U951H<F5A9"YR=6XH0V]M<&EL960@0V]D92D-"B @
M(" B06-T:79E475E=654:')E860W.#DR,C$B("A4240Z,'@Q,CDT-#EB."P@
M<WES7W1H<F5A9%]T.C!X8C9C,68P+"!7:6XS,DE$.C!X964L('-T871E.D-7
M*2!P<FEO/34-"@EJ879A+FQA;F<N3V)J96-T+G=A:70H0V]M<&EL960@0V]D
M92D-"@ES>6YC+G-E<G9E<BYM;VYI=&]R+D%C=&EV95%U975E26YF;RYG970H
M0V]M<&EL960@0V]D92D-"@ES>6YC+G-E<G9E<BYM;VYI=&]R+D%C=&EV95%U
M975E5&AR96%D+G)U;BA#;VUP:6QE9"!#;V1E*0T*(" @(")!8W1I=F51=65U
M951H<F5A9#<X.#@R,2(@*%1)1#HP>#$S8S$Y,3DP+"!S>7-?=&AR96%D7W0Z
M,'AB-F,T9# L(%=I;C,R240Z,'@Q.#@L('-T871E.D-7*2!P<FEO/34-"@EJ
M879A+FQA;F<N3V)J96-T+G=A:70H0V]M<&EL960@0V]D92D-"@ES>6YC+G-E
M<G9E<BYM;VYI=&]R+D%C=&EV95%U975E26YF;RYG970H0V]M<&EL960@0V]D
M92D-"@ES>6YC+G-E<G9E<BYM;VYI=&]R+D%C=&EV95%U975E5&AR96%D+G)U
M;BA#;VUP:6QE9"!#;V1E*0T*(" @(")!8W1I=F51=65U951H<F5A9#<X.#0R
M,2(@*%1)1#HP>#$W,&0P,V,X+"!S>7-?=&AR96%D7W0Z,'AB-F,U.# L(%=I
M;C,R240Z,'@Q9#$L('-T871E.D-7*2!P<FEO/34-"@EJ879A+FQA;F<N3V)J
M96-T+G=A:70H0V]M<&EL960@0V]D92D-"@ES>6YC+G-E<G9E<BYM;VYI=&]R
M+D%C=&EV95%U975E26YF;RYG970H0V]M<&EL960@0V]D92D-"@ES>6YC+G-E
M<G9E<BYM;VYI=&]R+D%C=&EV95%U975E5&AR96%D+G)U;BA#;VUP:6QE9"!#
M;V1E*0T*(" @(")!8W1I=F51=65U951H<F5A9#<X.# R,2(@*%1)1#HP>#$X
M.3EA-F$X+"!S>7-?=&AR96%D7W0Z,'AB-C1B93 L(%=I;C,R240Z,'@R,3@L
M('-T871E.D-7*2!P<FEO/34-"@EJ879A+FQA;F<N3V)J96-T+G=A:70H0V]M
M<&EL960@0V]D92D-"@ES>6YC+G-E<G9E<BYM;VYI=&]R+D%C=&EV95%U975E
M26YF;RYG970H0V]M<&EL960@0V]D92D-"@ES>6YC+G-E<G9E<BYM;VYI=&]R
M+D%C=&EV95%U975E5&AR96%D+G)U;BA#;VUP:6QE9"!#;V1E*0T*(" @(")!
M8W1I=F51=65U951H<F5A9#<X-S8R,2(@*%1)1#HP>#$X-3EB9&(P+"!S>7-?
M=&AR96%D7W0Z,'AB-C$X.# L(%=I;C,R240Z,'@X92P@<W1A=&4Z0U<I('!R
M:6\]-0T*"6IA=F$N;&%N9RY/8FIE8W0N=V%I="A#;VUP:6QE9"!#;V1E*0T*
M"7-Y;F,N<V5R=F5R+FUO;FET;W(N06-T:79E475E=65);F9O+F=E="A#;VUP
M:6QE9"!#;V1E*0T*"7-Y;F,N<V5R=F5R+FUO;FET;W(N06-T:79E475E=654
M:')E860N<G5N*$-O;7!I;&5D($-O9&4I#0H@(" @(D%C=&EV95%U975E5&AR
M96%D-S@W,C(Q(B H5$E$.C!X,30W-#ED8S@L('-Y<U]T:')E861?=#HP>&(V
M,6-D,"P@5VEN,S))1#HP>#$V."P@<W1A=&4Z0U<I('!R:6\]-0T*"6IA=F$N
M;&%N9RY/8FIE8W0N=V%I="A#;VUP:6QE9"!#;V1E*0T*"7-Y;F,N<V5R=F5R
M+FUO;FET;W(N06-T:79E475E=65);F9O+F=E="A#;VUP:6QE9"!#;V1E*0T*
M"7-Y;F,N<V5R=F5R+FUO;FET;W(N06-T:79E475E=654:')E860N<G5N*$-O
M;7!I;&5D($-O9&4I#0H@(" @(D%C=&EV95%U975E5&AR96%D-S@V.#(Q(B H
M5$E$.C!X,3DY86(X-S@L('-Y<U]T:')E861?=#HP>&(V-C<W,"P@5VEN,S))
M1#HP>#$S-2P@<W1A=&4Z0U<I('!R:6\]-0T*"6IA=F$N;&%N9RY/8FIE8W0N
M=V%I="A#;VUP:6QE9"!#;V1E*0T*"7-Y;F,N<V5R=F5R+FUO;FET;W(N06-T
M:79E475E=65);F9O+F=E="A#;VUP:6QE9"!#;V1E*0T*"7-Y;F,N<V5R=F5R
M+FUO;FET;W(N06-T:79E475E=654:')E860N<G5N*$-O;7!I;&5D($-O9&4I
M#0H@(" @(D%C=&EV95%U975E5&AR96%D-S@V-#(Q(B H5$E$.C!X,38P,S1E
M.#@L('-Y<U]T:')E861?=#HP>&(V-C4R,"P@5VEN,S))1#HP>&,U+"!S=&%T
M93I#5RD@<')I;STU#0H):F%V82YL86YG+D]B:F5C="YW86ET*$-O;7!I;&5D
M($-O9&4I#0H)<WEN8RYS97)V97(N;6]N:71O<BY!8W1I=F51=65U94EN9F\N
M9V5T*$-O;7!I;&5D($-O9&4I#0H)<WEN8RYS97)V97(N;6]N:71O<BY!8W1I
M=F51=65U951H<F5A9"YR=6XH0V]M<&EL960@0V]D92D-"B @(" B06-T:79E
M475E=654:')E860W.#8P,C$B("A4240Z,'@Q-S5E83(V."P@<WES7W1H<F5A
M9%]T.C!X8C8V-#0P+"!7:6XS,DE$.C!X,6(Q+"!S=&%T93I#5RD@<')I;STU
M#0H):F%V82YL86YG+D]B:F5C="YW86ET*$-O;7!I;&5D($-O9&4I#0H)<WEN
M8RYS97)V97(N;6]N:71O<BY!8W1I=F51=65U94EN9F\N9V5T*$-O;7!I;&5D
M($-O9&4I#0H)<WEN8RYS97)V97(N;6]N:71O<BY!8W1I=F51=65U951H<F5A
M9"YR=6XH0V]M<&EL960@0V]D92D-"B @(" B06-T:79E475E=654:')E860W
M.#0T,C$B("A4240Z,'@Q.#AF8C,S."P@<WES7W1H<F5A9%]T.C!X8C1E9&,P
M+"!7:6XS,DE$.C!X,3,X+"!S=&%T93I#5RD@<')I;STU#0H):F%V82YL86YG
M+D]B:F5C="YW86ET*$-O;7!I;&5D($-O9&4I#0H)<WEN8RYS97)V97(N;6]N
M:71O<BY!8W1I=F51=65U94EN9F\N9V5T*$-O;7!I;&5D($-O9&4I#0H)<WEN
M8RYS97)V97(N;6]N:71O<BY!8W1I=F51=65U951H<F5A9"YR=6XH0V]M<&EL
M960@0V]D92D-"B @(" B06-T:79E475E=654:')E860W.#,V,C$B("A4240Z
M,'@Q-F4X9C(R."P@<WES7W1H<F5A9%]T.C!X8C1D-S0P+"!7:6XS,DE$.C!X
M,6(V+"!S=&%T93I#5RD@<')I;STU#0H):F%V82YL86YG+D]B:F5C="YW86ET
M*$-O;7!I;&5D($-O9&4I#0H)<WEN8RYS97)V97(N;6]N:71O<BY!8W1I=F51
M=65U94EN9F\N9V5T*$-O;7!I;&5D($-O9&4I#0H)<WEN8RYS97)V97(N;6]N
M:71O<BY!8W1I=F51=65U951H<F5A9"YR=6XH0V]M<&EL960@0V]D92D-"B @
M(" B06-T:79E475E=654:')E860W.#,R-#$B("A4240Z,'@Q-S<W-3DW,"P@
M<WES7W1H<F5A9%]T.C!X8C1D830P+"!7:6XS,DE$.C!X,C,V+"!S=&%T93I#
M5RD@<')I;STU#0H):F%V82YL86YG+D]B:F5C="YW86ET*$-O;7!I;&5D($-O
M9&4I#0H)<WEN8RYS97)V97(N;6]N:71O<BY!8W1I=F51=65U94EN9F\N9V5T
M*$-O;7!I;&5D($-O9&4I#0H)<WEN8RYS97)V97(N;6]N:71O<BY!8W1I=F51
M=65U951H<F5A9"YR=6XH0V]M<&EL960@0V]D92D-"B @(" B06-T:79E475E
M=654:')E860W.#(X,C$B("A4240Z,'@Q.#<Q8SDQ,"P@<WES7W1H<F5A9%]T
M.C!X8C1D9C$P+"!7:6XS,DE$.C!X,3)F+"!S=&%T93I#5RD@<')I;STU#0H)
M:F%V82YL86YG+D]B:F5C="YW86ET*$-O;7!I;&5D($-O9&4I#0H)<WEN8RYS
M97)V97(N;6]N:71O<BY!8W1I=F51=65U94EN9F\N9V5T*$-O;7!I;&5D($-O
M9&4I#0H)<WEN8RYS97)V97(N;6]N:71O<BY!8W1I=F51=65U951H<F5A9"YR
M=6XH0V]M<&EL960@0V]D92D-"B @(" B06-T:79E475E=654:')E860W.#$V
M,C$B("A4240Z,'@Q.#=C.&5A,"P@<WES7W1H<F5A9%]T.C!X8C5B.#8P+"!7
M:6XS,DE$.C!X,3EE+"!S=&%T93I#5RD@<')I;STU#0H):F%V82YL86YG+D]B
M:F5C="YW86ET*$-O;7!I;&5D($-O9&4I#0H)<WEN8RYS97)V97(N;6]N:71O
M<BY!8W1I=F51=65U94EN9F\N9V5T*$-O;7!I;&5D($-O9&4I#0H)<WEN8RYS
M97)V97(N;6]N:71O<BY!8W1I=F51=65U951H<F5A9"YR=6XH0V]M<&EL960@
M0V]D92D-"B @(" B06-T:79E475E=654:')E860W.# X,C$B("A4240Z,'@Q
M.38P9&9C."P@<WES7W1H<F5A9%]T.C!X8C5A93(P+"!7:6XS,DE$.C!X,3)C
M+"!S=&%T93I#5RD@<')I;STU#0H):F%V82YL86YG+D]B:F5C="YW86ET*$-O
M;7!I;&5D($-O9&4I#0H)<WEN8RYS97)V97(N;6]N:71O<BY!8W1I=F51=65U
M94EN9F\N9V5T*$-O;7!I;&5D($-O9&4I#0H)<WEN8RYS97)V97(N;6]N:71O
M<BY!8W1I=F51=65U951H<F5A9"YR=6XH0V]M<&EL960@0V]D92D-"B @(" B
M06-T:79E475E=654:')E860W.# T,C$B("A4240Z,'@Q,S)C,F4X,"P@<WES
M7W1H<F5A9%]T.C!X8C5A9C8P+"!7:6XS,DE$.C!X9C(L('-T871E.D-7*2!P
M<FEO/34-"@EJ879A+FQA;F<N3V)J96-T+G=A:70H0V]M<&EL960@0V]D92D-
M"@ES>6YC+G-E<G9E<BYM;VYI=&]R+D%C=&EV95%U975E26YF;RYG970H0V]M
M<&EL960@0V]D92D-"@ES>6YC+G-E<G9E<BYM;VYI=&]R+D%C=&EV95%U975E
M5&AR96%D+G)U;BA#;VUP:6QE9"!#;V1E*0T*(" @(")!8W1I=F51=65U951H
M<F5A9#<X,# R,2(@*%1)1#HP>#$V,F8Q8F8X+"!S>7-?=&AR96%D7W0Z,'AB
M-6$X,# L(%=I;C,R240Z,'@Q8S(L('-T871E.D-7*2!P<FEO/34-"@EJ879A
M+FQA;F<N3V)J96-T+G=A:70H0V]M<&EL960@0V]D92D-"@ES>6YC+G-E<G9E
M<BYM;VYI=&]R+D%C=&EV95%U975E26YF;RYG970H0V]M<&EL960@0V]D92D-
M"@ES>6YC+G-E<G9E<BYM;VYI=&]R+D%C=&EV95%U975E5&AR96%D+G)U;BA#
M;VUP:6QE9"!#;V1E*0T*(" @(")!8W1I=F51=65U951H<F5A9#<W.38T,2(@
M*%1)1#HP>#$X-69D9# X+"!S>7-?=&AR96%D7W0Z,'AB-6$W,C L(%=I;C,R
M240Z,'@R,38L('-T871E.D-7*2!P<FEO/34-"@EJ879A+FQA;F<N3V)J96-T
M+G=A:70H0V]M<&EL960@0V]D92D-"@ES>6YC+G-E<G9E<BYM;VYI=&]R+D%C
M=&EV95%U975E26YF;RYG970H0V]M<&EL960@0V]D92D-"@ES>6YC+G-E<G9E
M<BYM;VYI=&]R+D%C=&EV95%U975E5&AR96%D+G)U;BA#;VUP:6QE9"!#;V1E
M*0T*(" @(")!8W1I=F51=65U951H<F5A9#<W.38R,2(@*%1)1#HP>#$X.#<V
M.#8P+"!S>7-?=&AR96%D7W0Z,'AB-6%B9C L(%=I;C,R240Z,'@R,F,L('-T
M871E.D-7*2!P<FEO/34-"@EJ879A+FQA;F<N3V)J96-T+G=A:70H0V]M<&EL
M960@0V]D92D-"@ES>6YC+G-E<G9E<BYM;VYI=&]R+D%C=&EV95%U975E26YF
M;RYG970H0V]M<&EL960@0V]D92D-"@ES>6YC+G-E<G9E<BYM;VYI=&]R+D%C
M=&EV95%U975E5&AR96%D+G)U;BA#;VUP:6QE9"!#;V1E*0T*(" @(")!8W1I
M=F51=65U951H<F5A9#<W.3(R,2(@*%1)1#HP>#$Y,CDS938X+"!S>7-?=&AR
M96%D7W0Z,'AB-3EA-3 L(%=I;C,R240Z,'@X-RP@<W1A=&4Z0U<I('!R:6\]
M-0T*"6IA=F$N;&%N9RY/8FIE8W0N=V%I="A#;VUP:6QE9"!#;V1E*0T*"7-Y
M;F,N<V5R=F5R+FUO;FET;W(N06-T:79E475E=65);F9O+F=E="A#;VUP:6QE
M9"!#;V1E*0T*"7-Y;F,N<V5R=F5R+FUO;FET;W(N06-T:79E475E=654:')E
M860N<G5N*$-O;7!I;&5D($-O9&4I#0H@(" @(D%C=&EV95%U975E5&AR96%D
M-S<X.#(Q(B H5$E$.C!X,3DV,F0W-#@L('-Y<U]T:')E861?=#HP>&(U.3DW
M,"P@5VEN,S))1#HP>#(U,"P@<W1A=&4Z0U<I('!R:6\]-0T*"6IA=F$N;&%N
M9RY/8FIE8W0N=V%I="A#;VUP:6QE9"!#;V1E*0T*"7-Y;F,N<V5R=F5R+FUO
M;FET;W(N06-T:79E475E=65);F9O+F=E="A#;VUP:6QE9"!#;V1E*0T*"7-Y
M;F,N<V5R=F5R+FUO;FET;W(N06-T:79E475E=654:')E860N<G5N*$-O;7!I
M;&5D($-O9&4I#0H@(" @(D%C=&EV95%U975E5&AR96%D-S<X-#(Q(B H5$E$
M.C!X,3@R8V9E93 L('-Y<U]T:')E861?=#HP>&(U.6)D,"P@5VEN,S))1#HP
M>#$R,"P@<W1A=&4Z0U<I('!R:6\]-0T*"6IA=F$N;&%N9RY/8FIE8W0N=V%I
M="A#;VUP:6QE9"!#;V1E*0T*"7-Y;F,N<V5R=F5R+FUO;FET;W(N06-T:79E
M475E=65);F9O+F=E="A#;VUP:6QE9"!#;V1E*0T*"7-Y;F,N<V5R=F5R+FUO
M;FET;W(N06-T:79E475E=654:')E860N<G5N*$-O;7!I;&5D($-O9&4I#0H@
M(" @(D%C=&EV95%U975E5&AR96%D-S<X,#(Q(B H5$E$.C!X,30Q860T9C L
M('-Y<U]T:')E861?=#HP>&(U-F8U,"P@5VEN,S))1#HP>#(T,BP@<W1A=&4Z
M0U<I('!R:6\]-0T*"6IA=F$N;&%N9RY/8FIE8W0N=V%I="A#;VUP:6QE9"!#
M;V1E*0T*"7-Y;F,N<V5R=F5R+FUO;FET;W(N06-T:79E475E=65);F9O+F=E
M="A#;VUP:6QE9"!#;V1E*0T*"7-Y;F,N<V5R=F5R+FUO;FET;W(N06-T:79E
M475E=654:')E860N<G5N*$-O;7!I;&5D($-O9&4I#0H@(" @(D%C=&EV95%U
M975E5&AR96%D-S<W-C(Q(B H5$E$.C!X,35F.3(P-3 L('-Y<U]T:')E861?
M=#HP>&(U-F,X,"P@5VEN,S))1#HP>#$U,"P@<W1A=&4Z0U<I('!R:6\]-0T*
M"6IA=F$N;&%N9RY/8FIE8W0N=V%I="A#;VUP:6QE9"!#;V1E*0T*"7-Y;F,N
M<V5R=F5R+FUO;FET;W(N06-T:79E475E=65);F9O+F=E="A#;VUP:6QE9"!#
M;V1E*0T*"7-Y;F,N<V5R=F5R+FUO;FET;W(N06-T:79E475E=654:')E860N
M<G5N*$-O;7!I;&5D($-O9&4I#0H@(" @(D%C=&EV95%U975E5&AR96%D-S<V
M.#(Q(B H5$E$.C!X,3)C-#4V,3 L('-Y<U]T:')E861?=#HP>&(U-CDQ,"P@
M5VEN,S))1#HP>#$X-2P@<W1A=&4Z0U<I('!R:6\]-0T*"6IA=F$N;&%N9RY/
M8FIE8W0N=V%I="A#;VUP:6QE9"!#;V1E*0T*"7-Y;F,N<V5R=F5R+FUO;FET
M;W(N06-T:79E475E=65);F9O+F=E="A#;VUP:6QE9"!#;V1E*0T*"7-Y;F,N
M<V5R=F5R+FUO;FET;W(N06-T:79E475E=654:')E860N<G5N*$-O;7!I;&5D
M($-O9&4I#0H@(" @(D%C=&EV95%U975E5&AR96%D-S<V-#(Q(B H5$E$.C!X
M,34P8V8V9# L('-Y<U]T:')E861?=#HP>&(U-C-D,"P@5VEN,S))1#HP>#$W
M9BP@<W1A=&4Z0U<I('!R:6\]-0T*"6IA=F$N;&%N9RY/8FIE8W0N=V%I="A#
M;VUP:6QE9"!#;V1E*0T*"7-Y;F,N<V5R=F5R+FUO;FET;W(N06-T:79E475E
M=65);F9O+F=E="A#;VUP:6QE9"!#;V1E*0T*"7-Y;F,N<V5R=F5R+FUO;FET
M;W(N06-T:79E475E=654:')E860N<G5N*$-O;7!I;&5D($-O9&4I#0H@(" @
M(D%C=&EV95%U975E5&AR96%D-S<V,#(Q(B H5$E$.C!X,34W,C(Q8C@L('-Y
M<U]T:')E861?=#HP>&(U-F4W,"P@5VEN,S))1#HP>&9F+"!S=&%T93I#5RD@
M<')I;STU#0H):F%V82YL86YG+D]B:F5C="YW86ET*$-O;7!I;&5D($-O9&4I
M#0H)<WEN8RYS97)V97(N;6]N:71O<BY!8W1I=F51=65U94EN9F\N9V5T*$-O
M;7!I;&5D($-O9&4I#0H)<WEN8RYS97)V97(N;6]N:71O<BY!8W1I=F51=65U
M951H<F5A9"YR=6XH0V]M<&EL960@0V]D92D-"B @(" B06-T:79E475E=654
M:')E860W-S4V,C$B("A4240Z,'@Q-69A935A."P@<WES7W1H<F5A9%]T.C!X
M8C4U8C<P+"!7:6XS,DE$.C!X.3<L('-T871E.D-7*2!P<FEO/34-"@EJ879A
M+FQA;F<N3V)J96-T+G=A:70H0V]M<&EL960@0V]D92D-"@ES>6YC+G-E<G9E
M<BYM;VYI=&]R+D%C=&EV95%U975E26YF;RYG970H0V]M<&EL960@0V]D92D-
M"@ES>6YC+G-E<G9E<BYM;VYI=&]R+D%C=&EV95%U975E5&AR96%D+G)U;BA#
M;VUP:6QE9"!#;V1E*0T*(" @(")!8W1I=F51=65U951H<F5A9#<W-3(R,2(@
M*%1)1#HP>#$V9F(W8V(X+"!S>7-?=&AR96%D7W0Z,'AB-3,Y-3 L(%=I;C,R
M240Z,'@Q8S0L('-T871E.D-7*2!P<FEO/34-"@EJ879A+FQA;F<N3V)J96-T
M+G=A:70H0V]M<&EL960@0V]D92D-"@ES>6YC+G-E<G9E<BYM;VYI=&]R+D%C
M=&EV95%U975E26YF;RYG970H0V]M<&EL960@0V]D92D-"@ES>6YC+G-E<G9E
M<BYM;VYI=&]R+D%C=&EV95%U975E5&AR96%D+G)U;BA#;VUP:6QE9"!#;V1E
M*0T*(" @(")!8W1I=F51=65U951H<F5A9#<W-#@R,2(@*%1)1#HP>#$Y-3(X
M-C@X+"!S>7-?=&AR96%D7W0Z,'AB-3,Q83 L(%=I;C,R240Z,'@R-&8L('-T
M871E.D-7*2!P<FEO/34-"@EJ879A+FQA;F<N3V)J96-T+G=A:70H0V]M<&EL
M960@0V]D92D-"@ES>6YC+G-E<G9E<BYM;VYI=&]R+D%C=&EV95%U975E26YF
M;RYG970H0V]M<&EL960@0V]D92D-"@ES>6YC+G-E<G9E<BYM;VYI=&]R+D%C
M=&EV95%U975E5&AR96%D+G)U;BA#;VUP:6QE9"!#;V1E*0T*(" @(")!8W1I
M=F51=65U951H<F5A9#<W-#0R,2(@*%1)1#HP>#$T-&8U-SDX+"!S>7-?=&AR
M96%D7W0Z,'AB-3-B9# L(%=I;C,R240Z,'@R,6(L('-T871E.D-7*2!P<FEO
M/34-"@EJ879A+FQA;F<N3V)J96-T+G=A:70H0V]M<&EL960@0V]D92D-"@ES
M>6YC+G-E<G9E<BYM;VYI=&]R+D%C=&EV95%U975E26YF;RYG970H0V]M<&EL
M960@0V]D92D-"@ES>6YC+G-E<G9E<BYM;VYI=&]R+D%C=&EV95%U975E5&AR
M96%D+G)U;BA#;VUP:6QE9"!#;V1E*0T*(" @(")!8W1I=F51=65U951H<F5A
M9#<W-# R,2(@*%1)1#HP>#$U8F,S.6,X+"!S>7-?=&AR96%D7W0Z,'AB-3-D
M,3 L(%=I;C,R240Z,'@Q,C0L('-T871E.D-7*2!P<FEO/34-"@EJ879A+FQA
M;F<N3V)J96-T+G=A:70H0V]M<&EL960@0V]D92D-"@ES>6YC+G-E<G9E<BYM
M;VYI=&]R+D%C=&EV95%U975E26YF;RYG970H0V]M<&EL960@0V]D92D-"@ES
M>6YC+G-E<G9E<BYM;VYI=&]R+D%C=&EV95%U975E5&AR96%D+G)U;BA#;VUP
M:6QE9"!#;V1E*0T*(" @(")!8W1I=F51=65U951H<F5A9#<W,S8R,2(@*%1)
M1#HP>#$Y,C<U,S$P+"!S>7-?=&AR96%D7W0Z,'AB-3,X-S L(%=I;C,R240Z
M,'@R-6$L('-T871E.D-7*2!P<FEO/34-"@EJ879A+FQA;F<N3V)J96-T+G=A
M:70H0V]M<&EL960@0V]D92D-"@ES>6YC+G-E<G9E<BYM;VYI=&]R+D%C=&EV
M95%U975E26YF;RYG970H0V]M<&EL960@0V]D92D-"@ES>6YC+G-E<G9E<BYM
M;VYI=&]R+D%C=&EV95%U975E5&AR96%D+G)U;BA#;VUP:6QE9"!#;V1E*0T*
M(" @(")!8W1I=F51=65U951H<F5A9#<W,S(R,2(@*%1)1#HP>#$S8CDY-3 P
M+"!S>7-?=&AR96%D7W0Z,'AB-3-C,S L(%=I;C,R240Z,'@Q-C8L('-T871E
M.D-7*2!P<FEO/34-"@EJ879A+FQA;F<N3V)J96-T+G=A:70H0V]M<&EL960@
M0V]D92D-"@ES>6YC+G-E<G9E<BYM;VYI=&]R+D%C=&EV95%U975E26YF;RYG
M970H0V]M<&EL960@0V]D92D-"@ES>6YC+G-E<G9E<BYM;VYI=&]R+D%C=&EV
M95%U975E5&AR96%D+G)U;BA#;VUP:6QE9"!#;V1E*0T*(" @(")!8W1I=F51
M=65U951H<F5A9#<W,C@R,2(@*%1)1#HP>#$W,S=C.#<P+"!S>7-?=&AR96%D
M7W0Z,'AB-3-E-S L(%=I;C,R240Z,'@Q.6,L('-T871E.D-7*2!P<FEO/34-
M"@EJ879A+FQA;F<N3V)J96-T+G=A:70H0V]M<&EL960@0V]D92D-"@ES>6YC
M+G-E<G9E<BYM;VYI=&]R+D%C=&EV95%U975E26YF;RYG970H0V]M<&EL960@
M0V]D92D-"@ES>6YC+G-E<G9E<BYM;VYI=&]R+D%C=&EV95%U975E5&AR96%D
M+G)U;BA#;VUP:6QE9"!#;V1E*0T*(" @(")!8W1I=F51=65U951H<F5A9#<W
M,C R,2(@*%1)1#HP>#$V.3$T-V,X+"!S>7-?=&AR96%D7W0Z,'AB-6(S93 L
M(%=I;C,R240Z,'@R,S0L('-T871E.D-7*2!P<FEO/34-"@EJ879A+FQA;F<N
M3V)J96-T+G=A:70H0V]M<&EL960@0V]D92D-"@ES>6YC+G-E<G9E<BYM;VYI
M=&]R+D%C=&EV95%U975E26YF;RYG970H0V]M<&EL960@0V]D92D-"@ES>6YC
M+G-E<G9E<BYM;VYI=&]R+D%C=&EV95%U975E5&AR96%D+G)U;BA#;VUP:6QE
M9"!#;V1E*0T*(" @(")!8W1I=F51=65U951H<F5A9#<W,38R,2(@*%1)1#HP
M>#$X-V0U-F$P+"!S>7-?=&AR96%D7W0Z,'AB-3AA.3 L(%=I;C,R240Z,'@Q
M-F,L('-T871E.D-7*2!P<FEO/34-"@EJ879A+FQA;F<N3V)J96-T+G=A:70H
M0V]M<&EL960@0V]D92D-"@ES>6YC+G-E<G9E<BYM;VYI=&]R+D%C=&EV95%U
M975E26YF;RYG970H0V]M<&EL960@0V]D92D-"@ES>6YC+G-E<G9E<BYM;VYI
M=&]R+D%C=&EV95%U975E5&AR96%D+G)U;BA#;VUP:6QE9"!#;V1E*0T*(" @
M(")!8W1I=F51=65U951H<F5A9#<W,3(R,2(@*%1)1#HP>#$Y-V%F.30P+"!S
M>7-?=&AR96%D7W0Z,'AB-3<R.3 L(%=I;C,R240Z,'@Q9C(L('-T871E.D-7
M*2!P<FEO/34-"@EJ879A+FQA;F<N3V)J96-T+G=A:70H0V]M<&EL960@0V]D
M92D-"@ES>6YC+G-E<G9E<BYM;VYI=&]R+D%C=&EV95%U975E26YF;RYG970H
M0V]M<&EL960@0V]D92D-"@ES>6YC+G-E<G9E<BYM;VYI=&]R+D%C=&EV95%U
M975E5&AR96%D+G)U;BA#;VUP:6QE9"!#;V1E*0T*(" @(")!8W1I=F51=65U
M951H<F5A9#<W,#@R,2(@*%1)1#HP>#$X,S,R.#DP+"!S>7-?=&AR96%D7W0Z
M,'AB-3$R.# L(%=I;C,R240Z,'@Y.2P@<W1A=&4Z0U<I('!R:6\]-0T*"6IA
M=F$N;&%N9RY/8FIE8W0N=V%I="A#;VUP:6QE9"!#;V1E*0T*"7-Y;F,N<V5R
M=F5R+FUO;FET;W(N06-T:79E475E=65);F9O+F=E="A#;VUP:6QE9"!#;V1E
M*0T*"7-Y;F,N<V5R=F5R+FUO;FET;W(N06-T:79E475E=654:')E860N<G5N
M*$-O;7!I;&5D($-O9&4I#0H@(" @(D%C=&EV95%U975E5&AR96%D-S<P-#0Q
M(B H5$E$.C!X,38Q,F,V-3@L('-Y<U]T:')E861?=#HP>&(U-C<P,"P@5VEN
M,S))1#HP>#%F.2P@<W1A=&4Z0U<I('!R:6\]-0T*"6IA=F$N;&%N9RY/8FIE
M8W0N=V%I="A#;VUP:6QE9"!#;V1E*0T*"7-Y;F,N<V5R=F5R+FUO;FET;W(N
M06-T:79E475E=65);F9O+F=E="A#;VUP:6QE9"!#;V1E*0T*"7-Y;F,N<V5R
M=F5R+FUO;FET;W(N06-T:79E475E=654:')E860N<G5N*$-O;7!I;&5D($-O
M9&4I#0H@(" @(D%C=&EV95%U975E5&AR96%D-S<P-#(Q(B H5$E$.C!X,38T
M8V9A,C L('-Y<U]T:')E861?=#HP>&(U-C4T,"P@5VEN,S))1#HP>#%D9"P@
M<W1A=&4Z0U<I('!R:6\]-0T*"6IA=F$N;&%N9RY/8FIE8W0N=V%I="A#;VUP
M:6QE9"!#;V1E*0T*"7-Y;F,N<V5R=F5R+FUO;FET;W(N06-T:79E475E=65)
M;F9O+F=E="A#;VUP:6QE9"!#;V1E*0T*"7-Y;F,N<V5R=F5R+FUO;FET;W(N
M06-T:79E475E=654:')E860N<G5N*$-O;7!I;&5D($-O9&4I#0H@(" @(D%C
M=&EV95%U975E5&AR96%D-S<P,#(Q(B H5$E$.C!X,3%D9#-F.3 L('-Y<U]T
M:')E861?=#HP>&(U,3ED,"P@5VEN,S))1#HP>&5C+"!S=&%T93I#5RD@<')I
M;STU#0H):F%V82YL86YG+D]B:F5C="YW86ET*$-O;7!I;&5D($-O9&4I#0H)
M<WEN8RYS97)V97(N;6]N:71O<BY!8W1I=F51=65U94EN9F\N9V5T*$-O;7!I
M;&5D($-O9&4I#0H)<WEN8RYS97)V97(N;6]N:71O<BY!8W1I=F51=65U951H
M<F5A9"YR=6XH0V]M<&EL960@0V]D92D-"B @(" B06-T:79E475E=654:')E
M860W-CDV,C$B("A4240Z,'@Q-&8T,60T,"P@<WES7W1H<F5A9%]T.C!X8C1D
M8C0P+"!7:6XS,DE$.C!X,3%E+"!S=&%T93I#5RD@<')I;STU#0H):F%V82YL
M86YG+D]B:F5C="YW86ET*$-O;7!I;&5D($-O9&4I#0H)<WEN8RYS97)V97(N
M;6]N:71O<BY!8W1I=F51=65U94EN9F\N9V5T*$-O;7!I;&5D($-O9&4I#0H)
M<WEN8RYS97)V97(N;6]N:71O<BY!8W1I=F51=65U951H<F5A9"YR=6XH0V]M
M<&EL960@0V]D92D-"B @(" B06-T:79E475E=654:')E860W-CDR,C$B("A4
M240Z,'@Q-6%C9#0Q,"P@<WES7W1H<F5A9%]T.C!X8C0X-S$P+"!7:6XS,DE$
M.C!X,6,S+"!S=&%T93I#5RD@<')I;STU#0H):F%V82YL86YG+D]B:F5C="YW
M86ET*$-O;7!I;&5D($-O9&4I#0H)<WEN8RYS97)V97(N;6]N:71O<BY!8W1I
M=F51=65U94EN9F\N9V5T*$-O;7!I;&5D($-O9&4I#0H)<WEN8RYS97)V97(N
M;6]N:71O<BY!8W1I=F51=65U951H<F5A9"YR=6XH0V]M<&EL960@0V]D92D-
M"B @(" B06-T:79E475E=654:')E860W-C@X,C$B("A4240Z,'@Q.#4P96-E
M,"P@<WES7W1H<F5A9%]T.C!X8C0U-&$P+"!7:6XS,DE$.C!X,C0S+"!S=&%T
M93I#5RD@<')I;STU#0H):F%V82YL86YG+D]B:F5C="YW86ET*$-O;7!I;&5D
M($-O9&4I#0H)<WEN8RYS97)V97(N;6]N:71O<BY!8W1I=F51=65U94EN9F\N
M9V5T*$-O;7!I;&5D($-O9&4I#0H)<WEN8RYS97)V97(N;6]N:71O<BY!8W1I
M=F51=65U951H<F5A9"YR=6XH0V]M<&EL960@0V]D92D-"B @(" B06-T:79E
M475E=654:')E860W-C@T,C$B("A4240Z,'@Q-&8T969E."P@<WES7W1H<F5A
M9%]T.C!X8C0U.64P+"!7:6XS,DE$.C!X,3@Y+"!S=&%T93I#5RD@<')I;STU
M#0H):F%V82YL86YG+D]B:F5C="YW86ET*$-O;7!I;&5D($-O9&4I#0H)<WEN
M8RYS97)V97(N;6]N:71O<BY!8W1I=F51=65U94EN9F\N9V5T*$-O;7!I;&5D
M($-O9&4I#0H)<WEN8RYS97)V97(N;6]N:71O<BY!8W1I=F51=65U951H<F5A
M9"YR=6XH0V]M<&EL960@0V]D92D-"B @(" B06-T:79E475E=654:')E860W
M-C@P,C$B("A4240Z,'@Q,CDR,#,Y,"P@<WES7W1H<F5A9%]T.C!X8C-A,C@P
M+"!7:6XS,DE$.C!X,C%A+"!S=&%T93I#5RD@<')I;STU#0H):F%V82YL86YG
M+D]B:F5C="YW86ET*$-O;7!I;&5D($-O9&4I#0H)<WEN8RYS97)V97(N;6]N
M:71O<BY!8W1I=F51=65U94EN9F\N9V5T*$-O;7!I;&5D($-O9&4I#0H)<WEN
M8RYS97)V97(N;6]N:71O<BY!8W1I=F51=65U951H<F5A9"YR=6XH0V]M<&EL
M960@0V]D92D-"B @(" B06-T:79E475E=654:')E860W-C<V,C$B("A4240Z
M,'@Q-&)B9C-C,"P@<WES7W1H<F5A9%]T.C!X8C0X.64P+"!7:6XS,DE$.C!X
M,30Q+"!S=&%T93I#5RD@<')I;STU#0H):F%V82YL86YG+D]B:F5C="YW86ET
M*$-O;7!I;&5D($-O9&4I#0H)<WEN8RYS97)V97(N;6]N:71O<BY!8W1I=F51
M=65U94EN9F\N9V5T*$-O;7!I;&5D($-O9&4I#0H)<WEN8RYS97)V97(N;6]N
M:71O<BY!8W1I=F51=65U951H<F5A9"YR=6XH0V]M<&EL960@0V]D92D-"B @
M(" B06-T:79E475E=654:')E860W-C<R,C$B("A4240Z,'@Q-C(V,S(R."P@
M<WES7W1H<F5A9%]T.C!X8C0X8S(P+"!7:6XS,DE$.C!X,3@Q+"!S=&%T93I#
M5RD@<')I;STU#0H):F%V82YL86YG+D]B:F5C="YW86ET*$-O;7!I;&5D($-O
M9&4I#0H)<WEN8RYS97)V97(N;6]N:71O<BY!8W1I=F51=65U94EN9F\N9V5T
M*$-O;7!I;&5D($-O9&4I#0H)<WEN8RYS97)V97(N;6]N:71O<BY!8W1I=F51
M=65U951H<F5A9"YR=6XH0V]M<&EL960@0V]D92D-"B @(" B06-T:79E475E
M=654:')E860W-C8X,C$B("A4240Z,'@Q.&)C,C@V,"P@<WES7W1H<F5A9%]T
M.C!X8C0W-#0P+"!7:6XS,DE$.C!X,3%F+"!S=&%T93I#5RD@<')I;STU#0H)
M:F%V82YL86YG+D]B:F5C="YW86ET*$-O;7!I;&5D($-O9&4I#0H)<WEN8RYS
M97)V97(N;6]N:71O<BY!8W1I=F51=65U94EN9F\N9V5T*$-O;7!I;&5D($-O
M9&4I#0H)<WEN8RYS97)V97(N;6]N:71O<BY!8W1I=F51=65U951H<F5A9"YR
M=6XH0V]M<&EL960@0V]D92D-"B @(" B06-T:79E475E=654:')E860W-C8T
M,C$B("A4240Z,'@Q-C-D-V(R,"P@<WES7W1H<F5A9%]T.C!X8C0W,S8P+"!7
M:6XS,DE$.C!X9C,L('-T871E.D-7*2!P<FEO/34-"@EJ879A+FQA;F<N3V)J
M96-T+G=A:70H0V]M<&EL960@0V]D92D-"@ES>6YC+G-E<G9E<BYM;VYI=&]R
M+D%C=&EV95%U975E26YF;RYG970H0V]M<&EL960@0V]D92D-"@ES>6YC+G-E
M<G9E<BYM;VYI=&]R+D%C=&EV95%U975E5&AR96%D+G)U;BA#;VUP:6QE9"!#
M;V1E*0T*(" @(")!8W1I=F51=65U951H<F5A9#<V-C R,2(@*%1)1#HP>#$W
M8V(Y-#@P+"!S>7-?=&AR96%D7W0Z,'AB-#<R,3 L(%=I;C,R240Z,'@R-3DL
M('-T871E.D-7*2!P<FEO/34-"@EJ879A+FQA;F<N3V)J96-T+G=A:70H0V]M
M<&EL960@0V]D92D-"@ES>6YC+G-E<G9E<BYM;VYI=&]R+D%C=&EV95%U975E
M26YF;RYG970H0V]M<&EL960@0V]D92D-"@ES>6YC+G-E<G9E<BYM;VYI=&]R
M+D%C=&EV95%U975E5&AR96%D+G)U;BA#;VUP:6QE9"!#;V1E*0T*(" @(")!
M8W1I=F51=65U951H<F5A9#<V-38R,2(@*%1)1#HP>#$T8V-C-#$P+"!S>7-?
M=&AR96%D7W0Z,'AB-#,U83 L(%=I;C,R240Z,'@Q8S<L('-T871E.D-7*2!P
M<FEO/34-"@EJ879A+FQA;F<N3V)J96-T+G=A:70H0V]M<&EL960@0V]D92D-
M"@ES>6YC+G-E<G9E<BYM;VYI=&]R+D%C=&EV95%U975E26YF;RYG970H0V]M
M<&EL960@0V]D92D-"@ES>6YC+G-E<G9E<BYM;VYI=&]R+D%C=&EV95%U975E
M5&AR96%D+G)U;BA#;VUP:6QE9"!#;V1E*0T*(" @(")!8W1I=F51=65U951H
M<F5A9#<V-3(R,2(@*%1)1#HP>#$R,3EA,V4X+"!S>7-?=&AR96%D7W0Z,'AB
M-#,W93 L(%=I;C,R240Z,'@Q-F8L('-T871E.D-7*2!P<FEO/34-"@EJ879A
M+FQA;F<N3V)J96-T+G=A:70H0V]M<&EL960@0V]D92D-"@ES>6YC+G-E<G9E
M<BYM;VYI=&]R+D%C=&EV95%U975E26YF;RYG970H0V]M<&EL960@0V]D92D-
M"@ES>6YC+G-E<G9E<BYM;VYI=&]R+D%C=&EV95%U975E5&AR96%D+G)U;BA#
M;VUP:6QE9"!#;V1E*0T*(" @(")!8W1I=F51=65U951H<F5A9#<V-#@R,2(@
M*%1)1#HP>#$W,C0Y-&4X+"!S>7-?=&AR96%D7W0Z,'AB-#,W,# L(%=I;C,R
M240Z,'@R,#,L('-T871E.D-7*2!P<FEO/34-"@EJ879A+FQA;F<N3V)J96-T
M+G=A:70H0V]M<&EL960@0V]D92D-"@ES>6YC+G-E<G9E<BYM;VYI=&]R+D%C
M=&EV95%U975E26YF;RYG970H0V]M<&EL960@0V]D92D-"@ES>6YC+G-E<G9E
M<BYM;VYI=&]R+D%C=&EV95%U975E5&AR96%D+G)U;BA#;VUP:6QE9"!#;V1E
M*0T*(" @(")!8W1I=F51=65U951H<F5A9#<V-#0R,2(@*%1)1#HP>#$W-6%D
M934P+"!S>7-?=&AR96%D7W0Z,'AB,V$T,C L(%=I;C,R240Z,'@Q-C L('-T
M871E.D-7*2!P<FEO/34-"@EJ879A+FQA;F<N3V)J96-T+G=A:70H0V]M<&EL
M960@0V]D92D-"@ES>6YC+G-E<G9E<BYM;VYI=&]R+D%C=&EV95%U975E26YF
M;RYG970H0V]M<&EL960@0V]D92D-"@ES>6YC+G-E<G9E<BYM;VYI=&]R+D%C
M=&EV95%U975E5&AR96%D+G)U;BA#;VUP:6QE9"!#;V1E*0T*(" @(")!8W1I
M=F51=65U951H<F5A9#<V-# R,2(@*%1)1#HP>#$R96$W,68P+"!S>7-?=&AR
M96%D7W0Z,'AB,V$U,# L(%=I;C,R240Z,'@Q9F8L('-T871E.D-7*2!P<FEO
M/34-"@EJ879A+FQA;F<N3V)J96-T+G=A:70H0V]M<&EL960@0V]D92D-"@ES
M>6YC+G-E<G9E<BYM;VYI=&]R+D%C=&EV95%U975E26YF;RYG970H0V]M<&EL
M960@0V]D92D-"@ES>6YC+G-E<G9E<BYM;VYI=&]R+D%C=&EV95%U975E5&AR
M96%D+G)U;BA#;VUP:6QE9"!#;V1E*0T*(" @(")!8W1I=F51=65U951H<F5A
M9#<V,S8R,2(@*%1)1#HP>#$W9#!C-3DP+"!S>7-?=&AR96%D7W0Z,'AB,V4X
M83 L(%=I;C,R240Z,'@Q9C L('-T871E.D-7*2!P<FEO/34-"@EJ879A+FQA
M;F<N3V)J96-T+G=A:70H0V]M<&EL960@0V]D92D-"@ES>6YC+G-E<G9E<BYM
M;VYI=&]R+D%C=&EV95%U975E26YF;RYG970H0V]M<&EL960@0V]D92D-"@ES
M>6YC+G-E<G9E<BYM;VYI=&]R+D%C=&EV95%U975E5&AR96%D+G)U;BA#;VUP
M:6QE9"!#;V1E*0T*(" @(")!8W1I=F51=65U951H<F5A9#<V,S(R,2(@*%1)
M1#HP>#$Y-#=F-C<P+"!S>7-?=&AR96%D7W0Z,'AB,V4T,C L(%=I;C,R240Z
M,'@Q,V$L('-T871E.D-7*2!P<FEO/34-"@EJ879A+FQA;F<N3V)J96-T+G=A
M:70H0V]M<&EL960@0V]D92D-"@ES>6YC+G-E<G9E<BYM;VYI=&]R+D%C=&EV
M95%U975E26YF;RYG970H0V]M<&EL960@0V]D92D-"@ES>6YC+G-E<G9E<BYM
M;VYI=&]R+D%C=&EV95%U975E5&AR96%D+G)U;BA#;VUP:6QE9"!#;V1E*0T*
M(" @(")!8W1I=F51=65U951H<F5A9#<V,C@T,2(@*%1)1#HP>#$T8F0U,3$P
M+"!S>7-?=&AR96%D7W0Z,'AB,V$Y-3 L(%=I;C,R240Z,'@Q,CDL('-T871E
M.D-7*2!P<FEO/34-"@EJ879A+FQA;F<N3V)J96-T+G=A:70H0V]M<&EL960@
M0V]D92D-"@ES>6YC+G-E<G9E<BYM;VYI=&]R+D%C=&EV95%U975E26YF;RYG
M970H0V]M<&EL960@0V]D92D-"@ES>6YC+G-E<G9E<BYM;VYI=&]R+D%C=&EV
M95%U975E5&AR96%D+G)U;BA#;VUP:6QE9"!#;V1E*0T*(" @(")!8W1I=F51
M=65U951H<F5A9#<V,C@R,2(@*%1)1#HP>#$T9C)E,C$X+"!S>7-?=&AR96%D
M7W0Z,'AB,V$Q83 L(%=I;C,R240Z,'@Q-60L('-T871E.D-7*2!P<FEO/34-
M"@EJ879A+FQA;F<N3V)J96-T+G=A:70H0V]M<&EL960@0V]D92D-"@ES>6YC
M+G-E<G9E<BYM;VYI=&]R+D%C=&EV95%U975E26YF;RYG970H0V]M<&EL960@
M0V]D92D-"@ES>6YC+G-E<G9E<BYM;VYI=&]R+D%C=&EV95%U975E5&AR96%D
M+G)U;BA#;VUP:6QE9"!#;V1E*0T*(" @(")!8W1I=F51=65U951H<F5A9#<V
M,C0R,2(@*%1)1#HP>#$V-64Y,C@X+"!S>7-?=&AR96%D7W0Z,'AB,SEC-# L
M(%=I;C,R240Z,'@Q.&,L('-T871E.D-7*2!P<FEO/34-"@EJ879A+FQA;F<N
M3V)J96-T+G=A:70H0V]M<&EL960@0V]D92D-"@ES>6YC+G-E<G9E<BYM;VYI
M=&]R+D%C=&EV95%U975E26YF;RYG970H0V]M<&EL960@0V]D92D-"@ES>6YC
M+G-E<G9E<BYM;VYI=&]R+D%C=&EV95%U975E5&AR96%D+G)U;BA#;VUP:6QE
M9"!#;V1E*0T*(" @(")!8W1I=F51=65U951H<F5A9#<V,C R,2(@*%1)1#HP
M>#$X,F%D,S0X+"!S>7-?=&AR96%D7W0Z,'AB,S4V-C L(%=I;C,R240Z,'@Q
M934L('-T871E.D-7*2!P<FEO/34-"@EJ879A+FQA;F<N3V)J96-T+G=A:70H
M0V] -
Issue in production environment
Hi,
We got an issue in the production environment on last Friday and yet not able to find out the root cause of the issue.
Environment
GUI : .Net
Server : Java
Application Server: Jboss
Database: Sqlserver
The Java application is deployed into Jboss application server which connects to SQL server as back end. GUI makes the web service call to server for data communication
Issue
Friday at 2 PM users reported the slowness of the application (Not getting the response from server in GUI), all the request from GUI were getting timed out.
Restart of Jboss didn't help
Restarted the Jboss and sqlserver for the second time and than the environment became stable
Analysis
1. From the thread dump of Jboss log we see that there are many threads waiting on a socket for database connection(according to database team all the connections are open and available at that particular time).
2. The size of transaction log is almost doubled during this period (when the issue was reported).
We couldn't find a reason why this issue is happened. Is it a database issue or something else? Please suggest...
Thanks,
ManojHi Manoj,
According to your description, when running the web application, many threads are waiting for the socket from database which cause requests from front GUI time out. Right?
In this scenario, user can access the front GUI, which means the application server is working. Since the connections are all open at that particular time, it should not be issue on your JDBC. It seems to be a deadlock issue which cause other threads
waiting and hang. I recommend check and optimize your code. On database side, I suggest you open the SQL profiler, select DeadLock Graph, it will record when deadlock occurs. Please refer to links below:
Detecting and Ending Deadlocks
Analyze Deadlocks with SQL Server Profiler
If you have any question, please feel free to ask.
Simon Hou
TechNet Community Support
Maybe you are looking for
-
How to hide the select-options fields on radiobutton select in webdynpro
hi Could you please help how to hide the select-option fields in webdynpro there are four radio buttons for every radiobutton there is corresponding the select_option filed is associated to it user as only provision select one radio button and reset
-
Hello, I develop my web site with a html editor. When I use preview, everything is fine. If I use Explorer, everything is fine also but if I use Firefox to display the page, the images and text are not where they should be.
-
No internet access after Snow upgrade
Was very upset yesterday after upgrading my 13" Macbook Pro to Snow Leopard only to find out I couldn't get online via ethernet or Airport. Went through all kinds of troubleshooting and just got off a 90 minute long convo with an Apple phone genius.
-
I need to know how to delete my free membership under one name (variation of my real name). because using Chrome browser, I paid for a membership using my real name and now I can't access that account using Firefox. I can't install any products.
-
Scrollbar on a tab canvas.
Hi All, How to show scrollbar on a tab canvas? I changed the window property "Show Vertical Scroll Bar" to yes, but even then there is no scrollbar on tab canvas. thanks in advance.