ADF BC: Optimal Application Module Runtime Configuration

Question

Have implemented an intranet application for 1000 users using the following technologies implemented on Oracle JDeveloper 10.1.3.0.4:
ADF BC for my Services layer
JSF for my Controller layer
JSF and ADF for my View layer
ADF Model for my Data layer
The application is entirely designed by .jspx and .jsp pages (for reports), handling 106 application modules, with several entity and view objects and without any step-by-step stateful scenario. At each page, there appears a commit button to commit the end-user transaction, and a rollback button to return to the previous page.
The application is database-backed from an Oracle Database 10g, deployed on three application servers Oracle 10.1.3.1 on three different hosts on Suse Linux OS, used data-source connection and AM pooling, and load-balanced by one webcacher in front of the 3 servers.
Under stress test scenario, on a heavy system load in production environment, several ADF State Management errors were recognized that pushed the webacher and application server with the maximum load to automatic restarts due to either demand for JVM increased heap size or errors from the database that the TNS listener complained it could not handle any more connections.
Thus, we started a safety/reliability tess for all our application module components (disabling "Enable application module pooling"), and a stress test using a third-party stress tool at work, but we're still faced with several questions.
(a) What is the best strategy for AM configuration, to quit State Management at all since our application doesn't need it, and if so can it be done globally or declaratively and not individually and programmatically?
(b) Testing on application module runtime configuration, we observed that not to "hang-on" on a connection -default functionality- but to release it at the end of each HTTP request, as each deployment is concentrated on a single data-source connection pool/single user, brought less open JDBC connections on A.S.? Is this the optimal run-time configuration than the default for our case?
(c) If State Management errors are resolved, and using a software balancer before 3 application servers, regulate the traffice by 1/3 to each, is it possible that a system crash will be avoided next time?
So, our problems related to ADF tunning and State Management Errors, but we need a good advice on this topic as the overall documentation is case-specific.

Answer 1

Chapter 29 "Understanding Application Module Pooling" in the ADF Developer's Guide for Forms/4GL Developers on the ADF Learning Center at http://www.oracle.com/technology/products/adf/learnadf.html is the best place to start. There is no "magic" optimal configuration that I (or anyone here, really) can suggest to you. You need to understand the various "knobs and dials" that control the pool sizing and pool cleanup behavior, and then apply application-specific information to this information to derive what the optimal configuration for your particular application will be. The chapter explains how the application module pool and database connection pool interact, and in which situations it is best to change the default setting of the jbo.doconnectionpooling property to true.
Chapter 28 "Application Module State Management" describes the functionality of that facility of the framework, as well as how to release the application module in stateless, unmanaged mode if you need to.
By following the tips in the section "28.3.2.4 Setting Release Level in an Custom ADF PageLifecycle" (which builds on information introduced in section "10.5.4.1 Globally Customizing the ADF Page Lifecycle"), you can generically impose a different release level in your application without "touching" every page.
As a general rule, if you have:
(*) Lots of distinct application modules
(*) Each of which is used as a separate data control (rather than being nested inside a containing application module as described in section "8.9.3 Root Application Modules Versus Nested Application Module Usages" of the dev guide)
(*) Each of which uses the same database connection
Then as described in section "29.8 How Database and Application Module Pools Cooperate" in the guide, your application can decrease the total overall number of connections used by using the non-default setting of jbo.doconnectionpooling=true.
Applications using many distinct data controls in the span of a single user's session need to be aware of Bug# 4566186 ("PERF: ADFBINDINGFILTER CHECKS OUT ALL ADF BC DATA CONTROLS USED AT LEAST ONCE"). This issue has been fixed in JDev/ADF 11g, but apparently the fix was too complex to accommodate a 10.1.X backport. I'm trying to document a workaround and publish it on my blog, but I'm not finished with it yet.