Strange classloader experience with toplink-essentials on Oc4j-10.1.3.3.0

Question

Hi,
I am teaching classes at the Eotvos University/Budapest on a topic where OC4J has been used for 2 years. Students use jdeveloper (regularly updated) to create webapps with EJB3.0+toplink backends. They test their projects first with the JDeveloper embedded OC4J, but after that they deploy it on a central server running standalone OC4J.
Students make quite an intensive and versatile use of the OC4J server, a noteworthy stress-test you may find interesting! The server is a Ubuntu Linux with JDK_1.5.0_14.
1. The first known issue is that after about 10 projects are deployed (small class projects 3-4 EJBs, 3-4 JSP-s each), oc4j regularl goes into a "runaway" mode, i.e. the CPU utilization of the JVM process goes up to 99-100%. Static web content is still available, but all applications stop responding. I can state pretty safely that the applications deployed are fairly harmless and it is not even necessary to run them to make this happen. I.e. after 10-15 apps are depoyed it is enough to start the admin-client.jar or the admin console to experience this problem. (The applications have no problem individually: last year after these issues occurred, we installed them individually for grading and nothing like this happened).
Let me point out that SO FAR this happened with Oc4j-10.1.3.1 and Oc4j-10.1.3.2 only, and this was the main reason we swithed to Oc4j-10.1.3.3.0
2. Another issue that exists both with the earlier versions and 10.1.3.3.0 is that applications need to include the toplink-essentials.jar explicitely if they use EJB3.0 entities. It appears to me that this should not be neessary as there are default definitions for that in config/server.xml (a shared-library definition for toplink-essentials), and in config/system-application.xml (a corresponding import-shared-library tag), but still their applications cannot be deployed because of the follwing error:
Deploy error: Deploy error: Operation failed with error:
Hiányzó osztály: oracle.toplink.essentials.ejb.cmp3.EntityManagerFactoryProvider
FĂĽggĹ' osztály: com.evermind.server.ejb.persistence.PersistenceUnitManagerImpl
BetöltĹ': oc4j:10.1.3
Kódforrás: /opt/oc4j_101330/j2ee/home/lib/oc4j-internal.jar
(On our machines, JDK displays some errors in Hungarian: HiányzĂł osztály=Missing class; FĂĽggĹ' osztály=Dependent class; BetöltĹ'=Loader; KĂłdforrás=Code source.)
To fix this issue, we used a workaround, i.e. their orion-application.xml is regularly extended with something like:
<library path="/opt/oc4j/toplink/jlib/toplink-essentials.jar"/>
And this solved the problem. Can you explain this?
3. Now, with 10.1.3.3.0 this did not help much either. Although the toplink-essentials.jar was found, another nasty exception happened in the server still during deployment:
Caused by: java.lang.NoSuchMethodException: oracle.toplink.essentials.platform.server.oc4j.Oc4jPlatform.<init>(oracle.toplink.essentials.internal.sessions.DatabaseSessionImpl)
at java.lang.Class.getConstructor0(Class.java:2678)
at java.lang.Class.getConstructor(Class.java:1629)
at oracle.toplink.essentials.internal.ejb.cmp3.EntityManagerSetupImpl.updateServerPlatform(EntityManagerSetupImpl.java:309)
Apparently a class in toplink-essentials.jar (version 2, build 41, i.e. the one that comes with oc4j) cannot properly invoke a the constructor of another class IN THE SAME JAR.
I checked everything, even unzipped and checked the class signatures, and also made sure that there is no other toplink-essentials.jar in the classpath. Still the error remained.
FInally, I tried to add the above libary tag to $OC4J_HOME/config/apllication.xml instead of the applications local orion-application.xml, and - voila!!! - it WORKS now!
Can you explain all this to me? From the documentation the location of the <library> tag in the descriptor files should not amke any difference, but still it seems to do.
Regards: Arpad Bakay

Answer 1

Seems to be a bug. That was the response from Oracle support (Metalink)
Response:
Your issue seems to be related to bug 5594702 - Abstract: EJB30 ENTITY BEAN WITH @ID AND @COLUMN ANNOTATION FAILS TO DEPLOY ON AIX.
There is an issue with the IBM JDK/JRE 1.5's processing of annotations.
Links:
http://www.theserverside.com/discussions/thread.tss?thread_id=37764
http://www-128.ibm.com/developerworks/forums/dw_thread.jsp?forum=367&thread=112543&cat=10
When processing annotations it returns boolean values as false.
Work-around:
Fully specify the @Column annotation's boolean values. If insertable and updatable are set to false (which will happen due to this bug) then TopLink sets the PK
mapping to read-only and the exception seen is expected.
Note: nullable attribute of the @Colmun is not used in the EJB3/JPA preview of 10.1.3.0 If the customer MUST override the default column name then they should use:
@Column(name="column-name", insertable=true, updatable=true)
If they do not wish to override the default column name then simply do not use an @Column annotation.
It can be deleted or commented out in the JDev generated code.
There are two reported annotation processing issues with the AIX JVM. One was fixed in SR1 and the other is fixed
in SR3 (due out Oc 11 - today). Upgrading to these more recent JVM releases may also address this issue.
RECOMMENDED SOLUTIONS:
1. Upgrade the IBM AIX JVM to SR3.
OR
2. Fix all generated @Column annotations as described above