GC performance and Class Loading/Unloading
We have EP 6.0, SP11 on Solaris with JDK 1.4.8_02. We are running Web Dynpro version of MSS/ESS and Adobe Document Services. This is a Java stack only Web AS.
We are experiencing very uneven performance on the Portal. Usually, when the Portal grinds to a halt, the server log shows GC entries or Class unloading entries for the entire time the Portal stops working.
I am thinking about setting the GC parameters to the same size to try and eliminate sudden GC interruptions. Also, what parameter can I set to allow as many classes to be loaded at startup and stay in memory for as long as possible?
Thanks,
Rob Bartlett
Hi Robert
Also, if the host running the WebAS is a multi processor machine, then setting the flags
-XX:+UseConcMarkSweepGC and
-XX:+UseParNewGC
will help reduce the pause time during the GC collection in old generation and the young genereation respectively, as the GC will happen using multiple threads
I can suggest you to check if the GC performs a minor collection or major collection by enabling the flags
-verbose:gc
-XX:+PrintGCTimeStamps
-XX:+PrintGCDetails. Based on this, try to tune the young or old generation.
Regards
Madhu
Similar Messages
-
JVM PermGen Class Loading & Unloading Concern
Hi All,
My application is spitting out lot of following statements periodically:
[Loaded sun.reflect.GeneratedMethodAccessor833 from __JVM_DefineClass__] ..
and ocassionally,
[Unloading class sun.reflect.GeneratedMethodAccessor1220]
My application uses lot of RMI and serialization & desrialization process. Initially, it was causing my application to slow down and later on we did the following:
- Increased the PermGen space.
- Tune the Garbage Collection Policy for Heap (Enabled ConcurrentMarkSweepGC and ParNewGC)
After making above changes, we have restarted the application and not experienced slowness again (though we are still monitoring the application). But above statements are still visible, I have following questions:
- Why "GeneratedMethodAccessor" is being loaded again and again? I guess these are required for accessing or invoking methods on serialized objects and proxy. But aren't they cached as reference in the main class that was loaded by JVM?
- Does "Unloading class sun.reflect.GeneratedMethodAccessor1220" happen only after full GC runs? In actual, when it actually starts unloading these references?
Any help shall be really appreciated?To add to the above, the classes get unloaded if they are collected. To be collected they must be eligible. And the only way that happens if the following are both true.
1. The class loader is eligible for collection.
2. There are no active references. -
QUERY PERFORMANCE AND DATA LOADING PERFORMANCE ISSUES
WHAT ARE QUERY PERFORMANCE ISSUES WE NEED TO TAKE CARE PLEASE EXPLAIN AND LET ME KNOW T CODES...PLZ URGENT
WHAT ARE DATALOADING PERFORMANCE ISSUES WE NEED TO TAKE CARE PLEASE EXPLAIN AND LET ME KNOW T CODES PLZ URGENT
WILL REWARD FULL POINT S
REGARDS
GURUBW Back end
Some Tips -
1)Identify long-running extraction processes on the source system. Extraction processes are performed by several extraction jobs running on the source system. The run-time of these jobs affects the performance. Use transaction code SM37 Background Processing Job Management to analyze the run-times of these jobs. If the run-time of data collection jobs lasts for several hours, schedule these jobs to run more frequently. This way, less data is written into update tables for each run and extraction performance increases.
2)Identify high run-times for ABAP code, especially for user exits. The quality of any custom ABAP programs used in data extraction affects the extraction performance. Use transaction code SE30 ABAP/4 Run-time Analysis and then run the analysis for the transaction code RSA3 Extractor Checker. The system then records the activities of the extraction program so you can review them to identify time-consuming activities. Eliminate those long-running activities or substitute them with alternative program logic.
3)Identify expensive SQL statements. If database run-time is high for extraction jobs, use transaction code ST05 Performance Trace. On this screen, select ALEREMOTE user and then select SQL trace to record the SQL statements. Identify the time-consuming sections from the results. If the data-selection times are high on a particular SQL statement, index the DataSource tables to increase the performance of selection (see no. 6 below). While using ST05, make sure that no other extraction job is running with ALEREMOTE user.
4)Balance loads by distributing processes onto different servers if possible. If your site uses more than one BW application server, distribute the extraction processes to different servers using transaction code SM59 Maintain RFC Destination. Load balancing is possible only if the extraction program allows the option
5)Set optimum parameters for data-packet size. Packet size affects the number of data requests to the database. Set the data-packet size to optimum values for an efficient data-extraction mechanism. To find the optimum value, start with a packet size in the range of 50,000 to 100,000 and gradually increase it. At some point, you will reach the threshold at which increasing packet size further does not provide any performance increase. To set the packet size, use transaction code SBIW BW IMG Menu on the source system. To set the data load parameters for flat-file uploads, use transaction code RSCUSTV6 in BW.
6)Build indexes on DataSource tables based on selection criteria. Indexing DataSource tables improves the extraction performance, because it reduces the read times of those tables.
7)Execute collection jobs in parallel. Like the Business Content extractors, generic extractors have a number of collection jobs to retrieve relevant data from DataSource tables. Scheduling these collection jobs to run in parallel reduces the total extraction time, and they can be scheduled via transaction code SM37 in the source system.
8). Break up your data selections for InfoPackages and schedule the portions to run in parallel. This parallel upload mechanism sends different portions of the data to BW at the same time, and as a result the total upload time is reduced. You can schedule InfoPackages in the Administrator Workbench.
You can upload data from a data target (InfoCube and ODS) to another data target within the BW system. While uploading, you can schedule more than one InfoPackage with different selection options in each one. For example, fiscal year or fiscal year period can be used as selection options. Avoid using parallel uploads for high volumes of data if hardware resources are constrained. Each InfoPacket uses one background process (if scheduled to run in the background) or dialog process (if scheduled to run online) of the application server, and too many processes could overwhelm a slow server.
9). Building secondary indexes on the tables for the selection fields optimizes these tables for reading, reducing extraction time. If your selection fields are not key fields on the table, primary indexes are not much of a help when accessing data. In this case it is better to create secondary indexes with selection fields on the associated table using ABAP Dictionary to improve better selection performance.
10)Analyze upload times to the PSA and identify long-running uploads. When you extract the data using PSA method, data is written into PSA tables in the BW system. If your data is on the order of tens of millions, consider partitioning these PSA tables for better performance, but pay attention to the partition sizes. Partitioning PSA tables improves data-load performance because it's faster to insert data into smaller database tables. Partitioning also provides increased performance for maintenance of PSA tables for example, you can delete a portion of data faster. You can set the size of each partition in the PSA parameters screen, in transaction code SPRO or RSCUSTV6, so that BW creates a new partition automatically when a threshold value is reached.
11)Debug any routines in the transfer and update rules and eliminate single selects from the routines. Using single selects in custom ABAP routines for selecting data from database tables reduces performance considerably. It is better to use buffers and array operations. When you use buffers or array operations, the system reads data from the database tables and stores it in the memory for manipulation, improving performance. If you do not use buffers or array operations, the whole reading process is performed on the database with many table accesses, and performance deteriorates. Also, extensive use of library transformations in the ABAP code reduces performance; since these transformations are not compiled in advance, they are carried out during run-time.
12)Before uploading a high volume of transaction data into InfoCubes, activate the number-range buffer for dimension IDs. The number-range buffer is a parameter that identifies the number of sequential dimension IDs stored in the memory. If you increase the number range before high-volume data upload, you reduce the number of reads from the dimension tables and hence increase the upload performance. Do not forget to set the number-range values back to their original values after the upload. Use transaction code SNRO to maintain the number range buffer values for InfoCubes.
13)Drop the indexes before uploading high-volume data into InfoCubes. Regenerate them after the upload. Indexes on InfoCubes are optimized for reading data from the InfoCubes. If the indexes exist during the upload, BW reads the indexes and tries to insert the records according to the indexes, resulting in poor upload performance. You can automate the dropping and regeneration of the indexes through InfoPackage scheduling. You can drop indexes in the Manage InfoCube screen in the Administrator Workbench.
14)IDoc (intermediate document) archiving improves the extraction and loading performance and can be applied on both BW and R/3 systems. In addition to IDoc archiving, data archiving is available for InfoCubes and ODS objects.
Hope it Helps
Chetan
@CP.. -
ConversionManager - Override and Class Loader Issues
Hi
We are using Toplink 10.1.3 deployed withiin 10gAS 10.1.3.
We have overridden the ConversionManger as documented in tips.
1. MyConversionManager extends ConversionManager.
2. We have overriden convertObject
3. We have a pre-login SessionEventAdapter that sets the ConversionManager.
We have had issues of classloading within the application previously, particularly where we have two versions of our application deployed on the same App Server instance (potentially with differing versions of MyConversionManager).
For this reason we only set the ConversionManager within the preLogin event on the Session as follows:
*//Just set the conversion manager for the session.*
event.getSession().getLogin().getPlatform().setConversionManager(getConversionManager());
We no longer set the default conversion manager, as my understanding was that this would set it at the root level within the OC4J instance (meaning that each deployment would override the previous).
Additionally, we have had to sepcify the class loader to use, within the ovverriden convertObject method, as otherwise start up of Toplink falied, as it was unable to solve references to application classes specified within class indicator mappings. We do this as follows:
*@Override*
*public Object convertObject(final Object source, final Class javaClass) {*
super.setShouldUseClassLoaderFromCurrentThread(true);
The only know issue with this, is that if I try and use OEM to veiw the Toplink Cache, I just get and AnnotatedClassNotFoundException as it is attempting to use the system class loader (that does not contain our Application classes).
Two questions therefore:
1. Is the deployment we have now stable, i.e. overriding the class loader within the overriden Conversion Manager, and only loading this against the session??
2. How can I get mutiple versions of the application to still work, together with being able to view the Toplink Cache from within OE:M.
Any help or insight, would be greatly appreciated.
MarcLooks fine, you may also wish to investigate using Converters in your mappings instead of customizing the ConversionManager.
For the OEM issue, try setting the class loader to be your application class loader instead of the thread one, i.e. MyAppClass.getLoader().
James : http://www.eclipselink.org -
ObjectInputStream and Class Loading
Hello,
I have a serializable object that I have to pass as an argument to a session bean. It therefore is serialised/deserialised with a default object inputstream, which uses the bootstrap class loader as default. However, the object contains objects of which the classes cannot be loaded by the bootstrap class loader, in fact the class definitions are in a .jar file. I thus would need to be able to deserialise these objects with a custom class loader at the receiver's end. The receiver has the .jar file and the object that is to be deserialised is a trusted object.
I could manually control the deserialization of the object by defining the following method:
private void readObject(ObjectInputStream in) throws IOException, ClassNotFoundException {
}I could also deserialize objects from a self created ObjectInputStream with a class loader of choice with the following class:
public class CustomObjectInputStream extends ObjectInputStream {
private ClassLoader classLoader;
public CustomObjectInputStream(InputStream in, ClassLoader classLoader) throws IOException {
super(in);
this.classLoader = classLoader;
protected Class<?> resolveClass(ObjectStreamClass desc) throws ClassNotFoundException {
return Class.forName(desc.getName(), false, classLoader);
}However I have no control over the type of ObjectInputstream passed as argument to the readObject method, which uses the bootstrap class loader.
I give 5 duke stars to who can explain me a working solution.Thanks for the suggestion, but it doesn't work. Thread.currentThread().setContextClassLoader() appears to have no effect whatsoever on the class loader. Your suggestion about latestUserDefinedLoader() made me realise though that the situation is serious. This is a native method and it appears to be conceptually impossible to crack the standard ObjectInputStream for it to use the class loader of your choice. I filed a bug request for this.
However, after a bit of thought I found a work around: if you can't control the way the EJB container serializes and deserializes objects, then serialize the objects before the EJB serializes them, pass the arguments as byte arrays and deserialize them only after the EJB container has deserialized them as byte arrays.
You didn't give me the solution, but you made me realise that it wouldn't make sense to look for a conventional solution anymore, as there wouldn't be one so I give you one duke star. Thanks for the suggestion! -
Question:
Does the gc mechanism collect unreferenced objects only, or does it also unload classes that serve as blueprint for those unreferenced objects?
Does anybody know any reference answering this question?The problem is that if I load all the classes that I might need unloaded in the future using my custom classloader, and follow the nullifying references/nuking classloader technique, I'll have to nullify all the unloadable objects based on several different classes each time I want to reload one class.
So I'll need to serialize all those objects, and than reload all the classes loaded/unloaded by the custom classloader, than recreate all the objects from their serialized version.
And if I create a seperate custom classloader object for each potentially unloadable class, the memory will be crowded.
So what can I do to reload a class without unloading all the classes loaded by the custom classloader? -
Dynamic Class Loading and Unloading
I am trying to create a system where the class name and method name is
picked up from a meta-data database and executed.
This was accompanied using Dynamic Class loading. I tried to extend this to
support versioning of meta-data. Here depending on the version of meta-data
different libraries can be loaded and different implementations of object
with the same name can be executed. This does not seem to work with Forte
3.0.
When the second Library is loaded the method execution does not work.
(Even though the unload flag on the LoadLibrary is set)
If the application is stopped and restarted pointing to the second library
there is no problem. In a running application a dynamically loaded library
does not seem to unload and reload a new library for the same class names.
Has anyone tried sometime similar, is there a workaround for this type of
problem.
- VivekI am trying to create a system where the class name and method name is
picked up from a meta-data database and executed.
This was accompanied using Dynamic Class loading. I tried to extend this to
support versioning of meta-data. Here depending on the version of meta-data
different libraries can be loaded and different implementations of object
with the same name can be executed. This does not seem to work with Forte
3.0.
When the second Library is loaded the method execution does not work.
(Even though the unload flag on the LoadLibrary is set)
If the application is stopped and restarted pointing to the second library
there is no problem. In a running application a dynamically loaded library
does not seem to unload and reload a new library for the same class names.
Has anyone tried sometime similar, is there a workaround for this type of
problem.
- Vivek -
Performance issues with class loader on Windows server
We are observing some performance issues in our application. We are Using weblogic 11g with Java6 on a windows 2003 server
The thread dumps indicate many threads are waiting in queue for the native file methods:
"[ACTIVE] ExecuteThread: '106' for queue: 'weblogic.kernel.Default (self-tuning)'" RUNNABLE
java.io.WinNTFileSystem.getBooleanAttributes(Native Method)
java.io.File.exists(Unknown Source)
weblogic.utils.classloaders.ClasspathClassFinder.getFileSource(ClasspathClassFinder.java:398)
weblogic.utils.classloaders.ClasspathClassFinder.getSourcesInternal(ClasspathClassFinder.java:347)
weblogic.utils.classloaders.ClasspathClassFinder.getSource(ClasspathClassFinder.java:316)
weblogic.application.io.ManifestFinder.getSource(ManifestFinder.java:75)
weblogic.utils.classloaders.MultiClassFinder.getSource(MultiClassFinder.java:67)
weblogic.application.utils.CompositeWebAppFinder.getSource(CompositeWebAppFinder.java:71)
weblogic.utils.classloaders.MultiClassFinder.getSource(MultiClassFinder.java:67)
weblogic.utils.classloaders.MultiClassFinder.getSource(MultiClassFinder.java:67)
weblogic.utils.classloaders.CodeGenClassFinder.getSource(CodeGenClassFinder.java:33)
weblogic.utils.classloaders.GenericClassLoader.findResource(GenericClassLoader.java:210)
weblogic.utils.classloaders.GenericClassLoader.getResourceInternal(GenericClassLoader.java:160)
weblogic.utils.classloaders.GenericClassLoader.getResource(GenericClassLoader.java:182)
java.lang.ClassLoader.getResourceAsStream(Unknown Source)
javax.xml.parsers.SecuritySupport$4.run(Unknown Source)
java.security.AccessController.doPrivileged(Native Method)
javax.xml.parsers.SecuritySupport.getResourceAsStream(Unknown Source)
javax.xml.parsers.FactoryFinder.findJarServiceProvider(Unknown Source)
javax.xml.parsers.FactoryFinder.find(Unknown Source)
javax.xml.parsers.DocumentBuilderFactory.newInstance(Unknown Source)
org.ajax4jsf.context.ResponseWriterContentHandler.<init>(ResponseWriterContentHandler.java:48)
org.ajax4jsf.context.ViewResources$HeadResponseWriter.<init>(ViewResources.java:259)
org.ajax4jsf.context.ViewResources.processHeadResources(ViewResources.java:445)
org.ajax4jsf.application.AjaxViewHandler.renderView(AjaxViewHandler.java:193)
org.apache.myfaces.lifecycle.RenderResponseExecutor.execute(RenderResponseExecutor.java:41)
org.apache.myfaces.lifecycle.LifecycleImpl.render(LifecycleImpl.java:140)
On googling this seems to be an issue with java file handling on windows servers and I couldn't find a solution yet. Any recommendation or pointer is appreciatedHi shubhu,
I just analyzed your partial Thread Dump data, the problem is that the ajax4jsf framework ResponseWriterContentHandler triggers internally a new instance of the DocumentBuilderFactory; every time; triggering heavy IO contention because of Class loader / JAR file search operations.
Too many of these IO operations under heavy load will create excessive contention and severe performance degradation; regardless of the OS you are running your JVM on.
Please review the link below and see if this is related to your problem.. This is a known issue in JBOSS JIRA when using RichFaces / ajaxJSF.
https://issues.jboss.org/browse/JBPAPP-6166
Regards,
P-H
http://javaeesupportpatterns.blogspot.com/ -
Diff. times are taken to complete load unload archive and unarchive
Hi All,
I am facing a problem with loading unloading archiving and unarchiving repository.
Problem is Everyday it is taking different to complete the process. Is the time deneds on the data and images availavle in the repository?
If large amount of data is there then it will take more time to complete all these process or large amount of user are log in it will take more time to unload?
Please Suggest
ThanksHi Shalini,
The repository Archieving and Unarchieving operations involves the data base.
I mean Repository tables and fields and the data of the repository are in the underlying database and You can backup an MDM repository using the MDM archiving mechanism.That will save the repository scema .a2a file for furthere refrence.This schema contains the tables and fields inside the tables and all these things are storesd on the DBMS level.So suppose next time when you unarchieve the repository and made some changes like
1.You have added some new Tables and Fields.
2.You have added new Data.
And all this is getting stored at the database leve at the end.
In this case the amount of data of your repository in terms of Tables and Fields or Data at the DBMS Level.So next time when you take archieve of your Repository It will take some extra time as compare to the previous one.And same thing will happen in case of Unarchieving because this time some new tables and their fields and some new data(images,pdfs etc) they all will be unarchieved as a Part of Repository so it will take a bit extra time to Unarchieve.
Same thing will happen in case of Loading and Unloading as the tables ,fields and amount of data increases the time to load and unload will also increases.
At the time of loading the repository You can go to your MDM Server and check that repository in status it will show checking image ID's ,processing tables,processing sound tables,loading search indexes etc.
Making a few minor adjustment with MDM and/or the DBMS server can enhance search and update performance by 10% to 20%.
SQL Serveru2019s performance increases by about 10% when the main data file (.mdf) and the transaction log file (.ldf) are located on separate spindles. Remember that this means using different drives, not just
different drive letters.
Reward if Helpfull.
Regards,
Vinay Yadav -
VB6 and Excel loading then unloading immediately
First off I posted this here
http://social.msdn.microsoft.com/Forums/office/en-US/4c07879b-e033-4e8f-8266-95f9f606923a/vb6-and-excel-loading-then-unloading-immediately?forum=exceldev However I think this is the right forum for this.
We have an old VB6 application that uses data from and to Excel 97-2003 XLS and
Access MDB. This application is slated to be updated to VS2012 and VB.NET. In
the mean time I have a glitch that I cannot seem to find the issue. When we
load the VB6 app it opens Excel 2010 in the background for getting and setting
data. We are set to roll out Office 2010 on everyone’s system and I am still
unable to find why it works fine on two systems and not on others. With Office
2007 everything seems to work just fine.
What happens is when I look at the processes in Task Manager on my Windows 7
Enterprise edition with Office Professional Plus 2010 system while loading
the VB6 app I see?
EXCEL.EXE *32 | "My user name" | 00 CPU | 10,132k | Microsoft Excel
This is normal and stays in process the entire time I run it on my system. I also have
this working on another end users system. However the issue and the fix I used
on our two systems does not seem to work on other systems. I launch the VB6 app
and it comes up.
EXCEL.EXE *32 | "My user name" | 00 CPU | 10,132k | Microsoft Excel
This loads in the processes and then immediately unloads without any errors. Then when I
execute something that needs to use Excel I get.
Run-Time error ‘91’:
Object variable or With block variable not set.
This only happens if Excel unloads immediately otherwise the program works great. There
are no events in any of the event logs and our other IT guy said it is loading
and unloading normally with out crashing.
I have applied this to the systems http://support.microsoft.com/kb/926430/en-us . In
all of my searching this looked like the best answer. However only two systems
are working with Office Professional Plus 2010.
Here are the references and components.
References:
Visual Basic For Applications
Visual Basic runtime objects and procedures
Visual Basic objects and procedures
OLE Automation
Microsoft DAO 3.6 Object Library
Microsoft Excel 5.0 Object Library
Components:
Controls
Desaware Animated Button Control
Microsoft Common Dialog Control 6.0 (SP6)
Microsoft Data Bound Grid Control 5.0 (SP6)
Designers
Dat Environment
Data Report
DHTML Page
WebClass
Code:
Public xlApp As Excel.Application
Public WBSetup As Excel.Workbook
Public objWbCALC As Excel.Workbook
Public objShtLEVELS As Excel.Worksheet
Sub Main()
' There is more code here that is not specific to Excel issue
Set xlApp = Excel.Application
' Get the workbook.
Set objWbCALC = xlApp.Workbooks.Open("c:\temp\Calc.xls")
Set WBSetup = xlApp.Workbooks.Open("c:\temp\Setup.xls")
' Get worksheet
Set objShtLEVELS = objWbCALC.Worksheets("Levels")
' There is more code here that is not specific to Excel issue
End Sub
This is working on one system that had Office 2007 and was upgraded to Office 2010. Working on my system with Office 2010. The EXE file is ran from the server you can install it or add the OCX files to local system and register them with regsvr32. No
new components have been added or changed to the actual VB6 application. It seems to work on two systems and not on other systems.
However our test system have Office 2010 loaded on it and never had office 2007 on it. Maybe there is a residual left over file on it.
Any ideas?
www.ipdg3.comHello,
Thank you for your post.
I am afraid that the issue is out of support range of VS General Question forum which mainly discusses
WPF & SL designer, Visual Studio Guidance Automation Toolkit, Developer Documentation and Help System
and Visual Studio Editor.
I would like to know whether you can get the same issue when you create/load a Visual Basic application rather than VB6 in VS2012. If the issue only occurs when you load VB6 app, as Andrew said, there are other better forums for
VB6 app provided in this thread:
http://social.msdn.microsoft.com/Forums/vstudio/en-US/6a0719fe-14af-47f7-9f51-a8ea2b9c8d6b/where-to-post-your-vb-6-questions?forum=vbgeneral
For example:
VB Forums
VB City
Best regards,
We are trying to better understand customer views on social support experience, so your participation in this interview project would be greatly appreciated if you have time. Thanks for helping make community forums a great place.
Click
HERE to participate the survey. -
My application was designed based on MVC Architecture. But I made some changes to HMV base on my requirements. Servlet invoke helper classes, helper class uses EJBs to communicate with the database. Jsps also uses EJBs to backtrack the results.
I have two EJBs(Stateless), one Servlet, nearly 70 helperclasses, and nearly 800 jsps. Servlet acts as Controler and all database transactions done through EJBs only. Helper classes are having business logic. Based on the request relevant helper classed is invoked by the Servlet, and all database transactions are done through EJBs. Session scope is 'Page' only.
Now I am planning to use EJBs(for business logic) instead on Helper Classes. But before going to do that I need some clarification regarding Network traffic and for better usage of Container resources.
Please suggest me which method (is Helper classes or Using EJBs) is perferable
1) to get better performance and.
2) for less network traffic
3) for better container resource utilization
I thought if I use EJBs, then the network traffic will increase. Because every time it make a remote call to EJBs.
Please give detailed explanation.
thank you,
sudheer<i>Please suggest me which method (is Helper classes or Using EJBs) is perferable :
1) to get better performance</i>
EJB's have quite a lot of overhead associated with them to support transactions and remoteability. A non-EJB helper class will almost always outperform an EJB. Often considerably. If you plan on making your 70 helper classes EJB's you should expect to see a dramatic decrease in maximum throughput.
<i>2) for less network traffic</i>
There should be no difference. Both architectures will probably make the exact same JDBC calls from the RDBMS's perspective. And since the EJB's and JSP's are co-located there won't be any other additional overhead there either. (You are co-locating your JSP's and EJB's, aren't you?)
<i>3) for better container resource utilization</i>
Again, the EJB version will consume a lot more container resources. -
Dinamyc class loading and jar files
I'm new to java. I would like to load some plugins in my application (it's going to be packaged as a jar at the end). I managed to find some dinamyc class loading examples but they search for the classes to load in a directory.
This is the code:
public static void main(String[] args) {
File pluginDirectory = new File("src/pluginsreloaded/plugins");
if (!pluginDirectory.exists()) { // the plugin directory does not exist
System.out.println("The plugins directory does not exist!");
return;
FilenameFilter filter = new FilenameFilter() {
public boolean accept(File dir, String name) {
return name.endsWith(".class");
String[] pluginFiles = pluginDirectory.list(filter);
for (int i = 0; i < pluginFiles.length; i++) {
if (pluginFiles.indexOf("$") == -1) {
System.out.println("Loading: " + pluginFiles[i].substring(0, pluginFiles[i].length() - 6));
IPlugin plugin = pm.loadPlugin(pluginFiles[i].substring(0, pluginFiles[i].length() - 6));
System.out.println(plugin.description());
protected static IPlugin loadPlugin(String name) {
// Query the plugin list for the plugin
PluginFactory _plugin = (PluginFactory) pluginList.get(name);
if (_plugin == null) { // the plugin is not loaded
try {
Class.forName("pluginsReloaded.plugins." + name);
// The plugin makes an entry in the plugin list
// when loaded
_plugin = (PluginFactory) pluginList.get(name);
if (_plugin == null) {
return null;
} catch (ClassNotFoundException e) {
System.out.println("Plugin " + name + " not found!");
return _plugin.create();
}IPlugin is an interface. I am using netbeans 5.0. The error I get is this:
Exception in thread "main" java.lang.NoClassDefFoundError: pluginsReloaded/plugins/plugin1 (wrong name: pluginsreloaded/plugins/plugin1)
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:620)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:124)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:260)
at java.net.URLClassLoader.access$100(URLClassLoader.java:56)
at java.net.URLClassLoader$1.run(URLClassLoader.java:195)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:268)
at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:164)
at pluginsreloaded.PluginManager.loadPlugin(PluginManager.java:30)
at pluginsreloaded.Main.main(Main.java:44)
Java Result: 1As far as I can see it can't find the class. My question is how can I load the class/where can I find it?
Thanks.You can use the java.util.jar.JarFile class to enumerate the contents of a jar. It's not good practice to search for plugins in this way though. A plugin may well involve serveral classes, which don't all have to be internal. Furthermore its wise to avoid any comitment about the mechaism by which class files are to be fetched. You might want to do it remotely, some time, or load them from a database.
What seesms to be the standard approach is to put a list of plugin classes into the jar as a text file.
Plugins for a given purpose usually implement some specified interface, or extend some abstract class to which their objects can be cast once loaded (without this they aren't really much use).
Say your plugins implment org.myorg.WidgetFactory then you put a list of them, one fully qualified name to a line, in a file or jar entry called META-INF/services/org.myorg.WidgetFactory. You framework picks up all such files from the classpath using ClassLoader.getResources() and loads all the classes whose names if finds, hence you can add new sets of plugins just by adding a new jar or class directory to the class path. -
Dynamic class loading problem using unknown JAR archive and directory names
I read the following article, which enlightened me a lot:
Ted Neward: Understanding Class.forName().
However, it took me some while to understand that my problem is the other way around:
I know the name of the class, I know the name of the method,
but my program/JVM does not know where to load the classes from.
Shortly, my problem is that the server engine that I am writing
uses two different versions of the same library.
So I am trying out the following solution:
My program is named TestClassPathMain.java
Assume the two libraries are named JAR1.jar and JAR2.jar
and the class/instance method that should
be exposed to TestClassPathMain.java by them is named
TestClass1.testMethod().
As long as I was depending on just one library,
I put JAR1.jar in the classpath before starting java,
and I was happy for a while.
At the moment I got the need to use another version of
TestClass1.testMethod() packaged in JAR2.jar,
a call would always access JAR1.jar's
TestClass1.testMethod().
I then decided to remove JAR1.jar from the classpath,
and programmatically define two separate ClassLoaders, one for use
with JAR1.jar and the other for use with JAR2.jar.
However, the problem is only partly solved.
Please refer to the enclosed code for details.
(The code in the JAR1.jar/JAR2.jar is extremely simple,
it just tells (by hardcoding) the name of the jar it is packaged in
and instantiates another class packaged in the same jar using
the "new" operator and calls a method on it. I don't enclose it.)
The TestClassPathMain.java/UC1.java/UC2.java code suite was
successfully compiled with an arbitrary of JAR1 or JAR2 in the classpath,
however removed from the classpath at runtime.
(I know that this could have been done (more elegantly...?) by producing an Interface,
but I think the main problem principle is still untouched by this potential lack of elegancy(?))
1) This problem should not be unknown to you experts out there,
how is it generally and/or most elegantly solved?
The "*** UC2: Variant 2" is the solution I would like best, had it only worked.
2) And why arent "*** UC2: Variant 2" and
"*** static UC2: Variant 2" working,
while "*** Main: Variant 2" is?
3) And a mal-apropos:
Why can't I catch the NoClassDefFoundError?
The output:
*** Main: Variant 1 JAR 1 ***:
Entering TestClass1.testMethod() packaged in JAR1.jar
About to instantiate TestClass2 with the new operator
About to call TestClass2.testMethod()
Entering TestClass2.testMethod() packaged in JAR1.jar
*** Main: Variant 1 JAR 2 ***:
Entering TestClass1.testMethod() packaged in JAR2.jar
About to instantiate TestClass2 with the new operator
About to call TestClass2.testMethod()
Entering TestClass2.testMethod() packaged in JAR2.jar
*** Main: Variant 2 JAR 1 ***:
Entering TestClass1.testMethod() packaged in JAR1.jar
About to instantiate TestClass2 with the new operator
About to call TestClass2.testMethod()
Entering TestClass2.testMethod() packaged in JAR1.jar
*** Main: Variant 2 JAR 2 ***:
Entering TestClass1.testMethod() packaged in JAR2.jar
About to instantiate TestClass2 with the new operator
About to call TestClass2.testMethod()
Entering TestClass2.testMethod() packaged in JAR2.jar
*** UC1: Variant 1 JAR 1 ***:
Entering TestClass1.testMethod() packaged in JAR1.jar
About to instantiate TestClass2 with the new operator
About to call TestClass2.testMethod()
Entering TestClass2.testMethod() packaged in JAR1.jar
*** UC1: Variant 1 JAR 2 ***:
Entering TestClass1.testMethod() packaged in JAR2.jar
About to instantiate TestClass2 with the new operator
About to call TestClass2.testMethod()
Entering TestClass2.testMethod() packaged in JAR2.jar
*** static UC2: Variant 2 JAR 1 ***:
Exception in thread "main" java.lang.NoClassDefFoundError: TestClass1
at UC2.runFromJarVariant2_static(UC2.java:56)
at TestClassPathMain.main(TestClassPathMain.java:52)
TestClassPathMain.java
import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;
import java.net.MalformedURLException;
import java.net.URL;
import java.net.URLClassLoader;
public class TestClassPathMain {
public static void main(final String args[]) throws MalformedURLException, ClassNotFoundException, InstantiationException, SecurityException, NoSuchMethodException, IllegalArgumentException, IllegalAccessException, InvocationTargetException {
// Commented out because I cannot catch the NoClassDefFoundError.
// Why?
try {
final TestClass1 testClass1 = new TestClass1();
System.out.println(
"\nThe class TestClass1 is of some unexplicable reason available." +
"\nFor the purpose of the test, it shouldn't have been!" +
"\nExiting");
System.exit(1);
} catch (NoClassDefFoundError e) {
System.out.println("\nPositively confirmed that the class TestClass1 is not available:\n" + e);
System.out.println("\n\nREADY FOR THE TEST: ...");
// Works fine
System.out.println("\n*** Main: Variant 1 JAR 1 ***:");
runFromJarVariant1("file:/W:/java/eclipse/workspaces/simped_test/CP1/JAR1.jar");
System.out.println("\n*** Main: Variant 1 JAR 2 ***:");
runFromJarVariant1("file:/W:/java/eclipse/workspaces/simped_test/CP2/JAR2.jar");
// Works fine
System.out.println("\n*** Main: Variant 2 JAR 1 ***:");
runFromJarVariant1("file:/W:/java/eclipse/workspaces/simped_test/CP1/JAR1.jar");
System.out.println("\n*** Main: Variant 2 JAR 2 ***:");
runFromJarVariant1("file:/W:/java/eclipse/workspaces/simped_test/CP2/JAR2.jar");
// Works fine
final UC1 uc1 = new UC1();
System.out.println("\n*** UC1: Variant 1 JAR 1 ***:");
uc1.runFromJarVariant1("file:/W:/java/eclipse/workspaces/simped_test/CP1/JAR1.jar");
System.out.println("\n*** UC1: Variant 1 JAR 2 ***:");
uc1.runFromJarVariant1("file:/W:/java/eclipse/workspaces/simped_test/CP2/JAR2.jar");
// Crashes
System.out.println("\n*** static UC2: Variant 2 JAR 1 ***:");
UC2.runFromJarVariant2_static("file:/W:/java/eclipse/workspaces/simped_test/CP1/JAR1.jar");
System.out.println("\n*** static UC2: Variant 2 JAR 2 ***:");
UC2.runFromJarVariant2_static("file:/W:/java/eclipse/workspaces/simped_test/CP2/JAR2.jar");
// Crashes
final UC2 uc2 = new UC2();
System.out.println("\n*** UC2: Variant 2 JAR 1 ***:");
uc2.runFromJarVariant2("file:/W:/java/eclipse/workspaces/simped_test/CP1/JAR1.jar");
System.out.println("\n*** UC2: Variant 2 JAR 2 ***:");
uc2.runFromJarVariant2("file:/W:/java/eclipse/workspaces/simped_test/CP2/JAR2.jar");
private static void runFromJarVariant1(final String jarFileURL)
throws MalformedURLException,
ClassNotFoundException,
InstantiationException,
IllegalArgumentException,
IllegalAccessException,
InvocationTargetException,
SecurityException,
NoSuchMethodException {
final URL url = new URL(jarFileURL);
final URLClassLoader cl =
new URLClassLoader(new URL[]{url},
Thread.currentThread().getContextClassLoader());
final Class clazz = cl.loadClass("TestClass1");
final Object testClass1 = clazz.newInstance();
final Method testMethod1 = clazz.getMethod("testMethod", null);
testMethod1.invoke(testClass1, null);
private static void runFromJarVariant2(final String jarFileURL)
throws MalformedURLException,
ClassNotFoundException,
InstantiationException,
IllegalArgumentException,
IllegalAccessException,
InvocationTargetException,
SecurityException,
NoSuchMethodException {
final URL url = new URL(jarFileURL);
final URLClassLoader cl =
new URLClassLoader(new URL[]{url},
Thread.currentThread().getContextClassLoader());
final Class clazz = cl.loadClass("TestClass1");
final TestClass1 testClass1 = new TestClass1();
testClass1.testMethod();
UC1.java
import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;
import java.net.MalformedURLException;
import java.net.URL;
import java.net.URLClassLoader;
public class UC1 {
public void runFromJarVariant1(final String jarFileURL)
throws MalformedURLException,
ClassNotFoundException,
InstantiationException,
IllegalArgumentException,
IllegalAccessException,
InvocationTargetException,
SecurityException,
NoSuchMethodException {
final URL url = new URL(jarFileURL);
final URLClassLoader cl =
new URLClassLoader(new URL[]{url},
Thread.currentThread().getContextClassLoader());
final Class clazz = cl.loadClass("TestClass1");
final Object testClass1 = clazz.newInstance();
final Method testMethod1 = clazz.getMethod("testMethod", null);
testMethod1.invoke(testClass1, null);
UC2.java
import java.lang.reflect.InvocationTargetException;
import java.net.MalformedURLException;
import java.net.URL;
import java.net.URLClassLoader;
public class UC2 {
public void runFromJarVariant2(final String jarFileURL)
throws MalformedURLException,
ClassNotFoundException,
InstantiationException,
IllegalArgumentException,
IllegalAccessException,
InvocationTargetException,
SecurityException,
NoSuchMethodException {
final URL url = new URL(jarFileURL);
final URLClassLoader cl =
new URLClassLoader(new URL[]{url},
Thread.currentThread().getContextClassLoader());
final Class clazz = cl.loadClass("TestClass1");
final TestClass1 testClass1 = new TestClass1();
testClass1.testMethod();
* Identic to the "runFromJarVariant2" method,
* except that it is static
public static void runFromJarVariant2_static(final String jarFileURL)
throws MalformedURLException,
ClassNotFoundException,
InstantiationException,
IllegalArgumentException,
IllegalAccessException,
InvocationTargetException,
SecurityException,
NoSuchMethodException {
final URL url = new URL(jarFileURL);
final URLClassLoader cl =
new URLClassLoader(new URL[]{url},
Thread.currentThread().getContextClassLoader());
final Class clazz = cl.loadClass("TestClass1");
final TestClass1 testClass1 = new TestClass1();
testClass1.testMethod();
}2. i need to load the class to the same JVM (i.e. to
the same environment) of the current running
aplication, so that when the loaded class is run, it
would be able to invoke methods on it!!!
ClassLoader(s) do this. Try the URLClassLoader.
(I was talking about relatively esoteric "security"
issues when I mentioned the stuff about Class objects
"scope".) You might use the URLClassLoader kind of
like this.
Pseudo-code follows:
// setup the class loader
URL[] urls = new URL[1];
urls[0] = new URL("/path/to/dynamic/classes");
URLClassLoader ucl = new URLClassLoader(urls);
// load a class & use make an object with the default constructor
Object tmp = ucl.loadClass("dynamic.class.name").newInstance();
// Cast the object to a know interface so that you can use it.
// This may be used to further determine which interface to cast
// the class to. Or it may simply be the interface to which all
// dynamic classes have to conform in your program.
InterfaceImplementedByDynamicClass loadedObj =
(InterfaceImplementedByDynamicClass)tmp;It's really not as hard as it sounds, just write a little test of
this and you will see how it works. -
Dynamic Class Loading and Stubs
How Dynamic Class Loading is used on Java RMI, since stubs are generated on clients using Reflection API?
I was reading Dynamic code downloading using JavaTM RMI (http://java.sun.com/javase/6/docs/technotes/guides/rmi/codebase.html), it seems to be out of date.
For example, "The client requests the class definition from the codebase. The codebase the client uses is the URL that was annotated to the stub instance when the stub class was loaded by the registry. Back in step 1, the annotated stub for the exported object was then registered with the Java RMI registry bound to a name.""Enhancements in J2SETM 5.0
Dynamic Generation of Stub Classes
This release adds support for the dynamic generation of stub classes at runtime, obviating the need to use the Java(tm) Remote Method Invocation (Java RMI) stub compiler, rmic, to pregenerate stub classes for remote objects. *Note that rmic must still be used to pregenerate stub classes for remote objects that need to support clients running on _earlier versions_.*
When an application exports a remote object (using the constructors or static exportObject methods(1) of the classes java.rmi.server.UnicastRemoteObject or java.rmi.activation.Activatable) and a pregenerated stub class for the remote object's class cannot be loaded, the remote object's stub will be a java.lang.reflect.Proxy instance (whose class is dynamically generated) with a java.rmi.server.RemoteObjectInvocationHandler as its invocation handler.
An existing application can be deployed to use dynamically generated stub classes unconditionally (that is, whether or not pregenerated stub classes exist) by setting the system property java.rmi.server.ignoreStubClasses to "true". If this property is set to "true", pregenerated stub classes are never used.
Notes:
* If a remote object has pre-5.0 clients, that remote object should use a stub class pregenerated with rmic or the client will get an ClassNotFoundException deserializing the remote object's stub. Pre-5.0 clients will not be able to load an instance of a dynamically generated stub class, because such a class contains an instance of RemoteObjectInvocationHandler, which was not available prior to this release.
* Prior to the J2SE 5.0 release, exporting a remote object would throw a java.rmi.StubNotFoundException if the pregenerated stub class for the remote object's class could not be loaded. With the added support for dynamically generated stub classes, exporting a remote object that has no pregenerated stub class will silently succeed instead. A user deploying a server application to support pre-5.0 clients must still make sure to pregenerate stub classes for the server's remote object classes, even though missing stub classes are no longer reported at export time. Such errors will instead be reported to a pre-5.0 client when it deserializes a dynamically generated stub class.
(1) The static method UnicastRemoteObject.exportObject(Remote) is declared to return java.rmi.server.RemoteStub and therefore cannot be used to export a remote object to use a dynamically generated stub class for its stub. An instance of a dynamically generated stub class is a java.lang.reflect.Proxy instance which is not assignable to RemoteStub."
http://java.sun.com/j2se/1.5.0/docs/guide/rmi/relnotes.html -
Hello,
I am trying to write a application that will be able to load unknown classes and find out their methods and fields. My problem is that i don't know how to extraxt the necessary information(methods, fields) from the classes. The first answer would be probably reflection, but as i've seen from a quick look at its tutorial it can give me only the public methods-fields(i think also private fields but not certain) of a class, but i need also the private and protected methods-fields that the class may have.
Basically, i need someone to point me to the right direction(class loader-verifier or something) of how to do these things and after that i'll go in depth...
Thanks in advance,
AntonyReflection, i.e. Class.forName(..).get...(), will give you all you need, including privates. I believe the security policy determines how much access your code gets to privates.
Maybe you are looking for
-
my daughter synced her 3g in my account accidentally now since ive synced my iphone 4, her contacts and mine are combined and my contacts are a mess . example, my favorites have a few names and numbers but some are gone completely, and in contacts so
-
After upgrading on both a Mac Pro and MacBook Pro I encountered several problems with Mail. On MacBook Pro: Unable to receive email for certain accounts Was able to delete the offending accounts and add them and this solved the problem. On Mac Pro: U
-
Often when I click a link within a web page, a message bar appears at the top of the page saying "Firefox has blocked this site from loading another page" or something similar. This happens frequently on banking or credit card sites. This is particul
-
could anybody explain me the procedure to create ALV reports?in detail?
-
HI, We are using Sun IDM to do provisioning for an service provider. The data model differs from the standard user view, where one Person has many Accounts on many Resources. The data model we are trying to represent is complicated: but simply it is