Dynamic compilation in memory
Hello,
I have used javax.tools.ToolProvider to compile a simple class that comes from a CharSequence object in memory, and it works just fine.
Now, I need to reference another class that comes also from a CharSequence object in memory. Which method in the ForwardingJavaFileManager I should override to make this class available to the main class.
Thank you
The class I need to reference is in class (.class) format not the java source format (.java).
Edited by: Gen.Java on Apr 23, 2013 2:05 AM
CharSequence src = "public class DynaClass {public String toString (){return"hello";}}";Using javax.tools API, this compiles just fine. However,the following does not, complaining that it cannot find the class "another". The bytecode for the class "another" is in memory (not the file system). I just need to make it available to the main class "DynaClass".
CharSequence src = "public class DynaClass {public String toString (){return new another().get();}}";The following is my code:
import java.util.*;
import javax.tools.*;
import java.io.*;
public class DynaCompTest
public static void main(String[] args) throws Exception
JavaCompiler compiler;
ClassFileManager fileManager;
List<JavaFileObject> jfiles;
CharSequence src;
String fullName;
fullName = "DynaClass";
src = "public class DynaClass {public String toString (){return"hello";}}";
compiler = ToolProvider.getSystemJavaCompiler ();
fileManager = new ClassFileManager ( compiler.getStandardFileManager(null, null, null) );
jfiles = new ArrayList<JavaFileObject>();
jfiles.add ( new CharSequenceJavaFileObject(fullName,src) );
compiler.getTask ( null, fileManager, null, null, null, jfiles).call ();
import java.net.*;
import javax.tools.*;
public class CharSequenceJavaFileObject extends SimpleJavaFileObject
private CharSequence content;
public CharSequenceJavaFileObject (
String className,
CharSequence content )
super ( URI.create("string:///" + className.replace('.', '/')+ Kind.SOURCE.extension), Kind.SOURCE);
this.content = content;
@Override
public CharSequence getCharContent (
boolean ignoreEncodingErrors )
return content;
import java.security.*;
import java.io.*;
import java.net.*;
import java.util.*;
import javax.tools.*;
import javax.tools.JavaFileObject.*;
public class ClassFileManager extends ForwardingJavaFileManager<JavaFileManager>
private final Map<String,JavaClassObject> fileObjects;
public ClassFileManager (
StandardJavaFileManager standardManager )
super ( standardManager );
fileObjects = new HashMap<String,JavaClassObject>();
@Override
public ClassLoader getClassLoader (
Location location )
return new ClassLoader() {
@Override
protected Class<?> findClass(String name) throws ClassNotFoundException {
JavaClassObject jco;
jco = fileObjects.get(name);
byte[] b = jco.getByteCode();
return super.defineClass(name, b, 0, b.length);
@Override
public JavaFileObject getJavaFileForOutput (
Location location,
String className,
Kind kind,
FileObject sibling ) throws IOException
JavaClassObject jco;
jco = new JavaClassObject(className, kind);
fileObjects.put ( className,jco );
return jco;
import java.net.*;
import java.io.*;
import javax.tools.*;
public class JavaClassObject extends SimpleJavaFileObject
protected final ByteArrayOutputStream bos = new ByteArrayOutputStream();
public JavaClassObject (
String className,
Kind kind )
super ( URI.create("string:///" + className.replace('.', '/')+ kind.extension), kind );
public byte[] getByteCode ()
return bos.toByteArray();
@Override
public OutputStream openOutputStream () throws IOException
return bos;
}
Similar Messages
-
A Question for the Cracks - dynamic Compilation in Memory?
HI
i just wondered if it is possible to compile a java source file during runtime in MEMORY, means, without writing the class file to the filesystem but being able to create an instance of that compiled class file.
I guess, theoreticaly this must be possible, since i could try to take the output of the compiler as an input stream in my application an save that stream as a String (or somehow different) and then try to create the an instance using classloader and serialization (somehow.... problem is, i don't know yet how this excatly could be done).
I know, that question sounds a little bit strange, but I'm just thinking about a licening methode and this could do a very important part of it.
So i would be very thankfull for any comment about this idea!
joshYeah, but here's the thing. If you provide the source files to the program, then you've put the thief one step closer to having your program (they don't have to decompile it.) Even if you encrypt the source, once they've decrypted that they're still one step closer than if you just encrypted the bytecode.
In the encrypted class file solution, if the user doesn't have a legal license, they cannot run the program, because they cannot decrypt the classes. They have the encryption algorithm, but since they don't have a private key (which is in the license file) they can't load the code.
The license also, would be one-license-per-user, with a multitude of different licenses able to decrypt the same encrypted classes. If you don't want a particular party to have the program, you just don't give them a key, or give them a key which decrypts only a certain set of classes (effectively providing them with a functional demo.)
Even once the user has a license and your algorithm though, it still isn't trivial for them to make decrypted .class files out of them though, since your algorithm will load the classes as soon as they're created. They'd have to decompile and hack up the decryption code to write the classes back to files also.
You could probably also figure out some way to stamp an ID on each decrypted class at runtime, so you can tell which user is running the program so if someone does distribute decrypted versions of the classes, you can tell which user did it and start legal proceedings. :-) -
Compiling (in memory) and Loading classes on fly
Hiya,
I have following classes:
public class Vehicle {Protected Sting reg;}
public class Car extends Vehicle
(public void print(){System.out.println(reg);}}
These classes (or different ones) are stored in one text file. The program must read these files then compile and load them dynamically in memory.
The link below authored by DCHSW deals with one class only (compile and loading) however it fails with more then one class particularly when it is subclassed.
http://forum.java.sun.com/thread.jsp?thread=347467&forum=4&message=1439418
How can we get the code mentioned at link above to compile (in memory) and load subclasses like the ones mentioned above.
Thanks
DerikHiya,
I have following classes:
public class Vehicle {Protected Sting reg;}
public class Car extends Vehicle
(public void print(){System.out.println(reg);}}
These classes (or different ones) are stored in one
text file. The program must read these files then
compile and load them dynamically in memory.
The link below authored by DCHSW deals with one class
only (compile and loading) however it fails with more
then one class particularly when it is subclassed.
Read the following post for full description of the problem
http://forum.java.sun.com/thread.jsp?forum=4&thread=417724
Any ideas?
Thanks
Derik
http://forum.java.sun.com/thread.jsp?thread=347467&foru
=4&message=1439418
How can we get the code mentioned at link above to
compile (in memory) and load subclasses like the ones
mentioned above.
Thanks
Derik -
Dynamic compilation : getting number of errors
Hi guys,
in my application i'm dynamically compiling java files and then processing results later.
In this i opted for the eclipse jdt compiler that i embedded in my application.
At this stage i need to get a critical information from the compiler :
if compilation fails i need a way to get how many errors were generated by the compiler ( Errors without warnings)
Is it possible to get this information ?
can we do that with eclipse jdt compiler API ? and how ?
is there another way to solve this if we can't do it with jdt compiler ?
I appreciate a lot any help on this issue.
thanking you.Thanks, I got the answer. xfa.host.numPages provides me the n
umber of pageas. -
I am reading a book Java 6 Platform Revealed which describes using of Compiler API. Unfortunately Compiler API was changed during development of Mustang.
This sample stopped to work although I renamed names of classes into new ones.
I am able to compile the source of HelloWorld class, but I don't know how to invoke a new instance of this successfully compiled class (I get permanent ClassNotFoundException).
How to get it work?
Thank you for your help
Cavity
PS. I don't want to use extern files (as described here http://javainsel.blogspot.com/2006/03/example-for-using-java-6-compiler-api.html), I would like to make it as fast as possible (=in memory).
Java SE 6 final
import javax.tools.*;
import java.io.IOException;
import java.io.PrintWriter;
import java.io.StringWriter;
import java.lang.reflect.InvocationTargetException;
import java.net.URI;
import java.util.Arrays;
public class CompileSource {
private CompileSource() {
public static void main(String args[]) throws IOException {
JavaCompiler compiler = ToolProvider.getSystemJavaCompiler();
DiagnosticCollector<JavaFileObject> diagnostics =
new DiagnosticCollector<JavaFileObject>();
StringWriter writer = new StringWriter();
PrintWriter out = new PrintWriter(writer);
out.println("public class HelloWorld {");
out.println(" public static void main(String args[]) {");
out.println(" System.out.println(\"Hello, World\");");
out.println(" }");
out.println("}");
out.close();
JavaFileObject file =
new JavaSourceFromString("HelloWorld", writer.toString());
Iterable<? extends JavaFileObject> compilationUnits =
Arrays.asList(file);
JavaCompiler.CompilationTask task = compiler.getTask(
null, null, diagnostics, null, null, compilationUnits);
boolean success = task.call();
for (Diagnostic diagnostic : diagnostics.getDiagnostics())
System.console().printf(
"Code: %s%n" +
"Kind: %s%n" +
"Position: %s%n" +
"Start Position: %s%n" +
"End Position: %s%n" +
"Source: %s%n" +
"Message: %s%n",
diagnostic.getCode(), diagnostic.getKind(),
diagnostic.getPosition(), diagnostic.getStartPosition(),
diagnostic.getEndPosition(), diagnostic.getSource(),
diagnostic.getMessage(null));
System.out.println("Success: " + success);
if (success) {
try {
System.out.println("-----Output-----");
Class.forName("HelloWorld").getDeclaredMethod("main",
new Class[]{String[].class}).invoke(null, new Object[]{null});
System.out.println("-----Output-----");
} catch (ClassNotFoundException e) {
System.err.println("Class not found: " + e);
} catch (NoSuchMethodException e) {
System.err.println("No such method: " + e);
} catch (IllegalAccessException e) {
System.err.println("Illegal access: " + e);
} catch (InvocationTargetException e) {
System.err.println("Invocation target: " + e);
static public class JavaSourceFromString extends SimpleJavaFileObject {
final String code;
JavaSourceFromString(String name, String code) {
super(URI.create(
"string:///" + name.replace('.', '/') + Kind.SOURCE.extension),
Kind.SOURCE);
this.code = code;
@Override
public CharSequence getCharContent(boolean ignoreEncodingErrors) {
return code;
}Message was edited by:
CavityThis is exactly what I was looking for the whole day. Thank you very much Peter.
Unfortunately I cannot give you your deserved dukes :-((.
This forum is buggy, i cannot log in under my previous account :-((( and reseting password without "security" question does not work either. -
Hi,
I am currently writing a system in which i need to dynamically generate a java source file, compile it, and then run it, all through Java.
To clarify, during the running of the program, a copy is taken of a current Java source file, and then additional method calls are inserted into the copy of the source file. Then i need to compile the "new" Java source file, and run the comiled class file.
is this possible? if so, how?
thanks
jamie.It's not documented (or at least you can't download the documentation).
In my example, I'm importing from a subpackage v8. Open up your tools.jar and find out what the name of the packages are.
I've created a small app that gets the class info using reflection
package com.sun.tools.javac.v8;
import com.sun.tools.javac.v8.code.Symbol;
import com.sun.tools.javac.v8.tree.Tree;
import com.sun.tools.javac.v8.util.Context;
import com.sun.tools.javac.v8.util.List;
import java.io.InputStream;
public JavaCompiler implements ClassReader$SourceCompleter {
public boolean verbose;
public boolean sourceOutput;
public boolean attrParseOnly;
public boolean classOutput;
public boolean printFlat;
public boolean deprecation;
public boolean warnunchecked;
public boolean genCrt;
public String encoding;
public JavaCompiler(Context) {}
public com.sun.tools.javac.v8.util.List compile(List) throws Throwable {}
public void close() {}
public Tree$TopLevel parse(String) {}
public Tree$TopLevel parse(String, InputStream) {}
public static String version() {}
public void complete(Symbol$ClassSymbol, String, InputStream) throws Symbol$CompletionFailure {}
public int errorCount() {}
public static JavaCompiler make(Context) {}
public InputStream openSource(String) {}
public native int hashCode();
public final native Class getClass();
public final void wait() throws InterruptedException {}
public final native void wait(long) throws InterruptedException;
public final void wait(long, int) throws InterruptedException {}
public boolean equals(java.lang.Object) {}
public final native void notify();
public final native void notifyAll();
public java.lang.String toString() {}
} -
FPGA dynamic read of memory item
I am using a flex rio PXIe-7961R/NI-6585. When I try to complie my code i get
An internal software error has occurred. Please contact National Instruments technical support at ni.com/support with the following information:
Error -61175 occurred at
Possible reason(s):
LabVIEW FPGA: FPGA FIFO Node not wired with constant FPGA FIFO Name.
=========================
LabVIEW FPGA: FPGA FIFO Node not wired with constant FPGA FIFO Name.
I am trying to read from block memory in a subvi that gets a memory name passed into it. I get the error in the writeTxFifo vi. The tlmMessageBuilder is the calling the vi. Any ideas on how to fix this problem?
Attachments:
tlmMessageBuilder.vi 56 KB
writeTxFifo.vi 41 KBI just took another look at the VI you posted.
The error message you're getting probably has nothing to do with the case structure. The error is a result of using a control as the source of your reference rather than a constant.
National Instruments
FlexRIO Product Support Engineer -
Include an in-memory jar file for JSR-199 compilation
I want to compile a source file in memory, which requires a jar file that is also represented in memory. I used the JavaSourceFromString class recommended in the documentation for the class JavaCompiler and in a demo I found online that shows how to compile sources represented as String in memory. To represent the jar file, I used a similar trick to JavaSourceFromString:
public class JarJavaFileObjectFromByteArray extends SimpleJavaFileObject {
* The contents of this jar file.
private final byte[] contents;
* Constructs a new JarJavaFileObjectFromByteArray given the name and binary
* contents of a jar file.
* @param name the name of this jar file
* @param contents the contents of this jar file
public JarJavaFileObjectFromByteArray(String name, byte[] contents) {
super(newURI(name), Kind.OTHER);
this.contents = contents;
... // code not shown ensures that the URI returned from newURI is of the form,
// for instance, bytes:///MathConstants.jar, if the name of the jar file is MathConstants.jar
@Override
public InputStream openInputStream() throws IOException {
return new ByteArrayInputStream(contents);
}However, I do not know how to alert the compiler that this jar file should be on the classpath. I tried this:
List<String> options = Arrays.asList(" -classpath bytes:///MathConstants.jar ");
CompilationTask task = compiler.getTask(null, fileManager, null, options, null, compilationUnits);but I get the following error when calling getTask:
java.lang.IllegalArgumentException: invalid flag: -classpath bytes:///MathConstants.jarI assume there is some way to tell the compiler, "look at the MathConstants.jar file that I am storing in memory when searching the classpath", but I do not know how to do this. I assumed that the options parameter for getTask represents command-line flags that would be passed to the compiler if this were happening on the command line (such as "-cp .", which also does not work), but perhaps this assumption is wrong.Hi Bruce,
I have a question regarding loading a jar file by the compiler to dynamically compile with a source file. I hope you can probably offer me an idea on what has been missing or wrong with the source codes I have written for my application.
I am using Eclipse compiler to dynamically compile a class. In the class, I want it to make a reference to a jar file for compilation dynamically.
Here is the source of a test class I wrote:
import javax.servlet.http.HttpServlet;
class MyServlet extends HttpServlet {
}The import statement refers to the class javax.servlet.http.HttpServlet from the jar file C:\\Program Files\\Apache Software Foundation\\Tomcat 6.0\\lib\\servlet-api.jar placed in the file system.
In the method called compileClass (shown below), I used the path of the jar file to add to the option -classpath as you suggested.
private static CompileClassResult compileClass(Writer out, String className, String classSource) {
try {
JavaCompiler javac = new EclipseCompiler();
StandardJavaFileManager sjfm = javac.getStandardFileManager(null, null, null);
SpecialClassLoader scl = new SpecialClassLoader();
SpecialJavaFileManager fileManager = new SpecialJavaFileManager(sjfm, scl);
List<String> options = new ArrayList<String>();
options.addAll(Arrays.asList("-classpath", "C:\\Program Files\\Apache Software Foundation\\Tomcat 6.0\\lib\\servlet-api.jar"));
List<MemorySource> compilationUnits = Arrays.asList(new MemorySource(className, classSource));
DiagnosticListener<JavaFileObject> diagnosticListener = null;
Iterable<String> classes = null;
if (out == null) {
out = new PrintWriter(System.err);
JavaCompiler.CompilationTask compile = javac.getTask(out, fileManager, diagnosticListener, options, classes, compilationUnits);
boolean res = compile.call();
if (res) {
//Need to modify the api to return an array of two elements - one classes and other bytecodes for all classes in the same class file.
return CompileClassResult.newInstance(scl.findClasses(), scl.findByteCodes());
} catch (Exception e) {
e.printStackTrace();
return null;
}I also extended the class ForwardingJavaFileManager as you suggested and have it delegated to the StandardJavaFileManager sent to the compiler mentioned in the method compileClass above. The extended class (called SpecialJavaFileManager) is as follows:
public class SpecialJavaFileManager extends ForwardingJavaFileManager<StandardJavaFileManager> {
private SpecialClassLoader xcl;
public SpecialJavaFileManager(StandardJavaFileManager sjfm, SpecialClassLoader xcl) {
super(sjfm);
System.out.println("SpecialJavaFileManager");
this.xcl = xcl;
public JavaFileObject getJavaFileForOutput(Location location, String name, JavaFileObject.Kind kind, FileObject sibling) throws IOException {
System.out.println("getJavaFileForOutput");
MemoryByteCode mbc = new MemoryByteCode(name);
xcl.addClass(name, mbc);
return mbc;
public Iterable<JavaFileObject> list(JavaFileManager.Location loc, String pkg, Set kinds, boolean recurse) throws IOException {
System.out.println("list ");
List<JavaFileObject> result = new ArrayList<JavaFileObject>();
for (JavaFileObject f : super.list(loc, pkg, kinds, recurse)) {
System.out.println(f);
result.add(f);
return result;
}I run the application and the result shows that it didn't load the jar file into the memory as expected. From the output (below) I got, it doesn't seem to invoke the method list(...) in the class SpecialJavaFileManager.
SpecialJavaFileManager
1. ERROR in \MyServlet.java (at line 1)
import javax.servlet.http.*;
^^^^^^^^^^^^^
The import javax.servlet cannot be resolved
2. ERROR in \MyServlet.java (at line 3)
class MyServlet extends HttpServlet {
^^^^^^^^^^^
HttpServlet cannot be resolved to a typeWould you please let me know what has possibly be missing or wrong?
Thanks.
Jonathan
Edited by: jonathanlam on Aug 10, 2009 6:47 PM -
Add jar for in-memory compilation by Eclipse Compiler
Hi,
I have a question regarding loading a jar file by the compiler to dynamically compile with a source file. I hope someone can probably offer me an idea on what has been missing or wrong with the source codes I have written for my application.
I am using Eclipse compiler to dynamically compile a class. In the class, I want it to make a reference to a jar file for compilation dynamically.
Here is the source of a test class I wrote:
import javax.servlet.http.HttpServlet;
class MyServlet extends HttpServlet {
}The import statement refers to the class javax.servlet.http.HttpServlet from the jar file C:\\Program Files\\Apache Software Foundation\\Tomcat 6.0\\lib\\servlet-api.jar placed in the local file system.
In the method called compileClass (shown below), I used the path of the jar file to add to the option -classpath as suggested in another thread (http://forums.sun.com/thread.jspa?threadID=5306520&start=0&tstart=0).
private static CompileClassResult compileClass(Writer out, String className, String classSource) {
try {
JavaCompiler javac = new EclipseCompiler();
StandardJavaFileManager sjfm = javac.getStandardFileManager(null, null, null);
SpecialClassLoader scl = new SpecialClassLoader();
SpecialJavaFileManager fileManager = new SpecialJavaFileManager(sjfm, scl);
List<String> options = new ArrayList<String>();
options.addAll(Arrays.asList("-classpath", "C:\\Program Files\\Apache Software Foundation\\Tomcat 6.0\\lib\\servlet-api.jar"));
List<MemorySource> compilationUnits = Arrays.asList(new MemorySource(className, classSource));
DiagnosticListener<JavaFileObject> diagnosticListener = null;
Iterable<String> classes = null;
if (out == null) {
out = new PrintWriter(System.err);
JavaCompiler.CompilationTask compile = javac.getTask(out, fileManager, diagnosticListener, options, classes, compilationUnits);
boolean res = compile.call();
if (res) {
//Need to modify the api to return an array of two elements - one classes and other bytecodes for all classes in the same class file.
return CompileClassResult.newInstance(scl.findClasses(), scl.findByteCodes());
} catch (Exception e) {
e.printStackTrace();
return null;
}I also extended the class ForwardingJavaFileManager as suggested in the thread mentioned above and have it delegated to the StandardJavaFileManager sent to the compiler mentioned in the method compileClass above. The extended class (called SpecialJavaFileManager) is as follows:
public class SpecialJavaFileManager extends ForwardingJavaFileManager<StandardJavaFileManager> {
private SpecialClassLoader xcl;
public SpecialJavaFileManager(StandardJavaFileManager sjfm, SpecialClassLoader xcl) {
super(sjfm);
System.out.println("SpecialJavaFileManager");
this.xcl = xcl;
public JavaFileObject getJavaFileForOutput(Location location, String name, JavaFileObject.Kind kind, FileObject sibling) throws IOException {
System.out.println("getJavaFileForOutput");
MemoryByteCode mbc = new MemoryByteCode(name);
xcl.addClass(name, mbc);
return mbc;
public Iterable<JavaFileObject> list(JavaFileManager.Location loc, String pkg, Set kinds, boolean recurse) throws IOException {
System.out.println("list ");
List<JavaFileObject> result = new ArrayList<JavaFileObject>();
for (JavaFileObject f : super.list(loc, pkg, kinds, recurse)) {
System.out.println(f);
result.add(f);
return result;
}I run the application and the result shows that it didn't load the jar file into the memory as expected. From the output (below) I got, it doesn't seem to invoke the method list(...) in the class SpecialJavaFileManager.
SpecialJavaFileManager
1. ERROR in \MyServlet.java (at line 1)
import javax.servlet.http.*;
^^^^^^^^^^^^^
The import javax.servlet cannot be resolved
2. ERROR in \MyServlet.java (at line 3)
class MyServlet extends HttpServlet {
^^^^^^^^^^^
HttpServlet cannot be resolved to a typeDoes somebody know what I probably have missed or done wrong in the codes?
Thanks.
JonathanHi jschell,
I read the thread referred to by the link you gave and tried to add the path using the setLocation method:
JavaCompiler javac = new EclipseCompiler();
StandardJavaFileManager sjfm = javac.getStandardFileManager(null, null, null);
SpecialClassLoader scl = new SpecialClassLoader();
List<File> path = Arrays.asList(new File("C:\\Program Files\\Apache Software Foundation\\Tomcat 6.0\\lib\\servlet-api.jar"));
sjfm.setLocation(StandardLocation.CLASS_PATH, path);
SpecialJavaFileManager fileManager = new SpecialJavaFileManager(sjfm, scl);After I run the file with the change, I still didn't get the compiler to read the jar the file.
Anyone knows how to solve this issue?
Thank you.
Jonathan -
How to compile a java file Dynamically !!!!
Hi all
The problem is
I have a dynamically created java file ,and i want to compile that file dynamically itself. The file name is stored in a string,
I am using ECLIPSE editor..
How do i compile this java file (which is stored in a string) dynamically?
Pls help me, already wasted a lot of time,
Thanks in advanceIn every Tutorial regarding Dynamic Compilation,
It is told to import javax.tools.*..
But i cannot find this package
I am using jdk1.4, is this the reason?
Actually the following packages are needed fro this
import javax.tools.DiagnosticCollector;
import javax.tools.JavaCompiler;
import javax.tools.JavaFileObject;
import javax.tools.StandardJavaFileManager;
import javax.tools.ToolProvider;
but i cant finad any..!!!
Please help me -
The danger of memory target in Oracle 11g - request for discussion.
Hello, everyone.
This is not a question, but kind of request for discussion.
I believe that many of you heard something about automatic memory management in Oracle 11g.
The concept is that Oracle manages the target size of SGA and PGA. Yes, believe it or not, all we have to do is just to tell Oracle how much memory it can use.
But I have a big concern on this. The optimizer takes the PGA size into consideration when calculating the cost of sort-related operations.
So what would happen when Oracle dynamically changes the target size of PGA? Following is a simple demonstration of my concern.
UKJA@ukja116> select * from v$version;
BANNER
Oracle Database 11g Enterprise Edition Release 11.1.0.6.0 - Production
PL/SQL Release 11.1.0.6.0 - Production
CORE 11.1.0.6.0 Production
TNS for 32-bit Windows: Version 11.1.0.6.0 - Production
NLSRTL Version 11.1.0.6.0 - Production
-- Configuration
*.memory_target=350m
*.memory_max_target=350m
create table t1(c1 int, c2 char(100));
create table t2(c1 int, c2 char(100));
insert into t1 select level, level from dual connect by level <= 10000;
insert into t2 select level, level from dual connect by level <= 10000;
-- First 10053 trace
alter session set events '10053 trace name context forever, level 1';
select /*+ use_hash(t1 t2) */ count(*)
from t1, t2
where t1.c1 = t2.c1 and t1.c2 = t2.c2
alter session set events '10053 trace name context off';
-- Do aggressive hard parse to make Oracle dynamically change the size of memory segments.
declare
pat1 varchar2(1000);
pat2 varchar2(1000);
va number;
vc sys_refcursor;
vs varchar2(1000);
begin
select ksppstvl into pat1
from sys.xm$ksppi i, sys.xm$ksppcv v -- views for x$ table
where i.indx = v.indx
and i.ksppinm = '__pga_aggregate_target';
for idx in 1 .. 10000000 loop
execute immediate 'select count(*) from t1 where rownum = ' || (idx+1)
into va;
if mod(idx, 1000) = 0 then
sys.dbms_system.ksdwrt(2, idx || 'th execution');
select ksppstvl into pat2
from sys.xm$ksppi i, sys.xm$ksppcv v -- views for x$ table
where i.indx = v.indx
and i.ksppinm = '__pga_aggregate_target';
if pat1 <> pat2 then
sys.dbms_system.ksdwrt(2, 'yep, I got it!');
exit;
end if;
end if;
end loop;
end;
-- As to alert log file,
25000th execution
26000th execution
27000th execution
28000th execution
29000th execution
30000th execution
yep, I got it! <-- the pga target changed with 30000th hard parse
-- Second 10053 trace for same query
alter session set events '10053 trace name context forever, level 1';
select /*+ use_hash(t1 t2) */ count(*)
from t1, t2
where t1.c1 = t2.c1 and t1.c2 = t2.c2
alter session set events '10053 trace name context off';With above test case, I found that
1. Oracle invalidates the query when internal pga aggregate size changes, which is quite natural.
2. With changed pga aggregate size, Oracle recalculates the cost. These are excerpts from the both of the 10053 trace files.
-- First 10053 trace file
PARAMETERS USED BY THE OPTIMIZER
PARAMETERS WITH ALTERED VALUES
Compilation Environment Dump
_smm_max_size = 11468 KB
_smm_px_max_size = 28672 KB
optimizer_use_sql_plan_baselines = false
optimizer_use_invisible_indexes = true
-- Second 10053 trace file
PARAMETERS USED BY THE OPTIMIZER
PARAMETERS WITH ALTERED VALUES
Compilation Environment Dump
_smm_max_size = 13107 KB
_smm_px_max_size = 32768 KB
optimizer_use_sql_plan_baselines = false
optimizer_use_invisible_indexes = true
Bug Fix Control Environment10053 trace file clearly says that Oracle recalculates the cost of the query with the change of internal pga aggregate target size. So, there is a great danger of unexpected plan change while Oracle dynamically controls the memory segments.
I believe that this is a desinged behavior, but the negative side effect is not negligible.
I just like to hear your opinions on this behavior.
Do you think that this is acceptable? Or is this another great feature that nobody wants to use like automatic tuning advisor?
================================
Dion Cho - Oracle Performance Storyteller
http://dioncho.wordpress.com (english)
http://ukja.tistory.com (korean)
================================I made a slight modification with my test case to have mixed workloads of hard parse and logical reads.
*.memory_target=200m
*.memory_max_target=200m
create table t3(c1 int, c2 char(1000));
insert into t3 select level, level from dual connect by level <= 50000;
declare
pat1 varchar2(1000);
pat2 varchar2(1000);
va number;
begin
select ksppstvl into pat1
from sys.xm$ksppi i, sys.xm$ksppcv v
where i.indx = v.indx
and i.ksppinm = '__pga_aggregate_target';
for idx in 1 .. 1000000 loop
-- try many patterns here!
execute immediate 'select count(*) from t3 where 10 = mod('||idx||',10)+1' into va;
if mod(idx, 100) = 0 then
sys.dbms_system.ksdwrt(2, idx || 'th execution');
for p in (select ksppinm, ksppstvl
from sys.xm$ksppi i, sys.xm$ksppcv v
where i.indx = v.indx
and i.ksppinm in ('__shared_pool_size', '__db_cache_size', '__pga_aggregate_target')) loop
sys.dbms_system.ksdwrt(2, p.ksppinm || ' = ' || p.ksppstvl);
end loop;
select ksppstvl into pat2
from sys.xm$ksppi i, sys.xm$ksppcv v
where i.indx = v.indx
and i.ksppinm = '__pga_aggregate_target';
if pat1 <> pat2 then
sys.dbms_system.ksdwrt(2, 'yep, I got it! pat1=' || pat1 ||', pat2='||pat2);
exit;
end if;
end if;
end loop;
end;
/This test case showed expected and reasonable result, like following:
100th execution
__shared_pool_size = 92274688
__db_cache_size = 16777216
__pga_aggregate_target = 83886080
200th execution
__shared_pool_size = 92274688
__db_cache_size = 16777216
__pga_aggregate_target = 83886080
300th execution
__shared_pool_size = 88080384
__db_cache_size = 20971520
__pga_aggregate_target = 83886080
400th execution
__shared_pool_size = 92274688
__db_cache_size = 16777216
__pga_aggregate_target = 83886080
500th execution
__shared_pool_size = 88080384
__db_cache_size = 20971520
__pga_aggregate_target = 83886080
1100th execution
__shared_pool_size = 92274688
__db_cache_size = 20971520
__pga_aggregate_target = 83886080
1200th execution
__shared_pool_size = 92274688
__db_cache_size = 37748736
__pga_aggregate_target = 58720256
yep, I got it! pat1=83886080, pat2=58720256Oracle continued being bounced between shared pool and buffer cache size, and about 1200th execution Oracle suddenly stole some memory from PGA target area to increase db cache size.
(I'm still in dark age on this automatic memory target management of 11g. More research in need!)
I think that this is very clear and natural behavior. I just want to point out that this would result in unwanted catastrophe under special cases, especially with some logic holes and bugs.
================================
Dion Cho - Oracle Performance Storyteller
http://dioncho.wordpress.com (english)
http://ukja.tistory.com (korean)
================================ -
Is there a 64-bit version of the CC compiler itself?
Hello,
We have a large (~4k lines) .cpp file which brings in a very large number of templates. On our v440 we see the compiler using more and more memory until it reaches 2Gb when it falls over with the message
"Insufficient memory is available for compilation to continue."
sun02% uname -a
SunOS sun02 5.8 Generic_117350-05 sun4u sparc SUNW,Sun-Fire-V440
sun02% CC -V
CC: Sun C++ 5.8 Patch 121017-02 2006/04/19
sun02% file /opt/v11/SUNWspro/bin/CC
/opt/v11/SUNWspro/bin/CC: ELF 32-bit MSB executable SPARC32PLUS Version 1, V8+ Required, dynamically linked, stripped
Is there a 64-bit version of the compiler which could handle code of this size? The large .cpp will be very difficult to split up.
Interestingly, on our Xeon-based box running Solaris 10, the compilation is very very slow, but does complete OK.
sun03% uname -a
SunOS sun03 5.10 Generic_118855-33 i86pc i386 i86pc
sun03% CC -V
CC: Sun C++ 5.8 Patch 121018-10 2007/02/21
sun03% file /opt/v11/SUNWspro/bin/CC
/opt/v11/SUNWspro/bin/CC: ELF 32-bit LSB executable 80386 Version 1 [FPU], dynamically linked, stripped
Thanks
-- SteveUnfortunately, there is no 64-bit version of the compiler.
Some things you can try:
Reduce the size of individual compilation units (the total amount of source code seen by the compiler in one compilation run).
Re-factor very large functions into smaller ones. During optimized compilation, the memory usage and compilation time is super-linear in the number of basic blocks in one function.
Reduce the optimization level.
If you have a support contract with Sun, please file a bug report via your support channel. If not, please file a bug report at bugs.sun.com. -
Why does the JRE use so much memory?
I know this question has been asked a million times in one form or another but I have not seen a complete answer yet. When I compile and run the following code under WindowsXP pro, it appears that the JRE is using 3.1MB or RAM. When I run a similar program using C++ the program only uses 630k.
public class HelloWorld
public static void main(String args[])
System.out.println("Hello World");
Why 3MB to show "HelloWorld"? I love Java, it is much easier and more enjoyable to develop with but our applications appear to use large amounts of memory. I realize that making calls to the runtime object to check the amount of "heap" memory we are using will probably show much less memory being used but Task Manager shows 3MB. It is sometimes hard to convince my fellow developers of he benefits of Java when there are strange issues like this.
Thanks in advanceJava uses that much memory because every time you start a Java program, even this small, it loads into RAM the whole Java Virtual Machine. Which is itself a very large C++ program - on the order of several hundred thousand lines. Compare that to your C++ HelloWorld program - and you see the reason for a difference...
So a real question is, is it possible to write a modern Java VM such that it uses space proportional to the size/complexity of the Java program that it executes. Partially that's possible - for example I understand that say native GUI libraries are separate from the main JVM dll, so when you don't use AWT/Swing, you don't waste space with related native code. However, for the core JVM it's probably very difficult to separate parts of functionality such as the interpreter, the dynamic compiler, the runtime system, the garbage collector, the serviceability stuff, etc. They are very interdependent, and thus, probably unless you have this goal from the beginning (and ready to sacrifice some performance/maintainability for it), you can't do that.
There exist, however, a bunch of other JVMs of all sizes, say those used in mobile phones (or even smart cards!), that consume a lot less memory. They, however, typically implement only a subset of the full Java functionality. On the other hand, the HotSpot VM that comes with Sun's JDK, that you probably use, is a full-fledged JVM that implements everything, can be used for huge server apps, etc. - no wonder its minimum memory consumption is a few megs. -
Is there such thing as a "compile and execute statement" in Java?
I wanted to know if there's any way to compile and execute a program "within" another program.
Yes, but the classes to do so aren't officially supported by Sun as far as I know.
And you must be able to assume a thing or two about the stuff you compile, such as existing constructors or factory methods etc, otherwise it gets hairy.
Here's some example code (haven't tried it, so excuse any spelling errors etc) to compile the source code in a file called MyClass.java:
ByteArrayOutputStream baos = new ByteArrayOutputStream();
new sun.tools.javac.Main(baos,null).compile("MyClass.java");
String theMsg = baos.toString();
if (theMsg.length() > 0)
// NiftyErrorHandlingHere
}After that, you can load it with a simple ClassLoader.loadClass() call and create instances of it.
It must be compiled with the tools.jar file in the classpath, to find the sun.tools.javac.Main class.
However, for any such dynamically compiled class to be of any use to you, you must be able to assume something about it, such as implementing interfaces or whatever, otherwise it's a kind of pointless exercise.
HTH,
F -
Memory upgrade problems on Satellite L30-134
I have just bought 2 x 1GB memory PC4200 for my L30-134 and installed them they work fine in BIOS, show up and read as they should do, but when windows is booting up it will crash with both modules installed, only ever gets to the screen with the loading bar going across and then I get blue screen.
Forgot to make a note of the error code I am afraid... (Fool I know). If I remove one of the modules and install the original 512 module with just 1 of the 1GB ones it will work fine. BIOS has been updated to the latest 3.10 version and still I get the same problem...
because all seems fine from within BIOS I am thinking that its more likely to be a software (windows) problem of some kind maybe some corrupt data in the registry or a virus maybe... has anybody any ideas?
Also, the laptop is actually my step sons and he really wants to have the CPU upgraded from the Celeron 410 to one that's a little faster. When first looking into this we was told that several CPUs were compatible (although not really worth the hassle of installing) they were cel m 450 2 GHz, pent dual core t2060 and t2080, and also the core duo t2250,so he bought the t2250,and no surprises it wouldn't work, or should I say it powered up with no display and the shut off and just epeated the cycle. So did not take a genius to work out it wasn't compat...
I know the t2080 are used in the L30's and are 1.73 533 FSB with 1MB L2 cache- 65nm and do stepping. The t2250 is identical apart from the L2 cache is 2MB and the stepping varies from do to mo depends when u bought it... seeing as ow he already has the CPU he's gonna sell it on and use the cash to buy the one that will work. If anybody has any info that could help make sure that he does not end up going round in circles I would really appreciate it...
Oh and just to make you all laugh it was Toshiba tech line staff member that told him that the CPU would work and that the L30 was able to take 2 x 1GB memory and never once mentioned that there may be issues with both upgrades...
Issues as in neither soddin work....lmaoHi!
The specs. you posted are correct. Xpress 200 could support dual channel: "Variants available with single or dual channel system memory support ". But there is nothing about dual channel mode in Xpress 200M specs. I have identical modules, but EVEREST doesn't say the modules are running in dual channel mode. So I think the info you found on the website gives the answer, but here are some explanation from Wikipedia to clarify the definitions:
*Dual-channel:*
"Dual-channel-enabled memory controllers utilize two 64-bit data channels, resulting in a total bandwidth of 128-bits, to move data from RAM to the CPU. In order to achieve this, two or more DDR/DDR2 SDRAM memory modules must be installed into matching banks, which are usually color coded on the motherboard. These separate channels allow each memory module access to the memory controller, increasing throughput bandwidth."
*DDR:*
"DDR SDRAM or double data rate synchronous dynamic random access memory is a class of memory integrated circuit used in computers. It achieves greater bandwidth than the preceding single data rate SDRAM by transferring data on the rising and falling edges of the clock signal (double pumped). Effectively, it doubles the transfer rate without increasing the frequency of the memory bus."
*DDR2:*
"Like DDR before it, DDR2 cells transfer data both on the rising and falling edge of the clock (a technique called "dual pumping"). The key difference between DDR and DDR2 is that in DDR2 the bus is clocked at twice the speed of the memory cells, so four words of data can be transferred per memory cell cycle. Thus, without speeding up the memory cells themselves, DDR2 can effectively operate at twice the bus speed of DDR."
*UMA (Unified Memory Architectrue):*
"In computer architecture, Shared Memory Architecture (SMA) refers to a design where the graphics chip does not have its own dedicated memory, and instead shares the main system RAM with the CPU and other components."
Message was edited by: Abe83
Maybe you are looking for
-
Change FI document items created from MIRO
Hi All I'm creating invoices with MIRO transaction. I want to change some document items of FI document related to one invoice, assigning the vendor number of header invoice to the field bseg-lifnr created in FI document. I have tried with BTEs, subs
-
Script to Multiple computer accounts to existing SCCM collection
Hi, I am looking for a script to add multiple computers to a existing SCCM collection. But unable to find a proper one. Below link is talking about the same, but unable to find the script. http://social.technet.microsoft.com/wiki/contents/articles/20
-
DOCUMENTS, PHOTOS, CASH MEMOS
HOW DO I COPY DOCUMENTS AND PICTURES THAT ARE IN MY EXTERNAL PORTABLE HARD DRIVE? HOW DO I COPY THE DOCUMENTS AND PHOTOS THAT ARE IN MY COMPUTER? I AM NOT ABLE TO ADD NEW ITEMS TO THE CASH MEMOS THAT I HAVE ON TEXT EDIT PAGES, UNLESS I TRANSFER THE E
-
Different Colors by "Saving for Web" - why???
hello, i save pics for the web, incl. profile sRBG. but the pics look colder - warm colors get lost - in firefox, because ff is not able to use profile (but only with an extension as i know). safari is able to show the collors correct. bridge color p
-
Table - Conditional SQL possible?
Application Express 4.1.1.00.23 Just wondering if the following is possible. I have created users and groups, for example; user1 user2 user3 group1 group2 I have assigned the users to the groups as follows; Group 1 Members user1 user2 Group 2 Members