Runtime Compilation to Increase Performance

Question

Hi everyone,
I've been struggling with this problem for a few days now and would be glad if you could help. I'm working on a neural networks project. I've written a GUI to edit and train neural networks for a certain task. In order to be able to play with the network architecture in a flexible manner I've exploited the OOP concepts generously. But once the network architecture is set I don't need this flexibility anymore and I want to compile a specific class that has the same functionality as that network where all the computation is carried out over local variables (I even made them static) instead of array elements and object references. I've successfully written this code that generates a string where loops are unrolled and all the if, switch statements are resolved and the variables that are not supposed to vary at this point are inserted as constants at the (runtime) compile time.This should speed up right? Especially considering all the array bounds checking and stuff java is doing.
When I try to compile this string, everything is fine and the mathematical functionality is perfectly imitated. But the problem is, for small networks, the code works twice as fast, as I would expect but for large networks the speed drops to something like 4 times slower! For a typical large network the resulting binary is 80k, so I wouldnt expect it to fill the cache (or should I?) anyway, even if it did fill the cache that shouldnt slow it down by a factor of 8. The slowdown sounds more like the difference between JITted and interpreted code. I suspect that as that single method is 80k long it might not be JITted (sorry if I annoy you with that crappy term). If you think this is the case, is there any way to bypass that? Or what do you think the reason could be. I already tried to decrease the number of method calls before compilation to 1 (with the command line argument -XX:CompileThreshold:1) just to try, but it didnt help.
I converted the resulting string also to C++ and it runs 4 times faster than the unrolled original method(Even for large networks). (It serves also as a nice test showing that java is half as fast as such an arithmetic and trigonometric intensive task)
I would be so glad if you could help me attain the x2 performance I obtain with the small networks in the general case. And I would be super duper glad if you could suggest a way I could runtime compile it in C++ and attach it to my program through JNI. Runtime compiling C++ code sounds messy but I keep some hope since the operations are purely mathematical and they don't need any platform specific thing.
Thanks a lot
Edited by: enobayram on Mar 5, 2009 5:51 AM

Answer 1

Thanks for your answer, I was thinking about the same thing, but I am not exactly trying to out-smart it. During that runtime compilation I know much more than the VM does. I know that the neuron axon function family is constant, so I bypass a switch statement. I know that function flatness will not change after that point so I insert it as a constant. I also avoid using arrays since I know how many elements there should be, and I can use individual local variables instead. This also explains the speed increase in small networks and also in the C++ experiment.
I've checked those options, and the only relevant one I could find is that XX:CompileThreshold. By the way, why do you think runtime compiling C++ is less messy? With java everything needed is included in the standard library. JavaCompiler class and the ClassLoader class are sufficient. With C++ I guess I would have to excite some C++ compiler through a system call so that it generates a .dll (or .so) then I would have to have it loaded to memory and interface it to my program. It could even get messier if I tried to recompile it when the network changes as then I would have to unload the dll and rebuild it. If you had an easier way in mind I would be so glad to hear.