Reading/Writing Bulk Data, getting OutOfMemoryError: Java heap space

Question

Hello,
I am reading more than a million record using CursoredStream, following is my code snippet for reading data
ReadAllQuery query = new ReadAllQuery();
query.setReferenceClass(DataBk.class);
query.useCursoredStream();
CursoredStream stream;
stream = (CursoredStream)dbses.executeQuery(query);
       while (!stream.atEnd()) {
       vector1 = stream.read(stream.size());
       }       Problem is I am getting
Exception in thread "main" java.lang.OutOfMemoryError: Java heap spaceI was reading other forums posts about this and it is mentioned that memory heap size could be increased.
Where do I increase memory heap size?
java.lang.OutOfMemoryError while opening 9.0.4.5 mwp in 10.1.3.3
Besides, if my above code is not alright, someone could suggest best approach for reading and writing bulk data.
Regards
Edited by: user20090209 on Oct 14, 2009 10:17 AM

Answer 1

Hello,
If you are reading a million records and running out of heap, you will still get the exception by reading it using a cursor one at a time. It also depends on how the driver is processing the million records. Using a scrollable cursor might help ( http://wiki.eclipse.org/Using_Advanced_Query_API_(ELUG) ), increasing the heap as suggested will (In Jdev, Java options can be added to the project properties->Run/Debug/Profile->Launch settings), or even changing your app design to use objects more efficiently all around. It depends on what your are using the objects for, but even using pagination (via setting the firstRows/maxResults on a query) to limit the results returned can help depending on how the application requires the objects. What is important is that the objects be free for garbage collection when you are done processing them, and before the next batch needs to be brought in from the database.
Also, common problems are that reading in one object can bring in multiple trees if relationships aren't managed efficiently. Only use relationships where required, and use indirection where ever possible.
Best Regards,
chris