BI XML Publiser Issue java langOutOfMemoryError Java heap space

Question

Hi
I am getting 'java.lang.OutOfMemoryError: Java heap space' error in XML publisher report. Just i am running 15MB data report still its giving OutOfmemory issue.
Please help on this.
[5/22/12 9:08:00 AM] [98639:RT12903275] XML Publisher post-processing action complete.
[5/22/12 9:08:01 AM] [98639:RT12903275] Completed post-processing actions for request 12903275.
[5/22/12 10:20:15 AM] [OPPServiceThread1] Post-processing request 12903281.
[5/22/12 10:20:19 AM] [98639:RT12903281] Executing post-processing actions for request 12903281.
[5/22/12 10:20:19 AM] [98639:RT12903281] Starting XML Publisher post-processing action.
[5/22/12 10:20:19 AM] [98639:RT12903281]
Template code: XXGFS_AP_APMCGRN
Template app: XXGFS_AP
Language: en
Territory: US
Output type: EXCEL
Exception: java.lang.OutOfMemoryError: Java heap space
java.lang.OutOfMemoryError: Java heap space
     at oracle.jdbc.dbaccess.DBItem.allocBuffer(DBItem.java:204)
     at oracle.jdbc.dbaccess.DBItem.allocBuffer(DBItem.java:232)
     at oracle.jdbc.dbaccess.DBDataSetImpl._allocItemsAndBuffers(DBDataSetImpl.java:300)
     at oracle.jdbc.dbaccess.DBDataSetImpl._allocDataAndItems(DBDataSetImpl.java:261)
     at oracle.jdbc.dbaccess.DBDataSetImpl.setType(DBDataSetImpl.java:2068)
     at oracle.jdbc.dbaccess.DBDataSetImpl.setType(DBDataSetImpl.java:1951)
     at oracle.jdbc.driver.OracleCallableStatement.registerOutParameterBytes(OracleCallableStatement.java:278)
     at oracle.jdbc.driver.OracleCallableStatement.registerOutParameter(OracleCallableStatement.java:402)
     at oracle.jdbc.driver.OracleCallableStatement.registerOutParameter(OracleCallableStatement.java:477)
     at oracle.apps.fnd.cp.gsm.GenCartComm.getQueueMessage(GenCartComm.java:185)
     at oracle.apps.fnd.cp.gsf.GSMStateMonitor.waitForNextEvent(GSMStateMonitor.java:95)
     at oracle.apps.fnd.cp.gsf.GSMServiceController.mainLoop(GSMServiceController.java:236)
     at oracle.apps.fnd.cp.gsf.BaseServiceController.run(BaseServiceController.java:71)
     at java.lang.Thread.run(Thread.java:619)
Exception in thread "Cache Stats Writer" java.lang.OutOfMemoryError: GC overhead limit exceeded
Exception in thread "OPPAQMON:98639" java.lang.OutOfMemoryError: Java heap space
     at oracle.jdbc.dbaccess.DBItem.allocBuffer(DBItem.java:204)
     at oracle.jdbc.dbaccess.DBItem.allocBuffer(DBItem.java:232)
     at oracle.jdbc.dbaccess.DBDataSetImpl._allocItemsAndBuffers(DBDataSetImpl.java:300)
     at oracle.jdbc.dbaccess.DBDataSetImpl._allocDataAndItems(DBDataSetImpl.java:261)
     at oracle.jdbc.dbaccess.DBDataSetImpl.setType(DBDataSetImpl.java:2068)
     at oracle.jdbc.dbaccess.DBDataSetImpl.setType(DBDataSetImpl.java:1951)
     at oracle.jdbc.driver.OracleCallableStatement.registerOutParameterBytes(OracleCallableStatement.java:278)
     at oracle.jdbc.driver.OracleCallableStatement.registerOutParameter(OracleCallableStatement.java:402)
     at oracle.apps.fnd.cp.opp.OPPAQMonitor.waitForMessage(OPPAQMonitor.java:423)
     at oracle.apps.fnd.cp.opp.OPPAQMonitor.run(OPPAQMonitor.java:113)
     at java.lang.Thread.run(Thread.java:619)
[5/22/12 10:24:36 AM] [UNEXPECTED] [98639:RT12903281] java.lang.OutOfMemoryError: Java heap space

Answer 1

user972966 wrote:
There are two tables (tableA, tableB) in two different databases. They both have almost same schema/structure. There is mapping between tables on their columns.
I need to develop a Java application to compare data value between these two tables for a time frame (for example one day) and find missing records or mismatch data.
The volume of data, even for one day, is huge; almost 70M, 100M. Do you mean this many records? or this many MB. either way they are not that big for Java. If you have collections of up to 2 billion entries, you won't have a problem.
>
I have tried to use Thread to split the comparison process in 10 threads. Each thread is fetching portion of data from both tables (for example 7M each) and processing portion them. The issue that I am facing is, I am getting following exception after running:I would suggest you get the program working with one thread first. You will find this is simpler and possibly even faster. Having 10 threads isn't guarenteed to be faster.
>
To compare the data, I am fetching data from tableA and add it into List<MyDataObject> listA (the MyDataObject represent a one row data. It is a java bean) and then I am fetching data from tableB and add it into List<MyDataObject> listB
And then I compare the lists.
It seems there is limitation in size of List and I cannot add many elements to a List object, it’ll throws Out of Memory exception. The limit is 2^31-1. However an Out of Memory error means what it says. You are out of heap space, so you need to either increase it, or use less memory.
>
This method works for small volume, but for large one I get that exceptionIs there any reason you cannot divide the problem so you can process a small volume at a time?