JDBC ResultSet out of memory problem with Scrollable one

Question

Hey guys,
I'm facing the following problem when accessing an Oracle 10g database over oracle jdbc driver 1.4.
I need to access the rows of a resultset (millions of rows) at least twice. Forward only doesn't need much memory - but I can't do a rs.beforeFirst()
Switching to ResultSet.TYPE_SCROLL_SENSITIVE gives me always an java.lang.OutOfMemoryError: Java heap space exception after rs.next() and I can see the used memory constantly increasing.
Here is my test code:
conn = DriverManager.getConnection
("jdbc:oracle:thin:@host:1521:ORCL", "user", "pass");
Statement my_stmt = conn.createStatement(ResultSet.TYPE_SCROLL_SENSITIVE,
ResultSet.CONCUR_READ_ONLY);
ResultSet result = my_stmt.executeQuery("select * from stats_big");
System.out.println("Query back :)");
while(result.next()) {      //here happens the error after 200 000 rows
//make statistics
//ask user what to do
result.beforeFirst();
while(result.next()) {
     //apply function and deliver new values
conn.close();
Is the implementation caching all the already read rows in memory?
Any help would be great,
Alex

Answer 1

if and how there are ways to read a Resultset 2 or more times
forward-only and the data is not completelly cached on the client If the client reads a million rows, how can it read them a second time without either storing them somewhere on the client as in scrollable cursors, or rereading them from the server as in regular cursors. To me this looks like the laws of physics, rather than something that can be changed in a software release.
In order of preference I would
1. Not process a million rows on the client.
2. Use a regular cursor and fetch it twice, the second fetch will be faster as the database already caches as much of it as it can.
Fetching that much data somewhere is going to take some time anyway, which is why I prefer the first option..