Moving a ResultSet into a JTable, cheaply and efficiently.

Question

Hi,
This may be something interesting for the more advanced Java programmers.
I've recently run into an issue, with moving a ResultSet into a JTable. Now, this may seem simple, and believe me it is, but, all the ways I have seen it done, and tried, just don't suite my need.
DBSql dbsql = new DBSql(); // My database handler class
tableResults.setModel(new javax.swing.table.DefaultTableModel(dbsql.execute(script), dbsql.getColumns()));
* Queries the DataBase and populates
* the values into a two-dimensional object array
* @param SqlQry
* @return Object[][]
public Object[][] execute(String SqlQry) throws SQLException {
Object[][] obj = null;
select = connection.createStatement(ResultSet.TYPE_SCROLL_SENSITIVE,
ResultSet.CONCUR_READ_ONLY);
ResultSet rs = select.executeQuery(SqlQry);
rsmd = rs.getMetaData();
try {
if (rs.next()) {
boolean found = true;
rs.last();
recordCount = rs.getRow();
rs.first();
obj = new Object[recordCount][rsmd.getColumnCount()];
int i = 0;
while (found) {
for (int j = 0; j < rsmd.getColumnCount(); j++) {
obj[i][j] = rsmd.getColumnType(j + 1) != Constants.TYPE_INTEGER ? rs.getString(j + 1) : rs.getInt(j + 1);
i++;
found = rs.next();
} else {
recordCount = 0;
return null;
} catch(OutOfMemoryError oome) {
System.gc();
throw new SQLException("Out Of Memory");
if(rs != null) rs.close();
return obj;
The application I have created is used to connect to any database I want. I have tested it with JavaDB, Microsoft Access, and DB2. The problem is, most DB2's have tables with many records and columns, while most new Relational Databases, have small tables with few records. This code works fantastic with a couple thousand records, with very few columns. But, doesn't cut it when it comes to 50000 records or more. For, instance I queried a DB2 table that has 34000 records and 117 columns ("select * from table"), it doesn't take too long, but it uses way too much memory. If I run that query the application resources 298mb, then when I run it again, it uses a little more and throws an OutOfMemoryError.
The second issue I have is, I queried another table that has 147000 records and selected 4 columns. No OutOfMemoryError this time - 70mb resourcing - but, the application did take at least 20 minutes to collect those records.
I have tried using the Vector<Vector<String>> type into the JTable, but, frankly that's just asking for an OutOfMemoryError.
I have tried creating my own custom table model, with a created data object.
I have tried inserting rows into a table model in the method itself and returning the table model.
Eg.
while (found) {
Object[] obj = new Object[rsmd.getColumnCount()];
for (int j = 0; j < rsmd.getColumnCount(); j++) {
obj[j] = rsmd.getColumnType(j + 1) != Constants.TYPE_INTEGER ? rs.getString(j + 1) : rs.getInt(j + 1);
tablemodel.insertRow(obj);
found = rs.next();
^I think you can use a vector for this too.
So far, nothing has solved my problem.
One thing I have not tried however, is a different table component for this kind of thing.
I'm basically looking for, a method of doing this, with less overhead and quicker table population.
Any Ideas?
Regards,
That_Gui

Answer 1

Thanks for the reply.
"1) Swing related questions should be posted in the Swing forum."
Apologies, I was caught between swing and essentials, as it seemed to me as, more of a "better ways to do things" kind of question.
"Also, the approach you are using requires two copies of the data, one in the ResultSet and one in the TableModel."
I understand that transferring a ResultSet into an Object Array may probably be resource intensive, my delphi colleague made mention of that to me, not too long ago. That is why I'm trying to transfer the ResultSet directly into the table. That is why I used the ListTableModel, which also uses a lot of memory, I have looked at that - I had forgotten to mention that.
"I have seen approaches where you try to only read a specific number of records. Then as you scroll additional records are read as needed. I can't point you to any specific solution though."
Using an approach of reading the data as you scroll, sounds great, but also sounds like unnecessary work for the program to do, just like this Object/Vector story. In RPG, if you are working with subfiles (like a JTable), you will display 10 records at a time, and if the user hits the Page Down button you will display the next 10, that is how I'm understanding it.
"I rarely use DefaultTableModel, creating your own model from AbstractTableModel is a lot more flexible when it comes to how you organise your storage, and it's pretty simple once you get the hang of it."
This is the dream, I just need to design one that works as I need it.
"You'd do a select count(*) on your query first to get the total rows expected. Of course one of the problems is that the databased table may change after you've retrieved the length or the items."
Unfortunately, this is not an option for me, it wont work for what I need.
I'm going to give the ResultSetTableModel a go - which I think it is actually similar to the ListTableModel - and I will get back to you.