Retrieve specific range data from oracle table

Question

Hi, Dear friends,
I want to retrieve all the data from oracle table and then save them to mysql table using JDBC ResultSet. The problem is that some oracle table is too big, if I retrieve all of them to memory at a once time using excuteQuery, the program will become no response out of memory limitation. So my question is if I can retrieve just specific range data once a time. I can¡¯t find this function through JDBC API. Also, I don¡¯t want to use the specific sql sentence, for example ¡°select ¡ from¡ where someid>¡ and someid<¡¡±, because there are many different tables, I want to transfer them automatically. So I can¡¯t construct such sql sentence in advance. Does anyone know if oracle JDBC driver provide such kind of function or does there any other way?
Any suggestion will be greatly appreciated!
Sammy

Answer 1

Dear Justin,
thank you so much for your prompt reply!
as your suggestion, I do check the performance while my program is running, after the program become nearly no response, I found the memory usage is nearly 100%, while disk usage and process usage is pretty low. that's the reason why I guess maybe the memory limitation, but the strange thing is that no any error reported by JBuilder, it just nearly no response and don't transfer any data any more. my os is windows xp, the version of JBuilder is 7. my main memory is 768M. just as you said, the total 6,000 records in not very large, just no more than 400K. another strange thing is that why my program works well when there is little data records in the table.
the big picture of my program is first I retrieve oracle table metadata, according to this information, I construct DDL sql words and then create the corresponding table in mysql database. this part works well. in order to save your time, I will not paste the code here. then there is a method to transfer oracle data to mysql table, whenever creating the mysql table, then I will call this method to transfer datat to it. the following is the code of this method, I am very sorry to take your time. please read it when you are available.
thank you very much!!
Sammy
//transfer data from oracle to mysql!!!
private static void transferData(Connection oracleConn, Connection mysqlConn, String oracleTableName, String oracleSchemaName) throws SQLException{
Statement oracleStmt=oracleConn.createStatement();
Statement mysqlStmt=mysqlConn.createStatement();
// sending sql to oracle to retrieve data
String thisTableName=oracleTableName;
String oracleSQL="SELECT * FROM ".concat(thisTableName);
ResultSet oracleRS = oracleStmt.executeQuery(oracleSQL);
String sql="";
if (oracleRS.next()) {
ResultSetMetaData rsmd = oracleRS.getMetaData();
int colCount = rsmd.getColumnCount();
do {
String sqlBodyPart="";
String sqlValuePart="";
System.out.println("the number of column is "+colCount);
for (int i = 1; i <= colCount; i++) {
String columnValue = oracleRS.getString(i);
boolean b = oracleRS.wasNull();
String columnName =rsmd.getColumnName(i);
System.out.println("the value of column " + i + "is " + columnValue);
//construc the sql body part and value part
sqlBodyPart=sqlBodyPart.concat(" ").concat(columnName).concat(",");
if(b){ //if the value of the column i is null
sqlValuePart=sqlValuePart.concat(" null").concat(",");
}else{
sqlValuePart=sqlValuePart.concat(" '").concat(columnValue).concat("',");
//get rid of the last comma in sqlBodyPart and sqlValuePart
if(!sqlBodyPart.equalsIgnoreCase("")) sqlBodyPart=sqlBodyPart.substring(0,sqlBodyPart.length()-1);
if(!sqlValuePart.equalsIgnoreCase("")) sqlValuePart=sqlValuePart.substring(0,sqlValuePart.length()-1);
//construct the sql sentence!!!
sql="INSERT INTO ".concat(thisTableName).concat(" (").concat(sqlBodyPart).concat(") ").concat(" VALUES(").concat(sqlValuePart).concat(")");
System.out.println("the sql words for this column is");
System.out.println(sql);
System.out.println(" ");
if(mysqlStmt!=null){
int rows = mysqlStmt.executeUpdate(sql);
} else{
System.out.println("can't connect with mysql server");
System.exit(1);
while (oracleRS.next());
} else {
System.out.println("There are no data in the table...");
}//end of method data transfer!
//end of method data transfer!