PreparedStatement or executeBatch

I have a text file which has more than 100,000 rows. I have to read it line by line and updates some tables accordingly. My question is which one will give me best performance: PreparedStatement or executeBatch?
Thanks

Your question is strange.
Use PreparedStatement AND executeBatch().
But make sure to execute the batch every 1000 rows or so, if you try to do all the 100.000 rows in one go you might get problems.

Similar Messages

To Joe Schell - jschell... - PreparedStatement and executeBatch()

Hi,
Thanks for the reply to the PreparedStatement and executeBatch() query. Is there any workaround for it other than using the Statement object?
thanks,
Nish

Could be.
But since I have no idea about what you are talking about, I can't really comment.
I respond to between 10-30 threads a day and read more than that. Consequently I need a context to understand questions.
If I have previously (recently) responded to a previous thread then if you post a response to it I will generally see it in less than a day. If is an older thread you might want to start a new thread. But if you do then add a link or put enough content so I can follow it.

PreparedStatement and executeBatch()

Hi,
I am trying to insert records into the database using the addBatch() and executeBatch() methods. I am using PreparedStatement. And I am trying to insert 52 records at a time.
I have the records in an array, and I am looping through the array, setting the parameters for the prepared statement and doing addBatch(). And finally when I come out of the loop I call preparedStatement.executeBatch().
int[] updateCounts = ps.executeBatch();
Here it is throwing out the ArrayIndexOutOfBoundsException. Anybody has any idea why this is happening? This works fine with the Statement object, but it is with the PreparedStatement that I am having this problem. I am using JDK 1.3. My database is DB2 UDB 7.1 and my driver is IBM DB2 ODBC Driver.
thanks in advance,
Nish

Hi,
Thanks for the reply... Here is my code.
public void insertNetFcst(Connection con, PreparedStatement ps, NetForecast[] arrNF){
Connection conn = con;
PreparedStatement psIns = ps;
NetForecast[] arrNet = new NetForecast[52]; //Array of objects of type NetForecast
arrNet = arrNF;
int intLen = 0;
intLen = arrNet.length;
try{
for (int ins = 0; ins < intLen; ins++){
NetForecast nFct = new NetForecast();
nFct = (NetForecast)arrNet[ins];
psIns.setString(1, nFct.getCorpId());
psIns.setString(2, nFct.getDivId());
psIns.setString(3, nFct.getShipWhseId());
psIns.setString(4, nFct.getRecvWhseId());
psIns.setDouble(5, nFct.getUpcId());
psIns.setInt(6, nFct.getYearId());
psIns.setInt(7, nFct.getWeekNbr());
psIns.setInt(8, nFct.getDemandQty());
psIns.setString(9, nFct.getUomCode());
psIns.setInt(10, nFct.getReqdYear());
psIns.setInt(11, nFct.getReqdWeek());
psIns.setString(12, nFct.getFcstTypeInd());
psIns.setTimestamp(13, new Timestamp(System.currentTimeMillis()));
psIns.addBatch();
int[] updateCounts = psIns.executeBatch();
//I am getting the exception at this line... where I call executeBatch()
conn.commit();
catch(BatchUpdateException b){
System.err.println("SQLException: " + b.getMessage());
System.err.println("SQLState: " + b.getSQLState());
System.err.println("Message: " + b.getMessage());
System.err.println("Vendor: " + b.getErrorCode());
System.err.print("Update counts: ");
int [] updateCounts = b.getUpdateCounts();
for (int i = 0; i < updateCounts.length; i++) {
System.err.print(updateCounts[i] + " ");
b.printStackTrace();
catch(SQLException insExp){
System.out.println("Duplicate record exception..." + insExp);
thanks,
Nish

PreparedStatement.executeBatch() and BatchUpdateException

Hi,
I am using PreparedStatement's executeBatch() method to make batch operations,
for such a scenario is there a way to continue the operation 3 and 4 when the operation 2 fails?
operation 1 (will succeed alone)
operation 2 (will fail alone)
operation 3 (will succeed alone)
operation 4 (will succeed alone)

Hi, I've already looked the link you've sent.
Let me give more detail about the result of my scenario.
The first two successfull insert operations really updates the database which is Oracle.
But it seems the failed 3rd operation prevents the operation 4 to update database.
My code is something like that:
int[] updateCounts = null;
try {
PreparedStatement stmtInsert = conn.prepareStatement("insert sql");
for (int i=0; i <SOMEVALUE; i++)
     stmtInsert.setString(1, SOMEVALUE);
     stmtInsert.setString(2, SOMEVALUE);
     stmtInsert.addBatch();
int[] counts = stmtInsert.executeBatch();
} catch (BatchUpdateException e)
updateCounts = e.getUpdateCounts();
checkUpdateCounts(updateCounts);
finally {
conn.close();
public static void checkUpdateCounts(int[] updateCounts) {
for (int i = 0; i < updateCounts.length; i++) {
if (updateCounts[i] == Statement.SUCCESS_NO_INFO) {
// Successfully executed; number of affected rows not available
System.out.println("Statement.SUCCESS_NO_INFO");
} else if (updateCounts[i] == Statement.EXECUTE_FAILED) {
System.out.println("Statement.EXECUTE_FAILED");
The results are pretty confusing, as I said first two operations updates the db, but fourth is did not.
But system out says:
Statement.EXECUTE_FAILED
Statement.EXECUTE_FAILED
Statement.EXECUTE_FAILED
Statement.EXECUTE_FAILED
I would expect something like that:
Statement.SUCCESS_NO_INFO");
Statement.SUCCESS_NO_INFO");
Statement.EXECUTE_FAILED
Statement.EXECUTE_FAILED
or even better (assuming operation 4 updated the db)
Statement.SUCCESS_NO_INFO");
Statement.SUCCESS_NO_INFO");
Statement.EXECUTE_FAILED
Statement.SUCCESS_NO_INFO");
Since the size of the int array that the e.getUpdateCounts() is 4 which is also equal of the batch operations' size, I assume that the driver continues to process the commands in the batch after a BatchUpdateException is thrown.
I am totally confused, can anybody comment these results? Is there a way to succeed the fourth operation?

ExecuteBatch(): number of successfully updated rows

Hello everybody:
Here is a simple but often a repeated question in java forums:
Requirement:
1.To read a flat file that has many rows of data.
2.Parse the data and update the database accordingly.
3.Find the number of successfully updated rows.
Approach:
After reading the file and parsing its data,
- use PreparedStatement
- use executeBatch()
I found this as unadvisable to use executeBatch() as its implementation is
inherently driver specific. The executeBatch() returns an array of update counts.
Now,can any one tell me, what is the best way to trace the number of successfully
(and unsuccessfully) updated rows by using this count?
Is there any other way to achieve the same by not using executeBatch()?
Can any one share a snippet of code to achieve this specific functionality?
[Need is to log the number of unsuccessful attempts along with their
corresponding rows of data].
Thanks & regards,
Venkat Kosigi

executeBatch submits a batch of commands to the database for execution and if all commands execute successfully, returns an array of update counts. The int elements of the array that is returned are ordered to correspond to the commands in the batch, which are ordered according to the order in which they were added to the batch. The elements in the array returned by the method executeBatch may be one of the following:
-- A number greater than or equal to zero indicates that the command was processed successfully and is an update count giving the number of rows in the database that were affected by the command's execution
-- A value of -2 indicates that the command was processed successfully but that the number of rows affected is unknown
If one of the commands in a batch update fails to execute properly, this method throws a BatchUpdateException, and a JDBC driver may or may not continue to process the remaining commands in the batch. However, the driver's behavior must be consistent with a particular DBMS, either always continuing to process commands or never continuing to process commands.
If the driver continues processing after a failure, the array returned by the method BatchUpdateException.getUpdateCounts will contain as many elements as there are commands in the batch, and at least one of the elements will be the following:
-- A value of -3 indicates that the command failed to execute successfully and occurs only if a driver continues to process commands after a command fails.
return values have been modified in the Java 2 SDK, Standard Edition, version 1.3 to accommodate the option of continuing to proccess commands in a batch update after a BatchUpdateException obejct has been thrown.
Throws BatchUpdateException (a subclass of SQLException) if one of the commands sent to the database fails to execute properly or attempts to return a result set. The BatchUpdateException getUpdateCounts() method allows you to known the element who caused the fail identified by a -3 value.
-- So, if you have a succesfully result, look for at the executeBatch returned array ( #values >= 0 ) + ( #values == -2 ) = successes
and if you have not a succesfully result, catching the BatchUpdateException take the array returned by the getUpdateCounts() method, and look for the position in which array values are -3. You could take the data at this position on batch and log it.
-- Other way to insert a bulk copy on database is to use a bcp command ( it�s not java, bcp is an independent command ) that allows you to do bulk inserts from file, indicate an error file, bcp will give to you as result a file with those lines not where inserted.
I hope have help you.;)

JDBC Statement.executeBatch() sometimes retrun NULL????

WHY???? DataBase is ORACLE8.1.7
i use PreparedStatement to executeBatch()
and DatabaseMetaData.supportsBatchUpdates() = true
but sometimes executeBatch() = NULL !!!!
is this respect with ORACLE parameter?

thanx for nobody's help.
i solved problem. it is a synchronization problem.
public synchronized void getTable(JTable theTable){
try {
String sql="SELECT ";
stmt=connection.createStatement();
rs=stmt.executeQuery(sql);
displayRS(rs,theTable);
stmt.close();
catch (Exception e ) {
e.printStackTrace();
}

Jdbc:BatchUpdate!help:)

I 'm writing a program where I have to read many lines of record from a table and write it to another.
I use PreparedStatement.addBatch()/executeBatch().
But the efficiency is so low that writing 7562 lines of records have taken 45 minutes.How to improve it?Lend me a hand.Thank you in advance.

When dealing with large sets of database records, I'm tend to look at any database performance tuning options. The solution depends a fair bit on what you are trying to do.
If the SQL statement reads in x number of records based on given parametera nd extracts the raw data, I suggest using a database view on top of the base table.This eliminates the huge traffic incurred reading/writing to the database.
Database triggers are another option to consider. Triggers also have the advantage of spreading the database load over a more even period, rather than slugging the db in one go.
Also ask yourself if you really do need to read the currect data being extracted ? Are there any large columns that can be ignored, or removed ? Can you make do with just the primary keys which you can later access ?
Lazy load the result sets if possible. No point running the query if there is a very good chance it isn't needed.

PreparedStatement.executeBatch() Not Working

Hello,
I'm having a problem with the executeBatch() command. It seems that whenever I used it, my program just hangs up. What this program do is that it first read a text file and place it in an Array. Then it would also read the Database and place the contenent of the table in an array as well. Each Array has at least 100,000 records. If the record from the textfile and the database matches, it would update the table using a preparedStatement.addBatch() command. After the said loop, it is then that the int[] cnt = preparedStatement.executeBatch() command is called. and that is when the Hanging part begins. Can you help me.
Thanks.

Hello,
Yes, i'm using a JDBC-ODBC bridge. The dirver is (sun.jdbc.odbc.jdbcodbcdriver). I tried examining the program patiently waited, and eventually it did not hang anymore. At first instance, I did gave the program some time and look if it really hangs, but after 15-30 mins, it work but the the second executeBatch() didn't work. There were no errors or exception that were caught.
Thanks.
Hi,
What driver are you using? And are you using the JDBC
- ODBC bridge? Please post the exact error message you
are getting when you try to do executeBatch().
Nish

PreparedStatement executeBatch

Hi Guys!
I'll be executing multiple sql statement and I'm using executeBatch of PreparedStatement. But I'm having some problem when I get errors during the execution (e.g. UNIQUE CONSTRAINT). The process stops on that point. What I want to happen is to continue the process even I encountered errors. Please help me! Thanks!

Hi,
Have you read the documentation?

SetString/executeBatch fails in Oracle 10g OCI JDBC driver

Hi,
I am using Oracle 10g OCI jdbc driver for batch updates.
Following is the the code that I am using
import java.sql.*;
import oracle.jdbc.*;
import oracle.jdbc.pool.OracleDataSource;
public class BatchUpdates
public static void main(String[] args)
Connection conn = null;
Statement stmt = null;
PreparedStatement pstmt = null;
ResultSet rset = null;
int i = 0;
try
String url = "jdbc:oracle:oci:@kctutf8";
try {
String url1 = System.getProperty("JDBC_URL");
if (url1 != null)
url = url1;
} catch (Exception e) {
OracleDataSource ods = new OracleDataSource();
ods.setUser("kctuser");
ods.setPassword("kana");
ods.setURL(url);
conn = ods.getConnection ();
stmt = conn.createStatement();
try { stmt.execute(
"create table mytest_table (col1 number, col2 varchar2(20))");
} catch (Exception e1) {}
pstmt = conn.prepareStatement("insert into mytest_table values (?, ?)");
pstmt.setInt(1, 1);
pstmt.setString(2, "row 1");
pstmt.addBatch();
pstmt.setInt(1, 2);
pstmt.setString(2, "row 2");
pstmt.addBatch();
pstmt.setInt(1, 3);
pstmt.setString(2, "row 3");
pstmt.addBatch();
pstmt.setInt(1, 4);
pstmt.setString(2, "row 4");
pstmt.addBatch();
pstmt.setInt(1, 5);
pstmt.setString(2, "row 5");
pstmt.addBatch();
pstmt.executeBatch();
rset = stmt.executeQuery("select * from mytest_table");
while (rset.next())
System.out.println(rset.getInt(1) + ", " + rset.getString(2));
catch (Exception e)
e.printStackTrace();
finally
if (stmt != null)
try { stmt.execute("drop table mytest_table"); } catch (Exception e) {}
try { stmt.close(); } catch (Exception e) {}
if (pstmt != null)
try { pstmt.close(); } catch (Exception e) {}
if (conn != null)
try { conn.close(); } catch (Exception e) {}
When I run this class I get the following output
1, row 1
2, row 3
3, row 5
4, null
5,
But It should have been
1, row 1
2, row 2
3, row 3
4, row 4
5, row 5
The same class runs fine if I use Thin driver.
Can anyone please help me solve this issue.
Note: This happens only in case we use setString with Varchar2 in the DB. This works fine if I have two number columns
Thanks,
Raja.S

Please post this question to the Java forum. It is located under "Technologies".

PreparedStatement and batchExecute

Hi,
I am trying to use PreparedStatement and batchExecution for inserting a few rows into database. I am using JTurbo driver.
But it only inserts the last set of values. For Example
Class.forName( "com.ashna.jturbo.driver.Driver" );
Connection con = DriverManager.getConnection( "jdbc:JTurbo://server","username","password" );
PreparedStatement pstm = con.prepareStatement( "INSERT INTO Test( ids, age, wt ) VALUES(?,?,?)" );
int ids[] = {1000,1001,1002,1003};
String[] age = { "12","13","14","15" };
String[] wt = { "45","46","47","48" };
for( int i=0 ; i<ids.length ; i++ )
pstm.setInt( 1, ids[i] );
pstm.setString( 2, age[i] );
pstm.setString( 3, wt[i] );
pstm.addBatch();
int[] rows = pstm.executeBatch();
pstm.clearBatch();
con.close();
In database, it shows
1003, 15, 48
1003, 15, 48
1003, 15, 48
1003, 15, 48
Pls..pls..help me with this...i'm having a hard time figuring out this problem.. :(
Thanks a lot...

Maybe you just wrote this short-hand but if not try this:
pstm.setInt( 1, ids[i] );
pstm.setString( 2, age[i] );
pstm.setString( 3, wt[i] );
//...........

Preparedstatement usage

Hi Everybody,
Is preparedStatement better than statement only in case of executeBatch method???
Kindly shed some light on this.
Regards
Ashwin

WorkForFood,
>
Gosh, your post seems a bit harsh towards me, you
must feel very strongly about this issue. I read
your post, and we seem to agree on most points, does
that make you nonsensical too? I know that in an
open forum, I wouldn't use that term to describe you
(or in a closed one either). I have seen your
comments in other posts, and I would suggest that you
are both knowledgable and helpful to others. Perhaps
I failed to communicate my points well and perhaps I
interpreted your response to personally.
No. In retrospect the fault is mine. I get tired of some of the same questions and misconeptions (as raised by the OP) raised time and time again.
More than that I took your response too personally and for that I am sorry. Please accept my sincere apologies.
The singular salient point that we appear to disagree
on, is this one.
To be clear, I only use Satement when
PreparedStatement will not provide adequate
performance, which is also what I would recommendto
others.
This is a very dangerous recommendation.
You as a developer cannot and should not rely onthe
database never changing. If that were the casethen
what is the point of having JDBC at all? The point
ultimatley is that you can write code that worksnow
AND in the future.
If you adopt bad practices (and sacrificing
portability and maintainability for nanoseconds of
performance is a bad practice) then you havedefeated
this point and your application may well sufferfor
it.You make excellent points here, I cannot find fault
in them when applied to new development efforts, most
often focused on transactional processing. However,
being a DBA and only a part time Java programmer, I
have run into many circumstances where the ability to
alter the design of the application stretched beyond
what was practical, and "Statements", or looking at
this more generically, "dynamic SQL", was used to
resolve a performance issues caused by weaknesses in
the database optimizer. In my case, we are talking
about the difference between useable and unusable
applications, with the difference being hours, not
simply shaving nanoseconds from performance.
It is certainly possible that my many years in
database and development have culminated to nothing
more then nonsense, but I sure hope not. I'm sorry
if I offended you with what you describe as a
nonsense post, I was only attempting to share my
experiences with the OP, and let the OP know that
perhaps "PreparedStatement is always better then
Statement" should be considered as a guide, not an
unbreakable rule. And on that point, I can only
assume that we will continue to disagree.
Thank you for sharing your thoughts on this.To be frank I agree with you in theory, however, I disagree with giving this advice on this forum.
To me people who ask these kinds of questions have not grasped enough of the issues at stake to make a qualified determination of what they should be doing and when. I truly believe that 99.99% of those who are asking questions in this forum should almost always be using PreparedStatements and thus I believe the use of PreparedStatements should always be encouraged. If as a result of experience and knowledge one comes to the conclusion that the use of Statement would be more appropriate than so be it.
The real issue I have is that I know that if someone doesn't say use PreparedStatements alot the unexperienced may assume that they should only use PreparedStatements under some circumstances and that their only benefit is for performance and only sometimes for that.
It is the same reason why most posts to do with memory size of objects as asked in the New to Java Programming forum are best answered by saying "dont do that" and "premature optimization is evil". Are there some cases in which knowing such a thing would be useful. Yes. But each one of the relative neophytes wrongly assumes that theirs is the right case and the fact is that almost always it isn't.

ExecuteBatch() with java.lang.NegativeArraySizeException

Friends,
I'm using this code :
try {
conn.setAutoCommit(false);
PreparedStatement pstat = conn.prepareStatement("INSERT INTO tb_fre_tran_in (fre_tran_id, fre_tran_in_tlv_seq, fre_tran_in_tlv_value) VALUES (?, ?, ?)");
for (iCount=0; iCount<in.getSizeFields(); iCount++) {
pstat.setInt(1, iFreTranId);
pstat.setInt(2, Integer.parseInt(in.getFieldTag(iCount)));
pstat.setString(3, in.getFieldValue(iCount));
pstat.addBatch();
System.out.println(iCount + "-> " + iFreTranId + "Passei pelo addBatch()");
int[] res = pstat.executeBatch();
System.out.println(iFreTranId + "-->" + res.length);
conn.commit();
pstat.close();
} catch (Exception e) {
e.printStackTrace();
When the program going to execute the "int[] res = pstat.executeBatch();" line I got a error telling that "java.lang.NegativeArraySizeException"
Somebody has some idea whats happening ???
Thanks a lot,
Juliano

Anyone how solved this problem?
I get the same one. I run with jdbc-odbc bridge.
Not good att all. It fails always in my application.
while((txtFileRow=reader.readLine())!=null & txtFileRow.length()>0)
System.out.println(txtFileRow);
char firstChar=txtFileRow.charAt(0);
if(firstChar=='1') {
stmt.setInt(1, nextIdInsert); //id
stmt.setInt(2, seqnrInsert++); //sequence
else if(firstChar=='2') {
stmt.setInt(1, nextIdUpdate); //id
stmt.setInt(2, seqnrUpdate++); //sequence
stmt.setInt(3, 1); //status
stmt.setString(4, "DATA"); //record_type
stmt.setInt(5, 63); //length ????????????????????????
stmt.setString(6, txtFileRow); //message_part1
stmt.addBatch();
rownr++;
if(rownr%100==0){
int[] result=stmt.executeBatch(); //!!!!!!!!!!!!!!!!!!!!
stmt.clearBatch();
int[] result=stmt.executeBatch(); //!!!!!!!!!!!!!!!!!!

Should PreparedStatement close before continue next PreparedStatement ?

Hi guys,
For instance, I need to use PreparedStatement for several insert and update like below:
Connection conn; // got the connection
PreparedStatement pstmt = conn.prepareStatement("insert ......");
pstmt.addBatch();
pstmt.executeBatch();
*// should I do a pstmt.close() here ?????????*
pstmt = conn.prepareStatement("update......");
pstmt.addBatch();
pstmt.executeBatch();
Thanks & Regards,
Mark

kmthien wrote:
Yes, I am using apache DBCP and using the example code here :
http://www.freshblurbs.com/jakarta-commons-dbcp-tutorial#comment-544
However, In my class, I need to connect to 2 database, 1 is local and another 1 is remote.Location doesn't matter.
I don't know how to create and initialize 2 connection pooling as when I initialize 2 connection pooling, it jus treated it as 1 only.This is of course completely different from your first question. Did you google?

More than 1 preparedStatement object using batch update

Hey how can I execute more than 1 preparedStatement object using batch update..Pls explain with a code(java)
Thanks

// turn off autocommit
con.setAutoCommit(false);
PreparedStatement stmt = con.prepareStatement(
"INSERT INTO employees VALUES (?, ?)");
stmt.setInt(1, 2000);
stmt.setString(2, "Kelly Kaufmann");
stmt.addBatch();
stmt.setInt(1, 3000);
stmt.setString(2, "Bill Barnes");
stmt.addBatch();
// submit the batch for execution
int[] updateCounts = stmt.executeBatch();
search in google for more information

PreparedStatement or executeBatch

Similar Messages

Maybe you are looking for