Having problems with new WebLogic MS SQL driver WLS 8.1

Question

Hi,
We've recently migrated our application from WLS 6.1 to WLS 8.1. We use an MS SQL database. In 6.1 we used the (now deprecated) weblogic.jdbc.mssqlserver4.Driver driver. In 8.1 we now use the new weblogic.jdbc.sqlserver.SQLServerDriver driver. We now are seeing NullPointerExceptions with the new driver that we never saw when running in 6.1 (with the old driver). The exception is occuring at the driver level. I'll paste some of the stack traces below but here is a quick synopsis of our usage pattern:
-standard get-use-close pattern on the connection for each request (connection is closed in finally block).
-pool size: min 1, max 7, increment 5
-test on reserver enabled
-we are simply reading data (no updates)
-we use a single thread that hits the database so there is only ever one active application thread hitting the DB.
-we are using a stored proc.
-we setTimeout on the statement (initial testing seems to indicate that query timeout seems to be at the root of the problem)
The problem seems to occur when statement timeouts occur. I can recreate the problem by setting the statement timeout to 1 sec and throwing a lot data at the DB. What seems to occur is a query hits the timeout and a SocketException is thrown, subsequent queries then throw a NullPointer in the driver layer. Again, this is the behavior under WLS 8.1 with the new weblogic.jdbc.sqlserver.SQLServerDriver driver. If I use the old driver (weblogic.jdbc.mssqlserver4.Driver) under WLS 8.1 all I get is the SocketException (which is what I want). This application ran under WLS 6.1 with the old driver processing millions of queries without any problems.
We are wondering:
(1) Is this a known problem?
(2) Is there a recommended workaround?
Cheers, and as always thanks in advance for any help.
Stack traces follow.
Thank,
Mich
Here are some failure traces from the WLS 8.1 new driver:
java.sql.SQLException: [BEA][SQLServer JDBC Driver]Execution timeout expired.
     at weblogic.jdbc.base.BaseExceptions.createException(Unknown Source)
     at weblogic.jdbc.base.BaseExceptions.getException(Unknown Source)
     at weblogic.jdbc.base.BaseStatement.postProcessExceptionFromCommonExecute(Unknown Source)
     at weblogic.jdbc.base.BaseStatement.executeQueryInternal(Unknown Source)
     at weblogic.jdbc.base.BasePreparedStatement.executeQuery(Unknown Source)
     at weblogic.jdbc.wrapper.PreparedStatement.executeQuery(PreparedStatement.java:124)
     at com.our.QueryHandler.execute(QueryHandler.java:79)
THEN WHEN A SUBSEQUENT QUERY IS MADE:
java.lang.NullPointerException
     at weblogic.jdbc.sqlserver.tds.TDSRequest.switchColumnDescriptions(Unknown Source)
     at weblogic.jdbc.sqlserver.tds.TDSRequest.processRow(Unknown Source)
     at weblogic.jdbc.sqlserver.tds.TDSRequest.processReplyToken(Unknown Source)
     at weblogic.jdbc.sqlserver.tds.TDSRPCRequest.processReplyToken(Unknown Source)
     at weblogic.jdbc.sqlserver.tds.TDSRequest.processReply(Unknown Source)
     at weblogic.jdbc.sqlserver.tds.TDSRPCNonCursorExecuteRequest.submitPrepare(Unknown Source)
     at weblogic.jdbc.sqlserver.tds.TDSRPCExecuteRequest.doPrepExec(Unknown Source)
     at weblogic.jdbc.sqlserver.tds.TDSRPCExecuteRequest.execute(Unknown Source)
     at weblogic.jdbc.sqlserver.SQLServerImplStatement.execute(Unknown Source)
     at weblogic.jdbc.base.BaseStatement.commonExecute(Unknown Source)
     at weblogic.jdbc.base.BaseStatement.executeQueryInternal(Unknown Source)
     at weblogic.jdbc.base.BasePreparedStatement.executeQuery(Unknown Source)
     at weblogic.jdbc.wrapper.PreparedStatement.executeQuery(PreparedStatement.java:124)
     at com.our.QueryHandler.execute(QueryHandler.java:79)
Just to be complete here is the stacktrace generated by a timeout using the old driver on WLS 8.1 (again, this one behaves as expected):
weblogic.jdbc.mssqlserver4.TdsException: I/O exception while talking to the server, java.net.SocketTimeoutException: Read timed out
     at weblogic.jdbc.mssqlserver4.TdsStatement.getMoreResults(TdsStatement.java:813)
     at weblogic.jdbc.mssqlserver4.TdsStatement.execute(TdsStatement.java:210)
     at weblogic.jdbc.mssqlserver4.TdsStatement.executeQuery(TdsStatement.java:54)
     at weblogic.jdbc.mssqlserver4.TdsStatement.executeQuery(TdsStatement.java:1424)
     at weblogic.jdbc.wrapper.PreparedStatement.executeQuery(PreparedStatement.java:124)

Answer 1

Michel Crichton wrote:
Hi,
Thanks for the prompt reply. In answer to your questions:
What I would like to know is the exact code/sequence that causes the subsequent exception. See the sequence of calls at the end of this reply.
Is this in the same thread in the very next call?
I replicated the problem by going through the Web container (5 clients) therefore WebLogic threads are used in this case. I see from the logs 4-5 different execute threads at work (ex. ExecuteThread: '10' for queue: 'weblogic.kernel.Default').
Does it happen all the time?
Although it is tough to tell for sure, once it starts happening it seems to happen most if not all the time.
Is it an attempted re-use of the same statement, or trying to use a different statement?
We do a connection.prepareStatement(QUERY_STRING) call everytime (there is only one per request/response cycle). I believe WLS caches statements behind the scenes though so I suspect at that level there is reuse. There is only one stored proc used (we vary the params per call).
Please show me your pool definition from the config.xml
file, and I will send you instructions to produce some
debug info that will get us to the heart of this matter.
<JDBCConnectionPool
DriverName="weblogic.jdbc.sqlserver.SQLServerDriver"
MaxCapacity="5" Name="THE POOL"
PasswordEncrypted="XXX"
Properties="user=XXX;portNumber=XXX;databaseName=XX;serverName=X.X.X.X"
Targets="Server"
TestConnectionsOnReserve="true" TestTableName="TEST_TABLE" URL="jdbc:bea:sqlserver://X.X.X.X:XXX"/>Ok, do these things:
1 - Add our spy attribute to your pool definition, like this:
<JDBCConnectionPool
DriverName="weblogic.jdbc.sqlserver.SQLServerDriver"
MaxCapacity="5" Name="THE POOL"
PasswordEncrypted="XXX"
Properties="user=XXX;portNumber=XXX;databaseName=XX;serverName=X.X.X.X"
Targets="Server"
TestConnectionsOnReserve="true"
TestTableName="TEST_TABLE"
URL="jdbc:bea:sqlserver://X.X.X.X:XXX;spyAttributes=(log=(file)spy.log;timestamp=yes)"/>
2 - Edit the start-weblogic script to add the spy jar (wlspy.jar) to the
classpath that the script creates for the server, eg:
set CLASSPATH=%WL_HOME%\server\lib\wlspy.jar;%CLASSPATH%
3 - Reboot and duplicate the problem as briefly as possible.
4 - In the same directory where you start the server, you will
now find a file named 'spy.log'. Please send it to me. ( j o e AT b e a DOT c o m )
I will try to reproduce the problem.
5 - Ask BEA support for the latest BEA driver package and install it according to
their instructions. Let me know asap if the probem is still there.
thanks
Joe
>
Also, I forgot to post in the original message some of the WLS stdout related logging we see. Here it is (sorry, it's a little messy):
11:38:37 o'clock AM EST> <Error> <JDBC> <ExecuteThread: '11' for queue: 'weblogic.kernel.Default'> <> <BEA-001131> <Received an exception when closing a cached statement for the pool " THE POOL": java.sql.SQLException: [BEA][SQLServer JDBC Driver]The operation was cancelled at the user's request..>
11:38:37 o'clock AM EST> <Error> <JDBC> <ExecuteThread: '13' for queue: 'weblogic.kernel.Default'> <> <BEA-001131> <Received an exception when closing a cached statement for the pool " THE POOL": java.sql.SQLException: [BEA][SQLServer JDBC Driver]The operation was cancelled at the user's request..>
11:38:37 o'clock AM EST> <Error> <JDBC> <ExecuteThread: '10' for queue: 'weblogic.kernel.Default'> <> <BEA-001131> <Received an exception when closing a cached statement for the pool " THE POOL": java.sql.SQLException: [BEA][SQLServer JDBC Driver]The operation was cancelled at the user's request..>
11:38:38 o'clock AM EST> <Error> <JDBC> <ExecuteThread: '14' for queue: 'weblogic.kernel.Default'> <> <BEA-001112> <Test "select count(*) from TEST_TABLE" set up for pool " THE POOL" failed with exception: "java.sql.SQLException: [BEA][SQLServer JDBC Driver]A problem occurred when attempting to contact the server (Server returned: Connection reset). Please ensure that the server parameters passed to the driver are correct and that the server is running. Also ensure that the maximum number of connections
have not been exceeded for this server.".>
11:38:38 o'clock AM EST> <Info> <JDBC> <ExecuteThread: '14' for queue: 'weblogic.kernel.Default'> <> <BEA-001128> <Connection for pool " THE POOL" closed.>
11:38:38 o'clock AM EST> <Info> <JDBC> <ExecuteThread: '14' for queue: 'weblogic.kernel.Default'> <> <BEA-001067> <Connection for pool " THE POOL" refreshed.>
11:38:40 o'clock AM EST> <Error> <JDBC> <ExecuteThread: '10' for queue: 'weblogic.kernel.Default'> <> <BEA-001112> <Test "select count(*) from TEST_TABLE" set up for pool " THE POOL" failed with exception: "java.sql.SQLException: [BEA][SQLServer JDBC Driver]A problem occurred when attempting to contact the server (Server returned: Connection reset). Please ensure that the server parameters passed to the driver are correct and that the server is running. Also ensure that the maximum number of connections
have not been exceeded for this server.".>
11:38:40 o'clock AM EST> <Info> <JDBC> <ExecuteThread: '10' for queue: 'weblogic.kernel.Default'> <> <BEA-001128> <Connection for pool " THE POOL" closed.>
11:38:40 o'clock AM EST> <Info> <JDBC> <ExecuteThread: '10' for queue: 'weblogic.kernel.Default'> <> <BEA-001067> <Connection for pool " THE POOL" refreshed.>
11:38:41 o'clock AM EST> <Error> <JDBC> <ExecuteThread: '13' for queue: 'weblogic.kernel.Default'> <> <BEA-001131> <Received an exception when closing a cached statement for the pool " THE POOL": java.sql.SQLException: [BEA][SQLServer JDBC Driver]The operation was cancelled at the user's request..>
11:38:41 o'clock AM EST> <Error> <JDBC> <ExecuteThread: '10' for queue: 'weblogic.kernel.Default'> <> <BEA-001131> <Received an exception when closing a cached statement for the pool " THE POOL": java.sql.SQLException: [BEA][SQLServer JDBC Driver]The operation was cancelled at the user's request..>
Here is the sequence of calls we make (notice the statement is closed outside the finally however we assume the connection.close should close associated resources):
PreparedStatement statement = null;
try {
connection = dataSource.getConnection();
statement = connection.prepareStatement(QUERY_STRING);
statement.setString(1, name1);
statement.setString(2, name2);
statement.setQueryTimeout(timeout);
ResultSet resultSet = statement.executeQuery();     
//iterate over result set
statement.close();
return result;               
} catch (SQLException e1) {
//log
} finally {
try {
if(connection != null)
connection.close();
} catch (SQLException e2) {
//log
Thanks again,
Mich