Memory leak in OCCI/libclnts on Linux RHEL 5.5 ? Please advise...

Question

Hi all,
Starting with some background:
I have an application that has been running on 32-bit Linux server for years without problem, a few weeks ago it was ported to 64-bit RHEL 5.5 (compiled with -m32). Now with the application in the new environment some processes are growing in memorysize.
I have tried to pinpoint the memory leak with Valgrind and have numerous leaks reported in "libclntsh.so", "libocci.so" and "libnnz11.so"
Examples:
==30114== Use of uninitialised value of size 4
==30114== at 0xA61BAD7: ztcedecb (in /soe3/opt/oracle-1/product/32bit-client-11.2.0.2/lib/libclntsh.so.11.1)
==30114== by 0xA61B1E2: ztcedencbk (in /soe3/opt/oracle-1/product/32bit-client-11.2.0.2/lib/libclntsh.so.11.1)
==30114== by 0xA61AA0A: ztcebn (in /soe3/opt/oracle-1/product/32bit-client-11.2.0.2/lib/libclntsh.so.11.1)
==30114== by 0xA61A561: ztcen (in /soe3/opt/oracle-1/product/32bit-client-11.2.0.2/lib/libclntsh.so.11.1)
==30114== by 0x8DA9725: ztceenc (in /soe3/opt/oracle-1/product/32bit-client-11.2.0.2/lib/libclntsh.so.11.1)
==30114== by 0x8E68920: ztcrbm (in /soe3/opt/oracle-1/product/32bit-client-11.2.0.2/lib/libclntsh.so.11.1)
==30114== by 0x8E684D5: ztcrbh (in /soe3/opt/oracle-1/product/32bit-client-11.2.0.2/lib/libclntsh.so.11.1)
==30114== by 0x8E68393: ztcrbp (in /soe3/opt/oracle-1/product/32bit-client-11.2.0.2/lib/libclntsh.so.11.1)
==30114== by 0x8E682C6: ztcr2seed (in /soe3/opt/oracle-1/product/32bit-client-11.2.0.2/lib/libclntsh.so.11.1)
==30114== by 0x8E6827F: ztcrseed3 (in /soe3/opt/oracle-1/product/32bit-client-11.2.0.2/lib/libclntsh.so.11.1)
==30114== by 0x8DAA24A: ztcsh (in /soe3/opt/oracle-1/product/32bit-client-11.2.0.2/lib/libclntsh.so.11.1)
==30114== by 0x99727D7: kpucpcreate (in /soe3/opt/oracle-1/product/32bit-client-11.2.0.2/lib/libclntsh.so.11.1)
And:
==30114== Conditional jump or move depends on uninitialised value(s)
==30114== at 0x8E69BBC: ztvo5ke (in /soe3/opt/oracle-1/product/32bit-client-11.2.0.2/lib/libclntsh.so.11.1)
==30114== by 0x8D55E68: kpu8lgn (in /soe3/opt/oracle-1/product/32bit-client-11.2.0.2/lib/libclntsh.so.11.1)
==30114== by 0x8D53E56: kpuauthxa (in /soe3/opt/oracle-1/product/32bit-client-11.2.0.2/lib/libclntsh.so.11.1)
==30114== by 0x8D536DB: kpuauth (in /soe3/opt/oracle-1/product/32bit-client-11.2.0.2/lib/libclntsh.so.11.1)
==30114== by 0x996EB99: kpucpinithstnode (in /soe3/opt/oracle-1/product/32bit-client-11.2.0.2/lib/libclntsh.so.11.1)
==30114== by 0x996FFB1: kpucpcrecons (in /soe3/opt/oracle-1/product/32bit-client-11.2.0.2/lib/libclntsh.so.11.1)
==30114== by 0x9972871: kpucpcreate (in /soe3/opt/oracle-1/product/32bit-client-11.2.0.2/lib/libclntsh.so.11.1)
==30114== by 0x9821048: OCIConnectionPoolCreate (in /soe3/opt/oracle-1/product/32bit-client-11.2.0.2/lib/libclntsh.so.11.1)
==30114== by 0x7BFB896: oracle::occi::ConnectionPoolImpl::initialise(oracle::occi::EnvironmentImpl*, void*, unsigned int, void*, unsigned int, void*, unsigned int, unsigned int, unsigned int, unsigned int) (in /soe3/opt/oracle-1/product/32bit-client-11.2.0.2/lib/libocci.so.11.1)
==30114== by 0x7BFBB79: ZN6oracle4occi18ConnectionPoolImplC9EPNS015EnvironmentImplERKSsS5_S5_jjj (in /soe3/opt/oracle-1/product/32bit-client-11.2.0.2/lib/libocci.so.11.1)
==30114== by 0x7BFBAC7: oracle::occi::ConnectionPoolImpl::ConnectionPoolImpl(oracle::occi::EnvironmentImpl*, std::string const&, std::string const&, std::string const&, unsigned int, unsigned int, unsigned int) (in /soe3/opt/oracle-1/product/32bit-client-11.2.0.2/lib/libocci.so.11.1)
==30114== by 0x7BF2C13: oracle::occi::EnvironmentImpl::createConnectionPool(std::string const&, std::string const&, std::string const&, unsigned int, unsigned int, unsigned int) (in /soe3/opt/oracle-1/product/32bit-client-11.2.0.2/lib/libocci.so.11.1)
And:
==30114== Conditional jump or move depends on uninitialised value(s)
==30114== at 0xAC3B2A5: CMP_RecomputeLength (in /soe3/opt/oracle-1/product/32bit-client-11.2.0.2/lib/libnnz11.so)
==30114== by 0xAC3B9EC: CMP_Divide (in /soe3/opt/oracle-1/product/32bit-client-11.2.0.2/lib/libnnz11.so)
==30114== by 0xAC3B5BA: CMP_ModularReduce (in /soe3/opt/oracle-1/product/32bit-client-11.2.0.2/lib/libnnz11.so)
==30114== by 0xAC3A2E6: Alg_ComputeModQ_GHash (in /soe3/opt/oracle-1/product/32bit-client-11.2.0.2/lib/libnnz11.so)
==30114== by 0xAC3AA30: A_X931RandomGenerateBytes (in /soe3/opt/oracle-1/product/32bit-client-11.2.0.2/lib/libnnz11.so)
==30114== by 0x8E681F0: ztcr2rnd (in /soe3/opt/oracle-1/product/32bit-client-11.2.0.2/lib/libclntsh.so.11.1)
==30114== by 0x8DAA260: ztcsh (in /soe3/opt/oracle-1/product/32bit-client-11.2.0.2/lib/libclntsh.so.11.1)
==30114== by 0x8CD0125: kpusattr (in /soe3/opt/oracle-1/product/32bit-client-11.2.0.2/lib/libclntsh.so.11.1)
==30114== by 0x8CE96CD: OCIAttrSet (in /soe3/opt/oracle-1/product/32bit-client-11.2.0.2/lib/libclntsh.so.11.1)
==30114== by 0x7BF6184: oracle::occi::ConnectionImpl::openConnection(OCIEnv*, OCIError*, void*, unsigned int, void*, unsigned int, void*, unsigned int, void*, unsigned int, unsigned int, void*, unsigned int, oracle::occi::Connection::Purity, oracle::occi::StatelessConnectionPool::PoolType) (in /soe3/opt/oracle-1/product/32bit-client-11.2.0.2/lib/libocci.so.11.1)
==30114== by 0x7BF9C29: ZN6oracle4occi14ConnectionImplC9EPNS018ConnectionPoolImplERKSsS5_ (in /soe3/opt/oracle-1/product/32bit-client-11.2.0.2/lib/libocci.so.11.1)
==30114== by 0x7BF9AD1: oracle::occi::ConnectionImpl::ConnectionImpl(oracle::occi::ConnectionPoolImpl*, std::string const&, std::string const&) (in /soe3/opt/oracle-1/product/32bit-client-11.2.0.2/lib/libocci.so.11.1)
There has been quite a few changes due to the new server. New compiler (gcc) from version 2.96 to 4.1.2, new version of oracle client 10g to 11g (11.2.0.2), new version of OCCI (also 11.2.0.2). The applications sourcecode is pretty much unchanged though, and that is why I turn to all of you for help.
Is there anyone who knows anything about this problem and how to even start to fix this leak (if it even is in OCCI)?
I can supply sourcecode and/or logs if needed. It is quite a lot of complex code so it might take me a while to sort out the relevant code though...

Answer 1

Thanks for your input! Tough this post is from some time ago this is still an issue for me I'm afraid, so I appreciate all the help I can get.
I have stripped down the code to just the basic functions needed to replicate the problem. The pool creation and connections are the same as in our application (which is unchanged since before the porting). Though the following code makes no indications of a leak when run through Valgrind, it still grows when checked in "top". It could be that I stripped the code too much, but this is pretty much what our application does. It creates a database lock (not in this code), creates connection, (read table), terminates connection and release lock every 6 seconds to check if there is any new assignments added...
#include <iostream>
#include <string>
#include <occi.h>
using namespace oracle::occi;
using namespace std;
Environment *env;
ConnectionPool *connPool;
Connection *con;
int x, loopnr;
string usr = "test_user";
string pwd = "test_pw";
string url = "dbtest";
void run(int x) { // Run loops
for(int i=0; i<x; i++){ // Loop connect and disconnect x number of times
try {
fflush(stdout); // Update console row for count value
con = connPool->createConnection (usr, pwd); // Connect
usleep(100000);
connPool->terminateConnection (con); // Disconnect
cout << "\r"; // Update console row for count value
cout << loopnr << endl; // Print number of loops done
usleep(100000);
loopnr++; // Count number of loops done
catch(SQLException ex) { // Exception handling
cout<<"SQLException"<<endl;
cout<<"Error number: "<< ex.getErrorCode() << endl;
cout<<ex.getMessage() << endl;
int main(int argc,char* argv[]) {
cout << "IN --- main()\n";
if(argc == 2) { // Read number of loops as argument...
x = atoi(argv[1]);
else { // ...or run loop 200 times if no args
x = 200;
try{
env = Environment::createEnvironment (Environment::DEFAULT); // Create environment
loopnr = 1;
connPool = env->createConnectionPool(usr, pwd, url, 1, 2, 1); // Create pool
run(x); // Run loops
catch(SQLException ex) { // Exception handling
cout<<"SQLException"<<endl;
cout<<"Error number: "<< ex.getErrorCode() << endl;
cout<<ex.getMessage() << endl;
env->terminateConnectionPool (connPool); // Cleanup Pool when done
Environment::terminateEnvironment (env); // Cleanup Environment when done
cout << "\nUT --- main()\n";
return 0;
Above code is built with the following makefile:
ORACLE_HOME=/opt/oracle/product/32bit-client-11.2.0.2
CXXFLAGS = -Wall -m32 -g -fPIC \
-I$(ORACLE_HOME)/rdbms/public
LDLIBS = -L$(ORACLE_HOME)/lib/ -locci \
-L$(ORACLE_HOME)/rdbms/lib/ -lclntsh
LDFLAGS = -m32
default: all
Sample: Sample.cc
all: Sample
clean:
rm -f Sample *.o
Hopefully someone can make sense out of this....
Edited by: StLa on 2011-nov-23 04:57