Memory leak in query preparation in dbxml-2.3.10
Hi,
We are using dbxml-2.3.10 in our production. I have ran valgrind and see big memory leaks under two categories:
Definetly lost:
The complete stack trace is as below:
==25482== 13,932 bytes in 129 blocks are definitely lost in loss record 32 of 35
2547 ==25482== at 0x4004790: operator new(unsigned) (vg_replace_malloc.c:164)
2548 ==25482== by 0x4144131: XQSort::SortSpec::staticResolution(StaticContext*, StaticResolutionContext&) (in /usr/netscreen/GuiSvr/utils /dbxml-2.3.10/lib/libxqilla.so.1.0.0)
2549 ==25482== by 0x4144BEC: XQSort::staticResolution(StaticContext*, StaticResolutionContext&) (in /usr/netscreen/GuiSvr/utils/dbxml-2.3 .10/lib/libxqilla.so.1.0.0)
2550 ==25482== by 0x4145A2A: XQFLWOR::staticResolutionImpl(StaticContext*) (in /usr/netscreen/GuiSvr/utils/dbxml-2.3.10/lib/libxqilla.so. 1.0.0)
2551 ==25482== by 0x4146018: XQFLWOR::staticResolution(StaticContext*) (in /usr/netscreen/GuiSvr/utils/dbxml-2.3.10/lib/libxqilla.so.1.0. 0)
2552 ==25482== by 0x41780CC: XQQuery::staticResolution(StaticContext*) (in /usr/netscreen/GuiSvr/utils/dbxml-2.3.10/lib/libxqilla.so.1.0. 0)
2553 ==25482== by 0x4563D6E: DbXml::StaticResolver::optimize(XQQuery*) (Optimizer.cpp:64)
2554 ==25482== by 0x4563C42: DbXml::Optimizer::startOptimize(XQQuery*) (Optimizer.cpp:42)
2555 ==25482== by 0x4563C5B: DbXml::Optimizer::startOptimize(XQQuery*) (Optimizer.cpp:39)
2556 ==25482== by 0x4563C5B: DbXml::Optimizer::startOptimize(XQQuery*) (Optimizer.cpp:39)
2557 ==25482== by 0x4563C5B: DbXml::Optimizer::startOptimize(XQQuery*) (Optimizer.cpp:39)
2558 ==25482== by 0x4563C5B: DbXml::Optimizer::startOptimize(XQQuery*) (Optimizer.cpp:39)
2559 ==25482== by 0x4563C5B: DbXml::Optimizer::startOptimize(XQQuery*) (Optimizer.cpp:39)
2560 ==25482== by 0x4563C5B: DbXml::Optimizer::startOptimize(XQQuery*) (Optimizer.cpp:39)
2561 ==25482== by 0x4563C5B: DbXml::Optimizer::startOptimize(XQQuery*) (Optimizer.cpp:39)
2562 ==25482== by 0x4563C5B: DbXml::Optimizer::startOptimize(XQQuery*) (Optimizer.cpp:39)
2563 ==25482== by 0x4563C5B: DbXml::Optimizer::startOptimize(XQQuery*) (Optimizer.cpp:39)
2564 ==25482== by 0x4446CE9: DbXml::QueryExpression::QueryExpression(std::string const&, DbXml::XmlQueryContext&, DbXml::Transaction*) (S copedPtr.hpp:41)
2565 ==25482== by 0x44A3F63: DbXml::XmlManager::prepare(std::string const&, DbXml::XmlQueryContext&) (XmlManager.cpp:601)
2566 ==25482== by 0x82B3152: XQuery::prepare(unsigned, unsigned short, char const*, char const*, char const*, char const*, char const*, R efCountedAutoPtr<XdbQueryContext>, unsigned) (XQuery.cpp:152)
We see another huge leak in possiibly lost category:
371,895 bytes in 121 blocks are possibly lost in loss record 33 of 35
2570 ==25482== at 0x4004405: malloc (vg_replace_malloc.c:149)
2571 ==25482== by 0x818C330: malloc (guiDaemon.c:783)
2572 ==25482== by 0x44A5B0C: DbXml::SimpleMemoryManager::allocate(unsigned) (Globals.cpp:67)
2573 ==25482== by 0x497CCFC: xercesc_2_7::XMemory::operator new(unsigned, xercesc_2_7::MemoryManager*) (in /usr/netscreen/GuiSvr/utils/db xml-2.3.10/lib/libxerces-c.so.27.0)
2574 ==25482== by 0x48D681A: xercesc_2_7::XMLPlatformUtils::makeMutex(xercesc_2_7::MemoryManager*) (in /usr/netscreen/GuiSvr/utils/dbxml- 2.3.10/lib/libxerces-c.so.27.0)
2575 ==25482== by 0x44A61B6: DbXml::Globals::initialize(DbEnv*) (Globals.cpp:78)
2576 ==25482== by 0x44A766C: DbXml::Manager::initialize(DbEnv*) (Manager.cpp:167)
2577 ==25482== by 0x44A8CBB: DbXml::Manager::Manager(DbEnv*, unsigned) (Manager.cpp:98)
2578 ==25482== by 0x44A2EAD: DbXml::XmlManager::XmlManager(DbEnv*, unsigned) (XmlManager.cpp:58)
2579 ==25482== by 0x83398EA: XdbImpl::initDb(bool, int) (XdbImpl.cpp:478)
2580 ==25482== by 0x8337407: XdbImpl::start(char const*, int) (XdbImpl.cpp:159)
2581 ==25482== by 0x8321123: Xdb::start(char const*, int) (Xdb.cpp:56)
Are these leaks addressed in some 2.3.10 patch? Please suggest way to solve the same.
PS : We are trying to upgrade to 2.5.16 but there are certain issues already reported in another thread due to which we are not able to migrate.
Have you tried turning on diagnostic logging in BDBXML and trying to parse the output? The library gives out some pretty detailed output. It might be helpful to see what the query optimizer is trying to do, as well as see what the XQuery looks like that you're running, to see if we can either pinpoint the bug or find a suitable workaround that doesn't trigger the memory leak.
Similar Messages
-
Memory leak in xquery code in DbXml?
While running a xml db which is update and xquery heavy, we are seeing the memory foot print grow to multiple gigs in a few minutes.
One of the sample output from running it in valgrind is as follows.
==6847== 792,931,233 bytes in 4,695,090 blocks are still reachable in loss record 323 of 323
==6847== at 0x4A19007: malloc (in /usr/lib64/valgrind/amd64-linux/vgpreload_memcheck.so)
==6847== by 0x64BE0C5: BaseMemoryManager::allocate(unsigned long) (in /usr/local/maui/lib/libxqilla.so.4.0.1)
==6847== by 0x554A0C0: xercesc_2_8::XMemory::operator new(unsigned long, xercesc_2_8::MemoryManager*) (in /usr/local/maui/lib/libxerces-c.so.28.0)
==6847== by 0x64E2BC8: xercesc_2_8::RefHash2KeysTableOf<int>::RefHash2KeysTableOf(unsigned, bool, xercesc_2_8::MemoryManager*) (in /usr/local/maui/lib/libxqilla.so.4.0.1)
==6847== by 0x64E181E: StaticAnalysis::StaticAnalysis(XPath2MemoryManager*) (in /usr/local/maui/lib/libxqilla.so.4.0.1)
==6847== by 0x61A7076: DbXml::SequentialScanQP::SequentialScanQP(DbXml::ImpliedSchemaNode::Type, DbXml::ImpliedSchemaNode*, DbXml::ContainerBase*, unsigned, XPath2MemoryManager*) (QueryPlan.hpp:160)
==6847== by 0x61A7136: DbXml::SequentialScanQP::copy(XPath2MemoryManager*) const (XPath2MemoryManager.hpp:175)
==6847== by 0x619474F: DbXml::QueryPlan::createCombinations(DbXml::OptimizationContext&, std::vector<DbXml::QueryPlan*, std::allocator<DbXml::QueryPlan*> >&) const (stl_vector.h:557)
==6847== by 0x618DC2E: DbXml::QueryPlan::createAlternatives(DbXml::OptimizationContext&, std::vector<DbXml::QueryPlan*, std::allocator<DbXml::QueryPlan*> >&) const (QueryPlan.cpp:128)
==6847== by 0x61CC1D5: DbXml::LevelFilterQP::createCombinations(DbXml::OptimizationContext&, std::vector<DbXml::QueryPlan*, std::allocator<DbXml::QueryPlan*> >&) const (LevelFilterQP.cpp:101)
==6847== by 0x61994A0: DbXml::QueryPlan::createReducedAlternatives(double, unsigned, DbXml::OptimizationContext&, std::vector<DbXml::QueryPlan*, std::allocator<DbXml::QueryPlan*> >&) const (QueryPlan.cpp:157)
==6847== by 0x6182852: DbXml::StructuralJoinQP::createCombinations(DbXml::OptimizationContext&, std::vector<DbXml::QueryPlan*, std::allocator<DbXml::QueryPlan*> >&) const (StructuralJoinQP.cpp:260)
==6847== by 0x61994A0: DbXml::QueryPlan::createReducedAlternatives(double, unsigned, DbXml::OptimizationContext&, std::vector<DbXml::QueryPlan*, std::allocator<DbXml::QueryPlan*> >&) const (QueryPlan.cpp:157)
==6847== by 0x61B2C13: DbXml::NodePredicateFilterQP::createCombinations(DbXml::OptimizationContext&, std::vector<DbXml::QueryPlan*, std::allocator<DbXml::QueryPlan*> >&) const (NodePredicateFilterQP.cpp:290)
==6847== by 0x61994A0: DbXml::QueryPlan::createReducedAlternatives(double, unsigned, DbXml::OptimizationContext&, std::vector<DbXml::QueryPlan*, std::allocator<DbXml::QueryPlan*> >&) const (QueryPlan.cpp:157)
==6847== by 0x618258C: DbXml::StructuralJoinQP::applyConversionRules(DbXml::OptimizationContext&, std::vector<DbXml::QueryPlan*, std::allocator<DbXml::QueryPlan*> >&) (XPath2MemoryManager.hpp:202)
==6847== by 0x618355D: DbXml::ChildJoinQP::applyConversionRules(DbXml::OptimizationContext&, std::vector<DbXml::QueryPlan*, std::allocator<DbXml::QueryPlan*> >&) (ChildJoinQP.cpp:46)
==6847== by 0x619958D: DbXml::QueryPlan::createReducedAlternatives(double, unsigned, DbXml::OptimizationContext&, std::vector<DbXml::QueryPlan*, std::allocator<DbXml::QueryPlan*> >&) const (stl_iterator.h:614)
==6847== by 0x61824FE: DbXml::StructuralJoinQP::applyConversionRules(DbXml::OptimizationContext&, std::vector<DbXml::QueryPlan*, std::allocator<DbXml::QueryPlan*> >&) (XPath2MemoryManager.hpp:202)
==6847== by 0x619958D: DbXml::QueryPlan::createReducedAlternatives(double, unsigned, DbXml::OptimizationContext&, std::vector<DbXml::QueryPlan*, std::allocator<DbXml::QueryPlan*> >&) const (stl_iterator.h:614)
==6847== by 0x61B2C13: DbXml::NodePredicateFilterQP::createCombinations(DbXml::OptimizationContext&, std::vector<DbXml::QueryPlan*, std::allocator<DbXml::QueryPlan*> >&) const (NodePredicateFilterQP.cpp:290)
==6847== by 0x618DC2E: DbXml::QueryPlan::createAlternatives(DbXml::OptimizationContext&, std::vector<DbXml::QueryPlan*, std::allocator<DbXml::QueryPlan*> >&) const (QueryPlan.cpp:128)
==6847== by 0x61B25A0: DbXml::NodePredicateFilterQP::applyConversionRules(DbXml::OptimizationContext&, std::vector<DbXml::QueryPlan*, std::allocator<DbXml::QueryPlan*> >&) (XPath2MemoryManager.hpp:202)
==6847== by 0x619958D: DbXml::QueryPlan::createReducedAlternatives(double, unsigned, DbXml::OptimizationContext&, std::vector<DbXml::QueryPlan*, std::allocator<DbXml::QueryPlan*> >&) const (stl_iterator.h:614)
==6847== by 0x618258C: DbXml::StructuralJoinQP::applyConversionRules(DbXml::OptimizationContext&, std::vector<DbXml::QueryPlan*, std::allocator<DbXml::QueryPlan*> >&) (XPath2MemoryManager.hpp:202)
==6847== by 0x618355D: DbXml::ChildJoinQP::applyConversionRules(DbXml::OptimizationContext&, std::vector<DbXml::QueryPlan*, std::allocator<DbXml::QueryPlan*> >&) (ChildJoinQP.cpp:46)
==6847== by 0x619958D: DbXml::QueryPlan::createReducedAlternatives(double, unsigned, DbXml::OptimizationContext&, std::vector<DbXml::QueryPlan*, std::allocator<DbXml::QueryPlan*> >&) const (stl_iterator.h:614)
==6847== by 0x6182892: DbXml::StructuralJoinQP::createCombinations(DbXml::OptimizationContext&, std::vector<DbXml::QueryPlan*, std::allocator<DbXml::QueryPlan*> >&) const (StructuralJoinQP.cpp:264)
==6847== by 0x61CAF39: DbXml::BufferQP::createCombinations(DbXml::OptimizationContext&, std::vector<DbXml::QueryPlan*, std::allocator<DbXml::QueryPlan*> >&) const (XPath2MemoryManager.hpp:202)
==6847== by 0x6198964: DbXml::QueryPlan::chooseAlternative(DbXml::OptimizationContext&, char const*, bool) const (QueryPlan.cpp:291)
==6847== by 0x61CAD75: DbXml::BufferQP::createCombinations(DbXml::OptimizationContext&, std::vector<DbXml::QueryPlan*, std::allocator<DbXml::QueryPlan*> >&) const (BufferQP.cpp:230)
==6847== by 0x6198964: DbXml::QueryPlan::chooseAlternative(DbXml::OptimizationContext&, char const*, bool) const (QueryPlan.cpp:291)
==6847== by 0x61567BE: DbXml::QueryPlanOptimizer::optimizeQueryPlanToAST(DbXml::QueryPlanToAST*) (QueryPlanToAST.hpp:29)
==6847== by 0x66506E6: ASTVisitor::optimizeForTuple(ForTuple*) (in /usr/local/maui/lib/libxqilla.so.4.0.1)
==6847== by 0x665066A: ASTVisitor::optimizeTupleNode(TupleNode*) (in /usr/local/maui/lib/libxqilla.so.4.0.1)
==6847== by 0x664FF63: ASTVisitor::optimizeReturn(XQReturn*) (in /usr/local/maui/lib/libxqilla.so.4.0.1)
==6847== by 0x664F9B3: ASTVisitor::optimize(ASTNode*) (in /usr/local/maui/lib/libxqilla.so.4.0.1)
==6847== by 0x6650DBF: ASTVisitor::optimize(XQQuery*) (in /usr/local/maui/lib/libxqilla.so.4.0.1)
==6847== by 0x664F7A4: Optimizer::startOptimize(XQQuery*) (in /usr/local/maui/lib/libxqilla.so.4.0.1)
==6847== by 0x664F7A4: Optimizer::startOptimize(XQQuery*) (in /usr/local/maui/lib/libxqilla.so.4.0.1)
==6847== by 0x664F7A4: Optimizer::startOptimize(XQQuery*) (in /usr/local/maui/lib/libxqilla.so.4.0.1)
==6847== by 0x664F7A4: Optimizer::startOptimize(XQQuery*) (in /usr/local/maui/lib/libxqilla.so.4.0.1)
==6847== by 0x664F7A4: Optimizer::startOptimize(XQQuery*) (in /usr/local/maui/lib/libxqilla.so.4.0.1)
==6847== by 0x6079D98: DbXml::QueryExpression::QueryExpression(std::string const&, DbXml::XmlQueryContext&, DbXml::Transaction*) (ScopedPtr.hpp:41)
==6847== by 0x60D6A5B: DbXml::XmlManager::prepare(DbXml::XmlTransaction&, std::string const&, DbXml::XmlQueryContext&) (XmlManager.cpp:563)
==6847== by 0x60D6B70: DbXml::XmlManager::query(DbXml::XmlTransaction&, std::string const&, DbXml::XmlQueryContext&, unsigned) (XmlManager.cpp:581)
Message was edited by:
user626230This looks like a memory leak. Please correct me if I am wrong. Also, since the library uses a memory manager would'nt it be reported differently?
They allocate a a new QueryPlan when they do a copy
QueryPlan SequentialScanQP::copy(XPath2MemoryManager mm) const
if(!mm) {
mm = memMgr_;
SequentialScanQP *result = new (mm) SequentialScanQP(nodeType_, isn_, container_, flags_, mm);
result->nameid_ = nameid_;
result->nsUriID_ = nsUriID_;
result->cost_ = cost_;
result->setLocationInfo(this);
return result;
Which is called from
void QueryPlan::createCombinations(OptimizationContext &opt, QueryPlans &combinations) const
combinations.push_back(copy(opt.getMemoryManager()));
which is never freed here.
void QueryPlan::createAlternatives(OptimizationContext &opt, QueryPlans &alternatives) const
QueryPlans combinations;
createCombinations(opt, combinations);
// Generate the alternatives by applying conversion rules to the combinations
for(QueryPlans::iterator it = combinations.begin(); it != combinations.end(); ++it) {
(*it)->applyConversionRules(opt, alternatives);
} -
SQL Server 2008R2 SP2 Query optimizer memory leak ?
It looks like we are facing a SQL Server 2008R2 queery optimizer memory leak.
We have below version of SQL Server
Microsoft SQL Server 2008 R2 (SP2) - 10.50.4000.0 (X64)
Jun 28 2012 08:36:30
Copyright (c) Microsoft Corporation
Standard Edition (64-bit) on Windows NT 6.1 <X64> (Build 7601: Service Pack 1)
The instance is set MAximum memory tro 20 GB.
After executing a huge query (2277 kB generated by IBM SPSS Clementine) with tons of CASE and a lot of AND/OR statements in the WHERE and CASE statements and muliple subqueries the server stops responding on Out of memory in the internal pool
and the query optimizer has allocated all the memory.
From Management Data Warehouse we can find that the query was executed at
7.11.2014 22:40:57
Then at 1:22:48 we recieve FAIL_PACE_ALLOCATION 1
2014-11-08 01:22:48.70 spid75 Failed allocate pages: FAIL_PAGE_ALLOCATION 1
And then tons of below errors
2014-11-08 01:24:02.22 spid87 There is insufficient system memory in resource pool 'internal' to run this query.
2014-11-08 01:24:02.22 Server Error: 17300, Severity: 16, State: 1. (Params:). The error is printed in terse mode because there was error during formatting. Tracing, ETW, notifications etc are skipped.
2014-11-08 01:24:02.22 Server Error: 17300, Severity: 16, State: 1. (Params:). The error is printed in terse mode because there was error during formatting. Tracing, ETW, notifications etc are skipped.
2014-11-08 01:24:02.30 Server Error: 17312, Severity: 16, State: 1.
2014-11-08 01:24:02.30 Server SQL Server is terminating a system or background task Fulltext Host Controller Timer Task due to errors in starting up the task (setup state 1).
2014-11-08 01:24:02.22 spid74 Error: 701, Severity: 17, State: 123.
2014-11-08 01:24:02.22 spid74 There is insufficient system memory in resource pool 'internal' to run this query.
2014-11-08 01:24:13.22 Server Error: 17312, Severity: 16, State: 1. (Params:). The error is printed in terse mode because there was error during formatting. Tracing, ETW, notifications etc are skipped.
2014-11-08 01:24:13.22 spid87 Error: 701, Severity: 17, State: 123.
2014-11-08 01:24:13.22 spid87 There is insufficient system memory in resource pool 'internal' to run this query.
2014-11-08 01:24:13.22 spid63 Error: 701, Severity: 17, State: 130.
2014-11-08 01:24:13.22 spid63 There is insufficient system memory in resource pool 'internal' to run this query.
2014-11-08 01:24:13.22 spid57 Error: 701, Severity: 17, State: 123.
2014-11-08 01:24:13.22 spid57 There is insufficient system memory in resource pool 'internal' to run this query.
2014-11-08 01:24:13.22 Server Error: 17300, Severity: 16, State: 1. (Params:). The error is printed in terse mode because there was error during formatting. Tracing, ETW, notifications etc are skipped.
2014-11-08 01:24:18.26 Server Error: 17300, Severity: 16, State: 1. (Params:). The error is printed in terse mode because there was error during formatting. Tracing, ETW, notifications etc are skipped.
2014-11-08 01:24:24.43 spid81 Error: 701, Severity: 17, State: 123.
2014-11-08 01:24:24.43 spid81 There is insufficient system memory in resource pool 'internal' to run this query.
2014-11-08 01:24:18.25 Server Error: 18052, Severity: -1, State: 0. (Params:). The error is printed in terse mode because there was error during formatting. Tracing, ETW, notifications etc are skipped.
2014-11-08 01:24:18.25 Server BRKR TASK: Operating system error Exception 0x1 encountered.
2014-11-08 01:24:30.11 Server Error: 17300, Severity: 16, State: 1. (Params:). The error is printed in terse mode because there was error during formatting. Tracing, ETW, notifications etc are skipped.
2014-11-08 01:24:30.11 Server Error: 17300, Severity: 16, State: 1. (Params:). The error is printed in terse mode because there was error during formatting. Tracing, ETW, notifications etc are skipped.
2014-11-08 01:24:35.18 spid57 Error: 701, Severity: 17, State: 131.
2014-11-08 01:24:35.18 spid57 There is insufficient system memory in resource pool 'internal' to run this query.
2014-11-08 01:24:35.18 spid71 Error: 701, Severity: 17, State: 193.
2014-11-08 01:24:35.18 spid71 There is insufficient system memory in resource pool 'internal' to run this query.
2014-11-08 01:24:35.18 Server Error: 17312, Severity: 16, State: 1. (Params:). The error is printed in terse mode because there was error during formatting. Tracing, ETW, notifications etc are skipped.
2014-11-08 01:24:35.41 Server Error: 17312, Severity: 16, State: 1.
2014-11-08 01:24:35.41 Server SQL Server is terminating a system or background task SSB Task due to errors in starting up the task (setup state 1).
2014-11-08 01:24:35.71 Server Error: 17053, Severity: 16, State: 1.
2014-11-08 01:24:35.71 Server BRKR TASK: Operating system error Exception 0x1 encountered.
2014-11-08 01:24:35.71 spid73 Error: 701, Severity: 17, State: 123.
2014-11-08 01:24:35.71 spid73 There is insufficient system memory in resource pool 'internal' to run this query.
2014-11-08 01:24:46.30 Server Error: 17312, Severity: 16, State: 1. (Params:). The error is printed in terse mode because there was error during formatting. Tracing, ETW, notifications etc are skipped.
2014-11-08 01:24:51.31 Server Error: 17053, Severity: 16, State: 1. (Params:). The error is printed in terse mode because there was error during formatting. Tracing, ETW, notifications etc are skipped.
2014-11-08 01:24:51.31 Server Error: 17300, Severity: 16, State: 1. (Params:). The error is printed in terse mode because there was error during formatting. Tracing, ETW, notifications etc are skipped.
2014-11-08 01:24:51.31 Logon Error: 18052, Severity: -1, State: 0. (Params:). The error is printed in terse mode because there was error during formatting. Tracing, ETW, notifications etc are skipped.
Last error message is half an hour after the inital Out of memory at 2014-11-08 01:52:54.03. Then the Instance is completely shut down
From the memory information in the error log we can see that all the memory is consumed by the QUERY_OPTIMIZER
Buffer Pool Value
Committed 2621440
Target 2621440
Database 130726
Dirty 3682
In IO
0
Latched
1
Free
346
Stolen 2490368
Reserved 0
Visible 2621440
Stolen Potential 0
Limiting Factor 17
Last OOM Factor 0
Last OS Error 0
Page Life Expectancy 28
2014-11-08 01:22:48.90 spid75
Process/System Counts Value
Available Physical Memory 29361627136
Available Virtual Memory 8691842715648
Available Paging File 51593969664
Working Set 628932608
Percent of Committed Memory in WS 100
Page Faults 48955000
System physical memory high 1
System physical memory low 0
Process physical memory low 1
Process virtual memory low 0
MEMORYCLERK_SQLOPTIMIZER (node 1) KB
VM Reserved 0
VM Committed 0
Locked Pages Allocated 0
SM Reserved 0
SM Committed 0
SinglePage Allocator 19419712
MultiPage Allocator 128
Memory Manager KB
VM Reserved 100960236
VM Committed 277664
Locked Pages Allocated 21483904
Reserved Memory 1024
Reserved Memory In Use 0
On the other side MDW reports that the MEMORYCLERK_SQLOPTIMIZER increases since the execution of the query up to the point of OUTOF MEMORY, but the Average value is 54.7 MB during that period as can be seen on attached graph.
We have encountered this issue already two times (every time the critical query is executed).Hi,
This does seems to me kind of memory Leak and actually it is from SQL Optimizer which leaked memory from buffer pool so much that it did not had any memory to be allocated for new page.
MEMORYCLERK_SQLOPTIMIZER (node 1) KB
VM Reserved 0
VM Committed 0
Locked Pages Allocated 0
SM Reserved 0
SM Committed 0
SinglePage Allocator 19419712
MultiPage Allocator 128
Can you post complete DBCC MEMORYSTATUS output which was generated in errorlog. Is this the only message in errorlog or there are some more messages before and after it.
select (SUM(single_pages_kb)*1024)/8192 as total_stolen_pages, type
from sys.dm_os_memory_clerks
group by typeorder by total_stolen_pages desc
and
select sum(pages_allocated_count * page_size_in_bytes)/1024,type from sys.dm_os_memory_objects
group by type
If you can post the output of above two queries with dbcc memorystaus output on some shared drive and share location with us here. I would try to find out what is leaking memory.
You can very well apply SQL Server 2008 r2 SP3 and see if this issue subsides but I am not sure whether this is fixed or actually it is a bug.
Please mark this reply as answer if it solved your issue or vote as helpful if it helped so that other forum members can benefit from it
My Technet Wiki Article
MVP -
Memory Leak During Spatial Query
Platform:
Oracle 8.1.7 EE with Spatial
Windows NT 4.0
I'm seeing a memory leak that I have determined has something to do with spatial. Here is the PL/SQL code that generates the leak:
DECLARE
v_id NUMBER(16);
v_longitude NUMBER(15,10);
v_latitude NUMBER(15,10);
v_wirecenter_name VARCHAR2(11);
CURSOR c_Source IS
SELECT id, longitude, latitude
FROM prospects_geocode_info;
CURSOR c_WC IS
SELECT wc.wirecenter_name
FROM ion_info.ion_wirecenters wc
WHERE mdsys.sdo_relate(wc.geoloc,
mdsys.sdo_geometry(2001,
8265,
mdsys.sdo_point_type(v_longitude,
v_latitude,
NULL),
NULL,
NULL),
'mask=contains querytype=window') = 'TRUE';
BEGIN
open c_Source;
loop
fetch c_Source into v_id, v_longitude, v_latitude;
exit when c_Source%NOTFOUND;
if v_longitude is not NULL and v_latitude is not NULL THEN
open c_WC;
loop
fetch c_WC into v_wirecenter_name;
exit when c_WC%NOTFOUND;
end loop;
close c_WC;
end if;
end loop;
close c_Source;
END;
I'm attributing the link to spatial because if I change the c_WC cursor's SELECT statement to not use a spatial query, I get no leak. As near as I can tell I'm leaking a little bit of memory with each spatial query. I can watch the consumed memory jump in 64K blocks about every second or two on the machine I am running on.
I have tried pinning the code in memory, as well as pinning the spatial index and it's table in memory. None of this helps. I am continuing to experiment with this, but wanted to see if anyone had any ideas.
Thanks.
Matt.I have been able to reproduce the memory leak now using a very simple anonymous PL/SQL block. All it is doing is looping through a range of lon/lat and issuing a given spatial query. Here it is:
DECLARE
v_longitude NUMBER(15,10) := 0;
v_latitude NUMBER(15,10) := 0;
v_miprinx NUMBER;
v_lon_start NUMBER(15,10) := -150.0000000000;
v_lon_end NUMBER(15,10) := -30.0000000000;
v_lat_start NUMBER(15,10) := 20.0000000000;
v_lat_end NUMBER(15,10) := 70.0000000000;
CURSOR c_test IS
SELECT mi_prinx
FROM ion_info.ion_markets
WHERE mdsys.sdo_relate(geoloc,
mdsys.sdo_geometry(2001,
8265,
mdsys.sdo_point_type(v_longitude,
v_latitude,
NULL),
NULL,
NULL),
'mask=contains querytype=window') = 'TRUE';
BEGIN
FOR v_longitude IN v_lon_start..v_lon_end LOOP
FOR v_latitude IN v_lat_start .. v_lat_end LOOP
OPEN c_test;
LOOP
FETCH c_test
INTO v_miprinx;
EXIT WHEN c_test%NOTFOUND;
END LOOP;
CLOSE c_test;
END LOOP;
END LOOP;
END;
On my 8.1.7 database on NT 4.0 I get the same results on all three tables that have spatial indexes. I tried this on a different spatial table on 8.1.6 on NT 4.0 and saw no memory leak. It doesn't matter whether I execute the script using sqlplus or sqlplusw.
Matt. -
Querying BOE security - getObjectPrincipals() has memory leak?
Hi,
I am querying the BOBJ Enterprise security and noticed that when I introduce the getObjectPrincipals() method in my code, my program consumes a great deal of additional memory, and will eventually fail due to insufficient heap space as the code iterates through the reports. The issue does seem to be tied to specifically this method, as the memory is handled fine with everything else the same. Is there a known memory leak with this method? I am currently testing against XI R2 SP2 and noticed in release notes for SP5 there was a documented memory leak with a similiar method in the COM SDK - getAnyPrincipals().
Can anyone shed any light on this? I can provide code samples too if need be...
Thanks!Thanks for the responses guys.
Ruben - I did try to use remove() for both of my iterators in use (the first of which contains all principals who have rights for the given InfoObject, and the 2nd contains all of the explicit security rights for the principal) - this unfortunately didn't have any affect on the memory usage. What I tried was as follows:
ISecurityInfo objSecurityInfo = boInfoObject.getSecurityInfo();
IObjectPrincipals objPrincipals = objSecurityInfo.getObjectPrincipals(); //THIS I BELIEVE TO HAVE MEMORY LEAK
Iterator objPrincipalIterator = objPrincipals.iterator();
//retrieve rights applied to object
while (objPrincipalIterator.hasNext())
...get security Role information for each principal
//now get explicit rights information
ISecurityRights objPrincipalExpRights = objPrincipal.getRights();
Iterator objPrincipalExpRightsIterator = objPrincipalExpRights.iterator();
while (objPrincipalExpRightsIterator.hasNext()
..process explicit rights
objPrincipalExpRightsIterator.remove();
obPrincipalIterator.remove();
Ted,
I was already batching the # of InfoObjects to retrieve security information for, but just to be sure I greatly decreased this number and did not see any improvement in memory usage. I actually worked a case with SAP on this question, and they confirmed there was a memory leak with the getObjectPrincipals() method. They confirmed this was fixed for XI R2 in SP5 for the .Net and COM SDKs, but not for Java. Is there anything else I can try? Using getObjectPrincipals(1) for only explicit rights works great as far as memory consumption, but only exposes users with custom roles and I need to retrieve principals with any type of role.
Thanks! -
Pro*c multithreaded application has memory leak
Hi there,
I posted this message a week ago in OCI section, nobody answer me.
I am really curious if my application has a bug or the pro*c has a bug.
Anyone can compile the sample code and test it easily.
I made multithreaded application which queries dynamic SQL, it works.
But the memory leaks when i query the SQL statement.
The more memory leaks, the more i query the SQL statement, even same SQL
statement.
I check it with top, shell command.
My machine is SUN E450, Solaris 8. Oracle 9.2.0.1
Compiler : gcc (GCC) 3.2.2
I changed source code which is from
$(ORACLE_HOME)/precomp/demo/proc/sample10.pc
the sample10 doesn't need to be multithreaded. But i think it has to work
correctly if i changed it to multithreaded application.
the make file and source code will be placed below.
I have to figure out the problem.
Please help
Thanks in advance,
the make file is below
HOME = /user/jkku
ORA = $(ORACLE_HOME)
CC = gcc
PROC = proc
LC_INCL = -I$(HOME)/work/dbmss/libs/include
lc_incl = include=$(HOME)/work/dbmss/libs/include
SYS_INCL =
sys_incl =
ORA_INCL = -I. \
-I$(ORA)/precomp/public \
-I$(ORA)/rdbms/public \
-I$(ORA)/rdbms/demo \
-I$(ORA)/rdbms/pbsql/public \
-I$(ORA)/network/public \
-DSLMXMX_ENABLE -DSLTS_ENABLE -D_SVID_GETTOD
INCLUDES = $(LC_INCL) $(SYS_INCL) $(ORA_INCL)
includes = $(lc_incl) $(sys_incl)
LC_LIBS =
SYS_LIBS = -lpthread -lsocket -lnsl -lrt
ORA_LIBS = -L$(ORA)/lib/ -lclntsh
LIBS = $(LC_LIBS) $(SYS_LIBS) $(ORA_LIBS)
# Define C Compiler flags
CFLAGS += -D_Solaris64_ -m64
CFLAGS += -g -D_REENTRANT
# Define pro*c Compiler flags
PROCFLAGS += THREADS=YES
PROCFLAGS += CPOOL=YES
# Our object files
PRECOMPS = sample10.c
OBJS = sample10.o
.SUFFIXES: .o .c .pc
.c.o:
$(CC) -c $(CFLAGS) $(INCLUDES) $*.c
.pc.c:
$(PROC) $(PROCFLAGS) $(includes) $*.pc $*.c
all: sample10
sample10: $(PRECOMPS) $(OBJS)
$(CC) $(CFLAGS) -o sample10 $(OBJS) $(LIBS)
clean:
rm -rf *.o sample10 sample10.c
the source code is below which i changed the oracle sample10.pc to
multithreaded application.
Sample Program 10: Dynamic SQL Method 4
This program connects you to ORACLE using your username and
password, then prompts you for a SQL statement. You can enter
any legal SQL statement. Use regular SQL syntax, not embedded SQL.
Your statement will be processed. If it is a query, the rows
fetched are displayed.
You can enter multi-line statements. The limit is 1023 characters.
This sample program only processes up to MAX_ITEMS bind variables and
MAX_ITEMS select-list items. MAX_ITEMS is #defined to be 40.
#include <stdio.h>
#include <string.h>
#include <setjmp.h>
#include <sqlda.h>
#include <stdlib.h>
#include <sqlcpr.h>
/* Maximum number of select-list items or bind variables. */
#define MAX_ITEMS 40
/* Maximum lengths of the names of the
select-list items or indicator variables. */
#define MAX_VNAME_LEN 30
#define MAX_INAME_LEN 30
#ifndef NULL
#define NULL 0
#endif
/* Prototypes */
#if defined(__STDC__)
void sql_error(void);
int oracle_connect(void);
int alloc_descriptors(int, int, int);
int get_dyn_statement(void);
void set_bind_variables(void);
void process_select_list(void);
void help(void);
#else
void sql_error(/*_ void _*/);
int oracle_connect(/*_ void _*/);
int alloc_descriptors(/*_ int, int, int _*/);
int get_dyn_statement(/* void _*/);
void set_bind_variables(/*_ void -*/);
void process_select_list(/*_ void _*/);
void help(/*_ void _*/);
#endif
char *dml_commands[] = {"SELECT", "select", "INSERT", "insert",
"UPDATE", "update", "DELETE", "delete"};
EXEC SQL INCLUDE sqlda;
EXEC SQL INCLUDE sqlca;
EXEC SQL BEGIN DECLARE SECTION;
char dyn_statement[1024];
EXEC SQL VAR dyn_statement IS STRING(1024);
EXEC SQL END DECLARE SECTION;
EXEC ORACLE OPTION (ORACA=YES);
EXEC ORACLE OPTION (RELEASE_CURSOR=YES);
SQLDA *bind_dp;
SQLDA *select_dp;
/* Define a buffer to hold longjmp state info. */
jmp_buf jmp_continue;
char *db_uid="dbmuser/dbmuser@dbmdb";
sql_context ctx;
int err_sql;
enum{
SQL_SUCC=0,
SQL_ERR,
SQL_NOTFOUND,
SQL_UNIQUE,
SQL_DISCONNECT,
SQL_NOTNULL
int main()
int i;
EXEC SQL ENABLE THREADS;
EXEC SQL WHENEVER SQLERROR DO sql_error();
EXEC SQL WHENEVER NOT FOUND DO sql_not_found();
/* Connect to the database. */
if (connect_database() < 0)
exit(1);
EXEC SQL CONTEXT USE :ctx;
/* Process SQL statements. */
for (;;)
/* Allocate memory for the select and bind descriptors. */
if (alloc_descriptors(MAX_ITEMS, MAX_VNAME_LEN, NAME_LEN) != 0)
exit(1);
(void) setjmp(jmp_continue);
/* Get the statement. Break on "exit". */
if (get_dyn_statement() != 0)
break;
EXEC SQL PREPARE S FROM :dyn_statement;
EXEC SQL DECLARE C CURSOR FOR S;
/* Set the bind variables for any placeholders in the
SQL statement. */
set_bind_variables();
/* Open the cursor and execute the statement.
* If the statement is not a query (SELECT), the
* statement processing is completed after the
* OPEN.
EXEC SQL OPEN C USING DESCRIPTOR bind_dp;
/* Call the function that processes the select-list.
* If the statement is not a query, this function
* just returns, doing nothing.
process_select_list();
/* Tell user how many rows processed. */
for (i = 0; i < 8; i++)
if (strncmp(dyn_statement, dml_commands, 6) == 0)
printf("\n\n%d row%c processed.\n", sqlca.sqlerrd[2], sqlca.sqlerrd[2] == 1 ? '\0' : 's');
break;
/* Close the cursor. */
EXEC SQL CLOSE C;
/* When done, free the memory allocated for pointers in the bind and
select descriptors. */
for (i = 0; i < MAX_ITEMS; i++)
if (bind_dp->V != (char *) 0)
free(bind_dp->V);
free(bind_dp->I); /* MAX_ITEMS were allocated. */
if (select_dp->V != (char *) 0)
free(select_dp->V);
free(select_dp->I); /* MAX_ITEMS were allocated. */
/* Free space used by the descriptors themselves. */
SQLSQLDAFree(ctx, bind_dp);
SQLSQLDAFree(ctx, select_dp);
} /* end of for(;;) statement-processing loop */
disconnect_database();
EXEC SQL WHENEVER SQLERROR CONTINUE;
EXEC SQL COMMIT WORK RELEASE;
puts("\nHave a good day!\n");
return;
* Allocate the BIND and SELECT descriptors using sqlald().
* Also allocate the pointers to indicator variables
* in each descriptor. The pointers to the actual bind
* variables and the select-list items are realloc'ed in
* the set_bind_variables() or process_select_list()
* routines. This routine allocates 1 byte for select_dp->V
* and bind_dp->V, so the realloc will work correctly.
alloc_descriptors(size, max_vname_len, max_iname_len)
int size;
int max_vname_len;
int max_iname_len;
int i;
* The first sqlald parameter determines the maximum number of
* array elements in each variable in the descriptor. In
* other words, it determines the maximum number of bind
* variables or select-list items in the SQL statement.
* The second parameter determines the maximum length of
* strings used to hold the names of select-list items
* or placeholders. The maximum length of column
* names in ORACLE is 30, but you can allocate more or less
* as needed.
* The third parameter determines the maximum length of
* strings used to hold the names of any indicator
* variables. To follow ORACLE standards, the maximum
* length of these should be 30. But, you can allocate
* more or less as needed.
if ((bind_dp =
SQLSQLDAAlloc(ctx, size, max_vname_len, max_iname_len)) ==
(SQLDA *) 0)
fprintf(stderr,
"Cannot allocate memory for bind descriptor.");
return -1; /* Have to exit in this case. */
if ((select_dp =
SQLSQLDAAlloc(ctx, size, max_vname_len, max_iname_len)) == (SQLDA *)
0)
fprintf(stderr,
"Cannot allocate memory for select descriptor.");
return -1;
select_dp->N = MAX_ITEMS;
/* Allocate the pointers to the indicator variables, and the
actual data. */
for (i = 0; i < MAX_ITEMS; i++) {
bind_dp->I = (short *) malloc(sizeof (short));
select_dp->I = (short *) malloc(sizeof(short));
bind_dp->V = (char *) malloc(1);
select_dp->V = (char *) malloc(1);
return 0;
int get_dyn_statement()
char *cp, linebuf[256];
int iter, plsql;
for (plsql = 0, iter = 1; ;)
if (iter == 1)
printf("\nSQL> ");
dyn_statement[0] = '\0';
fgets(linebuf, sizeof linebuf, stdin);
cp = strrchr(linebuf, '\n');
if (cp && cp != linebuf)
*cp = ' ';
else if (cp == linebuf)
continue;
if ((strncmp(linebuf, "EXIT", 4) == 0) ||
(strncmp(linebuf, "exit", 4) == 0))
return -1;
else if (linebuf[0] == '?' ||
(strncmp(linebuf, "HELP", 4) == 0) ||
(strncmp(linebuf, "help", 4) == 0))
help();
iter = 1;
continue;
if (strstr(linebuf, "BEGIN") ||
(strstr(linebuf, "begin")))
plsql = 1;
strcat(dyn_statement, linebuf);
if ((plsql && (cp = strrchr(dyn_statement, '/'))) ||
(!plsql && (cp = strrchr(dyn_statement, ';'))))
*cp = '\0';
break;
else
iter++;
printf("%3d ", iter);
return 0;
void set_bind_variables()
int i, n;
char bind_var[64];
/* Describe any bind variables (input host variables) */
EXEC SQL WHENEVER SQLERROR DO sql_error();
bind_dp->N = MAX_ITEMS; /* Initialize count of array elements. */
EXEC SQL DESCRIBE BIND VARIABLES FOR S INTO bind_dp;
/* If F is negative, there were more bind variables
than originally allocated by sqlald(). */
if (bind_dp->F < 0)
printf ("\nToo many bind variables (%d), maximum is %d\n.",
-bind_dp->F, MAX_ITEMS);
return;
/* Set the maximum number of array elements in the
descriptor to the number found. */
bind_dp->N = bind_dp->F;
/* Get the value of each bind variable as a
* character string.
* C contains the length of the bind variable
* name used in the SQL statement.
* S contains the actual name of the bind variable
* used in the SQL statement.
* L will contain the length of the data value
* entered.
* V will contain the address of the data value
* entered.
* T is always set to 1 because in this sample program
* data values for all bind variables are entered
* as character strings.
* ORACLE converts to the table value from CHAR.
* I will point to the indicator value, which is
* set to -1 when the bind variable value is "null".
for (i = 0; i < bind_dp->F; i++)
printf ("\nEnter value for bind variable %.*s: ",
(int)bind_dp->C, bind_dp->S);
fgets(bind_var, sizeof bind_var, stdin);
/* Get length and remove the new line character. */
n = strlen(bind_var) - 1;
/* Set it in the descriptor. */
bind_dp->L = n;
/* (re-)allocate the buffer for the value.
sqlald() reserves a pointer location for
V but does not allocate the full space for
the pointer. */
bind_dp->V = (char *) realloc(bind_dp->V, (bind_dp->L + 1));
/* And copy it in. */
strncpy(bind_dp->V, bind_var, n);
/* Set the indicator variable's value. */
if ((strncmp(bind_dp->V, "NULL", 4) == 0) ||
(strncmp(bind_dp->V, "null", 4) == 0))
*bind_dp->I = -1;
else
*bind_dp->I = 0;
/* Set the bind datatype to 1 for CHAR. */
bind_dp->T = 1;
return;
void process_select_list()
int i, null_ok, precision, scale;
if ((strncmp(dyn_statement, "SELECT", 6) != 0) &&
(strncmp(dyn_statement, "select", 6) != 0))
select_dp->F = 0;
return;
/* If the SQL statement is a SELECT, describe the
select-list items. The DESCRIBE function returns
their names, datatypes, lengths (including precision
and scale), and NULL/NOT NULL statuses. */
select_dp->N = MAX_ITEMS;
EXEC SQL DESCRIBE SELECT LIST FOR S INTO select_dp;
/* If F is negative, there were more select-list
items than originally allocated by sqlald(). */
if (select_dp->F < 0)
printf ("\nToo many select-list items (%d), maximum is %d\n",
-(select_dp->F), MAX_ITEMS);
return;
/* Set the maximum number of array elements in the
descriptor to the number found. */
select_dp->N = select_dp->F;
/* Allocate storage for each select-list item.
sqlprc() is used to extract precision and scale
from the length (select_dp->L).
sqlnul() is used to reset the high-order bit of
the datatype and to check whether the column
is NOT NULL.
CHAR datatypes have length, but zero precision and
scale. The length is defined at CREATE time.
NUMBER datatypes have precision and scale only if
defined at CREATE time. If the column
definition was just NUMBER, the precision
and scale are zero, and you must allocate
the required maximum length.
DATE datatypes return a length of 7 if the default
format is used. This should be increased to
9 to store the actual date character string.
If you use the TO_CHAR function, the maximum
length could be 75, but will probably be less
(you can see the effects of this in SQL*Plus).
ROWID datatype always returns a fixed length of 18 if
coerced to CHAR.
LONG and
LONG RAW datatypes return a length of 0 (zero),
so you need to set a maximum. In this example,
it is 240 characters.
printf ("\n");
for (i = 0; i < select_dp->F; i++)
char title[MAX_VNAME_LEN];
/* Turn off high-order bit of datatype (in this example,
it does not matter if the column is NOT NULL). */
sqlnul ((unsigned short *)&(select_dp->T), (unsigned short
*)&(select_dp->T), &null_ok);
switch (select_dp->T)
case 1 : /* CHAR datatype: no change in length
needed, except possibly for TO_CHAR
conversions (not handled here). */
break;
case 2 : /* NUMBER datatype: use sqlprc() to
extract precision and scale. */
sqlprc ((unsigned int *)&(select_dp->L), &precision,
&scale);
/* Allow for maximum size of NUMBER. */
if (precision == 0) precision = 40;
/* Also allow for decimal point and
possible sign. */
/* convert NUMBER datatype to FLOAT if scale > 0,
INT otherwise. */
if (scale > 0)
select_dp->L = sizeof(float);
else
select_dp->L = sizeof(int);
break;
case 8 : /* LONG datatype */
select_dp->L = 240;
break;
case 11 : /* ROWID datatype */
case 104 : /* Universal ROWID datatype */
select_dp->L = 18;
break;
case 12 : /* DATE datatype */
select_dp->L = 9;
break;
case 23 : /* RAW datatype */
break;
case 24 : /* LONG RAW datatype */
select_dp->L = 240;
break;
/* Allocate space for the select-list data values.
sqlald() reserves a pointer location for
V but does not allocate the full space for
the pointer. */
if (select_dp->T != 2)
select_dp->V = (char *) realloc(select_dp->V,
select_dp->L + 1);
else
select_dp->V = (char *) realloc(select_dp->V,
select_dp->L);
/* Print column headings, right-justifying number
column headings. */
/* Copy to temporary buffer in case name is null-terminated */
memset(title, ' ', MAX_VNAME_LEN);
strncpy(title, select_dp->S, select_dp->C);
if (select_dp->T == 2)
if (scale > 0)
printf ("%.*s ", select_dp->L+3, title);
else
printf ("%.*s ", select_dp->L, title);
else
printf("%-.*s ", select_dp->L, title);
/* Coerce ALL datatypes except for LONG RAW and NUMBER to
character. */
if (select_dp->T != 24 && select_dp->T != 2)
select_dp->T = 1;
/* Coerce the datatypes of NUMBERs to float or int depending on
the scale. */
if (select_dp->T == 2)
if (scale > 0)
select_dp->T = 4; /* float */
else
select_dp->T = 3; /* int */
printf ("\n\n");
/* FETCH each row selected and print the column values. */
EXEC SQL WHENEVER NOT FOUND GOTO end_select_loop;
for (;;)
EXEC SQL FETCH C USING DESCRIPTOR select_dp;
/* Since each variable returned has been coerced to a
character string, int, or float very little processing
is required here. This routine just prints out the
values on the terminal. */
for (i = 0; i < select_dp->F; i++)
if (*select_dp->I < 0)
if (select_dp->T == 4)
printf ("%-*c ",(int)select_dp->L+3, ' ');
else
printf ("%-*c ",(int)select_dp->L, ' ');
else
if (select_dp->T == 3) /* int datatype */
printf ("%*d ", (int)select_dp->L,
*(int *)select_dp->V);
else if (select_dp->T == 4) /* float datatype */
printf ("%*.2f ", (int)select_dp->L,
*(float *)select_dp->V);
else /* character string */
printf ("%-*.*s ", (int)select_dp->L,
(int)select_dp->L, select_dp->V);
printf ("\n");
end_select_loop:
return;
void help()
puts("\n\nEnter a SQL statement or a PL/SQL block at the SQL> prompt.");
puts("Statements can be continued over several lines, except");
puts("within string literals.");
puts("Terminate a SQL statement with a semicolon.");
puts("Terminate a PL/SQL block (which can contain embedded
semicolons)");
puts("with a slash (/).");
puts("Typing \"exit\" (no semicolon needed) exits the program.");
puts("You typed \"?\" or \"help\" to get this message.\n\n");
int connect_database()
err_sql = SQL_SUCC;
EXEC SQL WHENEVER SQLERROR DO sql_error();
EXEC SQL WHENEVER NOT FOUND DO sql_not_found();
EXEC SQL CONTEXT ALLOCATE :ctx;
EXEC SQL CONTEXT USE :ctx;
EXEC SQL CONNECT :db_uid;
if(err_sql != SQL_SUCC){
printf("err => connect database(ctx:%ld, uid:%s) failed!\n", ctx, db_uid);
return -1;
return 1;
int disconnect_database()
err_sql = SQL_SUCC;
EXEC SQL WHENEVER SQLERROR DO sql_error();
EXEC SQL WHENEVER NOT FOUND DO sql_not_found();
EXEC SQL CONTEXT USE :ctx;
EXEC SQL COMMIT WORK RELEASE;
EXEC SQL CONTEXT FREE:ctx;
return 1;
void sql_error()
printf("err => %.*s", sqlca.sqlerrm.sqlerrml, sqlca.sqlerrm.sqlerrmc);
printf("in \"%.*s...\'\n", oraca.orastxt.orastxtl, oraca.orastxt.orastxtc);
printf("on line %d of %.*s.\n\n", oraca.oraslnr, oraca.orasfnm.orasfnml,
oraca.orasfnm.orasfnmc);
switch(sqlca.sqlcode) {
case -1: /* unique constraint violated */
err_sql = SQL_UNIQUE;
break;
case -1012: /* not logged on */
case -1089:
case -3133:
case -1041:
case -3114:
case -3113:
/* �6�Ŭ�� shutdown�ǰų� �α��� ���°� �ƴҶ� ��b�� �õ� */
/* immediate shutdown in progress - no operations are permitted */
/* end-of-file on communication channel */
/* internal error. hostdef extension doesn't exist */
err_sql = SQL_DISCONNECT;
break;
case -1400:
err_sql = SQL_NOTNULL;
break;
default:
err_sql = SQL_ERR;
break;
EXEC SQL CONTEXT USE :ctx;
EXEC SQL WHENEVER SQLERROR CONTINUE;
EXEC SQL ROLLBACK WORK;
void sql_not_found()
err_sql = SQL_NOTFOUND;Hi Jane,
What version of Berkeley DB XML are you using?
What is your operating system and your hardware platform?
For how long have been the application running?
What is your current container size?
What's set for EnvironmentConfig.setThreaded?
Do you know if containers have previously not been closed correctly?
Can you please post the entire error output?
What's the JDK version, 1.4 or 1.5?
Thanks,
Bogdan -
Hello guys,
I'm having a very, very difficult time debugging this memory leak on my server. I'm pretty sure it's related to berkeley DB XML, bu I just couldn't prove this in numbers.
My server: Debian GNU/Linux 2.4.27-2-386 #1 Wed Aug 17 09:33:35 UTC 2005 i686 GNU/Linux, 512 MB of RAM, Pentium 4-class processor.
Here's the behaviour. I start Tomcat 5.5.15, the application is loaded, current memory status:
delegaciainterativa:~# free -m
total used free shared buffers cached
Mem: 440 346 94 0 14 282
-/+ buffers/cache: 49 391
Swap: 909 4 905
So, I've got 94 MBs of free memory. Now, here's this JSP I'm running:
<%@ page language="java" %>
<%@ page session="true" %>
<%@ page import="br.gov.al.delegaciainterativa.admin.*"%>
<%@ page import="br.gov.al.delegaciainterativa.controles.*"%>
<%@ page import="br.gov.al.delegaciainterativa.admin.areas.*"%>
<%@ page import="br.gov.al.delegaciainterativa.controles.BdbXmlAmbiente"%>
<%@ page import="java.util.*"%>
<%@ page import="javax.servlet.RequestDispatcher"%>
<%@ page import="com.sleepycat.dbxml.XmlContainer"%>
<%@ page import="com.sleepycat.dbxml.XmlDocument"%>
<%@ page import="com.sleepycat.dbxml.XmlException"%>
<%@ page import="com.sleepycat.dbxml.XmlIndexDeclaration"%>
<%@ page import="com.sleepycat.dbxml.XmlIndexSpecification"%>
<%@ page import="com.sleepycat.dbxml.XmlQueryContext"%>
<%@ page import="com.sleepycat.dbxml.XmlQueryExpression"%>
<%@ page import="com.sleepycat.dbxml.XmlResults"%>
<%@ page import="com.sleepycat.dbxml.XmlTransaction"%>
<%@ page import="com.sleepycat.dbxml.XmlUpdateContext"%>
<%@ page import="com.sleepycat.dbxml.XmlValue"%>
<%@ page import="com.sleepycat.dbxml.XmlUpdateContext"%>
<%@ page import="com.sleepycat.dbxml.XmlQueryContext"%>
<%@ include file="config.jsp"%>
<%!
final String _grupoPagina = "visualizaLogsUsuario";
%>
<font face="verdana" size="1">
<%
HttpSession sessao = request.getSession(true);
BdbXmlAmbiente bdbxmlAmbiente = new BdbXmlAmbiente();
XmlContainer container = bdbxmlAmbiente.abrirContainer(Config.getNomeContainerBO());
String fullquery = "";
fullquery = "for $i in collection('"+ Config.getNomeContainerBO() +"')/BO \n";
fullquery += "where $i/cadastro/fim/data/date >= '2006-00-00' \n";
fullquery += "return $i";
ArrayList docs = new ArrayList();
XmlResults results = null;
XmlValue value = null;
XmlDocument document = null;
try {
XmlQueryContext context = bdbxmlAmbiente.getEnvironmentInit().getManager().createQueryContext();
context.setEvaluationType(XmlQueryContext.Eager);
XmlQueryExpression xqe = bdbxmlAmbiente.getEnvironmentInit().getManager().prepare(fullquery, context);
results = xqe.execute(context);
out.println("<p>fez busca, deletando...</p>");
while (results.hasNext()) {
value = results.next();
document = value.asDocument();
document.fetchAllData();
out.println(document.getName() + "<br>");
//out.println("<p>total: " + results.size() + "</p>");
} catch (Throwable e) {
out.println(e);
e.printStackTrace();
} finally {
results.delete();
%>
This basically loads about 1600 small documents (~5KB) from the container, and guess how memory is now:
delegaciainterativa:~# free -m
total used free shared buffers cached
Mem: 440 429 11 0 14 282
-/+ buffers/cache: 132 308
Swap: 909 4 905
That's about 83Mbs consumed!! And if I perform other searches like this, this keeps eating up more and more memory, then swapping and getting slow... and sometimes crashing Tomcat.
Here's the big mistery: the memory NEVER GETS RELEASED. Even calling delete(), as I did explicitily, it doesnt release the memory. It keeps increasing as searches are performed.
Now, let's go to the profiler. I've installed JProfiler and run it on this server, and here're the results: http://freeunix.com.br/All_Objects.html
Even more puzzling, I havent got any impressive amount of memory consumption.
So, what can I do to figure this out? The only way to release memory is to restart the server once and a while, which far from ideal.
I'd really would appreciate any help about this, since I'm on production with this server.
cheers,
-- Breno JacintoHello George,
I tried this:
<%
HttpSession sessao = request.getSession(true);
BdbXmlAmbiente bdbxmlAmbiente = new BdbXmlAmbiente();
XmlContainer container = bdbxmlAmbiente.abrirContainer(Config.getNomeContainerBO());
String fullquery = "";
fullquery = "for $i in collection('"+ Config.getNomeContainerBO() +"')/BO \n";
fullquery += "where $i/cadastro/fim/data/date >= '2006-00-00' \n";
fullquery += "return $i";
ArrayList docs = new ArrayList();
XmlResults results = null;
XmlValue value = null;
XmlDocument document = null;
try {
XmlQueryContext context = bdbxmlAmbiente.getEnvironmentInit().getManager().createQueryContext();
context.setEvaluationType(XmlQueryContext.Eager);
XmlQueryExpression xqe = bdbxmlAmbiente.getEnvironmentInit().getManager().prepare(fullquery, context);
results = xqe.execute(context);
out.println("<p>fez busca, deletando...</p>");
while (results.hasNext()) {
value = results.next();
document = value.asDocument();
document.fetchAllData();
out.println(document.getName() + "<br>");
value.delete();
document.delete();
//out.println("<p>total: " + results.size() + "</p>");
context.delete();
xqe.delete();
results.delete();
} catch (Throwable e) {
out.println(e);
e.printStackTrace();
%>
Don't you think that it's too much to consume ~80 MBs of RAM, in a search that returns ~1600 5 KB documents? And worse, this amount is never released.
I'm wondering if there's any configuration issue here? Such as JVM's Heap size, or BDBXML Environment Initialization?
thanks,
-- Breno -
Bug:4705928 PLSQL: Memory leak using small varrays
We have Oracle version 10.2.0.1.0
We have a problem with a procedure.
In our scenario we make use of VARRAY in the procedure to pass some filter parameters to a select distinct querying a view made on three tables.
Unfotunately not always execution it is successful.
Sometimes it returns wrong value (0 for the count parameter), sometimes (rarely) the server stops working.
We suspect that this is caused by a bug fixed in versione 10.2.0.3.0
Bug:4705928 PLSQL: Memory leak using small varrays when trimming the whole collection and inserting into it in a loop
We suspect this becasue we made two procedure the first (spProductCount) uses a function (fnProductFilter) to calculate the values of a varray and passes them into the select,
while in the second procedure (spProductCount2) parameters are passed directly into the statement without varray
and there are failures only in the first procedure.
On the other hand on another server 10.2.0.1.0 we never have this problem.
The instance manifesting the bug runs under shared mode, while the other is under dedicated mode.
Turning the first one to dedicated mode makes the bugs disapear.
Unfortunately this is not a solution.
In the sample there are the three table with all constraints, the view, tha varray custom type, the function and the two procedures.
Is there someone that may examine our sample and tell us if the pl/sql code corresponds to the bug desciption.
We also want to know if it's possibile that the same server running under different mode (SHARED/DEDICATED) doesn't behave the same way.
The tables:
--Products
CREATE TABLE "Products" (
"Image" BLOB
, "CatalogId" RAW(16)
, "ProductId" RAW(16)
, "MnemonicId" NVARCHAR2(50) DEFAULT ''
, "ProductParentId" RAW(16)
ALTER TABLE "Products"
ADD CONSTRAINT "NN_Products_M04" CHECK ("CatalogId" IS NOT NULL)
ALTER TABLE "Products"
ADD CONSTRAINT "NN_Products_M05" CHECK ("ProductId" IS NOT NULL)
ALTER TABLE "Products"
ADD CONSTRAINT "PK_Products"
PRIMARY KEY ("ProductId")
CREATE INDEX "IX_Products"
ON "Products" ("CatalogId", "MnemonicId")
CREATE UNIQUE INDEX "UK_Products"
ON "Products" (DECODE("MnemonicId", NULL, NULL, RAWTOHEX("CatalogId") || "MnemonicId"))
--Languages
CREATE TABLE "Languages" (
"Description" NVARCHAR2(250)
, "IsStandard" NUMBER(1)
, "LanguageId" RAW(16)
, "MnemonicId" NVARCHAR2(12)
ALTER TABLE "Languages"
ADD CONSTRAINT "NN_Languages_M01" CHECK ("LanguageId" IS NOT NULL)
ALTER TABLE "Languages"
ADD CONSTRAINT "NN_Languages_M05" CHECK ("MnemonicId" IS NOT NULL)
ALTER TABLE "Languages"
ADD CONSTRAINT "PK_Languages"
PRIMARY KEY ("LanguageId")
ALTER TABLE "Languages"
ADD CONSTRAINT "UK_Languages"
UNIQUE ("MnemonicId")
--ProductDesc
CREATE TABLE "ProductDesc" (
"Comment" NCLOB
, "PlainComment" NCLOB
, "Description" NVARCHAR2(250)
, "DescriptionText" NCLOB
, "PlainDescriptionText" NCLOB
, "LanguageId" NVARCHAR2(12)
, "ProductId" RAW(16)
ALTER TABLE "ProductDesc"
ADD CONSTRAINT "NN_ProductDescM01" CHECK ("LanguageId" IS NOT NULL)
ALTER TABLE "ProductDesc"
ADD CONSTRAINT "NN_ProductDescM02" CHECK ("ProductId" IS NOT NULL)
ALTER TABLE "ProductDesc"
ADD CONSTRAINT "PK_ProductDesc"
PRIMARY KEY ("ProductId", "LanguageId")
ALTER TABLE "ProductDesc"
ADD CONSTRAINT "FK_ProductDesc1"
FOREIGN KEY("ProductId") REFERENCES "Products" ("ProductId")
ALTER TABLE "ProductDesc"
ADD CONSTRAINT "FK_ProductDesc2"
FOREIGN KEY("LanguageId") REFERENCES "Languages" ("MnemonicId")
/The view:
--ProductView
CREATE OR REPLACE VIEW "vwProducts"
AS
SELECT
"Products"."CatalogId"
, "ProductDesc"."Comment"
, "ProductDesc"."PlainComment"
, "ProductDesc"."Description"
, "ProductDesc"."DescriptionText"
, "ProductDesc"."PlainDescriptionText"
, "Products"."Image"
, "Languages"."MnemonicId" "LanguageId"
, "Products"."MnemonicId"
, "Products"."ProductId"
, "Products"."ProductParentId"
, TRIM(NVL("ProductDesc"."Description" || ' ', '') || NVL("ParentDescriptions"."Description", '')) "FullDescription"
FROM "Products"
CROSS JOIN "Languages"
LEFT OUTER JOIN "ProductDesc"
ON "Products"."ProductId" = "ProductDesc"."ProductId"
AND "ProductDesc"."LanguageId" = "Languages"."MnemonicId"
LEFT OUTER JOIN "ProductDesc" "ParentDescriptions"
ON "Products"."ProductParentId" = "ParentDescriptions"."ProductId"
AND ("ParentDescriptions"."LanguageId" = "Languages"."MnemonicId")
/The varray:
--CustomType VARRAY
CREATE OR REPLACE TYPE Varray_Params IS VARRAY(100) OF NVARCHAR2(1000);
/The function:
--FilterFunction
CREATE OR REPLACE FUNCTION "fnProductFilter" (
parCatalogId "Products"."CatalogId"%TYPE,
parLanguageId NVARCHAR2 := N'it-IT',
parFilterValues OUT Varray_Params
RETURN INTEGER
AS
varSqlCondition VARCHAR2(32000);
varSqlConditionValues NVARCHAR2(32000);
varSql NVARCHAR2(32000);
varDbmsCursor INTEGER;
varDbmsResult INTEGER;
varSeparator VARCHAR2(2);
varFilterValue NVARCHAR2(1000);
varCount INTEGER;
BEGIN
varSqlCondition := '(T_Product."CatalogId" = HEXTORAW(:parentId)) AND (T_Product."LanguageId" = :languageId )';
varSqlConditionValues := CHR(39) || TO_CHAR(parCatalogId) || CHR(39) || N', ' || CHR(39 USING NCHAR_CS) || parLanguageId || CHR(39 USING NCHAR_CS);
parFilterValues := Varray_Params();
varSql := N'SELECT FilterValues.column_value FilterValue FROM TABLE(Varray_Params(' || varSqlConditionValues || N')) FilterValues';
BEGIN
varDbmsCursor := dbms_sql.open_cursor;
dbms_sql.parse(varDbmsCursor, varSql, dbms_sql.native);
dbms_sql.define_column(varDbmsCursor, 1, varFilterValue, 1000);
varDbmsResult := dbms_sql.execute(varDbmsCursor);
varCount := 0;
LOOP
IF (dbms_sql.fetch_rows(varDbmsCursor) > 0) THEN
varCount := varCount + 1;
dbms_sql.column_value(varDbmsCursor, 1, varFilterValue);
parFilterValues.extend(1);
parFilterValues(varCount) := varFilterValue;
ELSE
-- No more rows to copy
EXIT;
END IF;
END LOOP;
dbms_sql.close_cursor(varDbmsCursor);
EXCEPTION WHEN OTHERS THEN
dbms_sql.close_cursor(varDbmsCursor);
RETURN 0;
END;
FOR i in parFilterValues.first .. parFilterValues.last LOOP
varSeparator := ', ';
END LOOP;
RETURN 1;
END;
/The procedures:
--Procedure presenting anomaly\bug
CREATE OR REPLACE PROCEDURE "spProductCount" (
parCatalogId "Products"."CatalogId"%TYPE,
parLanguageId NVARCHAR2 := N'it-IT',
parRecords OUT NUMBER
AS
varFilterValues Varray_Params;
varResult INTEGER;
varSqlTotal VARCHAR2(32000);
BEGIN
parRecords := 0;
varResult := "fnProductFilter"(parCatalogId, parLanguageId, varFilterValues);
varSqlTotal := 'BEGIN
SELECT count(DISTINCT T_Product."ProductId") INTO :parCount FROM "vwProducts" T_Product
WHERE ((T_Product."CatalogId" = HEXTORAW(:parentId)) AND (T_Product."LanguageId" = :languageId ));
END;';
EXECUTE IMMEDIATE varSqlTotal USING OUT parRecords, varFilterValues(1), varFilterValues(2);
END;
--Procedure NOT presenting anomaly\bug
CREATE OR REPLACE PROCEDURE "spProductCount2" (
parCatalogId "Products"."CatalogId"%TYPE,
parLanguageId NVARCHAR2 := N'it-IT',
parRecords OUT NUMBER
AS
varFilterValues Varray_Params;
varResult INTEGER;
varSqlTotal VARCHAR2(32000);
BEGIN
parRecords := 0;
varSqlTotal := 'BEGIN
SELECT count(DISTINCT T_Product."ProductId") INTO :parCount FROM "vwProducts" T_Product
WHERE ((T_Product."CatalogId" = HEXTORAW(:parentId)) AND (T_Product."LanguageId" = :languageId ));
END;';
EXECUTE IMMEDIATE varSqlTotal USING OUT parRecords, parCatalogId, parLanguageId;
END;Edited by: 835125 on 2011-2-9 1:31835125 wrote:
Using VARRAY was the only way I found to transform comma seprated text values (e.g. "'abc', 'def', '123'") in a collection of strings.A varray is just a functionally crippled version of a nested table collection type, with a defined limit you probably don't need. (Why 100 specifically?) Instead of
CREATE OR REPLACE TYPE varray_params AS VARRAY(100) OF NVARCHAR2(1000);try
CREATE OR REPLACE TYPE array_params AS TABLE OF NVARCHAR2(1000);I don't know whether that will solve the problem but at least it'll be a slightly more useful type.
What makes you think it's a memory leak specifically? Do you observe session PGA memory use going up more than it should?
btw good luck with all those quoted column names. I wouldn't like to have to work with those, although they do make the forum more colourful.
Edited by: William Robertson on Feb 11, 2011 7:54 AM -
Memory leak in weblogic 6.0 sp2 oracle 8.1.7 thin driver
Hi,
I have a simple client that opens a database connection, selects from
a table containing five rows of data (with four columns in each row)
and then closes all connections. On running this in a loop, I get the
following error after some time:
<Nov 28, 2001 5:57:40 PM GMT+06:00> <Error> <Adapter>
<OutOfMemoryError in
Adapter
java.lang.OutOfMemoryError
<<no stack trace available>>
>
<Nov 28, 2001 5:57:40 PM GMT+06:00> <Error> <Kernel> <ExecuteRequest
failed
java.lang.OutOfMemoryError
I am running with a heap size of 64 Mb. The java command that runs
the client is:
java -ms64m -mx64m -cp .:/opt/bea/wlserver6.0/lib/weblogic.jar
-Djava.naming.f
actory.initial=weblogic.jndi.WLInitialContextFactory
-Djava.naming.provider.url=
t3://garlic:7001 -verbose:gc Test
The following is the client code that opens the db connection and does
the select:
import java.util.*;
import java.sql.*;
import javax.naming.*;
import javax.sql.*;
public class Test {
private static final String strQuery = "SELECT * from tblPromotion";
public static void main(String argv[])
throws Exception
String ctxFactory = System.getProperty
("java.naming.factory.initial");
String providerUrl = System.getProperty
("java.naming.provider.url");
Properties jndiEnv = System.getProperties ();
System.out.println ("ctxFactory : " + ctxFactory);
System.out.println ("ProviderURL : " + providerUrl);
Context ctx = new InitialContext (jndiEnv);
for (int i=0; i <1000000; i++)
System.out.println("Running query for the "+i+" time");
Connection con = null;
Statement stmnt = null;
ResultSet rs = null;
try
DataSource ds = (DataSource) ctx.lookup
(System.getProperty("eaMDataStore", "jdbc/eaMarket"));
con = ds.getConnection ();
stmnt = con.createStatement();
rs = stmnt.executeQuery(strQuery);
while (rs.next ())
//System.out.print(".");
//System.out.println(".");
ds = null;
catch (java.sql.SQLException sqle)
System.out.println("SQL Exception : "+sqle.getMessage());
finally
try {
rs.close ();
rs = null;
//System.out.println("closed result set");
} catch (Exception e) {
System.out.println("Exception closing result set");
try {
stmnt.close ();
stmnt = null;
//System.out.println("closed statement");
} catch (Exception e) {
System.out.println("Exception closing result set");
try {
con.close();
con = null;
//System.out.println("closed connection");
} catch (Exception e) {
System.out.println("Exception closing connection");
I am using the Oracle 8.1.7 thin driver. Please let me know if this
memory leak is a known issue or if its something I am doing.
thanks,
rudyRepost in JDBC section ... very serious issue but it may be due to Oracle or
to WL ... does it happen if you test inside WL itself?
How many iterations does it take to blow? How long? Does changing to a
different driver (maybe Cloudscape) have the same result?
Peace,
Cameron Purdy
Tangosol Inc.
<< Tangosol Server: How Weblogic applications are customized >>
<< Download now from http://www.tangosol.com/download.jsp >>
"R.C." <[email protected]> wrote in message
news:[email protected]...
Hi,
I have a simple client that opens a database connection, selects from
a table containing five rows of data (with four columns in each row)
and then closes all connections. On running this in a loop, I get the
following error after some time:
<Nov 28, 2001 5:57:40 PM GMT+06:00> <Error> <Adapter>
<OutOfMemoryError in
Adapter
java.lang.OutOfMemoryError
<<no stack trace available>>
>
<Nov 28, 2001 5:57:40 PM GMT+06:00> <Error> <Kernel> <ExecuteRequest
failed
java.lang.OutOfMemoryError
I am running with a heap size of 64 Mb. The java command that runs
the client is:
java -ms64m -mx64m -cp .:/opt/bea/wlserver6.0/lib/weblogic.jar
-Djava.naming.f
actory.initial=weblogic.jndi.WLInitialContextFactory
-Djava.naming.provider.url=
t3://garlic:7001 -verbose:gc Test
The following is the client code that opens the db connection and does
the select:
import java.util.*;
import java.sql.*;
import javax.naming.*;
import javax.sql.*;
public class Test {
private static final String strQuery = "SELECT * from tblPromotion";
public static void main(String argv[])
throws Exception
String ctxFactory = System.getProperty
("java.naming.factory.initial");
String providerUrl = System.getProperty
("java.naming.provider.url");
Properties jndiEnv = System.getProperties ();
System.out.println ("ctxFactory : " + ctxFactory);
System.out.println ("ProviderURL : " + providerUrl);
Context ctx = new InitialContext (jndiEnv);
for (int i=0; i <1000000; i++)
System.out.println("Running query for the "+i+" time");
Connection con = null;
Statement stmnt = null;
ResultSet rs = null;
try
DataSource ds = (DataSource) ctx.lookup
(System.getProperty("eaMDataStore", "jdbc/eaMarket"));
con = ds.getConnection ();
stmnt = con.createStatement();
rs = stmnt.executeQuery(strQuery);
while (rs.next ())
//System.out.print(".");
//System.out.println(".");
ds = null;
catch (java.sql.SQLException sqle)
System.out.println("SQL Exception : "+sqle.getMessage());
finally
try {
rs.close ();
rs = null;
//System.out.println("closed result set");
} catch (Exception e) {
System.out.println("Exception closing result set");
try {
stmnt.close ();
stmnt = null;
//System.out.println("closed statement");
} catch (Exception e) {
System.out.println("Exception closing result set");
try {
con.close();
con = null;
//System.out.println("closed connection");
} catch (Exception e) {
System.out.println("Exception closing connection");
I am using the Oracle 8.1.7 thin driver. Please let me know if this
memory leak is a known issue or if its something I am doing.
thanks,
rudy -
Hi All,
I had posted this query to couple of other groups ..I am still struglling with this issue..Hope someone will help me out here
I have very basic application which uses DataSerives over RTMP channel to populate dataGrid , I have not used any user defined component.
This application has got LOAD data and Remove data Button just to simulate the actual application
What happens , when I click on LOAD data , It loads the data from server(JBOSS) then I Remove all the data and force the GC to run
If I continuously keep loading and removing the data , memory goes up every time and after some times It reaches to threshold and finally crashes the Browser.
In actual application as soon as user logs in a dedicated DS destination is created..
private function createDestination(destName:String):void{
var remoteService:RemoteObject = new RemoteObject("DestinationManager");
remoteService.addEventListener(ResultEvent.RESULT, resultHandler,false,0,true);
var operation:AbstractOperation = remoteService.getOperation("createDestination");
operation.send(destName);
var cs:ChannelSet = new ChannelSet();
var rtmpChannel:RTMPChannel = ServerConfig.getChannel("my-rtmp") as RTMPChannel;
cs.addChannel(rtmpChannel);
tradeService = new DataService() ;
tradeService.destination=destName;
tradeService.channelSet = cs;
Now when user hits on Load data button to load the trade data ...
public function loadData():void
var remoteService:RemoteObject = new RemoteObject("filterRemoteVO");
var operation:AbstractOperation = remoteService.getOperation("loadData");
tradeService.fill(tradeCollection);
operation.send(rowData.text ,GUIID);
tradeList.dataProvider = tradeCollection;
Remove Data hit I clean all the resource and released collection :-
public function removeAll():void{
if(tradeList.dataProvider){
tradeCollection.source = new Array();
tradeList.dataProvider.source = new Array();
tradeList.dataProvider = null;
tradeService.release();
tradeService.releaseCollection(tradeCollection);
tradeService.channelSet.disconnectAll();
tradeService.disconnect();
tradeService.logout();
System.gc();
Except these 3 function I don't have any code which could cause memory leak.
I m using
LCDS 2.5
Flex 3.0
JBoss4.2
Please see the mxml file below , If possible please have a look at it , I might not be using dataSerice API correctly.
I have been trying to solve this leak for last two week , I really need help from you guys.
My client has already evaluating other contemporary technology because of this memory issue.
I have also created scalled down war file which can be deployed on any java application server.
Any help will greatly be appreciated
Regards,
Dam
######################## MXML##########################
<?xml version="1.0" encoding="utf-8"?>
<mx:Application xmlns:mx="http://www.adobe.com/2006/mxml" layout="absolute"
creationComplete="init();" height="100%" width="100%"
>
<mx:Script >
<![CDATA[
import mx.containers.GridRow;
import mx.utils.Base64Decoder;
import mx.utils.StringUtil;
import mx.events.CollectionEvent;
import mx.formatters.Formatter;
import mx.events.ResizeEvent;
import mx.controls.DataGrid;
import mx.effects.Fade;
import mx.events.DataGridEvent;
import mx.utils.ObjectUtil;
import mx.events.ScrollEvent;
import mx.formatters.DateFormatter;
import mx.events.ListEvent;
import mx.data.mxml.DataService;
import mx.messaging.config.ServerConfig;
import mx.messaging.channels.RTMPChannel;
import mx.messaging.ChannelSet;
import mx.rpc.AbstractOperation;
import mx.rpc.remoting.RemoteObject;
import mx.collections.ArrayCollection;
import mx.controls.Alert;
import mx.rpc.events.ResultEvent;
import mx.messaging.messages.*;
import mx.messaging.events.*;
import mx.controls.Alert;
private var GUIID:String;
private var lastRollOverIndex:Number;
public var tradeService:DataService=null;
public var tradeCollection:ArrayCollection = new ArrayCollection();
public var flag:Boolean = false;
private function init():void {
var guiId:String ="TEST";
createDestination(GUIID);
public function getData():void{
// memory.text =""+ ((System.totalMemory / 1024)/1024);
public function releaseDataServices():void {
// tradeService.fill(null);
tradeService.release();
// tradeService.releaseCollection(tradeCollection);
tradeService.channelSet.disconnectAll();
tradeService.disconnect();
tradeService.logout();
//tradeService.dataStore = null;
/*Function for creating destination*/
private function createDestination(destName:String):void{
var remoteService:RemoteObject = new RemoteObject("DestinationManager");
remoteService.addEventListener(ResultEvent.RESULT, resultHandler,false,0,true);
var operation:AbstractOperation = remoteService.getOperation("createDestination");
operation.send(destName);
var cs:ChannelSet = new ChannelSet();
var rtmpChannel:RTMPChannel = ServerConfig.getChannel("my-rtmp") as RTMPChannel;
cs.addChannel(rtmpChannel);
tradeService = new DataService() ;
tradeService.destination=destName;
tradeService.channelSet = cs;
//This function calls dataservice fill method depending on wheather destination is created or not
private function resultHandler(event:ResultEvent):void{
//call data service as destination is created successfully
var status:Boolean = event.result as Boolean
// Alert.show("status="+status);
if(status){
getData();
else{
//TODO report error to MATS app
public function removeAll():void{
if(tradeList.dataProvider){
tradeCollection.source = new Array();
tradeList.dataProvider.source = new Array();
tradeList.dataProvider = null;
tradeService.release();
// tradeService.releaseCollection(tradeCollection);
tradeService.channelSet.disconnectAll();
tradeService.disconnect();
tradeService.logout();
System.gc();
public function loadData():void
var remoteService:RemoteObject = new RemoteObject("filterRemoteVO");
var operation:AbstractOperation = remoteService.getOperation("loadData");
tradeService.fill(tradeCollection);
operation.send(rowData.text ,GUIID);
tradeList.dataProvider = tradeCollection;
]]>
</mx:Script>
<mx:DataGrid id="tradeList"
initialize="tradeList.dataProvider = tradeCollection" horizontalScrollPolicy="auto"
height="659" x="0" y="84">
<mx:columns>
<mx:DataGridColumn headerText="Security Description" dataField="securityDesc"
/>
<mx:DataGridColumn headerText="B/S" dataField="transactionType"
/>
<mx:DataGridColumn headerText="Amount" dataField="amount"
/>
<mx:DataGridColumn headerText="Price Text" dataField="priceText"
/>
<mx:DataGridColumn headerText="Counterparty" dataField="counterParty"
/>
<mx:DataGridColumn headerText="Trade Date" dataField="tradeDate"
/>
<mx:DataGridColumn headerText="Settle Date" dataField="settleDate"
/>
<mx:DataGridColumn headerText="Trade Status" dataField="tradeStatusId"
/>
<mx:DataGridColumn headerText="Trader" dataField="trader"
/>
<mx:DataGridColumn headerText="Salesperson" dataField="salesperson"
/>
<mx:DataGridColumn headerText="Ticket Number" dataField="tradeNumber"
/>
<mx:DataGridColumn headerText="Entry Date" dataField="tradeTime"
/>
<mx:DataGridColumn headerText="Cancel Date" dataField="cancelDate"
/>
<mx:DataGridColumn headerText="Cancel Time" dataField="cancelTime"
/>
</mx:columns>
</mx:DataGrid >
<mx:Grid x="149" y="23" width="995">
<mx:GridRow width="100%" height="100%">
<mx:GridItem width="100%" height="100%">
<mx:Label text="Enter No of obejcts" width="111"/>
</mx:GridItem>
<mx:GridItem width="100%" height="100%">
<mx:TextInput name="row" id="rowData" width="97"/>
</mx:GridItem>
<mx:GridItem width="100%" height="100%">
<mx:Button label="Load Data" click='loadData()'/>
</mx:GridItem>
<mx:GridItem width="100%" height="100%">
<mx:Button label="removeAll" click='removeAll()'/>
</mx:GridItem>
<mx:GridItem width="100%" height="100%">
<mx:Label text="Total Memory"/>
</mx:GridItem>
<mx:GridItem width="100%" height="100%">
<mx:TextInput text="Total Memory" id="memory" width="97"/>
</mx:GridItem>
<mx:GridItem width="100%" height="100%">
<mx:Label text="Grid Row Count"/>
</mx:GridItem>
<mx:GridItem width="100%" height="100%">
<mx:TextInput name="GridRowCountN" id="GridRowCount" width="97"/>
</mx:GridItem>
</mx:GridRow>
</mx:Grid>
</mx:Application>Hi Natasha,
Sorry for late reply.
I tried your suggestion but It did not help to release memory. Actully,this app contains only one event handler which is for remote method call , I dont think this could cause any memory leak.
Profiling session reveals that It is the issue of DataService class.I dont know How do I convince Adobe to investigate this issue.
Regards,
Dharmendra -
Memory Leak in BI SDK with XMLA Connector ?
Hi,
I am using BI SDK 3.5 with XMLA Connector. I noticed that in com.sap.ip.bi.sdk.dac.connector.xmla.impl.Connection class, the LocalTransaction object is rollbacked when the close() method is called.
This makes a call to the endTransaction() method of Repository class with "true" parameter and leads to informations storage into a RepositoryDBMemory instance.
The only way I found to avoid this is to call the endTransaction() method with "false" as parameter, which is what is done in commit() method of the LocalTransaction class.
So, I tried to commit the transaction before closing the connection but in XMLA mode, the getLocalTransaction() method throws an BIResourceException so it is impossible to commit it. This is the first issue.
Second issue: it seems that data hold by the DataSetContentHandler is not freed. I tried using SAP Parser, Xerces, Java Parser but it has no effect. I hava a MDX query that returns 13000 lines, the XML file size is about 15Mo.
In a for loop with 10 operations, it leads to a OutOfMemory error quickly (only 7 iterations OK with -Xmx256m -Xms256m options). On each iteration, the memory grows about 25 to 30Mo !
For the first point, I do not call the close method on the connection object and it is garbage collected (but, in my opinion, it is quite strange not calling the close() method on a connection object).
For the second one (DataSetContentHandler), I have no idea. Has someone already encounter this problem ? The Netbeans profiler tells me it is this object that holds all java.lang.String objects. Is it right ?
I saw in DataSetContentHandler class that it has two static fields: one for the row cursor of the dataset, the other one for column cursor. Why ? Is it OK in a multi-threaded context ?
<b>This is urgent as we will not put the project in production mode with such a memory leak ! We must find the problem very quickly.</b>
Thanks in advance for any help you might give to me !
Regards,
David Degardin (Ubik-Ingenierie)I found the same thing. Tracing the execution back I found the only place this function was called was as part of the setup, so you shouldn't have to worry about it getting called multiple times and leaking memory. The trouble seems to be when the AudioPlayer dealloc method is called, the memory is not cleaned up.
My fix was to modify the dealloc in AudioQueueObject to check to see if audioLevels is nil and, if not, free it. This seems to have fixed the problem. -
Memory Leak in terminateConnection() function. How to resolve it ???
Hi,
We are facing a memory leak issue in terminateConnection() function. Here is a sample code regarding that. We have run around 1.5 hours simultaneously and it was consuming around 250MB for this simple program. How could I resolve this problem ?
bool test()
string userName = "mapserver";
string password = "a";
string connectString = "//localhost:1521/orcl";
string query = "SELECT GEOMETRY AS GEOM FROM AG_WATER";
Environment *env = Environment::createEnvironment(Environment::OBJECT);
Connection *conn;
Statement *stmt;
ResultSet *rs;
try{
conn = env->createConnection(userName, password, "");
catch (SQLException ex) {
printf("ORACLE execution error: [Error Code : %d] : %s\n", ex.getErrorCode(), ex.getMessage().c_str());
return false;
env->terminateConnection(conn);
stmt = NULL;
conn = NULL;
rs = NULL;
Environment::terminateEnvironment(env);
return true;
int main()
OracleResultSet pResultSet;
while (true) {
test();
return 0;
Does any body help me to find out this problem ???This is not a web dynpro related question. Please restrict the questions in this forum to the Web Dynpro ABAP topic.
-
Hello all,
Not sure if this is the right place to post this but here goes. I have a program that needs to change the font of at least 250,000 letters, however after doing this the program runs slow and laggy as if changing the fonts of each letter took up memory and never released it? Here is a compilable example, Click the button at the bottom of the window and watch the letter it is currently working on. You will notice that around 200,000 it will begin to not smoothly count up anymore but count in kind of a skipping pattern. This seems to get worse the more you do it and seems to indicate to me that something is using memory and not releasing it. I'm not sure why replacing a letter's font with a new font would cause more memory to be taken up I would think if I was doing it properly it would simply replace the old font with the new one not taking anymore memory then it did before. Here is the example: This program sometimes locks up so be prepared. If someone could maybe point out what is causing this to take up more memory after changing the fonts that would be great and hopefully find a solution :) Thanks in advance.
-neptune692
* To change this template, choose Tools | Templates
* and open the template in the editor.
package paintsurface;
import java.awt.*;
import java.awt.event.*;
import javax.swing.*;
import java.util.*;
import java.util.List;
public class PaintSurface implements Runnable, ActionListener {
public static void main(String[] args) {
SwingUtilities.invokeLater(new PaintSurface());
List<StringState> states = new ArrayList<StringState>();
Tableaux tableaux;
Random random = new Random();
Font font = new Font("Arial",Font.PLAIN,15);
// Point mouselocation = new Point(0,0);
static final int WIDTH = 1000;
static final int HEIGHT = 1000;
JFrame frame = new JFrame();
JButton add;
public void run() {
tableaux = new Tableaux();
for (int i=250000; --i>=0;)
addRandom();
frame.add(tableaux, BorderLayout.CENTER);
add = new JButton("Change Font of letters - memory leak?");
add.addActionListener(this);
frame.add(add, BorderLayout.SOUTH);
frame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
frame.setSize(WIDTH, HEIGHT);
frame.setLocationRelativeTo(null);
frame.setVisible(true);
public void actionPerformed(ActionEvent e) {
new Thread(new ChangeFonts()).start();
void addRandom() {
tableaux.add(
Character.toString((char)('a'+random.nextInt(26))),
UIManager.getFont("Button.font"),
random.nextInt(WIDTH), random.nextInt(HEIGHT));
//THIS CLASS SEEMS TO HAVE SOME KIND OF MEMORY LEAK I'M NOT SURE?
class ChangeFonts implements Runnable {
public void run() {
Random rand = new Random();
for(int i = 0; i<states.size(); i++) {
font = new Font("Arial",Font.PLAIN,rand.nextInt(50));
states.get(i).font = font;
add.setText("Working on letter - "+i);
class StringState extends Rectangle {
StringState(String str, Font font, int x, int y, int w, int h) {
super(x, y, w, h);
string = str;
this.font = font;
String string;
Font font;
class Tableaux extends JComponent {
Tableaux() {
this.enableEvents(MouseEvent.MOUSE_MOTION_EVENT_MASK);
lagState = createState("Lag", new Font("Arial",Font.BOLD,20), 0, 0);
protected void processMouseMotionEvent(MouseEvent e) {
repaint(lagState);
lagState.setLocation(e.getX(), e.getY());
repaint(lagState);
super.processMouseMotionEvent(e);
StringState lagState;
StringState createState(String str, Font font, int x, int y) {
FontMetrics metrics = getFontMetrics(font);
int w = metrics.stringWidth(str);
int h = metrics.getHeight();
return new StringState(str, font, x, y-metrics.getAscent(), w, h);
public void add(String str, Font font, int x, int y) {
StringState state = createState(str, font, x, y);
states.add(state);
repaint(state);
protected void paintComponent(Graphics g) {
Rectangle clip = g.getClipBounds();
FontMetrics metrics = g.getFontMetrics();
for (StringState state : states) {
if (state.intersects(clip)) {
if (!state.font.equals(g.getFont())) {
g.setFont(state.font);
metrics = g.getFontMetrics();
g.drawString(state.string, state.x, state.y+metrics.getAscent());
if (lagState.intersects(clip)) {
g.setColor(Color.red);
if (!lagState.font.equals(g.getFont())) {
g.setFont(lagState.font);
metrics = g.getFontMetrics();
g.drawString("Lag", lagState.x, lagState.y+metrics.getAscent());
}Here is the block of code that I think is causing the problem:
//THIS CLASS SEEMS TO HAVE SOME KIND OF MEMORY LEAK I'M NOT SURE?
class ChangeFonts implements Runnable {
public void run() {
Random rand = new Random();
for(int i = 0; i<states.size(); i++) {
font = new Font("Arial",Font.PLAIN,rand.nextInt(50));
states.get(i).font = font; // this line seems to cause the problem?
add.setText("Working on letter - "+i);
}neptune692 wrote:
jverd wrote:
You're creating a quarter million distinct Font objects, and obviously you must be hanging on to all of them because each character is having its font set to the newly created object. So if you have 250k chars, you're forcing it to have 250k Font objects.
Since the only difference is that rand.nextInt(50) parameter, just pre-create 50 Font objects with 0..49, stick 'em in the corresponding elements in an array, and use rand.nextInt to select the Font object to use.That does make sense but it does that when the the program is first launched and doesn't lag. But the second and third time you change the letters font it seems to lag so if it wasn't taking up more memory the second time it should perform like it did when it first launched. I don't care to investigate any further. The real problem is almost certainly the quarter million Font objects. It could be that 250k is fine, but by the time you get to 500k, it has to do a lot of GC, and that's where the slow down is coming. You might even be able to make it work better with the code you have just by tweaking the GC parameters at startup, but I wouldn't bother. Fix the code first, and then see if you have issues.
Does creating a new font for each of those letters not replace the old font object? If it didn't use more memory the second and third time I don't think you would see the skipping in the counter and the slowing down of the iterations. So it must be remembering some of the old font objects or am I wrong?Using new always creates a new object. When you do it the second time around, and call letter.setFont(newFont), the old Font object is eligible for GC. That doesn't mean it will be GCed right away though. The JVM can leave them all laying around until it runs out of memory, and then GC some or all of them. -
Memory Leak in my JDBC application.
Hi
I am experiencing a memory leak in a test application using the JDBC-ODBC bridge to access an MS Access DB.
I close the result set, statement and connection objects after each query. Even then the memory allocated to the process increases by about 20K after each query.
Is there anything else I should do apart from closing these objects??
Thanks a lot for your time.
Fred.I am having the same problem using JDBC-ODBC bridge with the MS SQL server DB. Even after closing all of the objects as specified.
Sorry couldn't be of much help but check the following link
http://www.allaire.com/Handlers/index.cfm?ID=12409&Method=Full
But I do not have a work around for this may be I am not looking at the right response.
Can some one please help. -
Memory Leak in 8.1.6.0.1 JDBC/OCI for Solaris
Hello,
there is a memory leak in the 8.1.6.0.1 JDBC-OCI driver for solaris.
The leak causes your jvm to eat up all memory
if you reuse callable statements
(calling one statement multiple times with
different values).
The thin driver has no such problem. Is
there any fix available ?Ok. The code spans multiple classes and
most of it comes from a customized version
of the Enhydra Java Application server.
I have a class called "StandardDBConnection"
which caches CallableStatements and is a
wrapperclass for java.sql.DBConnection. The
interesting method here is "prepareCall":
* Get a callable statement given an SQL string. If the statement is
* cached, return that statement, otherwise prepare and save in the
* cache.
* @param sql The SQL statement to be called.
* @return a new CallableStatement object containing the
* pre-compiled SQL statement.
* @exception java.sql.SQLException If a database access error occurs
* statement.
public synchronized CallableStatement prepareCall(String sql)
throws SQLException {
PreparedStatement preparedStmt;
logDebug ("Prepare call: " + sql);
validate();
preparedStmt = (PreparedStatement)preparedStmtCache.get(sql);
// Check if the object returned by the cache really is a
// callable statement. if it is not, someone did call first
// prepareStatement() and now prepareCall() with the same
// sql. Silently replace the existing cache entry by a
// callable statement in this case.
if (preparedStmt instanceof CallableStatement) {
preparedStmt.clearParameters();
else {
// Need to close the old PreparedStatement in case we have to
// replace it with a CallableStatement
if (preparedStmt != null) {
preparedStmt.close();
else if (preparedStmtCache.size() >= maxPreparedStmts) {
String key = (String)preparedStmtCache.keys().nextElement();
((PreparedStatement) preparedStmtCache.remove(key)).close();
preparedStmt = connection.prepareCall(sql);
preparedStmtCache.put(sql, preparedStmt);
return (CallableStatement)preparedStmt;
The statements get closed when I close the
connection:
boolean closeStmts = true;
// Close the prepared statements.
Enumeration e = preparedStmtCache.keys();
while (e.hasMoreElements() && closeStmts) {
String key = (String)e.nextElement();
try {
((PreparedStatement)
preparedStmtCache.remove(key)).close();
} catch (SQLException except) {
// Ignore errors, we maybe handling one.
closeStmts = false;
log.write(Logger.NOTICE,
"DBConnection[" + id + "]: " + url +
"\nUnable to close statements. Continuing....\n");
In my classes using database queries I just
use the prepareCall method of DBConnection
and do not have to care about anything.
Works perfectly with the thin driver, but
as soon as I switch to oci... :-|
<BLOCKQUOTE><font size="1" face="Verdana, Arial">quote:</font><HR>Originally posted by JDBC Dev Team:
Soda, Rupper,
Do you mind posting some code that shows us what your code was doing when you notice this leak?
Thanks.<HR></BLOCKQUOTE>
null
Maybe you are looking for
-
How to undo (open) a closed network?
I closed the network but can't figure out how to open it again. When I go into Admin Utility the name appears but the "close the network" box is NOT checked. But the name does not appear on the Airport drop-down menu Running OSX 10.3 & recently updat
-
HT4889 Transferring files from my old Macbook Pro to the new one takes over 30 hours?!?!
How is that even possible? I am trensferring the files using a tunderbolt cable and the wifi on the source computer is switched off to avoid trying to transfer the files over wifi. At first I thought the estimation was inaccurate and hope that when I
-
Changing my volume causes the application to minimize!!! How i can solve this???
Dear Friends, I purchased a HP Pavilion dv6-2180us and I'm having this problem. Whenever I'm gaming or watching a movie on full screen and try to change the volume (my volume is touch sensitive) sliding to increase or decrease the sound, the applicat
-
How CAN i get a refund after i purchase an app i do Not like?
-
How to implement the security notes in Java System.
Hi All, For the ABAP systems we use RSECNOTE to implement the security notes, but how do we do that in Java systems? Any reference or guidance will be of great help. Thanks, Akash.