BCP memory not releasing.
Hi, I am running BCP command to copy the data from a file to the database in a while loops. as the number of iterations goes up, the memory usage is grown to the extreme and it consumed all my 64 GB space. My system is crashed.
How could I release the memory or How could I force the SQL Server to release the memory used by the BCP
declare @currentIteration int
set @currentIteration = 1
while @currentIteration <=5000
begin
exec master..xp_cmdShell 'bcp MyDB.dbo.Invoice01 in "D:\DataFiles\DataToDump_100.txt" -e"D:\DataFiles\error100.txt" -b 100 -t"|" -T -c'
end
Hello,
Could you try to use bcp with the tablock hint instead?
http://msdn.microsoft.com/en-us/library/ms162802.aspx
Hope this helps.
Regards,
Alberto Morillo
SQLCoffee.com
Similar Messages
-
LR 2.3RC - Memory not released from Develop
I am running 2.3RC under Windows XP. It's going fine but it still has a memory issue that I observed under 2.2.
After going from Library Grid to Develop, then going back to Grid without doing any actions in Develop (actually, it doesn't matter whether you do anything or not), I see approximately an extra 180M in the VM size in Windows Task Manager. If I then go into Print the VM size stays about the same - about 180M bigger than it should be.
However, if instead, once I return to Grid from Develop, I then use the arrow key and select another image, the memory is released. I can then go back to the original image and the memory does not increase. Then I can go to Print and, once again, the memory does not increase (some minor change - in fact, on my machine, it goes down!).
Looks like as long as the original image is selected, memory from Develop is not being released. As I said at the outset, this is not new in 2.3RC - it also happened in 2.2.
Also - no local adjustments were or have been made to the image in question.
Anyone else see this on their machines?
Selby ShanlyIan and Simon - you are correct, it is not a leak. Please note that is 180M - not 180K.
Of course, as users, we simply don't know whether or not it is cached information that actually gets reused, or just a mistake.
The Print Spooler under XP takes a good chunk of memory - why fight for it with LR? What about Photoshop? Often the next thing I do after Develop is go to PS to do soft proofing. PS could use that memory.
The question to Adobe is whether or not this is an oversight. If it is then they alone are in the position to evaluate the risks of changing the code - it might be utterly trivial. If it is deliberate and provides efficiencies then at least it would be interesting to know!
Even with the size of current machines, 180M of memory (YMMV) is still not something to use up lightly for no benefit. And for users who are tight on memory and at the knee of the performance curve, it might make a noticable difference.
But all of this is speculation on my part - let's see what Adobe says.
I do appeciate the comments. Thanks.
Selby -
Memory not released on disconnections
Hi,
We are developing a pro*C++ application to connect to an Oracle 8i database running in Windows 2000 . The application will run in HP-UX. When the application try to connect to the database, either the connection fails or successes, a memory segment is allocated to the connection, and even when we do a rollback release before the disconnection, the resources allocated are not released, so the memory allocated to the application grows without a limit (every 60 connections it takes 128 kb).
We´ve tried with a rollback/commit release before every disconnection (depending on the failure or success of the associated processes), analising the tnsnames parameters to avoid any waiting time, increasing the timer period between connection tests, but the memory is not being released after disconnections... any idea about what is happening? why even when the connection with the database is not possible there is any memory allocated to it? how could we release it? do you know how to test that the database/listener are alive so the connection is possible, other than try to connect and examine the possible error messagges?
Thanks a lot!
AnaJim,
I don't think this is ID directly causing a problem, but some other interaction. I'm using 5.0.3 on XP pro SP2 and the exe disappears from processes about 2 or 3 seconds after close.
What version of Windows are you running?
Peter -
Dynamic memory not released to host
Dear Techies,
Ours is a small Hyper V virtual server infrastructure with three DELL power-edge physical hosts(Windows server 2012 Datacenter) and around 15 virtual machines running on top of it. The hosts are added to fail-over cluster. Each host has 95 GB RAM. All the
VMs are running Windows server 2012 standard edition.
We have installed terminal services(TS licensing, TS connection broker, TS session host) in four VMs with the following dynamic memory settings:
Start-up RAM : 2048 MB
Minimum RAM : 2048 MB
Maximum RAM : 8192 MB
Below mentioned applications are configured in the server:
Nova Application Installed
SQL Developer Tool is Configured (ODBC Connection established for Database communication)
FTPs communication allowed for File Transfer
McAfee Agent is configured (Virus Scanner)
Nimsoft Robot Agent Configured – Monitoring
Terminal Service
Enabled Multiple terminal sessions based on customer requirement
BGinfo tool configured through group policy for customized desktop background
The average memory utilization in the terminal servers are 3.6 GB. As per dynamic allocation the maximum RAM requirement/allocation till date is 4GB. As seen in Hyper V console, the current RAM demand is 2300 MB and assigned memory is 2800 MB.
However, the earlier assigned RAM in the server is ballooned/faked to the VM as driver locked memory. This is by design. Despite the memory being released back to the host, the server still shows up the 4Gb which makes the memory utilization report from
monitoring tools look 80% (3.2 GB out of 4 GB).
As a result, the memory utilization report is always based on the current dynamically allocated RAM and not calculated based on the maximum assigned RAM(8GB in this case). To make it clear: If the
currently assigned RAM is 4Gb and utilization is 3.2 GB the utilization % is reported as 80%. However, if calculated in accordance with maximum RAM capacity of the server it would be 40% ((3.2/8)*100).
Is there any way to release the driver locked memory from the VM.?
Regards,
Auditya NI am not really clear on the point of your question.
Allocated RAM is what is currently in use / allocated to a VM out of the physical RAM pool. It is Demand + Buffer. The demand is based on the applications in the VM and what they think they need and how efficiently they return unused memory
to the machine. This has nothing to do with in-application paging (which happens a lot with Terminal Servers).
So yes, the memory utilization is accurate in relation to physical RAM utilization.
Dynamic Memory is about efficiency, not about over-allocation. Hyper-V can never give VMs more RAM than is in the physical RAM pool. The VMs can be configured to have more RAM than is in the physical RAM pool - but the VMs will hit the top of
the pool and not be allowed to have any more. There is no ballooning or paging to disk.
So, you maximum allocated could go beyond what is possible. But that would mean that your utilization would be artificially low if this was used in the calculation.
Brian Ehlert
http://ITProctology.blogspot.com
Learn. Apply. Repeat. -
Interface operator : Memory not released
Hello,
In the interface operator provided by Ni “Full-Featured \ CVI”,
it would seem that the memory is not released correctly.
By adding the
following lines of codes at the end of the hand, one realizes that the
number of block and byte are not to 0.
#ifdef _CVI_DEBUG_
unsigned int
NbByte;
unsigned int NbBlock;
CVIDynamicMemoryInfo (“Fine of the Hand”,
&NbBlock, &NbByte, DYNAMIC_MEMORY_SHOW_ALLOCATED_MEMORY);
#endif
A priori that comes from the function:
errChk
(TS_LoadMenuBarResourceStrings (gMainWindow.engine, GetPanelMenuBar
(gMainWindow.panel), 0, “TSUI_OI_MAIN_PANEL”, errMsg));
In spite of my
research, I do not manage to solve the problem.
Somebody would have it
the solution or then a track to be exploited.
Thank you in advance for
your assistanceHello Dave.
I found this KB for your problem.
I hope it will help you.
Let me know if it's ok for you.
Have a nice day.
Regards.
Romain D.
National Instruments France
#adMrkt{text-align: center;font-size:11px; font-weight: bold;} #adMrkt a {text-decoration: none;} #adMrkt a:hover{font-size: 9px;} #adMrkt a span{display: none;} #adMrkt a:hover span{display: block;}
NIDays 2010 : Conférence mondiale de l'instrumentation virtuelle
>>Détails et Inscription<< -
Memory not released using DBMS_XMLPARSER
Hello,
I'm using Oracle 9.2.0.3 under HPUX 11 (64 bit)
I want to process XML messages stored as CLOB in a table
When I run several times the following piece of code I notice that the PGA keeps growing and is never released even when I call "dbms_session.free_unused_user_memory"
Here is the code
DECLARE
v_parser DBMS_XMLPARSER.PARSER := NULL ;
v_msg CLOB := NULL ;
CURSOR C_xml IS
SELECT myXml
FROM XML_TABLE ;
PROCEDURE processXml (
p_Xml IN CLOB,
p_parser IN DBMS_XMLPARSER.PARSER )
IS
v_doc DBMS_XMLDOM.DOMDOCUMENT := NULL ;
v_DocElt DBMS_XMLDOM.DOMElement := NULL ;
BEGIN
DBMS_XMLPARSER.parseClob(p_parser, p_Xml) ;
v_doc := DBMS_XMLPARSER.getDocument(p_parser) ;
DBMS_XMLPARSER.freeParser(p_parser) ;
v_DocElt := DBMS_XMLDOM.getDocumentElement(v_doc) ;
v_DocElt := NULL ;
DBMS_XMLDOM.freeDocument(v_doc) ;
v_doc := NULL ;
END processXml ;
BEGIN
v_parser := DBMS_XMLPARSER.newParser ;
DBMS_XMLPARSER.setValidationMode(v_parser, FALSE) ;
FOR R_xml IN C_xml LOOP
processXml(R_xml.myXml, v_parser) ;
END LOOP ;
v_parser := NULL ;
END ;
I checked the v$pgastat and v$mystat views I noticed that the memory is never released and that the freable_pga_memory stay at 0.
Could you please tell me if I should call some other method to release my memory or is it a leak in the DBMS_XML* packages
Thanks a lot
David CosteI ran this in 10g with the following results...
Oracle Database 10g Enterprise Edition Release 10.1.0.2.0 - Production
With the Partitioning, OLAP and Data Mining options
SQL> set long 100000
SQL> --
SQL> select resource_monitor.gather_statistics()
2 from dual
3 /
RESOURCE_MONITOR.GATHER_STATISTICS()
<statisticsSnapshot>
<operatingSystemPID>2880</operatingSystemPID>
<databasePID>18</databasePID>
<databaseSID>159</databaseSID>
<snapshotTimestamp>26-MAY-04 04.45.13.576000000 PM</snapshotTimestamp>
<tempSpaceUsed>257949696</tempSpaceUsed>
<PGA>
<Current>498252</Current>
<Maximum>563788</Maximum>
<Used>403610</Used>
<Allocated>730298</Allocated>
RESOURCE_MONITOR.GATHER_STATISTICS()
<Freeable>0</Freeable>
<Max>730298</Max>
</PGA>
<UGA>
<Current>152184</Current>
<Maximum>217648</Maximum>
</UGA>
<LOBS/>
</statisticsSnapshot>
SQL> --
SQL> DECLARE
2 aParser DBMS_XMLPARSER.PARSER ;
3 aDoc DBMS_XMLDOM.DOMDOCUMENT ;
4 aElement DBMS_XMLDOM.DOMELEMENT ;
5 myXml CLOB :=
6 '<?xml version=''1.0'' encoding=''UTF-8''?>
7 <TEST>
8 <subtest name="dummy">12</subtest>
9 <subtest name="dummy">12</subtest>
10 <subtest name="dummy">12</subtest>
11 <subtest name="dummy">12</subtest>
12 <subtest name="dummy">12</subtest>
13 <subtest name="dummy">12</subtest>
14 <subtest name="dummy">12</subtest>
15 <subtest name="dummy">12</subtest>
16 <subtest name="dummy">12</subtest>
17 <subtest name="dummy">12</subtest>
18 <subtest name="dummy">12</subtest>
19 <subtest name="dummy">12</subtest>
20 <subtest name="dummy">12</subtest>
21 <subtest name="dummy">12</subtest>
22 <subtest name="dummy">12</subtest>
23 <subtest name="dummy">12</subtest>
24 <subtest name="dummy">12</subtest>
25 <subtest name="dummy">12</subtest>
26 <subtest name="dummy">12</subtest>
27 <subtest name="dummy">12</subtest>
28 <subtest name="dummy">12</subtest>
29 <subtest name="dummy">12</subtest>
30 <subtest name="dummy">12</subtest>
31 <subtest name="dummy">12</subtest>
32 <subtest name="dummy">12</subtest>
33 <subtest name="dummy">12</subtest>
34 <subtest name="dummy">12</subtest>
35 <subtest name="dummy">12</subtest>
36 <subtest name="dummy">12</subtest>
37 <subtest name="dummy">12</subtest>
38 <subtest name="dummy">12</subtest>
39 <subtest name="dummy">12</subtest>
40 <subtest name="dummy">12</subtest>
41 <subtest name="dummy">12</subtest>
42 <subtest name="dummy">12</subtest>
43 <subtest name="dummy">12</subtest>
44 <subtest name="dummy">12</subtest>
45 <subtest name="dummy">12</subtest>
46 <subtest name="dummy">12</subtest>
47 <subtest name="dummy">12</subtest>
48 <subtest name="dummy">12</subtest>
49 </TEST>' ;
50 BEGIN
51 aParser := DBMS_XMLPARSER.newParser ;
52 FOR i IN 1..5000 LOOP
53 DBMS_XMLPARSER.parseClob(aParser, myXml) ;
54 aDoc := DBMS_XMLPARSER.getDocument(aParser) ;
55 aElement := DBMS_XMLDOM.getDocumentElement(aDoc) ;
56 aElement := NULL ;
57 DBMS_XMLDOM.freeDocument(aDoc) ;
58 aDoc := NULL ;
59 END LOOP ;
60 DBMS_XMLPARSER.freeParser(aParser) ;
61 aParser := NULL ;
62 END ;
63 /
PL/SQL procedure successfully completed.
SQL> show errors
No errors.
SQL> --
SQL> select resource_monitor.gather_statistics()
2 from dual
3 /
RESOURCE_MONITOR.GATHER_STATISTICS()
<statisticsSnapshot>
<operatingSystemPID>2880</operatingSystemPID>
<databasePID>18</databasePID>
<databaseSID>159</databaseSID>
<snapshotTimestamp>26-MAY-04 04.45.21.487000000 PM</snapshotTimestamp>
<tempSpaceUsed>257949696</tempSpaceUsed>
<PGA>
<Current>2136652</Current>
<Maximum>2202188</Maximum>
<Used>1752054</Used>
<Allocated>2958522</Allocated>
RESOURCE_MONITOR.GATHER_STATISTICS()
<Freeable>0</Freeable>
<Max>2958522</Max>
</PGA>
<UGA>
<Current>1199608</Current>
<Maximum>1265072</Maximum>
</UGA>
<LOBS>
<Cached>0</Cached>
<nonCached>0</nonCached>
<Abstract>0</Abstract>
RESOURCE_MONITOR.GATHER_STATISTICS()
</LOBS>
</statisticsSnapshot>
SQL> quit
Note I had to move freeParser() call outside of the loop otherwise the code failed in the second iteration...
The definition of gather_statistics follows...
alter session set current_schema = SYS
create or replace package resource_monitor
authid definer
as
function gather_statistics return XMLType;
function get_Lob_Stats(stats XMLTYPE) return XMLType;
function get_Time(stats XMLTYPE) return timestamp;
procedure initializeTimer;
function getElapsedTime return INTERVAL DAY TO SECOND;
end;
show errors
create or replace package body resource_monitor
as
START_TIME timestamp;
function gather_statistics
return XMLType
as
time timestamp;
ORA_PID number;
ORA_SID number;
OS_PID number;
SESSION_PGA_MEMORY number;
SESSION_PGA_MEMORY_MAX number;
SESSION_UGA_MEMORY number;
SESSION_UGA_MEMORY_MAX number;
PGA_USED_MEMORY number;
PGA_ALLOC_MEMORY number;
PGA_FREEABLE_MEMORY number;
PGA_MAX_MEMORY number;
CACHE_LOBS number;
NO_CACHE_LOBS number;
ABSTRACT_LOBS number;
TEMP_FILE_USAGE number;
statistics XMLType;
begin
select distinct SID
into ORA_SID
from V$MYSTAT;
select PID,SPID,
PGA_USED_MEM,PGA_ALLOC_MEM,PGA_FREEABLE_MEM,PGA_MAX_MEM
into ORA_PID, OS_PID,
PGA_USED_MEMORY, PGA_ALLOC_MEMORY, PGA_FREEABLE_MEMORY, PGA_MAX_MEMORY
from V$PROCESS p, V$SESSION s
where p.addr = s.paddr
and s.sid = ORA_SID;
select value
into SESSION_PGA_MEMORY
from v$mystat st,v$statname n
where n.name = 'session pga memory'
and n.statistic# = st.statistic#;
select value
into SESSION_PGA_MEMORY_MAX
from v$mystat st,v$statname n
where n.name = 'session pga memory max'
and n.statistic# = st.statistic#;
select value
into SESSION_UGA_MEMORY
from v$mystat st,v$statname n
where n.name = 'session uga memory'
and n.statistic# = st.statistic#;
select value
into SESSION_UGA_MEMORY_MAX
from v$mystat st,v$statname n
where n.name = 'session uga memory max'
and n.statistic# = st.statistic#;
begin
select l.cache_lobs, l.nocache_lobs, l.abstract_lobs
into CACHE_LOBS, NO_CACHE_LOBS, ABSTRACT_LOBS
from v$temporary_lobs l, v$session s
where s.audsid = SYS_CONTEXT('USERENV', 'SESSIONID')
and s.sid = l.sid;
exception
when others then
null;
end;
select sum(USER_BYTES)
into TEMP_FILE_USAGE
from DBA_TEMP_FILES;
select systimestamp
into TIME
from dual;
select xmlElement
"statisticsSnapshot",
xmlForest
OS_PID as "operatingSystemPID",
ORA_PID as "databasePID",
ORA_SID as "databaseSID",
TIME as "snapshotTimestamp",
TEMP_FILE_USAGE as "tempSpaceUsed"
xmlElement
"PGA",
xmlForest
SESSION_PGA_MEMORY as "Current",
SESSION_PGA_MEMORY_MAX as "Maximum",
PGA_USED_MEMORY as "Used",
PGA_ALLOC_MEMORY as "Allocated",
PGA_FREEABLE_MEMORY as "Freeable",
PGA_MAX_MEMORY as "Max"
xmlElement
"UGA",
xmlForest
SESSION_UGA_MEMORY as "Current",
SESSION_UGA_MEMORY_MAX as "Maximum"
xmlElement
"LOBS",
xmlForest
CACHE_LOBS as "Cached",
NO_CACHE_LOBS as "nonCached",
ABSTRACT_LOBS as "Abstract"
into statistics
from dual;
statistics := statistics.extract('/');
statistics := xmltype(xmltype.getCLOBVal(statistics));
return statistics;
end;
function get_Lob_Stats(stats XMLTYPE) return xmltype
as
val xmltype;
begin
return stats.extract('/statisticsSnapshot/LOBS');
end;
function get_Time(stats XMLTYPE) return timestamp
as
begin
return to_Timestamp
stats.extract('/statisticsSnapshot/snapshotTimestamp/text()').getStringVal()
end;
procedure initializeTimer
as
begin
START_TIME := sysTimeStamp;
end;
function getElapsedTime return interval DAY TO SECOND
as
END_TIME timestamp;
ELAPSED_TIME INTERVAL DAY (2) TO SECOND (6);
begin
END_TIME := sysTimeStamp;
ELAPSED_TIME := END_TIME - START_TIME;
START_TIME := sysTimeStamp;
return ELAPSED_TIME;
end;
end;
show errors
grant execute on resource_monitor to public
create or replace public synonym resource_monitor for resource_monitor
desc resource_monitor
set pages 50
set long 2048
select resource_monitor.gather_statistics
from dual
spool off
Can you run this and see what you get. For 9.2.x you'll need to remove the references to abstract_lobs... -
Vector renewed but memory not released
Hi,
Please see the following code:
Vector v = new Vector(10000000);
// from windows task manager, it seems it takes 40 Mb of memory.
// there is a need to do the following after some code
v = new Vector(10000000);
// now the memory taken is around 80 Mb
It would be useful if the program releases the earlier memory of 40 Mb. May be it does not work that way? Any thoughts are welcome.
Regards
Chandra.Thank you all folks. Each suggestion helps me understand vectors and memory in a better way than before. I have noted setting the heap and also the 'clear' function.
When I repeat the code several times, it seems that initially there is an increase in memory up to a certain limit (in my case it is 128 Mb) but there is no increase after that, how many times I repeat the code. (Repeating the code is to find when the out of memory problem occurs). It works okay for now.
Thanks to all again.
Regards
Chandra. -
Internal memory not released after moved to SD card
After installed from market, it was first it on the internal memory. I moved it to SD card and noticed the Manage applications page show the size changed to 128KB. But when I go to Settings -> Storage, the available space has not changed. It is still 12 - 13MB less than what is was before install Firefox Mobile.
Has any one else got the same problem?A restart solved this problem. After restart the available space amount is correct.
-
Memory not released using dbms_xmldom. free_document()
Hi:
I'm using Oracle 10.2.0 under Windows 2003 server 32 bits (Oracle Dedicated Connection)
I have a PL/SQL stored function that builds a Dom Document from the result of a SQL query and at the end store its content in a Clob variable which is returned to the caller. I’m using the DBMS_XMLDOM package to manipulate the Dom document and at the end I make a call the dbms_xmldom.free_document (domDoc) to free the resources associated to the Dom Doc.
For some reason when I run several times the PL/SQL stored function I’ve notice that the PGA keeps growing and is never released even when I call "dbms_xmldom.free_document"
Here is the pl/sql code:
FUNCTION sfLerColCenCarga (pId IN INTEGER, pTipoSerie IN VARCHAR2) RETURN CLOB AS
CURSOR cCenCarga IS
SELECT CgCen.IdCgCen, CgCen.IdX, CgCen.Nom, CgCen.TpS, CgCen.EtI, CgCen.EtF,
CgCen.Fnt, CgCen.DtR, CgCen.DtA, CgCen.Arq, CgCen.Obs,
PeCad.IdX PeCadIdX
FROM CgCen, PeCad
WHERE PeCad.IdPeCad = CgCen.IdPeCad
AND CgCen.IdCgCen = DECODE(pId, NULL, CgCen.IdCgCen, pId)
ORDER BY CgCen.DtA DESC;
regCenCarga cCenCarga%ROWTYPE;
-- Atributos para manipular o XML
domDoc DBMS_XMLDOM.DOMDocument;
noRaiz DBMS_XMLDOM.DOMNode;
noObjeto DBMS_XMLDOM.DOMNode;
noColecaoObjeto DBMS_XMLDOM.DOMNode;
e DBMS_XMLDOM.DOMElement;
xmlData CLOB;
BEGIN
dbms_session.free_unused_user_memory;
domDoc := DBMS_XMLDOM.newDOMDocument;
noRaiz := DBMS_XMLDOM.makeNode(domDoc);
e := DBMS_XMLDOM.createElement(domDoc, 'ColecaoCenCarga');
noColecaoObjeto := DBMS_XMLDOM.appendChild(noRaiz, DBMS_XMLDOM.makeNode(e));
OPEN cCenCarga;
LOOP
FETCH cCenCarga INTO regCenCarga;
EXIT WHEN cCenCarga%NOTFOUND;
e := DBMS_XMLDOM.createElement(domDoc, 'CenCarga');
-- Define valores dos atributos do elemento
DBMS_XMLDOM.setAttribute (e, 'Id', regCenCarga.IdCgCen);
DBMS_XMLDOM.setAttribute (e, 'IdX', TRIM(regCenCarga.IdX));
DBMS_XMLDOM.setAttribute (e, 'Nom', TRIM(regCenCarga.Nom));
DBMS_XMLDOM.setAttribute (e, 'Fnt', TRIM(regCenCarga.Fnt));
DBMS_XMLDOM.setAttribute (e, 'TpS', regCenCarga.TpS);
DBMS_XMLDOM.setAttribute (e, 'EtI', TRIM(regCenCarga.EtI));
DBMS_XMLDOM.setAttribute (e, 'EtF', TRIM(regCenCarga.EtF));
DBMS_XMLDOM.setAttribute (e, 'Aut', TRIM(regCenCarga.PeCadIdX));
DBMS_XMLDOM.setAttribute (e, 'Org', 'X');
DBMS_XMLDOM.setAttribute (e, 'OrgX', 'Externa');
DBMS_XMLDOM.setAttribute (e, 'DtR', TO_CHAR(regCenCarga.DtR,'dd/mm/yyyy HH24:MI'));
DBMS_XMLDOM.setAttribute (e, 'DtA', TO_CHAR(regCenCarga.DtA,'dd/mm/yyyy HH24:MI'));
DBMS_XMLDOM.setAttribute (e, 'Arq', TRIM(regCenCarga.Arq));
DBMS_XMLDOM.setAttribute (e, 'Obs', TRIM(regCenCarga.Obs));
noObjeto := DBMS_XMLDOM.appendChild(noColecaoObjeto, DBMS_XMLDOM.makeNode(e));
END LOOP;
CLOSE cCenCarga;
dbms_lob.createTemporary(xmlData, TRUE, DBMS_LOB.CALL);
DBMS_XMLDOM.writeToClob(domDoc, xmlData);
DBMS_XMLDOM.freeDocument(domDoc);
RETURN xmlData;
END;Could somebody please tell me if I should call some other method to release my memory or is it a leak in the DBMS_XMLDOM package?
Thanks in advance,
André GranvilleYou left off the final number in your version, but I'm going to assume it is .1 - .3. You are running into a bug in Oracle.
http://anononxml.blogspot.com/2010/09/memory-leaks.html
I would suggest upgrading to .4 or .5 (should be out for all versions) as the first step to fix your problem. -
Ever since jupdating to CS3 5.03, When I close down InDesign, about 30% of the it doesn't want to start up again and I have to go into Task Manager's "processes"( Not "applications," since it doesn't show there) and delete InDesign. Then it starts and works okay. This is getting annoying. What can be the cause?
Jim,
I don't think this is ID directly causing a problem, but some other interaction. I'm using 5.0.3 on XP pro SP2 and the exe disappears from processes about 2 or 3 seconds after close.
What version of Windows are you running?
Peter -
Applets and memory not being released by Java Plug-in
Hi.
I am experiencing a strange memory-management behavior of the Java Plug-in with Java Applets. The Java Plug-in seems not to release memory allocated for non-static member variables of the applet-derived class upon destroy() of the applet itself.
I have built a simple "TestMemory" applet, which allocates a 55-megabytes byte array upon init(). The byte array is a non-static member of the applet-derived class. With the standard Java Plug In configuration (64 MB of max JVM heap space), this applet executes correctly the first time, but it throws an OutOfMemoryException when pressing the "Reload / Refresh" browser button or if pressing the "Back" and then the "Forward" browser buttons. In my opionion, this is not an expected behavior. When the applet is destroyed, the non-static byte array member should be automatically invalidated and recollected. Isn't it?
Here is the complete applet code:
// ===================================================
import java.awt.*;
import javax.swing.*;
public class TestMemory extends JApplet
private JLabel label = null;
private byte[] testArray = null;
// Construct the applet
public TestMemory()
// Initialize the applet
public void init()
try
// Initialize the applet's GUI
guiInit();
// Instantiate a 55 MB array
// WARNING: with the standard Java Plug-in configuration (i.e., 64 MB of
// max JVM heap space) the following line of code runs fine the FIRST time the
// applet is executed. Then, if I press the "Back" button on the web browser,
// then press "Forward", an OutOfMemoryException is thrown. The same result
// is obtained by pressing the "Reload / Refresh" browser button.
// NOTE: the OutOfMemoryException is not thrown if I add "testArray = null;"
// to the destroy() applet method.
testArray = new byte[55 * 1024 * 1024];
// Do something on the array...
for (int i = 0; i < testArray.length; i++)
testArray[i] = 1;
System.out.println("Test Array Initialized!");
catch (Exception e)
e.printStackTrace();
// Component initialization
private void guiInit() throws Exception
setSize(new Dimension(400, 300));
getContentPane().setLayout(new BorderLayout());
label = new JLabel("Test Memory Applet");
getContentPane().add(label, BorderLayout.CENTER);
// Start the applet
public void start()
// Do nothing
// Stop the applet
public void stop()
// Do nothing
// Destroy the applet
public void destroy()
// If the line below is uncommented, the OutOfMemoryException is NOT thrown
// testArray = null;
//Get Applet information
public String getAppletInfo()
return "Test Memory Applet";
// ===================================================Everything works fine if I set the byte array to "null" upon destroy(), but does this mean that I have to manually set to null all applet's member variables upon destroy()? I believe this should not be a requirement for non-static members...
I am able to reproduce this problem on the following PC configurations:
* Windows XP, both JRE v1.6.0 and JRE v1.5.0_11, both with MSIE and with Firefox
* Linux (Sun Java Desktop), JRE v1.6.0, Mozilla browser
* Mac OS X v10.4, JRE v1.5.0_06, Safari browser
Your comments would be really appreciated.
Thank you in advance for your feedback.
Regards,
Marco.Hi Marco,
my guess as to why JPI would keep references around, if it does keep them, is that it propably is an implementation side effect. A lot of things are cached in the name of performance and it is easy to leave things laying around in your cache. Maybe the page with the associated images/applets is kept in the browser cache untill the browser needs some memory and if the browser memory manager is not co-operating with the JPI/JVM memory manager the browser is not out of memory, thus not releasing its caches but the JVM may be out of memory. Thus the browser indirectly keeps the reference that it realy does not need. This reference could be inderect through some 'applet context' or what ever the browser uses to interact with JPI, don't realy know any of these details, just imaging what must/could be going on there. Browser are amazingly complicated beast.
This behaviour that you are observing, weather the origin is something like I speculated or not, is not nice but I would not expect it to be fixed even if you filed a bug report. I guess we are left with relleasing all significatn memory structures in destroy. A simple way to code this is not to store anything in the member fields of the applet but in a separate class; then one has to do is to null that one reference from the applet to that class in the destroy method and everything will be relased when necessary. This way it is not easy to forget to release things.
Hey, here is a simple, imaginary, way in which the browser could cause this problem:
The browser, of course needs a reference to the applet, call it m_Applet here. Presume the following helper function:
Applet instantiateAndInit(Class appletClass) {
Applet applet=appletClass.newInstance();
applet.init();
return applet;
When the browser sees the applet tag it instantiates and inits the new applet as follows:
m_Applet=instantiateAndInit(appletClass);
As you can readily see, the second time the instantiation occurs, the m_Applet holds the reference to the old applet until after the new instance is created and initlized. This would not cause a memory leak but would require that twice the memory needed by the applet would be required to prevent OutOfMemory.I guess it is not fair to call this sort of thing a bug but it is questionable design.In real life this is propably not this blatant, but could happen You could try, if you like, by allocating less than 32 Megs in your init. If you then do not run out of memory it is an indication that there are at most two instances of your applet around and thus it could well be someting like I've speculated here.
br Kusti -
OIM 11g using too much memory and not releasing it when shutting down
Hello,
we have a problem with OIM 11g using too much memory (about 5gb) and it's not releasing it at shutdown (4gb still used).
We are using a VM with RedHat Linux 5.6. Originally we had 4gb RAM + 2gb swap file. We installed Admin Server, OAM, OIM and SOA on that machine but quickly realised we couldn't run all 4 programs at once in 6gb. AdminServer could run with 2 other products, but it was a tight fit in memory.
So we increased the RAM to 8gb (still 2gb swap file). But then our problem occured : I start the Admin Server (2.7gb total memory used), then OAM (4.6gb total memory used) and then OIM. After it started the server is now using 9.6gb of memory (~300mb of free memory) ! The problem gets even better : after I shut down everything (OIM, OAM, admin server) the "top" command show that there is still 4gb of memory used even tho nothing else is running on the server ! There is absolutely no other process (other than root stuff) running and no other users connected to the machine. After a reboot, it shows 400mb of memory used. I tried restarting the programs and it did the same thing.
Our intuition is that there might be a memory leak or some bug in OIM that might use up almost all the free memory and it's not releasing it upon shutdown. It might have been there before we increased the memory but have not noticed it since memory was already tight.
Anyone encountered the same problem ? Any idea ? Any suggestion to narrow down the problem ?
Thank youYou can adjust the memory settings for WLS by editing the setSOADomainEnv.sh file that can be found in your /middleware/user_projects/domains/<domain>/bin/ folder. There is an argument called PORT_MEM_ARGS which is used to set your Java memory arguments. This way you can decrease/increase the amount of memory used by each managed server.
I usually type "ps -ef | grep oracle" to see what processes are running by the oracle user. This way the output doesn't get cluttered with root processes.
Sunny Tsang -
XML,CLOB AND MEMORY : CONSUMED BUT NOT RELEASED !!!
Hi,
I'm working with XMLGEN Package and XSLT Processor to produce XML Document on Oracle 8.1.7.3 server, I use 9i XML and Java packages.
I'm facing the following BIG MEMORY problem :
Environment : I must generate an XML parsed with an XSL document of 80Mo (nearly 22 000 rows),
as XMLGEN is on DOM parsing method, I extract my XML by 500 Rows and I loop until 22 000 rows. I use DBMS_JOB with jobs who read and execute export each minute.
The algorithme is :
keeprow=22000
while keeprow>0
Create 3 clob (DBMS_LOB Package) : one for XSL Sheet, one for XML and one for result
GetXML (XMLGEN Package)
Transform in XSL (XSL Processor)
Write to disk (UTL_FILE Package)
Free the 3 Clob ((DBMS_LOB Package))
keeprow =keeprow-500
loop
The problem : The process start at 250Mo ot total memory and END at 1000 Mo of used memory and NEVER RELEASE 1 ko of memory.
So often I get a JavaOutOfMemoryError (I've allocated 1Go Ram to my JAVA process).
Any help will be very very appreciated !
My derived problem is 22 000 rows is not enough I've some export of 200 000 rows to do (And I cannot allocate 10 Go of RAM !!!)
Following My PL/SQL Code.
Regards
Fred
PROCEDURE DO_EXPORT_XML(
P_JOB_ID JOB_PARAMETRE.JOB_ID%TYPE,
P_JOB_ID_ORDRE JOB_PARAMETRE.JOB_ID_ORDRE%TYPE,
P_CLE_UP UPLOADREQ_TEMP.ID%TYPE,
P_LOAD_OR_DELOAD VARCHAR2)
IS
L_FILE_NAME JOB_PARAMETRE.JOB_FILE_NAME_DEST%TYPE;
L_REP_NAME JOB_PARAMETRE.JOB_REP_NAME_DEST%TYPE;
L_FILE_STYLESHEET JOB_PARAMETRE.JOB_STYLESHEET%TYPE;
L_VERSION_PDM JOB_PARAMETRE.JOB_VPDM%TYPE;
P_SELECT varchar2(4000):='';
P_CURSOR varchar2(4000):='';
l_filehandler_out UTL_FILE.FILE_TYPE;
--Variable pour le traitement par lot de 500
L_NBROW_TODO_ATONCE number := 500;
L_NBROW_MIN number := 1;
L_NBROW_MAX number := -1;
L_NBROWKEEPTODO number := -1;
xslString CLOB := null;
res number default -1;
xmlString CLOB := null;
li_ret number := 0;
li_faitle number := 0;
amount integer:= 255;
li_loop integer := 0;
charString varchar2(255);
ls_deload varchar2(255) default '';
ls_SQL varchar2(4000) default '';
ls_temp_file varchar2(255) default '';
text_file_dir varchar2(32) := 'e:\temporarydir';
l_par xmlparser.parser;
l_xml xmldom.domdocument;
l_pro xslprocessor.processor;
l_xsl xslprocessor.stylesheet;
docfragnode xmldom.DOMNode;
docfrag xmldom.DOMDocumentFragment;
l_parsedclob clob := null;
l_amount binary_integer := 32767;
l_ligne varchar2(32767);
l_offset number default 1;
l_pos number default null;
l_pos2 number default null;
l_lobsize number default null;
l_memsize number default 1073741824; --1024 Mo
l_mempipo number default 0;
type rc is ref cursor;
l_cursor rc;
cursor TEMPLATE is select UNSPSC,1 as NB from UPLOADREQ_TEMP where 1=2;
c1rec TEMPLATE%rowtype;
BEGIN
l_mempipo:=setmaxmemorysize(l_memsize);
dbms_lob.createtemporary(l_parsedclob, true, dbms_lob.session);
dbms_lob.createtemporary(xmlstring, true, dbms_lob.session);
--return the good select
GET_SELECT_TO_EXPORT_XML(P_JOB_ID , P_JOB_ID_ORDRE , P_CLE_UP , P_SELECT,P_CURSOR, P_LOAD_OR_DELOAD);
SELECT JOB_FILE_NAME_DEST,JOB_REP_NAME_DEST,JOB_STYLESHEET
INTO L_FILE_NAME,L_REP_NAME,L_FILE_STYLESHEET
FROM JOB_PARAMETRE
WHERE JOB_ID =P_JOB_ID AND JOB_ID_ORDRE=P_JOB_ID_ORDRE;
l_filehandler_out := UTL_FILE.FOPEN(text_file_dir, L_FILE_NAME, 'w',l_amount);
--Return XSL Sheet in a clob : cause of memory consumed but not released
xslString := METACAT.load_a_file( 1,L_FILE_STYLESHEET);
open l_cursor for P_CURSOR;
LOOP
fetch l_cursor into c1rec;
exit when l_cursor%notfound;
L_NBROW_MIN := 1;
L_NBROW_MAX := 0;
L_NBROWKEEPTODO:=c1rec.NB;
LOOP
begin
if(L_NBROWKEEPTODO > L_NBROW_TODO_ATONCE) THEN
begin
L_NBROW_MAX:= L_NBROW_TODO_ATONCE + L_NBROW_MAX;
L_NBROWKEEPTODO:= L_NBROWKEEPTODO - L_NBROW_TODO_ATONCE;
end;
else
begin
L_NBROW_MAX:= L_NBROW_MAX + L_NBROWKEEPTODO;
L_NBROWKEEPTODO:=0;
end;
end if;
--on ouvre le fichier de risultats
ls_SQL:= P_SELECT || ' AND ( ROWNUM BETWEEN ' || L_NBROW_MIN || ' AND ' || L_NBROW_MAX || ' ) and UNSPSC=''' || c1rec.UNSPSC || '''';
ls_temp_file := c1rec.UNSPSC || '_' || L_FILE_NAME;
L_NBROW_MIN:=L_NBROW_TODO_ATONCE + L_NBROW_MIN;
--CAT_AUTOLOAD.JOB_ADD_TRACE (P_JOB_ID,'UPLOAD REQUISITE : Export donnies REQUETE ' || to_char(li_loop), ls_SQL,'',0,0);
xmlgen.resetOptions;
xmlgen.setErrorTag('ERROR_RESULT');
xmlgen.setRowIdAttrName('NAH');
xmlgen.setRowIdColumn('NAH');
xmlgen.setEncodingTag('ISO-8859-1');
xmlgen.useNullAttributeIndicator(false);
if(xmlString is not null) then
dbms_lob.open(xmlString,dbms_lob.lob_readwrite);
l_lobsize:= dbms_lob.Getlength(xmlString);
if(l_lobsize>0) then
dbms_lob.erase(xmlString,l_lobsize,1);
end if;
dbms_lob.close(xmlString);
dbms_lob.freetemporary(xmlString);
dbms_lob.createtemporary(xmlstring, true, dbms_lob.session);
end if;
--Return XML in a clob : cause of memory consumed but not released
xmlString := xmlgen.getXML(ls_SQL,0);
l_par := xmlparser.newparser;
xmlparser.parseclob(l_par, xslString);
l_xsl := xslprocessor.newstylesheet(xmlparser.getdocument(l_par),null);
xmlparser.parseclob(l_par, xmlString);
l_xml := xmlparser.getdocument(l_par);
l_pro := xslprocessor.newprocessor;
xslprocessor.showWarnings(l_pro, true);
xslprocessor.setErrorLog(l_pro, text_file_dir || substr(ls_temp_file,0,length(ls_temp_file)-4) || '_logerreur.XML');
if(l_parsedclob is not null) then
dbms_lob.open(l_parsedclob,dbms_lob.lob_readwrite);
l_lobsize:= dbms_lob.Getlength(l_parsedclob);
if(l_lobsize>0) then
dbms_lob.erase(l_parsedclob,l_lobsize,1);
end if;
dbms_lob.close(l_parsedclob);
dbms_lob.freetemporary(l_parsedclob);
dbms_lob.createtemporary(l_parsedclob, true, dbms_lob.session);
end if;
--Return XML Processed with XSL in a clob : cause of memory consumed but not released
xslprocessor.processxsl(l_pro,l_xsl,l_xml,l_parsedclob);
--release NOTHING
xmlparser.freeparser(l_par);
xslprocessor.freeprocessor(l_pro);
l_ligne:='';
l_offset :=1;
l_pos := null;
l_pos2 := null;
if(li_loop=0) then
begin
--on ouvre le fichier et on sauve l'entete + les donnies dedans.
l_pos:=dbms_lob.instr(l_parsedclob,'</DATA>');
if ( nvl(l_pos,0) > 0 ) then
loop
if(l_pos-1>l_amount + l_offset ) then
l_ligne:=dbms_lob.SUBSTR(l_parsedclob,l_amount,l_offset);
UTL_FILE.PUT(l_filehandler_out,l_ligne);
UTL_FILE.fflush(l_filehandler_out);
l_offset:=l_offset+l_amount;
else
l_ligne:=dbms_lob.SUBSTR(l_parsedclob,l_pos-1 -l_offset ,l_offset);
UTL_FILE.PUT(l_filehandler_out,l_ligne);
UTL_FILE.fflush(l_filehandler_out);
exit;
end if;
end loop;
else
EXIT;
end if;
end;
else
--on met les donnies donc on ne repete pas le debut
begin
l_pos:=dbms_lob.instr(l_parsedclob,'<ITEM');
if ( nvl(l_pos,0) > 0 ) then
l_pos2:=dbms_lob.instr(l_parsedclob,'</DATA>');
if ( nvl(l_pos2,0) > 0 ) then
loop
if(l_pos + l_amount <= l_pos2 -1 ) then
l_ligne:=dbms_lob.SUBSTR(l_parsedclob,l_amount,l_pos);
UTL_FILE.PUT(l_filehandler_out,l_ligne);
UTL_FILE.fflush(l_filehandler_out);
l_pos:=l_pos +l_amount;
else
l_ligne:=dbms_lob.SUBSTR(l_parsedclob,l_pos2 -1 -l_pos,l_pos);
UTL_FILE.PUT(l_filehandler_out,l_ligne);
UTL_FILE.fflush(l_filehandler_out);
exit;
end if;
end loop;
else
exit;
end if;
end if;
end;
end if;
li_loop:=li_loop + 1 ;
--UTL_FILE.FCLOSE(l_filehandler_in);
JAVA_GC();
EXIT WHEN L_NBROWKEEPTODO=0;
Exception
when others then
begin
-- IF(utl_file.is_open(l_filehandler_in)) THEN
-- utl_file.fclose( l_filehandler_in);
-- END IF;
IF(utl_file.is_open(l_filehandler_out)) THEN
utl_file.fclose( l_filehandler_out);
END IF;
RAISE_APPLICATION_ERROR(-20001,'File with errors');
end;
END;
END LOOP;
END LOOP;
CLOSE l_cursor;
if ( xmlString is not null ) then
dbms_lob.open(xmlString,dbms_lob.lob_readwrite);
l_lobsize:= dbms_lob.Getlength(xmlString);
if(l_lobsize>0) then
dbms_lob.erase(xmlString,l_lobsize,1);
end if;
dbms_lob.close(xmlString);
dbms_lob.freeTemporary( xmlString);
end if;
if(l_parsedclob is not null) then
dbms_lob.open(l_parsedclob,dbms_lob.lob_readwrite);
l_lobsize:= dbms_lob.Getlength(l_parsedclob);
if(l_lobsize>0) then
dbms_lob.erase(l_parsedclob,l_lobsize,1);
end if;
dbms_lob.close(l_parsedclob);
dbms_lob.freetemporary(l_parsedclob);
end if;
UTL_FILE.NEW_LINE(l_filehandler_out);
l_ligne:='</DATA></CATALOG>';
UTL_FILE.PUT(l_filehandler_out,l_ligne);
UTL_FILE.FCLOSE(l_filehandler_out);
EXCEPTION
when others then
begin
IF(utl_file.is_open(l_filehandler_out)) THEN
utl_file.fclose( l_filehandler_out);
END IF;
end;
END;
******************************Thank you for the info - I had no idea I was puing myself in danger by cutting it so close. Since your post I have moved my iphoto library to an external drive and now have 165 GB of space on my HD. Following this I have 2 questions.
1. Since my available HD space was reduced by the size of the photo download it seems logical that the download is somewhere on my HD still. Is there a place where these photos might be hiding on my HD even though they are not available on the iphoto library?
2. I was able to recover the .jpg files which are fine. I also recovered the .mov files but they have been compromised. I am hoping I can find the originals still on the HD somewhere. If not, do you have any suggestions for recovery methods or programs? I have not used the SD card since the incident so I should be able to attempt another recovery to salvage the .mov files if there is an alternative method/program available.
Thanks again! -
Garbage Collection not releasing memory properly
Hello, In my web application memory is not releasing properly, after some point of time application throw out of memory error, we are using tomcat5, jdk5, my heap size is half of the RAM.Garbage Collection not releasing memory properly.
sabre150 wrote:
punter wrote:
georgemc wrote:
Yep. Couldn't be a problem with your code, it must be that garbage collection doesn't work and nobody else noticed. Raise a bug with Sun.Are you serious ?You need to replace your sarcasm detector.I have blowed my sarcasm meter and limping in the blindness !!!! :( -
Revision: 2692
Author: [email protected]
Date: 2008-07-31 13:05:35 -0700 (Thu, 31 Jul 2008)
Log Message:
Bug: BLZ-227 - When using JMS Destination, MessageClient and FlexClient not released from memory when the session times out.
QA: Yes
Doc: No
Checkintests: Pass
Details: Fixed a memory leak with JMS adapter. Also a minor tweak to QA build file to not to start the server if the server is already running.
Ticket Links:
http://bugs.adobe.com/jira/browse/BLZ-227
Modified Paths:
blazeds/branches/3.0.x/modules/core/src/java/flex/messaging/services/messaging/adapters/J MSAdapter.java
blazeds/branches/3.0.x/qa/build.xmlRevision: 2692
Author: [email protected]
Date: 2008-07-31 13:05:35 -0700 (Thu, 31 Jul 2008)
Log Message:
Bug: BLZ-227 - When using JMS Destination, MessageClient and FlexClient not released from memory when the session times out.
QA: Yes
Doc: No
Checkintests: Pass
Details: Fixed a memory leak with JMS adapter. Also a minor tweak to QA build file to not to start the server if the server is already running.
Ticket Links:
http://bugs.adobe.com/jira/browse/BLZ-227
Modified Paths:
blazeds/branches/3.0.x/modules/core/src/java/flex/messaging/services/messaging/adapters/J MSAdapter.java
blazeds/branches/3.0.x/qa/build.xml
Maybe you are looking for
-
Sending mail to internal mailadres to external smtp
Hi, I must have 1 email address from my email-domain (intern and external) that always send the mails to the external smtp-server. example: mail domain is novellrocks.com email address create is [email protected] So when we send internal our external
-
Sales order costing and valuated sales order stocks
SAP Gurus Can somebody explain the key configurations and account postings in sales order costing and valuated sales order stocks in product costing using the specific tcodes.How the process moves from one configuration to the other? Use of examples
-
Hello, I have a condition type the value of which is depending on the value of Condition Group1 found in the Additional data A tab in the Header. When creating the sales Order I manually enter the value in the Condition Group1 field. The problem here
-
Hide dynamic data if its not available
Hi, I've something like this: User Name: <cfoutput>#profile.username#</cfoutput> I want to hide it when the data is not available. I only want to hide this row nothing else thanks.
-
Capturing Microphone from Client- Side using C# Webform
Is their any way to capture audio from any device through a web form using C# and HTML5 ? Rogelio Rios