Constant increase of resident memroy while heap usage is constant

Why is it that the memory use (resident memory reported by top) grows to be much larger than the heap? I have a Linux Centos 5 server running JBoss 5 with JDK1.6_29. We are using the latest collectors and a maximum heap of 12GB. I can see that the young generation is never larger than 3.5 GB, the Old Generation is never larger than 300MB, adn the Perm Gen is never alrger than 100 MB. I can see the heap being collected when it aproaches 3.8GB and the JVM has set limits of 4GB and 8GB fro the young and old gen.
My settigns are as follows:
JAVA_OPTS="-d64 -Xms12288m -Xmx12288m -XX:MaxPermSize=512m -XX:+UseParallelGC -XX:+UseParallelOldGC -XX:GCTimeRatio=19 -XX:+AggressiveOpts -Dsun.lang.ClassLoader.allowArraySyntax=true -Dorg.jboss.resolver.warning=true -Dsun.rmi.dgc.client.gcInterval=3600000 -Dsun.rmi.dgc.server.gcInterval=3600000"
The physical memory of the server is about is about 16GB. When I re-start JBoss, the resident memory is about 4GB and during the week, it increases steadly up to 8.4GB at which pointwe start to see that the OS starts paging into swap. Is the resident memory mainly non-heap or for soft references, or both?
Why should the server page anything when the resident memory of the JBoss process reaches about 8.5GB whn there is still another 8GB of total memory available to the OS?

The answer to all of the above comes from understanding, in detail, how virtual memory works and what impact the management (by the OS) of that has on various tools.
You might want to start by research exactly what top tells you and what it doesn't.

Similar Messages

Crystal Report export to PDF cause high Heap usage ?

Hi all,
As part of our reporting integrated with our JSF/JSP application, Crystal report is converted to PDF then sent to browser for user to display. mean while during peak load our Heap usage could reach 3.5GB - 4GB. So I am suspecting the unclosed byteArrayInputStream is the cause.
(This is a production application so I am collecting information before change the code)
Is the unclosed() byteArrayInputStream really cause the problem ? (the codes is below)
Thank you,
Krist
ByteArrayInputStream byteArrayInputStream = (ByteArrayInputStream)
                                 reportClientDoc.getPrintOutputController().export(exportOptions);
reportClientDoc.close();
writeToBrowser(byteArrayInputStream, response, "application/csv", EXPORT_FILE);
private void writeToBrowser(ByteArrayInputStream byteArrayInputStream, HttpServletResponse
response, String mimetype, String exportFile)
   throws Exception {
      byte[] buffer = new byte[byteArrayInputStream.available()];
      int bytesRead = 0;
      response.reset();
      response.setHeader("Content-disposition", "inline;filename=" + exportFile);
      response.setContentType(mimetype);
      //Stream the byte array to the client.
      while((bytesRead = byteArrayInputStream.read(buffer)) != -1)
            { response.getOutputStream().write(buffer , 0, bytesRead);}
      //Flush and close the output stream.
      response.getOutputStream().flush();
      response.getOutputStream().close();

I do not know if my solution to my Heap problem will help any of you but I thought I would just post it here
just incase yourselves or others come looking for possible solutions.
I created a very simlpe report with 2 groups and not much in the way of complex functions. Whilst reporting against about
100 pages of output everything worked fine, but as soon as we pushed the report up to 500+ pages we got all sorts
of issues.
java.lang.OutOfMemoryError: Java heap space
After much hair pulling and trial and error I discovered that the issue came about where I did not declare formula variables as local. I was concatinating various street address details for of the envelope windows.
Stringvar Address;      //    I was using this declaration
Global Stringvar Address;    // Specific Global declaration
Local Stringvar Address;    // Changed to this declaration
After changing to Local, my report now runs with no hassels. And the memory usage whilst exporting the report has gone from maxing out at over 1GB to almost nothing (dont even get it registering)
Am sure someone can come up with a better explanation for this and give reasons but just thougth I would share.
Cheers
Darren

Could this unclosed() byteArrayInputStream cause high Heap usage ?

Hi all,
As part of our reporting integrated with our JSF/JSP application, the
report is converted to PDF then sent to browser for user to display.
mean while during peak load our Heap usage could reach 3.5GB - 4GB. So
I am suspecting the unclosed byteArrayInputStream is the cause.
(This is a production application so I am collecting information
before change the code)
Is the unclosed() byteArrayInputStream really cause the problem ?
(the codes is below)
Thank you,
Krist
ByteArrayInputStream byteArrayInputStream = (ByteArrayInputStream)
reportClientDoc.getPrintOutputController().export(exportOptions);
reportClientDoc.close();
writeToBrowser(byteArrayInputStream, response, "application/csv",
EXPORT_FILE);
private void writeToBrowser(ByteArrayInputStream byteArrayInputStream,
HttpServletResponse
response, String mimetype, String exportFile)
throws Exception {
byte[] buffer = new byte[byteArrayInputStream.available()];
int bytesRead = 0;
response.reset();
response.setHeader("Content-disposition", "inline;filename=" +
exportFile);
response.setContentType(mimetype);
//Stream the byte array to the client.
while((bytesRead = byteArrayInputStream.read(buffer)) != -1)
{ response.getOutputStream().write(buffer , 0,
bytesRead);}
//Flush and close the output stream.
response.getOutputStream().flush();
response.getOutputStream().close();
}

xtanto wrote:
Is the unclosed() byteArrayInputStream really cause the problem ? Source code from 1.6.12
class ByteArrayInputStream extends InputStream {
     * Closing a <tt>ByteArrayInputStream</tt> has no effect. The methods in
     * this class can be called after the stream has been closed without
     * generating an <tt>IOException</tt>.
     * <p>
    public void close() throws IOException {
    }

Keeping Heap Usage lower than a given threshold level

Hello,
We are trying to find out which are the most suitable parameters for the gc for our application.
We have an application using a 1GB heap.
We have tried different combinations of parameters, such as generational gc vs single spaced, different nursery sizes, dynamic gc with priority on memory throughput vs pause time, â€¦
We plot the usage of the heap before and after the gc has run.
There is one common thing that we see with all the different configurations: The heap usage always reaches levels of around 95% of the heap size.
We would like to find a way to reduce the heap usage (even if it increases the pause times. If the pause time is higher, we will analyze the final impact on our application responsiveness and decide afterwards).
The ideal would be to find a way to keep it under a given threshold level (letâ€™s say, 70% of the total heap size).
Your ideas will be very welcome.
Thanks and regards,
Ramiro

I'm not sure I understand your question, but I'll give you an answer and
you can tell me if it helped :-).
Basically the gc doesn't trigger until the heap is full. For
generational GCs, nursery collections will trigger when the nursery is
full. Some objects get promoted, and when the heap is entirely full an
old-space collection is performed.
So you can't really get JRockit to gc before the heap is full.
If you want to know how small heap you can use, look at how much space
is used after a full gc. Add some marigin of error, then use that value
for -Xmx.
I don't know what tool you're using for plotting the heap usage, but you
could use JRockit's Management Console (bin/jrcc). Also, if you want to
know where your heap memory is going, try the memleak detector from
http://dev2dev.bea.com/jrockit/tools.html.
Regards //Johan
Ramiro Alcazar wrote:
Hello,
We are trying to find out which are the most suitable parameters for the gc for our application.
We have an application using a 1GB heap.
We have tried different combinations of parameters, such as generational gc vs single spaced, different nursery sizes, dynamic gc with priority on memory throughput vs pause time, ???
We plot the usage of the heap before and after the gc has run.
There is one common thing that we see with all the different configurations: The heap usage always reaches levels of around 95% of the heap size.
We would like to find a way to reduce the heap usage (even if it increases the pause times. If the pause time is higher, we will analyze the final impact on our application responsiveness and decide afterwards).
The ideal would be to find a way to keep it under a given threshold level (let???s say, 70% of the total heap size).
Your ideas will be very welcome.
Thanks and regards,
Ramiro

Capturing the JVM heap usage information to a log

When using weblogic 6.1sp3 the console under monitoring/performance a graph is
displayed with the historical JVM heap usage information. Is there any way to
capture this information to a log?

For heap size before and after each gc, you could pass the -verbose:gc option to the JVM
on startup:
WLS C:\alex>java -verbose:gc weblogic.Admin PING 10 10
[GC 512K->154K(1984K), 0.0068905 secs]
[GC 666K->164K(1984K), 0.0069037 secs]
[GC 676K->329K(1984K), 0.0029822 secs]
[GC 841K->451K(1984K), 0.0038960 secs]
[GC 963K->500K(1984K), 0.0015452 secs]
[GC 1012K->598K(1984K), 0.0027509 secs]
[GC 1110K->608K(1984K), 0.0029370 secs]
[GC 1120K->754K(1984K), 0.0027361 secs]
[GC 1266K->791K(1984K), 0.0019639 secs]
[GC 1303K->869K(1984K), 0.0028314 secs]
[GC 1381K->859K(1984K), 0.0012957 secs]
[GC 1367K->867K(1984K), 0.0012504 secs]
[GC 1379K->879K(1984K), 0.0018592 secs]
[GC 1391K->941K(1984K), 0.0036871 secs]
[GC 1453K->988K(1984K), 0.0027143 secs]
Sending 10 pings of 10 bytes.
RTT = ~47 milliseconds, or ~4 milliseconds/packet
Looks like it might be too much info though...
Cheerio,
-alex
Fazle Khan wrote:
When using weblogic 6.1sp3 the console under monitoring/performance a graph is
displayed with the historical JVM heap usage information. Is there any way to
capture this information to a log?

Does entity cache cause high heap usage ? better setClearCacheOnCommit ?

Hi all,
During peak load (150-200 users) of our production ADF application (10.1.3.3), the heap usage can be reach 3GB, causing JVM very busy doing frequent GC.
Is this possibly because the 'by default uncleared' entity cache ?
What is the implication is I do 'setClearCacheOnCommit()' ?
Thank you for your help,
xtanto

The EO cache will be cleared when the AM is released in stateless mode. By default that would occur when your web session times out, but you can eagerly release it in stateless mode (when the user is finished with the task that uses that AM).
Using setClearCacheOnCommit() will more eagerly clear the EO cache, however doing so will clear the VO caches, too, for the VOs related to those EOs so it may end up causing more database requerying than you were doing before. Effectively, after a commit you'll need to requery any data that's needed for the subsequent pages the user visits. If your work flow is such that the user does not do a commit and then continue processing other rows that you've already queried, then it might be an overall slight win on memory usage, however if the user does issue a commit (say, from an Edit form) and then return back to a "list" page to process some other record, doing a clearCacheOnCommit=true will force your list page to requery the data (which it's not doing now when the entity cache isn't been eagerly cleared)
So, like many performance-related question, it depends on exactly what your app is doing.

Computer slows down after a while of usage

I just purchased a new Mac Mini (the one with no optical drive) & I noitce after a while of usage it starts slowing down. The most mundane tasks can take minutes (like closing a window) when it goes into its "slow down" mode. It's like it's moving through molassis. Before this, I had a Macbook laptop & I never had slow downs like this. I would have thought "newer technology" would translate into better but not worse functioning. I have 4 gig of ram. I do use Photoshop, which is a ram hog. But like I said, I had this same configuration with the Macbook & no problems. What has changed other than the computer to the Mac Mini. Any thoughts on how I might remedy this? Thanks.

I too am having that same issue and I have had mine only a week. Mine is the bare minimum ordered for the new release.

HCM Interactive forms - operation error while applying usage rights

Hi!
We have setup several scenarios using the new HCM Interactive forms on ERP6.0 and EP7.0. The scenarios include things like Transfer, pay change request.
In 9 out of 10 cases everything thing is working as expected so that the users can complete the forms and the process for their requests but occasionally we are seeing a random error that occurs for any user at any time without any apparent pattern.
The below is what is seen from the log file
Exception Stack Trace:
com.adobe.ProcessingError: PDF operation error while applying usage rights to PDF: /tmp/adobewa_PEP_17116350/DM-6192899546130604164.dir/DM8253967776737558967.tmp
Specific error information:
this document is signed
     at com.adobe.ads.request.UsageRights.execute(Unknown Source)
     at com.adobe.BaseADSRequest.doWork(Unknown Source)
     at com.adobe.AdobeDocumentServicesWorker.execute(Unknown Source)
     at com.adobe.AdobeDocumentServicesEJB.processRequest(Unknown Source)
     at com.adobe.AdobeDocumentServicesEJB.rpData(Unknown Source)
     at com.adobe.AdobeDocumentServicesLocalLocalObjectImpl0_0.rpData(AdobeDocumentServicesLocalLocalObjectImpl0_0.java:120)
     at sun.reflect.GeneratedMethodAccessor403.invoke(Unknown Source)
     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:62)
     at java.lang.reflect.Method.invoke(Method.java:391)
     at com.sap.engine.services.webservices.runtime.EJBImplementationContainer.invokeMethod(EJBImplementationContainer.java:126)
     at com.sap.engine.services.webservices.runtime.RuntimeProcessor.process(RuntimeProcessor.java:157)
     at com.sap.engine.services.webservices.runtime.RuntimeProcessor.process(RuntimeProcessor.java:79)
     at com.sap.engine.services.webservices.runtime.servlet.ServletDispatcherImpl.doPost(ServletDispatcherImpl.java:92)
     at SoapServlet.doPost(SoapServlet.java:51)
     at javax.servlet.http.HttpServlet.service(HttpServlet.java:760)
     at javax.servlet.http.HttpServlet.service(HttpServlet.java:853)
     at com.sap.engine.services.servlets_jsp.server.HttpHandlerImpl.runServlet(HttpHandlerImpl.java:401)
     at com.sap.engine.services.servlets_jsp.server.HttpHandlerImpl.handleRequest(HttpHandlerImpl.java:266)
     at com.sap.engine.services.httpserver.server.RequestAnalizer.startServlet(RequestAnalizer.java:387)
     at com.sap.engine.services.httpserver.server.RequestAnalizer.startServlet(RequestAnalizer.java:365)
     at com.sap.engine.services.httpserver.server.RequestAnalizer.invokeWebContainer(RequestAnalizer.java:944)
     at com.sap.engine.services.httpserver.server.RequestAnalizer.handle(RequestAnalizer.java:266)
     at com.sap.engine.services.httpserver.server.Client.handle(Client.java:95)
     at com.sap.engine.services.httpserver.server.Processor.request(Processor.java:175)
     at com.sap.engine.core.service630.context.cluster.session.ApplicationSessionMessageListener.process(ApplicationSessionMessageListener.java:33)
     at com.sap.engine.core.cluster.impl6.session.MessageRunner.run(MessageRunner.java:41)
     at com.sap.engine.core.thread.impl3.ActionObject.run(ActionObject.java:37)
     at java.security.AccessController.doPrivileged(AccessController.java:215)
     at com.sap.engine.core.thread.impl3.SingleThread.execute(SingleThread.java:100)
     at com.sap.engine.core.thread.impl3.SingleThread.run(SingleThread.java:170)
Caused by: com.adobe.document.pdf.PDFOperationFailure: IDL:com/adobe/document/pdf/PDFOperationFailure:1.0
     at com.adobe.EJB_PDFAgent.linearize(Unknown Source)
     at com.adobe.EJB_PDFAgent.setUsageRights(Unknown Source)
     ... 30 more
I am in the processes of raising a message to SAP about this but was wondering if any one else had experienced a similar error or knows what might cause this type of error.
Thanks
Matt

We had the same problem. The fix is to get a unique ID for every form. Given below
Instructions to generate unique ID
1. Modify template
Edit your template via t-code SFP
-> goto Layout tab -> Adobe LiveCycle Designer will be started
-> Goto 'XML source' tab
in line2, you will find following entry
<?xfa generator="AdobeLiveCycleDesigner_V8.0_SAP" APIVersion="2.5.8227.0"?>
<xdp:xdp xmlns:xdp="http://ns.adobe.com/xdp/" timeStamp="2008-09-11T14:
39:44Z" uuid="4ed773ff-6aeb-483e-a5ec-4af1a138c775">
from this line, delete uuid="..." part.
Go back to 'Design view' tab
Check if layout is not changed.
Save / activate template
2. Delete database table entry for table ADS_Licensed_forms
Schema SAP<SID>DB -> table ADS_LICENSED_FORMS
(you can delete all table entry from this table)
3. When you access the forms, they are aitomatically registered in this table.
Hope this helps.
Sunil

Find out current old heap usage from within the process

Hello!
We use the CMS garbage collector and need a way to find out how much memory is used of the old heap by reachable objects. This we have to do from within the process (not using jvmstat or jstat etc.).
Since there is no way to distinguish between reachable and non-reachable objects (except for traversing the entire heap... -- or are there other possibilities?) our idea is to get the amount of used memory right after a garbage collection in the old heap.
Using Java 1.5, this can be done by
java.lang.management.MemoryPoolMXBean pool = <Pool for Old Generation>;
pool.getUsage().getUsed();However, java.lang.management is only available in Java 1.5.
Therefore my first question: Is there a similar way of finding out old heap usage in Java 1.4?
There is another problem with this method: By calling pool.getUsage().getUsed();, one has to know when a GC has occurred (this could be done by calling it in an interval of x seconds -- if the current value is lower than the one before, a GC must hava occurred). A better way would be to use pool.getCollectionUsage().getUsed();, but this seems not to work for the CMS collector.
Second question: Is pool.getCollectionUsage().getUsed(); really not working with CMS, or are we just doing it in a wrong way? Are there other ways of finding out the used memory in the old heap after a GC even when using the CMS?
Thanks for any help!
Regards,
Nicolas Michael

Hi Nicolas,
There is no API in 1.4 to get the after GC memory usage of the old generation. The only thing close to it is (Runtime.totalMemory - Runtime.freeMemory) but it is the approx amount of memory used for the heap (not just the old generation).
MemoryPoolMXBean.getCollectionUsage() returns the after GC MemoryUsage. This method should work for all collectors. I have a simple test case that shows it working fine with CMS. It shows the same value as the -XX:+PrintGCDetails shows.
If you have a test case showing that this method doesn't work correctly, please submit a bug along with the test case. We'll investigate it.
Thanks
Mandy

High heap usage even when system idle, is this caused by AMPool setting ?

Hi All,
We are running ADF BC 10.1.3.3 with 2 JVM OC4J instance. Max heap is 3.5 GB.
I notice that when system is idle, the heap usage for each JVM can be between 1.5 to 2 GB. It is confusing me because relatively no user is accessing the application.
Then I am suspecting that the setting in Apps Module Pool may be the cause, i.e :
<jbo.recyclethreshold>50</jbo.recyclethreshold>
<jbo.ampool.maxavailablesize>90</jbo.ampool.maxavailablesize>
<jbo.ampool.minavailablesize>25</jbo.ampool.minavailablesize>
Is there anything wrong with the setting that may cause AM Pool Monitor busy doing wrong thing ?
Thank you for your help,
xtanto

mcmillan wrote:
rohankolay wrote:Right now i have kernel26-2.6.35.4 which i updated on 27 aug. That is around the same time the problem began.Should i downgrade it to kernel26-2.6.35.3 or kernel26-2.6.34.3?
I think most people have been seeing the problems in any of the 2.6.35 kernels, so 2.6.34.3 would be my recommendation
+1
And put kernel26 in IgnorePkg.

Websphere/Oracle 11 - much more Heap Usage than with Oracle 10

Hi all,
while testing our application with Oracle 11 (previously, we had Oracle 10), we saw that our server uses much more heap space.
It seems as it has something to do with T4CConnection/T4CPreparedStatement; there are 500 objects of T4CPreparedStatement allocated. Someone told me, that Oracle 11 is using SoftReferences to keep the connection pool; but we don't need that.
Is that correct? Could that be the problem for the increased heap space? If yes - how can we avoid connection pooling?
Thanks a lot!!
Edited by: 840550 on 28.02.2011 23:37

I suspect you are getting a connection for each user and holding onto it for the duration that he is logged in. Also, that you are not closing resultSet and preparedStatements correctly. Rewrite all your code to get/use/close a connection each time its needed as quickly as possible in a try/catch/finally block. There is no performance penalty opening and closing connections countless times in your code (actually, you are not really closing it, but returning it to the pool).
Here is a previous post I provided on how to get/use/close a connection:
Re: can not get the right query result using JDBC
Note: within the above example, you can pass the already open connection to another function within the above function for it to use. Within that other function, you create a new preparedStatement and resultSet. After using it, you close the resultSet and preparedStatement (in that second function), but not the connection (its closed in the outer function).

Heap usage

Hi
I am using JRockit JVM. I see during load tests that heap s used almost entirely and then gc happens. GC times are still in milliseconds and i do not see out of memory issue. I would like to knw if almost 99% of heap consumption calls for increase in heap size ? I checked the heap for memory leaks too. Couldnt find anythin out of usual
Thanks
Krupa

If the heap is 'used' then GC isn't going to be doing much. Conversely if the heap is cleaned up after a gc thus the active objects are using much less space then there is no problem.
Given that you are not get an out of memory problem and if you are testing the application in a way that mimics production usage for that then it would suggest there is no problem.

After upgrading to Lion, the Finder will increase the virtual memory while using external HD

after upgrading to Lion, the Finder will increase the "virtual memory size" while using external HD. I check it using Activity Monitor and found that "Finder" wll increase the "Virtual Memory Size" and "Private Memory Size". and then after a while. the system told me my HD is full and hanging there.
My External HD is 1TB. and I have installed new version Paragon NTFS for Mac V.9.01. I am not sure it is related or not.
but I attached my external HD. the "Finder" will start to increase the "Virtual Memory Size".
please help out this case and solve this issue.
thank you !!!

The one has nothing to do with the other. Virtual memory size and Private memory size are used by developers to analyze their software. it has nothing to do with anything related to the user.

Error while doing usage decison using moment type 321

Hi,
I have created a batch by confirming process order
(t code COR6). But when I do usage decision for this batch using transaction QA11, material stock from batch created by order gets transferred to new batch.
When I checked material document, there is complimentory movement for batch created with process order.
For ex.
I have created batch 000400 by confirmation of process order.
Batch stock is in quality stock. Now I want to move this stock to unrestricted, for this I did usage decision using transaction code QA11. After usage decision when I checked material documents then I found that there is only one movement for 321 movement for batch 000400 i.e. for debit no credit (that means there is no complimentary moment) & credit movement exists for new batch 000401 which is created by sytem itself.
My requirement is that system should not create new batch for material posting while using moment type 321, it shoud post material stock to same batch after usage decision.
It is urgent issue.

In the SAP IMG configuration,
Material Management ->Inventory Management and Physical Inventory -->Movement types --> Copy and Change Movement type(OMJJ)
In the field screen, select movement type and then "321" and getinto the definition.
Please verify the field "Create new batch" is empty.
Other reason could be user-exit or custom code which during material movement does the new batch creation.
Thanks,
Srinivas Karri

Re: Increasing Column Header Length While Printing in ALV Grid

Hi
For some column headers in a report, the translations in Turkey are more than 40 characters, because of which while executing the report in TR language, the column header is getting truncated.
So, I would like to know if we can increase the column header length while printing the ALV Grid(OO ALV) output.
Or else, Is there any way where we can divide the column header into 2 rows.
Please let me know.
Thanks and Regards,
Vishwa.

Jose and Sivaram,
Thanks for replying.
The problem we have is with the column headers, there is a limit of 40 characters for the column header in ALV reports. So, increasing the outputlen also did not help.
Regards,
Vishwa.

Constant increase of resident memroy while heap usage is constant

Similar Messages

Maybe you are looking for