Thread bottleneck?
Hi -
We have an application that is loading tens of thousands of xml messages
into a JMS queue backed by Oracle, and committing every 100. MDB's pick up
from that queue and start their own UserTransactions managing from the
stateless on. We're doing this to achieve execute concurrency in the
container even tho it is a batch application. It also will allow us to
handle near-realtime requests later using the same mechanism. (We know
we're really done with the load by monitoring the number of rows in the JMS
Oracle queue -- are handling poisen messages by starting a UserTransaction
in the MDBs and any that don't work get the container work rolled back, then
written in a new trans to a table with a CLOB.) It's working fine -- but
there is one place where the performance could be improved, I think.
It looks like there is a thread bottleneck. When the 100 transactions are
committed, only about 10 or so MDB's get to fire (about the number that are
in the pool), but after that they wait again until the next commit to handle
another 10 or so. Once the batch client finishes loading his thousands of
messages into the Queue and closes, the MDBs take off at full steam and we
get very good thruput.
So it looks like there is a single (or very few) threads for the JMS server,
and that they give priority to the JMS client doing the load and not to the
MDBs that handle them. Even tho lots are commited, the MDBs only get a
single shot at grabbing a few. Is there a way to tune this so that the
MDBs can get a bigger slice of the pie while the JMS server is loading?
The load seems to hog the JMS server.
Thanks for any help,
- Mark
Bob -
We are 6.1 SP2. Each message is a transaction when it gets inside of the
container. We are committing every 100 because we synchronize a counter in
case we need to restart, which works fine. But even without synchronizing
that counter, the load behavior is the same.
Since we have clustered servers, being able to have a queue and have
multiple MDBs working on these transactions should increase our thruput
because each transaction causes lots of database work to happen with of
course a commit at the end of each unit of work. Also we wanted a
message-based queueing solution for plugging in near-realtime workloads to
the same system.
Even tho the queue and JMS server are defined on one server, the MDBs launch
fine on both clustered app servers. The theory is that two threads (and
possibly several clustered boxes with multiple cpus) can wait on database IO
almost as fast as one can. If the amount of database work is small and you
have one cpu and you commit infrequently, then single-threading is
definitely the way to go as you mention. We'll be doing variations on the
load testing with MDBs very soon with different numbers of MDBs and servers.
Unfortunately our test setup has the clusters defined on a single,
multi-homed box so varying the servers won't be a good test. But we can
easily throttle the MDBs. The MDBs seem to launch so that the same ones
from the pool are launched as soon as there is work available even tho there
are a larger number of them available -- that's good because you can easily
track the workload. There seem to be about five active now at any given
time between the two servers even tho a total of 20 are available.
It's working fine and the performance is currently not an issue, but I was
just questioning the way that the intial load to JMS behaves.
Thanks,
- Mark.
"Bob Lee" <[email protected]> wrote in message
news:B960628D.190E%[email protected]...
> What SP are you using? Why does the client commit every 100 messages? Do
> these 100 messages represent an atomic unit? If so, then why not combine
> them into one message? It sounds to me like there could be some threaded
> code issues here (code as opposed to resource allocation issues).
>
> Also, are the MDB's taking the data and inserting it in a database or
> something? If so, I'd question the value of inserting the data
concurrently.
> Write a simple performance test program to prove this. The first test
should
> insert 1000 records sequentially. The second test can spawn 10 threads
that
> each insert 100 records. If the first tests performs close enough, I'd go
> with a sequential solution as the resource overhead will be a lot lighter
> and the coding can be simpler. Since you're using JMS, you don't have to
> worry about clients blocking each other.
>
> Also, when you write the test, connect to the database directly as opposed
> to accessing a DataSource running in WebLogic in another process. The
> overhead of the excess remote access might affect the results.
>
> Let me know,
> Bob
>
> On 7/20/02 8:57 AM, in article
> r%[email protected], "Mark Cassidy"
> <[email protected]> wrote:
>
> > Hi -
> >
> > We have an application that is loading tens of thousands of xml messages
> > into a JMS queue backed by Oracle, and committing every 100. MDB's
pick up
> > from that queue and start their own UserTransactions managing from the
> > stateless on. We're doing this to achieve execute concurrency in the
> > container even tho it is a batch application. It also will allow us to
> > handle near-realtime requests later using the same mechanism. (We know
> > we're really done with the load by monitoring the number of rows in the
JMS
> > Oracle queue -- are handling poisen messages by starting a
UserTransaction
> > in the MDBs and any that don't work get the container work rolled back,
then
> > written in a new trans to a table with a CLOB.) It's working fine --
but
> > there is one place where the performance could be improved, I think.
> >
> > It looks like there is a thread bottleneck. When the 100 transactions
are
> > committed, only about 10 or so MDB's get to fire (about the number that
are
> > in the pool), but after that they wait again until the next commit to
handle
> > another 10 or so. Once the batch client finishes loading his thousands
of
> > messages into the Queue and closes, the MDBs take off at full steam and
we
> > get very good thruput.
> >
> > So it looks like there is a single (or very few) threads for the JMS
server,
> > and that they give priority to the JMS client doing the load and not to
the
> > MDBs that handle them. Even tho lots are commited, the MDBs only get a
> > single shot at grabbing a few. Is there a way to tune this so that the
> > MDBs can get a bigger slice of the pie while the JMS server is loading?
> > The load seems to hog the JMS server.
> >
> > Thanks for any help,
> > - Mark
> >
> >
> >
>
Similar Messages
-
Thread bottleneck due to bug in JSF 1.1 : please help
Hi. We are using JSF 1.1 on a J2EE 1.4 server and have identified a critical a bottleneck because methods of class com.sun.faces.application.ApplicationAssociate are synchronized. This is a bug in JSF 1.1 identified by Sun (http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6223295) that has apparently been fixed in JSF 1.2.
However, JSF 1.2 requires Servlet 2.5 and JSP 2.1support, and our server is only J2EE 1.4 compatible.
So what can we do ? If you have any ideas, please post.
Thanks.
Fabien CoppensActually, I have just tried JSF 1.1_02 and it does seem to solve the thread bottleneck problem. I'll need verification once we get into performance testing, I'll keep you guys posted.
-
Thread behaves wierdly.. takes more time for less code...
Hi everyone,
I am stuck into this probel very badly..i have never seen such a problem before..Thread bottleneck somewhere is the issue i guess,,, plz have a look at the code patiently.. and do reply.. u guys are my last hope on this thing..Thank YOU...
The Bank method is called by another method which runs 50 threads , so at a time 50 threads could enter this method to record Debit...
class BankHelper {
// Some code of bank.. not relevant to my problem public Bank() {
long time = System.nanoTime();
abc.recordDebit(); // want to measure how much time this method takes
abc.recordTimeForDebit(System.nanoTime() - time); // accumulating the total time the above method takes to calculate
class abc {
// Some code of bank.. not relevant to my problem
public synchronized void recordDebit(){
debit++;
} // I had put timers aound the ++ and it took not more than 700 to 900 nanoseconds
public synchronized void recordTimeForDebit(long time){
record += time;
Log.debug("Time taken to record drop for DEBIT " + record + " Millis " + time);
}Answer:
Record time is 9014 millis for 5000 increments or 5000 calls to recordDebit()
One can see in the answer that its a huge number when one is expecting it to be somewhere around:
EXPECTED ANSWER:
5000 * 800 nanos( it takes 800 nanos for very increment) = 4000000 = 4 millis and its like 9014 millis
How is there such a huge difference ????????? I have bruised thorugh it for like 3 days... i have lost all my hope.. Plz help..Also look at the next code..
When i go to see in the log .. it shows me that every record took like from 2000 nanos to 6 to 7 millis i.e.7000000...
HOw is this even possibly possible ??? where did it get all this extra time from...it should not be more than some 700 to 900 nanos ..
Is there a bottleneck somewhere ????
Now part 2:
This thing has fazzed, dazzled , destroyed me.. I could not understand this and it has tossed all my concepts into the cupboard..
Same stuff: JUST HAVE A LOOK AND ENJOY AND TELL ME WHATS GOING ON...Same code.. different way of incrementing data i.e synchronization is at a different place..but results are very highly improved...
class BankHelper {
// Some code of bank.. not relevant to my problem
public Bank() {
long time = System.nanoTime();
abc.recordDebit(); // want to measure how much time this method takes
abc.recordTimeForDebit(System.nanoTime() - time); // accumulating the total time the above method takes to calculate
}The Bank method is called by another method which runs 50 threads , so at a time 50 threads could enter this method to record Debit...
class abc {
// Some code of bank.. not relevant to my problem
public void recordDebit(){
someotherclass.increment();
} // this is not synchronized nowwwwwwww
// I have put timers here too and it took like 1500 to 2500 nanos
public synchronized void recordTimeForDebit(long time){
record += time;
Log.debug("Time taken to record drop for DEBIT " + record + " Millis " + time);
class someotherclass{
// Not relevant code
public void increment(){
someotherclass1.increment1();
class someotherclass1{
// Not relevant code
public void increment1(){
someotherclass2.increment2();
class someotherclass2{
// Not relevant code
public synchronized void increment2(){
someotherclass3.increment3();
} //now its synchronized
class someotherclass3{
// Not relevat code
public synchronized void increment3(){
debit++;
} //now its synchronized
}ANSWER: Record is 135 millis for 5000 increments or 5000 calls to recordDebit()
Expected time was : 5000 * 2500 = 125000000 = 125 millis (WOW .. AS EXPECTED)
Please don't ask me why this code has been written this way..i know it goes and increment and does the same thing...but somehow when i measured it..
overall time or the accumulated time for both codes varied like in huge numbers...even though latter code is actually a superset of previous code..
HOW IS THERE SUCH A HUGE DIFFERENCE BETWEEN THE NUMBERS ???
COULD BE BECAUSE OF THE POINT OF SYNCHRONIZATION ???
Thank you..for going through all this..Triple post - http://forums.sun.com/thread.jspa?threadID=5334258&messageID=10438241#10438241
-
Threads behaving wierdly..Work of 5 months going for a toss.. plz help...?
Hi everyone,
I am stuck into this probel very badly..i have never seen such a problem before..Thread bottleneck somewhere is the issue i guess,,, plz have a look at the code patiently.. and do reply.. u guys are my last hope on this thing..Thank YOU...
The Bank method is called by another method which runs 50 threads , so
at a time 50 threads could enter this method to record Debit... class BankHelper {
// Some code of bank.. not relevant to my problem
public Bank() {
long time = System.nanoTime();
abc.recordDebit();_ *// want to measure how much time this method takes*_
abc.recordTimeForDebit(System.nanoTime() - time); *_// accumulating the total time the above method takes to calculate_*
class abc {
// Some code of bank.. not relevant to my problem
public synchronized void recordDebit(){
debit++;
} _*// I had put timers aound the ++ and it took not more than 700 to 900 nanoseconds*_
public synchronized void recordTimeForDebit(long time){
record += time;
Log.debug("Time taken to record drop for DEBIT " + record + " Millis " + time);
Answer:_
Record time is 9014 millis for 5000 increments or 5000 calls to recordDebit()_
One can see in the answer that its a huge number when one is expecting it to be somewhere around:
EXPECTED ANSWER:
*5000 * 800 nanos( it takes 800 nanos for very increment) = 4000000 = 4 millis and its like 9014 millis*
How is there such a huge difference ????????? I have bruised thorugh it for like 3 days... i have lost all my hope.. Plz help..Also look at the next code..
When i go to see in the log .. it shows me that every record took like from 2000 nanos to 6 to 7 millis i.e.7000000...
HOw is this even possibly possible ??? where did it get all this extra time from...it should not be more than some 700 to 900 nanos ..
Is there a bottleneck somewhere ????
Now part 2:
This thing has fazzed, dazzled , destroyed me.. I could not understand this and it has tossed all my concepts into the cupboard..
Same stuff: JUST HAVE A LOOK AND ENJOY AND TELL ME WHATS GOING ON...Same code.. different way of incrementing data i.e synchronization is at a different place..but results are very highly improved...
class BankHelper {
// Some code of bank.. not relevant to my problem
public Bank() {
long time = System.nanoTime();
abc.recordDebit(); // want to measure how much time this method takes
abc.recordTimeForDebit(System.nanoTime() - time); // accumulating the total time the above method takes to calculate
}The Bank method is called by another method which runs 50 threads , so at a time 50 threads could enter this method to record Debit...
class abc {
// Some code of bank.. not relevant to my problem
public void recordDebit(){
someotherclass.increment();
} // this is not synchronized nowwwwwwww
// I have put timers here too and it took like 1500 to 2500 nanos
public synchronized void recordTimeForDebit(long time){
record += time;
Log.debug("Time taken to record drop for DEBIT " + record + " Millis " + time);
class someotherclass{
// Not relevant code
public void increment(){
someotherclass1.increment1();
class someotherclass1{
// Not relevant code
public void increment1(){
someotherclass2.increment2();
class someotherclass2{
// Not relevant code
public synchronized void increment2(){
someotherclass3.increment3();
} //now its synchronized
class someotherclass3{
// Not relevat code
public synchronized void increment3(){
debit++;
} //now its synchronized
ANSWER: Record is 135 millis for 5000 increments or 5000 calls to recordDebit()_
Expected time was : 5000 2500 = 125000000 = 125 millis (WOW .. AS EXPECTED)*
Please don't ask me why this code has been written this way..i know it goes and increment and does the same thing...but somehow when i measured it..
overall time or the accumulated time for both codes varied like in huge numbers...even though latter code is actually a superset of previous code..
HOW IS THERE SUCH A HUGE DIFFERENCE BETWEEN THE NUMBERS ???
COULD BE BECAUSE OF THE POINT OF SYNCHRONIZATION ???
Thank you..for going through all this..Answered on JavaRanch. By synchronizing both the increment and the recording methods, you can only perform one of those functions at a time.Thanking you Steve, I shall read and inwardly digest.
My (humble) advise is:
1. WTF are you doing trying to measure the time taken to execute a [unirary operation|http://en.wikipedia.org/wiki/Unary_operation] (you putz!) which is just like so freeken small (on a 2Ghz CPU) that even a gazzillion invocations might take less time than it took you get your duds off in order to earn your reputation as a putz!
2. FFS, Don't write profiling code in your app... just download one. VisualVM is way cool!
3. As you would know, if you had bothered to actually RTFM at any stage in your (bugging;-) career... synchronized methods are slow(er) because of the farckin time taken to acquire and release the lock... as well as the inherent inefficiency of bottlenecking a whole method, as apposed to just the critical statements... which in the case of a simple incrementor is just about no near new never mind... so no never mind... but still the fact remains... you've turned an atomic operation (google it yourself) into a bottleneck... and then you're standing there whining about bottlenecks. Doh!
4. Seriously dude, read the effin manual.
Edited by: corlettk on 27/09/2008 14:19 -
This may seem like a very simple question -
The JVM spec suggests that threads have their own local copy of variables, which is copied between the main memory.
In terms of variables that are local to a method, do these variables reside locally within the context of each thread.
As an example :
public class A
public void methodA(SomeObject so)
int i = so.getMethodResult();
}The question is - if two methods were accessing the method concurrently, would any operation on the local variable i require synchronizing, of if all of the operations within methodA are purely local, would multiple threads interfere with each thread's local copy of those variables?
Thank You
JimNo. There is a dual dichotomy here.
Variables local to the method are allocated at the beginning of the method, deallocated at the end, and belong to the specific invocation of the method. Two threads calling the same method will get independent copies of the method-local variables and no synchronization of any kind is needed. This is perhaps clearer if you consider a local variableString s = new String("Some dynamically created string" + hashCode());The string is separately allocated as part of execution of the new method anytime the object or class method is called. If the object or class method is called four times, four separate String references will be allocated (and eventually garbage-collected). It doesn't matter if two of the invocations happened concurrently in two different threads.
The other side of the dual dichotomy is itself a dichotomy (that's what makes it "dual"). Consider the following variation on your classpublic class A
int i = 12; // Here we have an instance variable
public void methodA(SomeObject so)
// Set an object instance variable rather than a local variable
i = so.getMethodResult(); // Now the implications are broader
}Suppose this class is used in two different threadsA a = new a();
// The same instance a of object A is passed to two different threads...
// Thread 1 code
// Invoke methodA on our reference to the object of class A created in the parent code
thread1sa.methodA(new SomeObject(17); // thread1sa refers to the same object as thread2sa
// Thread 2 code
// Invoke methodA on our reference to the object of class A created in the parent code
thread2sa.methodA(new SomeObject(23); // thread1sa refers to the same object as thread2saNow i might need to be protected by a synchronize block because it could be an index into a shared array that could be corrupted by unprotected concurrent access. This is simply a coding consideration inherent with any multithreaded code
But, in addition, there are also JVM/underlying-hardware-implementation considerations to deal with, even if no modifications are made to shared data. This is the other half of the second dichotomy and is, I think, the implication that prompted your original question. Consider the following codepublic class A
int i = 12; // The shared instance variable
public void methodA(SomeObject so)
// Set an object instance variable rather than a local variable
i = so.getMethodResult();
public int sayHowBigIsi()
// Return where we are
if (i < 7)
Thread.Sleep(3000); // This raises the possibility that instance variable i will change while we're not looking
System.out.println("i is" + i);
else
System.out.println("i is greater than 7");
}The original intent of methodB is to print the value of i if it is less than 7, but there is a possibility that between the time i is tested and the time it is printed, another thread could change it to something else, causing the println to print out an incorrect value of i. This is because the generated bytecode keeps local copies of variables for performance (and in fact some values could on the bare metal level be kept in even more volatile storage, such as registers) and these local copies aren't guaranteed to be kept updated at all times. That is, the value of i printed by the println could be an old copy of i that is now wrong.
One solution would be to enclose the code that uses i in a synchronize block. This would alleviate the problem, but would inevitably incur a thread bottleneck at that point, and since no data is actually modified, control as strict as is provided by synchronize is not necessary. Instead, Java provides the volatile keyword, which signals to the Java compiler and JVM that a variable may be updated asynchronously and that implementation-local copies should either not be kept, or always be refreshed before use. This also incurs a performance cost, but does not result in a thread bottleneck.public class A
volatile int i = 12; // The shared instance variable now made volatile
public void methodA(SomeObject so)
// Set an object instance variable rather than a local variable
i = so.getMethodResult();
public int sayHowBigIsi()
// Return where we are
if (i < 7)
Thread.Sleep(3000); // This raises the possibility that instance variable i will change while we're not looking
// ...but now there's no problem because i is volatile.
System.out.println("i is" + i);
else
System.out.println("i is greater than 7");
} -
Problem updating row via OLEDB Provider (v10.2.0.1)
We have an application that either updates or inserts a row into an Oracle 10g table.
We found that every so often the connection hangs when trying to update a row. We soon discovered that during our testing we were deleting individual rows from the table and then the update of other rows failed.
After a long period of time the system seems to sort itself out and allow the updating of rows again.
Does anyone have any ideas what the problem is?
Is it worth updating to a later version of the ODAC software?
Regards
DAKWhen you say the update 'failed', you're referring to the hang right? No errors? And when it hangs, it pretty much hangs the whole app until it fixes itself? How long does the 'hang state' last?
Since the hang 'fixes itself', its not likely a hung or orphaned critical section, so my guess would be either 1) some sort of thread bottleneck on the client (if this is some sort of "multi user under load" scenario, or 2) the database is doing something goofy and the client is simply waiting for a response before it can continue.
The way I'd try to get more information is to enable sqlnet tracing at level 16, and make sure to enable timestamps as well. Then let the app do it's 'hang and fix itself' type of thing, note what time-ish the hang thing was happening, and check the traces timestamp gaps. What I suspect you may find is a gap between a NSPSEND and NSPRECV call in the Oracle client. I dont have a trace handy to give you the actual calls though so I may not have those names quite right. However, if you see something like the following,
[10:20:01.1234] nspsend ...
[10:21:34.4567] nsprecv ...that would indicated that the client send a packet and was waiting for a minute and half before the database sent a packet back, so you'd want to drag your dba in to have a look at that part.
If you see no timestamp gaps of appreciable size (more than a second or so), and this is a "multi user under load" type situation, then you may want to get a memory dump of the app while it's in it's hung state , and again a minute or two later, and look to see 1) if threads are bottlenecked waiting for a critical section, 2) if they've moved in the dump that was taken a minute later. If you see that the thread that everything is waiting on has WSARECV32 in the stack, that generally means its in socket receive state though, which leads back to the "waiting for the database to respond" situation.
Use debugdiag to get and do a first pass analysis on memory dumps, and you get get info on [sqlnet tracing here|http://www.oraclefaq.net/2007/05/02/how-to-enable-sqlnet-client-side-tracing/].
I don't have any compelling reason to believe that upgrading oledb will help, but if you're using 10201, and the 10201 client as well, I'd recommend going to 10204 purely on general principle, and you do that by applying the 10204 rdbms patch to the client machine, and that will patch both OraOledb as well as the underlying client librarires.
Hope it helps, I know its a little vague on the details.
If you need further help and have access to Oracle Support, we could certainly give you a hand sorting it out.
Cheers,
Greg -
Greetings,
I am on Solaris using TimesTen Release 7.0.5.0.0.
I fixed some threading bottlenecks in our application. Unfortunately this seems to put more pressure on TimesTen and degrades the overall throughput of our system. :-/
Someone suggested running "ttXactAdmin -latchstats show" so I did but I have a bit of trouble interpreting the results.
I ran the old software and the new software against the same performance test.
I discovered that many of the SpinCnt values are much bigger now (anywhere between 4x and 8x bigger) but what does it mean?
The number of collisions is also about 4x bigger (values range between 5 to 100) .
The important question is what does the ouput of ttXactAdmin mean and why is it worse now ? (well two questions.. ;-)
The second question is how can I improve this without touching the DB schema ? The DB schema does need some optimization work but if I could improve the performance a bit before making such changes it would be ideal.
Thanks,
SergeHi Serge,
Firstly, you need to be sure you are capturing accurate latchstats. Maybe you know this already but just inc case, what you need to do is, when you workload is running a reasonably steady state, run the commands:
ttXactAdmin -latchstats off dsnname
ttXactAdmin -latchstats clear dsnname
ttXactAdmin -latchstats on dsnname
sleep 60
ttXactAdmin -latchstats off dsnname
ttXactAdmin -latchstats show dsnname
This ensures you are capturing latchstats for just a 60 second period rather than any that may have been ther before. Do this for both cases (before app optimisation) and afer.
It is very likely that improving multi-thread / multi-CPu concurrency in the app will shift the concurrency issues into TimesTen. Basically high(er) values of colilisions, average spincnt and sleep count reflect increased contention within TimesTen. Depending on which objects these affect you may (or may not) be able to make some optimisations. There are several common contention points you may see depending on your workload:
1. Log buffer insertion latch. This is a common one if your workload has a lot of insert/update/delete operations. Not much you can do about this except to reduce the volume of write operations or use fewer concurrent application processes/threads.
2. Index latches (mainly for T-Tree indexes). Little to be done about these other than remove the indexes or split the table (and hence the indexes0 into multiple tables/indexes.
3. Command latches. Eliminate these by specifying PruvateCommands=1 in the ODBC attributes.
4. Heap latches. If these are on temp heaps not much can be done. if they are on Perm heaps you can try increase the number of perm heaps. the way to do this is undocumented but is sort of supported :-)
An important note; if you are using direct connection mode or local (intra machine) client/server then it is totally counterproductive to have more concurrent processes/threads accessing TimesTen than the number of CPU cores in your machine. Depending on the workload and what else is running on the machine you will see peak throughput and optimum response time with <= N concurrent processes/threads where N is the total number of CPU cores in the machine. if you are using remote client server then the optimal number of connections is obviously < N since each connection spends some time waiting for the network round trip.
Chris -
How to improve spreadsheet speed when single-threaded VBA is the bottleneck.
My brother works with massive Excel spreadsheets and needs to speed them up. Gigabytes in size and often with a million rows and many sheets within the workbook. He's already refined the sheets to take advantage of Excel's multi-thread recalculation and
seen significant improvements, but he's hit a stumbling block. He uses extensive VBA code to aid clarity, but the VB engine is single-threaded, and these relatively simple functions can be called millions of times. Some functions are trivial (e.g. conversion
functions) and just for clarity and easily unwound (at the expense of clarity), some could be unwound but that would make the spreadsheets much more complex, and others could not be unwound.
He's aware of http://www.analystcave.com/excel-vba-multithreading-tool/ and similar tools but they don't help as the granularity is insufficiently fine.
So what can he do? A search shows requests for multi-threaded VBA going back over a decade.
qtsHi,
>> The VB engine is single-threaded, and these relatively simple functions can be called millions of times.
The Office Object Model is
Single-Threaded Apartments, if the performance bottleneck is the Excel Object Model operation, the multiple-thread will not improve the performance significantly.
>> How to improve spreadsheet speed when single-threaded VBA is the bottleneck.
The performance optimization should be based on the business. Since I’m not familiar with your business, so I can only give you some general suggestions from the technical perspective. According to your description, the size of the spreadsheet had reached
Gigabytes and data volume is about 1 million rows. If so, I will suggest you storing the data to SQL Server and then use the analysis tools (e.g. Power Pivot).
Create a memory-efficient Data Model using Excel 2013
and the Power Pivot add-in
As
ryguy72 suggested, you can also leverage some other third party data processing tool
according to your business requirement.
Regards,
Jeffrey
We are trying to better understand customer views on social support experience, so your participation in this interview project would be greatly appreciated if you have time. Thanks for helping make community forums a great place.
Click
HERE to participate the survey. -
Struts Action threading model bottleneck
Hi,
Struts Action classes are only instantiated once. Then the controller servlet routes all perform/execute calls through the single instantiated Action class. Does this create a performance bottleneck? Any insights would be helpful.
ThanksIt's more performant, since it doesn't need to create new objects for each request. You just need to write your code in a thread-safe manner (watch the instance variable!).
Cheers -
Will there be a bottleneck with 16GB RAM and 32 threads CPU?
And if so how much of a bottleneck? Is it worth the money to upgrade to 48GB? I have 8x2GB in the HP Z820, but was thinking of getting more RAM from Mr Memory that is compatible with the Z820 for much cheaper than what HP are selling HP branded ram for. I was looking at 8x4GB for around £344 to make 48GB. Is it risky going unbranded or do people do this a lot?
Thanks
CostaHP can list that it's an option that doesn't mean it's recommended or the best way to do it without having issues. Since the ram controller was moved to the CPU, trying to run asynchronous timings on system boards with Quad channel is asking for problems. If you call the ram manufacturers they wont recommend it with their ram and likely wont support it if they have any clue. Due to how the ram market is so volatile and dimm modules change ram chips from 1 production run to the next , you cant even run 2 different sets of the same model of ram with different manufacturer chips on the dimms. The ram companies will tell you that you need to change them out. I have been dealing with all of the ram variables since tri-channel started on the I7 Gen 1 platform and it only gets more complex with Quad channel. We have our ram custom made just to ensure we have the same manufacturer chips on our dimms regardless of the production along with the same stepping of construction until the wafer has to change. There are far more timings than CAS rating and the most important to compatibility are the sub timings which the manufacturer does not list. HP also can say they only want Grade 1A ram modules for their dimms which is why they are more expensive while that ram manufacturer will use other grades for their own branded ram. Those modules will not work together most of the time if at all. Do what you will but dont say you were not warned. Much of this information is not available to anyone unless they talk to the Engineers over in TW that deal with the dram chip fabs.
Eric
ADK -
[WLS92] Thread synchronization bottleneck in RJVMManager
All,
First off, apologies for the long post -- there is a lot to get through.
h3. Environment:
Weblogic Server 9.2 MP3 deployed into Solaris 10 / Sparc hosts.
Sun JDK 1.5.0_12 JVM (shipped with WLS)
h3. Problem:
We are experiencing a problem with out WLS 9.2/MP3 (Solaris/Sparc) installation. We have a number of servers each running the same WLS configuration but some with different application: one server is running App A, 4 other servers are running App B (not in a cluster). We are using JMS to transfer messages from App A to App B in a strict failover configuration such that App A will always try and send to App B1, App B2, AppB3 and finally App B4, moving from one to the next if the transfer fails or takes longer than a given timeout (10 seconds). The dispatching happens in separate threads, which I will refer to as 'senders', so we can timeout the transfer attempt.
In the 'happy day' scenario where all four App B servers are up and running, everything works as expected.
Problems start to surface when we test the failover configurations.
In scenarios where the hosts are available but WLS is in either not running or in Admin mode, the failover proceeds as expected.
In scenarios where the hosts are unavailable (hardware is powered down, network is disrupted etc.), the failover does not proceed as expected. In a scenario where App B1 is down but the other three are available for example, the following are observed:
1. the time taken for the underlying socket connection to App B1 to timeout is very long (~7.5 minutes).
2. the connections to the other hosts appear to block until after the socket timeout to App B1 occurs.
The relevant section of the code is:
weblogic.security.Security.runAs(new Subject(), new PrivilegedAction<Void>() {
public Void run() {
InitialContext context = null;
try {
* During server-to-server sending of messages we must establish and maintain
* a new context that will contain the security information established by
* the initial context object. This has to be done per thread, hence why it
* is applied here to the dispatcher. The context will be closed when
* the thread is returned.
final Hashtable<?,?> environment = getEnvironment();
try {
if (log.isDebugEnabled()) {
log.debug(format("run(): Sender %d Creating initial context; env=%s", id, environment));
context = new InitialContext(environment);
if (log.isDebugEnabled()) {
log.debug(format("run(): Sender %d Created initial context in %d ms", id, System.currentTimeMillis() - start));
runWithContext();
} catch (NamingException e) {
log.error(format("run(): Sender %d cannot create initial context", id), e);
} finally {
try {
if (context != null) {
if (log.isDebugEnabled()) {
log.debug(format("run(): Sender %d closing initial context", id, context));
context.close();
} catch (NamingException e) {
log.error(format("run(): Sender %d cannot close initial context", id), e);
return null;
Thread stacks for two sample threads in this scenario are as follows. Thread '20' is the sender attempting to connect to App B1, Thread '8' is the sender attempting to connect to App B2 and is representative of a number of other threads all attempting to connect to known good hosts.
Daemon Thread [[ACTIVE] ExecuteThread: '20' for queue: 'weblogic.kernel.Default (self-tuning)'] (Suspended)
PlainSocketImpl.socketConnect(InetAddress, int, int) line: not available [native method]
SocksSocketImpl(PlainSocketImpl).doConnect(InetAddress, int, int) line: 333
SocksSocketImpl(PlainSocketImpl).connectToAddress(InetAddress, int, int) line: 195
SocksSocketImpl(PlainSocketImpl).connect(SocketAddress, int) line: 182
SocksSocketImpl.connect(SocketAddress, int) line: 366
SSLSocketImpl(Socket).connect(SocketAddress, int) line: 519
SSLSocketImpl(Socket).connect(SocketAddress) line: 469
SSLSocketImpl(Socket).<init>(SocketAddress, SocketAddress, boolean) line: 366
SSLSocketImpl(Socket).<init>(InetAddress, int, InetAddress, int) line: 266
SSLSocketImpl(SSLSocket).<init>(InetAddress, int, InetAddress, int) line: 195
SSLSocketImpl.<init>(TLSSystem, InetAddress, int, InetAddress, int) line: not available
SSLSocketFactoryImpl.createSocket(InetAddress, int, InetAddress, int) line: not available
ChannelSSLSocketFactory.createSocket(InetAddress, int) line: 77
MuxableSocketT3S(AbstractMuxableSocket).createSocket(InetAddress, int, int) line: 207
MuxableSocketT3S(MuxableSocketT3).newSocketWithRetry(InetAddress, int, int) line: 252
MuxableSocketT3S(MuxableSocketT3).connect(InetAddress, int, int) line: 421
MuxableSocketT3S.createConnection(InetAddress, int, ServerChannel, JVMID, int) line: 79
ConnectionManager.createConnection(Protocol, InetAddress, int, ServerChannel, JVMID, int) line: 1749
ConnectionManagerServer(ConnectionManager).findOrCreateConnection(ServerChannel, JVMID, int) line: 1410
ConnectionManagerServer(ConnectionManager).bootstrap(JVMID, ServerChannel, int) line: 448
ConnectionManagerServer(ConnectionManager).bootstrap(InetAddress, int, ServerChannel, int) line: 326
RJVMManager.findOrCreateRemoteInternal(InetAddress, int, String, String, int) line: 261
RJVMManager.findOrCreate(InetAddress, int, String, String, int) line: 204
RJVMFinder.findOrCreateRemoteServer(InetAddress, int, String, int) line: 226
RJVMFinder.findOrCreate(boolean, String, HostID, int) line: 189
ServerURL.findOrCreateRJVM(boolean, String, HostID, int) line: 154
WLInitialContextFactoryDelegate$1.run() line: 342
AuthenticatedSubject.doAs(AbstractSubject, PrivilegedExceptionAction) line: 363
SecurityManager.runAs(AuthenticatedSubject, AuthenticatedSubject, PrivilegedExceptionAction) line: 147
WLInitialContextFactoryDelegate.getInitialContext(Environment, String, HostID) line: 337
Environment.getContext(String, HostID) line: 307
Environment.getContext(String) line: 277
WLInitialContextFactory.getInitialContext(Hashtable) line: 117
NamingManager.getInitialContext(Hashtable<?,?>) line: 667
InitialContext.getDefaultInitCtx() line: 247
InitialContext.init(Hashtable<?,?>) line: 223
InitialContext.<init>(Hashtable<?,?>) line: 197
FailoverMessageSender$Sender$1.run() line: 475
FailoverMessageSender$Sender$1.run() line: 458
AuthenticatedSubject.doAs(AbstractSubject, PrivilegedAction) line: 321
SecurityManager.runAs(AuthenticatedSubject, AuthenticatedSubject, PrivilegedAction) line: 121
Security.runAs(Subject, PrivilegedAction) line: 41
FailoverMessageSender$Sender.run() line: 457
DelegatingWork.run() line: 61
J2EEWorkManager$WorkWithListener.run() line: 259
ExecuteThread.execute(Runnable) line: 209
ExecuteThread.run() line: 181
Daemon Thread [[ACTIVE] ExecuteThread: '8' for queue: 'weblogic.kernel.Default (self-tuning)'] (Suspended)
RJVMManager.findOrCreateRemoteInternal(InetAddress, int, String, String, int) line: 252
RJVMManager.findOrCreate(InetAddress, int, String, String, int) line: 204
RJVMFinder.findOrCreateRemoteServer(InetAddress, int, String, int) line: 226
RJVMFinder.findOrCreate(boolean, String, HostID, int) line: 189
ServerURL.findOrCreateRJVM(boolean, String, HostID, int) line: 154
WLInitialContextFactoryDelegate$1.run() line: 342
AuthenticatedSubject.doAs(AbstractSubject, PrivilegedExceptionAction) line: 363
SecurityManager.runAs(AuthenticatedSubject, AuthenticatedSubject, PrivilegedExceptionAction) line: 147
WLInitialContextFactoryDelegate.getInitialContext(Environment, String, HostID) line: 337
Environment.getContext(String, HostID) line: 307
Environment.getContext(String) line: 277
WLInitialContextFactory.getInitialContext(Hashtable) line: 117
NamingManager.getInitialContext(Hashtable<?,?>) line: 667
InitialContext.getDefaultInitCtx() line: 247
InitialContext.init(Hashtable<?,?>) line: 223
InitialContext.<init>(Hashtable<?,?>) line: 197
FailoverMessageSender$Sender$1.run() line: 475
FailoverMessageSender$Sender$1.run() line: 458
AuthenticatedSubject.doAs(AbstractSubject, PrivilegedAction) line: 321
SecurityManager.runAs(AuthenticatedSubject, AuthenticatedSubject, PrivilegedAction) line: 121
Security.runAs(Subject, PrivilegedAction) line: 41
FailoverMessageSender$Sender.run() line: 457
DelegatingWork.run() line: 61
J2EEWorkManager$WorkWithListener.run() line: 259
ExecuteThread.execute(Runnable) line: 209
ExecuteThread.run() line: 181
h3. Conclusion:
All of the threads are jamming up in RJVMManager.findOrCreateRemoteInternal(..) behind the one thread that is actually attempting to make a connection.
A quick disassemby of the offending method class shows the following snippet:
/* */ private RJVM findOrCreateRemoteInternal(InetAddress paramInetAddress, int paramInt1, String paramString1, String paramString2, int paramInt2)
/* */ throws UnrecoverableConnectException, IOException
/* 246 */ RJVM localRJVM = findExisting(paramInetAddress, paramInt1, (ServerChannel)localObject1);
/* 248 */ if (localRJVM != null) return localRJVM;
/* 250 */ synchronized (this.bootstrapLock)
/* 252 */ localRJVM = findExisting(paramInetAddress, paramInt1, (ServerChannel)localObject1);
/* 253 */ if (localRJVM != null) return localRJVM;
/* 256 */ if ((Kernel.DEBUG) && (debugConnection.isDebugEnabled())) {
/* 257 */ RJVMLogger.logDebug("Bootstrapping connection to: '" + paramInetAddress + ":" + paramInt1 + "' using: '" + localProtocol + "'");
/* 260 */ ConnectionManager localConnectionManager = ConnectionManager.create(null);
/* 261 */ return record(localConnectionManager.bootstrap(paramInetAddress, paramInt1, (ServerChannel)localObject1, paramInt2), paramInetAddress);
Now the problem becomes clear. I suspect that the reason for the synchronized block on line 250 is to prevent two threads attempting the relatively expensive operation on line 261 (see lines 248-250 and 252-253). However, the flaw here is that the synchronization blocks all threads irrespective of which InetAddress (param 1) the thread is attempting to connect to. My personal opinion is that the monitor used for thread synchronization should be timed to the host (InetAddress) for the connection, which incidentally is used in the record(..) method anyway.
The problem is exacerbated by the very long network connection timeout.
h3. Solutions:
We've tried setting the sun.net.defaultConnectTimeout JVM property to try and reduce the impact but haven't seen any noticeable improvement.
We've tried specifying the weblogic.jndi.clientTimeout and weblogic.jndi.requestTimeout values when creating the InitialContext. For some reason specifying these properties caused all connection attempts to fail -- even the happy day scenarios.
h3. Final Questions:
Is this a bug? If so, what do I need to do to get this fixed urgently?
Is this a bug other people have run into? If so, how was it resolved?
Any other suggestions?
Thanks!
Edited by: user11373704 on 10-Jul-2009 08:52Perhaps the relevant timeouts are socket/network related in this case.
To detect "silent" network failures, WebLogic uses a heartbeat mechanism, but I'm told that one should not tune the related settings too aggressively. I think this is tunable on a per server basis -- at least for the default network channels. Specifically there's heartbeat interval and period length on the console at "Server -> Configuration Tab --> Tuning Tab --> Advanced".
There's also "Complete Message Timeout" under "Protocol -> General".
In addition, TCP/IP timeouts are tunable on the operating system itself. I don't know the specifics for Solaris.
Tom -
Bridge is slow because it's only using two threads
I'm finding that Bridge (CS6) exhibits very poor performance when building 100% previews, I think the software is limited to only two threads.
If you load bridge and go into a folder with some images and then Tools->Cache->Purge you'll see that Bridge takes a long time to rebuild the previews even on a high end machine, especially if you have both "Always High Quality" and "100% Previews" selected. I think the problem is that it appears to build the previews using only two threads on the CPU, so while a 2005 dual core machine might get maxed out, a Quad Core, or 8 Thread, or even 12 Thread machine sits there mostly idle while Bridge grinds away for an hour or more.
It's easy to check this, just load up Windows Task Manager, and view the "Processes" tab and Bridge will only ever use 2 threads worth of CPU cycles, i.e. 50% on a Quad Core, and 17% on a 12 Thread high end desktop. I've seen the problem with JPG files, PSD files, and NEF files (Nikon RAW). The processing time of large Nikon D800E NEF files is very slow. After shooting 900 images I can expect the preview build time to take over an hour on a super high end machine with loads of memory, storage IO, and CPU cores and threads, all because it appears bottlenecked on software design. Meanwhile the machine is only running at roughly 20% utilization, the memory barely used, and the IO system close to idle.
I'm using Windows 7 Ultimate 64-bit, 64GB RAM, 12 Thread Core i7 3960X at 4Ghz, two Nvidia GTX470 graphics (in SLI), a 2-way RAID-0 SSD storage array for the OS, and an 8-way RAID-0 SSD array for the photoshop work files. The hardware is clearly not the issue. Although, just to check, I confirmed the same problem exists on a simple Win8 Core i7 system, and a Core i5 laptop.
Even worse, after waiting an hour to build previews for 900 images (3 seconds each), after adding GPS data, or after making edits in Camera Raw it then needs to rebuild all the previews again, this is understandable, but super fustrating when you know it's not working as fast as it really should be.
To be clear, I'm not upset it takes 6 seconds per image per thread - there's a lot of work to do, especially with D800 files. But I'm upset that Bridge's software design only uses 2 threads, and thus limits performance to building a preview every 3 seconds, when in fact it could be working 6 times faster on a PC with 12 thread, and I find it shocking that Adobe's latest software wouldn't be optimized for anything more than a dual core CPU, wasn't quad core launched in 2006?
Roland.
P.S. for any Adobe Tech's reading - I submitted this under case number 184020852 and there's an attached PDF in the ticket including several screenshots.There have been many discussions in Photoshop forum about processor speed and multicores. I know Bridge is not Photoshop but it may give you some insight as what is going on. Photoshop has supported multithreading since CS3. Here is a quote from Adobe Tech Note:
Photoshop generally runs faster with more processor cores, although some features take greater advantage of the additional cores than others. There is a law of diminishing returns with multiple processor cores: The more cores you use, the less you get from each additional core. Therefore, Photoshop doesn’t run four times as fast on a computer with 16 processor cores as on a computer with four cores. For most users, the increase in performance that more than six cores provides doesn't justify the increased cost.
I run embedded thumbnails (therefore 100% previews are off) as they take up less space and are the fastest to load. I ran a simple test with Bridge CS5 (32 bit) and CS6 (64 bit) on one folder of 200 video images totaling 16 gigs, or 80 meg per file. With folder loaded I clicked Tools/Cache/purge cache for xxx folder. I then timed the rebuild until the arrow stopped spinning in lower left hand corner. For CS5 the time was 35 seconds. For CS6 the time was 72 seconds.
I then took a folder of 1147 jpg images of 660 meg (427k/image). In CS5 it took 12 seconds to build the cache, and the same for CS6. However, the interesting part of this is when I loaded this folder in CS6 I had never been there before so it had to build it. THis took 80 seconds. When I purged it an it rebuilt it took 12 seconds. I have noticed this phenomemum before where the initial build takes 5-7 times longer than a rebuild of the cache.
I then ran the same test with the still images with 100% preview and Always High Quality. With CS5 it took 3 min 30 seconds and with CS6 it took 5 min. 20 seconds.
So with these very SIMPLE tests I conclude that CS6 has quite a bit slower cache process than CS5. As we have all learned with buying new products, new and improved does not always mean it is better. -
How to find where is bottleneck oracle 11.2
I am running oracle 11gr2 on windows server 2008 R2 having ram 20gb.
I am taking statspack report on database and following is it:
i am not getting where is actual bottleneck is and some time there is problem regarding procedure hangup which having business logic fetch by cursor.
i am generate statistics every day using :
begin
dbms_stats.gather_schema_stats(ownname => 'MFG',cascade => TRUE,no_invalidate => FALSE);
end;
and also using:
analyze table abc compute statistics;
stats pack report:
Load Profile Per Second Per Transaction Per Exec Per Call
~~~~~~~~~~~~ ------------------ ----------------- ----------- -----------
DB time(s): 2.0 0.8 0.00 0.01
DB CPU(s): 1.5 0.6 0.00 0.00
Redo size: 68,274.8 28,441.6
Logical reads: 83,672.4 34,855.8
Block changes: 633.0 263.7
Physical reads: 2,763.0 1,151.0
Physical writes: 37.4 15.6
User calls: 379.8 158.2
Parses: 342.6 142.7
Hard parses: 0.3 0.1
W/A MB processed: 9.9 4.1
Logons: 34.0 14.2
Executes: 2,702.3 1,125.7
Rollbacks: 0.0 0.0
Transactions: 2.4
Instance Efficiency Indicators
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Buffer Nowait %: 100.00 Redo NoWait %: 100.00
Buffer Hit %: 98.06 Optimal W/A Exec %: 99.99
Library Hit %: 100.03 Soft Parse %: 99.90
Execute to Parse %: 87.32 Latch Hit %: 99.97
Parse CPU to Parse Elapsd %: 91.04 % Non-Parse CPU: 99.19
Shared Pool Statistics Begin End
Memory Usage %: 68.94 69.12
% SQL with executions>1: 64.57 65.59
% Memory for SQL w/exec>1: 81.76 82.22
Top 5 Timed Events Avg %Total
~~~~~~~~~~~~~~~~~~ wait Call
Event Waits Time (s) (ms) Time
CPU time 2,194 71.4
db file sequential read 131,781 323 2 10.5
db file scattered read 89,404 206 2 6.7
Disk file operations I/O 91,788 176 2 5.7
direct path read 17,001 89 5 2.9
Host CPU (CPUs: 8 Cores: 4 Sockets: 1)
~~~~~~~~ Load Average
Begin End User System Idle WIO WCPU
16.82 5.62 77.56
Instance CPU
~~~~~~~~~~~~ % Time (seconds)
Host: Total time (s): 15,156.8
Host: Busy CPU time (s): 3,401.3
% of time Host is Busy: 22.4
Instance: Total CPU time (s): 2,827.1
% of Busy CPU used for Instance: 83.1
Instance: Total Database time (s): 3,753.3
%DB time waiting for CPU (Resource Mgr): 0.0
Memory Statistics Begin End
~~~~~~~~~~~~~~~~~ ------------ ------------
Host Mem (MB): 20,468.5 20,468.5
SGA use (MB): 11,022.6 11,022.6
PGA use (MB): 779.3 807.4
% Host Mem used for SGA+PGA: 57.7 57.8
Time Model System Stats DB/Inst: ORACLE/oracle Snaps: 511-512
-> Ordered by % of DB time desc, Statistic name
Statistic Time (s) % DB time
sql execute elapsed time 2,912.2 78.8
DB CPU 2,825.0 76.4
connection management call elapsed 343.6 9.3
PL/SQL execution elapsed time 56.6 1.5
parse time elapsed 42.8 1.2
hard parse elapsed time 25.1 .7
PL/SQL compilation elapsed time 1.1 .0
repeated bind elapsed time 1.0 .0
inbound PL/SQL rpc elapsed time 0.7 .0
hard parse (sharing criteria) elaps 0.5 .0
sequence load elapsed time 0.1 .0
failed parse elapsed time 0.0 .0
hard parse (bind mismatch) elapsed 0.0 .0
DB time 3,697.2
background elapsed time 56.1
background cpu time 2.1
Foreground Wait Events DB/Inst: ORACLE/oracle Snaps: 511-512
-> Only events with Total Wait Time (s) >= .001 are shown
-> ordered by Total Wait Time desc, Waits desc (idle events last)
Avg %Total
%Tim Total Wait wait Waits Call
Event Waits out Time (s) (ms) /txn Time
db file sequential read 131,324 0 321 2 28.9 10.5
db file scattered read 89,376 0 206 2 19.6 6.7
Disk file operations I/O 91,705 0 176 2 20.2 5.7
direct path read 16,992 0 89 5 3.7 2.9
log file sync 5,064 0 16 3 1.1 .5
db file parallel read 1,575 0 14 9 0.3 .4
enq: KO - fast object checkp 8 0 5 591 0.0 .2
control file sequential read 4,457 0 2 0 1.0 .1
direct path write temp 1,635 0 2 1 0.4 .1
SQL*Net more data to client 14,776 0 1 0 3.2 .0
SQL*Net message from dblink 603 0 0 1 0.1 .0
ADR block file read 91 0 0 4 0.0 .0
direct path read temp 713 0 0 0 0.2 .0
SQL*Net break/reset to clien 152 0 0 0 0.0 .0
asynch descriptor resize 8,239 100 0 0 1.8 .0
library cache: mutex X 1,238 0 0 0 0.3 .0
SQL*Net more data from dblin 345 0 0 0 0.1 .0
ADR block file write 5 0 0 0 0.0 .0
latch free 66 0 0 0 0.0 .0
direct path write 4 0 0 0 0.0 .0
cursor: pin S 10 0 0 0 0.0 .0
SQL*Net message from client 526,480 0 238,770 454 115.7
jobq slave wait 3,954 100 2,034 514 0.9
wait for unread message on b 1,896 98 1,894 999 0.4
Streams AQ: waiting for mess 379 100 1,892 4993 0.1
SQL*Net more data from clien 11,597 0 4 0 2.5
single-task message 29 0 1 21 0.0
SQL*Net message to client 526,478 0 1 0 115.7
Background Wait Events DB/Inst: ORACLE/oracle Snaps: 511-512
-> Only events with Total Wait Time (s) >= .001 are shown
-> ordered by Total Wait Time desc, Waits desc (idle events last)
Avg %Total
%Tim Total Wait wait Waits Call
Event Waits out Time (s) (ms) /txn Time
log file parallel write 7,020 0 16 2 1.5 .5
db file parallel write 5,529 0 14 3 1.2 .5
control file sequential read 5,966 0 5 1 1.3 .2
control file parallel write 1,618 0 3 2 0.4 .1
log file sequential read 66 0 3 50 0.0 .1
SQL*Net more data to client 64,218 0 2 0 14.1 .1
db file sequential read 457 0 2 4 0.1 .1
os thread startup 66 0 1 8 0.0 .0
Disk file operations I/O 83 0 0 1 0.0 .0
asynch descriptor resize 64,343 100 0 0 14.1 .0
direct path read 9 0 0 6 0.0 .0
db file scattered read 28 0 0 2 0.0 .0
rdbms ipc reply 8 0 0 0 0.0 .0
LGWR wait for redo copy 57 0 0 0 0.0 .0
log file single write 8 0 0 0 0.0 .0
db file single write 1 0 0 2 0.0 .0
rdbms ipc message 14,376 51 31,809 2213 3.2
DIAG idle wait 3,738 100 3,788 1013 0.8
smon timer 12 33 2,077 ###### 0.0
dispatcher timer 32 100 1,920 60012 0.0
Streams AQ: qmn coordinator 136 50 1,905 14009 0.0
Streams AQ: qmn slave idle w 68 0 1,905 28017 0.0
pmon timer 2,099 30 1,896 903 0.5
Space Manager: slave idle wa 381 98 1,894 4970 0.1
shared server idle wait 63 100 1,891 30014 0.0
SQL*Net message from client 257,103 0 1,741 7 56.5
SQL*Net more data from clien 64,218 0 88 1 14.1
SQL*Net message to client 192,827 0 0 0 42.4
-------------------------------------------------------------Now , I am perform only analyzed method for statics.
but even though dbms_stats is new method and cover all limit of analyzed method why should you not prefer it?..
second thing we don't have licence for performance tuning so,AWR does not work . I am using stats pack utility and some query to find top sql on load time which are as below:
high memory consumed:
110125 19483 5.65
begin CHECK_EMP_ISSUE(P_FOR_COMP=>:P_FOR_COMP, P_FOR_TRANS_DATE=>:P_FOR_TRANS_DATE, P_FOR_EMP_CODE=>:P_FOR_EMP_CODE, P_FOR_DEPT_CODE=>:P_FOR_DEPT_CODE, P_FOR_PROCESS=>:P_FOR_PROCESS, P_FOR_KAPAN_NO=>:P_FOR_KAPAN_NO, P_FOR_PACKET_NO=>:P_FOR_PACKET_NO, P_FOR_PACKET_ID=>:P_FOR_PACKET_ID, P_FOR_ISSUE_TYPE=>:P_FOR_ISSUE_TYPE, P_FOR_SHIFT_NO=>:P_FOR_SHIFT_NO, P_FOR_MC_CODE=>:P_FOR_MC_CODE, P_FOR_CHECK_PREF_EMP=>:P_FOR_CHECK_PREF_EMP, P_FOR_DUMMY=>:P_FOR_DUMMY, P_FOR_MSG=>:P_FOR_MSG, P_FOR_WARNING=>:P_FOR_WARNING, P_FOR_MSG_VALUE=>:P_FOR_MSG_VALUE, VREC=>:VREC); end; high i/o consumed:
EVENT WAIT_CLASS USER_IO_WAIT_TIME SQL_TEXT
asynch descriptor resize Other 32454263 SELECT /*+ result_cache */ COMP_CODE,KAPAN_GROUP,VKAPAN_GROUP,SEQ_NO,PACKET_ID,CHILD_ID,INW_DATE, VKAPAN_NO,KAPAN_NO,VPACKET_NO,PACKET_NO,SUB_ID,STONE_TYPE,PCS,WGT,CUR_WGT,P_SEQ_NO,L_SEQ_NO, STAGE,STATUS,PACKET_TYPE,:B1 DEPT_CODE,EMP_CODE,PROCESS,CLV_END_DATE,MFG_END_DATE,TRUNC(SYSDATE)-TRUNC(INW_DATE)VDAYS FROM ( SELECT /*+ leading(a) use_hash(b)*/ :B3 COMP_CODE,A.KAPAN_GROUP,A.VKAPAN_GROUP,:B1 DEPT_CODE,A.SEQ_NO,A.PACKET_ID,A.CHILD_ID,A.INW_DATE, A.VKAPAN_NO,A.KAPAN_NO,A.VPACKET_NO,A.PACKET_NO,A.SUB_ID,A.STONE_TYPE,A.PCS PCS,A.WGT,A.CUR_WGT,A.P_SEQ_NO,A.L_SEQ_NO, -
New session was refused due to memory bottleneck
Has anyone had experience with this error and know how to go about fixing it from occuring again?
Error when processing your request
What has happened?
The URL https://____________/zpro was not called due to an error.
Note
The following error text was processed in the system DEV : New session was refused due to memory bottleneck
The error occurred on the application server onebase_DEV_00 and in the work process 0 .
The termination type was: ABORT_MESSAGE_STATE
The ABAP call stack was:
Function: ICF_ATTACH_ITS_PLUGIN of program SAPLHTTP_RUNTIME
Method: EXECUTE_REQUEST_FROM_MEMORY of program CL_HTTP_SERVER================CP
Function: HTTP_DISPATCH_REQUEST of program SAPLHTTP_RUNTIME
Module: %_HTTP_START of program SAPMHTTP
What can I do?
If the termination type was RABAX_STATE, then you can find more information on the cause of the termination in the system DEV in transaction ST22.
If the termination type was ABORT_MESSAGE_STATE, then you can find more information on the cause of the termination on the application server onebase_DEV_00 in transaction SM21.
If the termination type was ERROR_MESSAGE_STATE, then you can search for more information in the trace file for the work process 0 in transaction ST11 on the application server onebase_DEV_00 . In some situations, you may also need to analyze the trace files of other work processes.
If you do not yet have a user ID, contact your system administrator.
Error code: ICF-IE-https -c: 160 -u: ****** -l: E -s: DEV -i: onebase_DEV_00 -w: 0 -d: 20080822 -t: 094930 -v: ABORT_MESSAGE_STATE -e: New session was refused due to memory bottleneck
HTTP 500 - Internal Server Error
Your SAP Internet Communication Framework TeamHi ,
I am having an issue related to the query in this thread. Can you please let me know if you any solution for this.
The following error text was processed in the system : Error occurred while sending data through the HTTP protocol.
The error occurred on the application server bwdbi_DIB_41 and in the work process 0 .
The termination type was: RABAX_STATE
The ABAP call stack was:
Method: SEND_RESPONSE of program CL_HTTP_SERVER_NET============CP
Function: HTTP_DISPATCH_REQUEST of program SAPLHTTP_RUNTIME
Module: %_HTTP_START of program SAPMHTTP
tHE ABOVE IS THE error message. This occurs when we are trying to call the a URL from web report using RRI, where in this url is used for RRI.
Regards
Sudha -
Any way to get CPU-seconds for process or a thread from within my app?
I'm trying to guage the work by different processes within our application. I'd like to be able to get the number of CPU-seconds before a long task and then again after the task finishes and compute the difference. I'm sure I could poll the % CPU every second and try estimate the value, but I don't think it would be very accurate.
Is there a MBean or some other way to access the CPU-seconds for the process?
It would even be better if I could get this at the thread level (which seems possible since JRA recordings can show the 'running' time on a graph drawn in milliseconds).
Thanks
-BillWe are considering making this available through a MBean in our upcoming R28 release.Do you have a program I can participate in to get beta or pre-releases of new JRockit versions? The purpose would be to do preliminary performance testing so we can get official releases to our development team more quickly once they are formally released.
I'm also hoping to find cumulative disk i/o bytes and cumulative network i/o bytes on a thread or even at the JVM levelI don't think we have that in JRockit today... What would you use it for?The use would be for performance monitoring. In a large system or a busy system, the effect of code changes can be hard to measure just based on clock time or traditional measurements like % CPU or bytes/sec at a specific instant. Knowing how much 'work' is getting done (together with the clock time) gives a better picture.
I like cumulative numbers like cpu-sec and # of bytes because I only need two measures (before and after) instead of continuous polling and summing. Some of my processes get created on their own threads, so I could potentially get these measures of work on my customer's production systems with a lot of other activity going on. By comparing samples over time, we can tell if processes are running longer due to 'more work' or due to resource bottlenecks.
I think it could be useful on shared threads too even though i don't know yet if I can see which threads specific requests are using.
-Bill
Maybe you are looking for
-
Treasury - Contract Expired - Any follow-up activities
We had our first treasury contract mature in SAP since we went live and I'm not sure if any follow-up activities need to be done. The contract is in status 'settlement.' Is there a way to get the contract into an expired/matured type status?
-
HT4527 my pc died how do i transfer my music from a iphone 3gs to itunes on a new pc
my old pc is dead but i need to know how to transfer music from a old iphone 3gs to itunes on my new pc and then on to another iphone 3gs?
-
Unexpected error for file explorer
I have a user that was trying to browse her document lib by using the explorer feature in sharepoint. At first we were unable to even get the explorer working.. but then I followed these steps: 1- patch IE For windows 7 x32 machines this update need
-
Passing parameter values to jsp
hi i have got a slight probelm that my jsp doesnt print out teh value that is being to it from teh wml my wml file is: <b><big> Subject:</big></b> <input type ="text" name="srch" height="20" format="*X,*x"/>
-
By default view the last instance for a specific report
Hello, if you went to CMC->Applications->Infoview->Properties the last property is Default Viewing Action on Listing Page: View the object View the latest successful instance of the object this will apply over all the reports in the system there is n