Rpcbind high CPU utilisation

Hi,
The issue is my CPU utilisation is unusually high. I've got a dual core laptop and one core is at 100% all of the time.  When I run top I can see that rpcbind is responsible for the high CPU usage. If I reboot the problem starts immediately.
I've only noticed this problem quite recently, but can't think of anything I've done that may have caused it.
This problem is really quite urgent because if I can't get it solved I will have to spend days installing another OS because at the moment my laptop battery is draining really fast.
Does anyone know what may be causing the problem, or how I can troubleshoot it?
Thanks in advance.

I figured out the problem. When I upgraded my kernel the other day it turns out that the kernel was installed on my root partition instead of on my boot partition because /boot was not mounted at the time. As a result my old kernel was still being booted, but parts of the system were affected by the new kernel supposedly being installed. So what I did was mount the /boot partition, reinstall the latest kernel and reboot the system - problem solved - no rpcbind taking up 100% of the CPU now.
In the end I decided to scrap fam and use gamin anyway, based on the wiki recommendation.

Similar Messages

  • How to restart IIS on high CPU utilisation

    Hi There,
    Heres some background on a problem I'm trying to mitigate with SCOM 2010.
    Got a problem with a web server, occasionally the web app causes the CPU to max out and stay there. We can't pick when it happens and often it happens out of hours when everyone is asleep so we don't know that the CPU is running maxed out until the morning.
    At this stage the only way to resolve CPU utilisation is to do iisreset on the affected server.
    We're still trying to figure out what is causing the problem but until then it would be really good to have SCOM detect the high CPU on the server and restart IIS for me. That way the site performance recovers quicker because we wouldn't need someone to
    notice the alert and issue the iisreset.
    I've been tooling around in SCOM trying to figure out how I can have SCOM restart IIS on the affected server when the CPU spikes but I'm just not sure how to do it.
    I only want this IIS Restart Recovery action to only affect the server that has been playing up, rather than affecting all servers that run IIS.
    It would be good to have SCOM detect that the IIS service (w3wp) is the service maxing out the CPU and to restart it on the affected server. But I'd also accept SCOM simply detecting high CPU (> 95%) on the CPU and restarting the IIS service because nothing
    else on the server has been seen to cause the CPU to max out like IIS does.
    Suggestions very much appreciated.

    Please try if this helps:
    Create a custom rule Performance Based->Windows Performance with Counter as Processor & threshold as 95%. Keep the rule disabled.
    Goto properties of that Rule & under Configuration, create a command Response to restart IIS service.
    Override the rule & enable it only for IIS computer group. 
    Hope it helps !!

  • High CPU utilisation on 2960X stack

    Hello !
    i have a lot of switches 2960-X stack in my network, but i downgraded the iOS version (15.0(2)EX4 to 15.0(2)EX3 be cause some bugs are on EX4 with SFP).
    Since I downgraded the version, i have a lot of troubles on some switches with a huge amount of CPU utilisation.
    One is "normal" and this is the famous one : Hulc LED
    but the other one is not "normal" for me :
    this line :
    190   207178513     1061976     195092  0.00% 31.06% 24.83%   0 Auth Manager
    i dont know why on some switches the "Auth Manager" CPU utilisation is too high.
    When the CPU is too high, we cannot join the switch, the ping is too high (between 500ms and 2100) and users complaining about network spikes.
    So, i dont want to upgrade the iOS to the EX5 (too recent, and i dont know if bug with SPF and CPU are fixed).
    About the Auth manager, what could raise this process to high for the CPU ?
    (i have some check with nagios and snmpwalk but every 5 min and not every minutes, and nagios works well with EX4, issues appear since EX3)
    thank for your help and advice

    15.2(2)E is the first main-line engineering software for the 2960X.  The "EX" versions you've been using are interim version and not really meant for long-term production network.  

  • High cpu utilisation while generating HUM reports

    Hi
    i am facing the issue that while generating the HUM reports the CPU utilisation going to 100%. i am using HUM 300 + LMS 100. right now i am using 2GB RAM

    you're right. i forgot that i made some changes to my igoogle page. the chat function is the culprit. adding it causes the spike. i've reported the problem. thanks for the help!

  • Firefox 13 High CPU Utilisation

    Hello I have installed the latest update and have realised that Firefox’s CPU has sky rocketed it sometimes dips to a normal level but the majority of the time it is around 8-13. With just firefox running it is around 20% utilisation. .
    I have tried using safe mode and have used CCleaner to completely remove history, compact database etc. This unfortunately hasn’t improved the problem.
    I would appreciate any advice on how to resolve the issue and wondered if anyone else has had similar issues.
    Thanks
    BTW this is on a Windows 7 x64 machine if that helps?

    Create a new profile as a test to check if your current profile is causing the problems.
    See "Basic Troubleshooting: Make a new profile":
    *https://support.mozilla.org/kb/Basic+Troubleshooting#w_8-make-a-new-profile
    There may be extensions and plugins installed by default in a new profile, so check that in "Tools > Add-ons > Extensions & Plugins" in case there are still problems.
    If the new profile works then you can transfer some files from the old profile to that new profile, but be careful not to copy corrupted files.
    See:
    *http://kb.mozillazine.org/Transferring_data_to_a_new_profile_-_Firefox

  • Solution to High CPU Utilisation With *Nothing* Running After CS4 Install on Mac OS 10.6.1.

    I'm posting this in case someone else has the same problem. (If anyone else even notices the problem.)
    Another possible title [to help with people searching for the solution] "Computer runs slow after installing CS4 on Mac OS 10.6.1 Snow Leopard"
    I just purchased CS4 Design Premium and have been trying to install it on a fresh install of Snow Leopard (10.6.1) and whilst everything appears to be fine, my CPU sits at 60% with *every* application closed. The instant I uninstall CS4 the CPU drops back to 1-2%. I've conducted this test twice already---that is installed CS4 on a fresh Snow Leopard twice over.
    To be clear this isn't resolved by restarting, as is the solution to one known bug mentioned in the CS4 read-me.
    I did eventually solve the problem. The tech support guy at Adobe was useless, even though I told him that I had re-installed twice and once re-installed Snow Leopard, he had me run the Adobe cleaning utility thingy. Three hours of re-installing (for the third time) and another 700MB of updates (from my precious download quota) and I was exactly back to where I started.
    Then when the updates for Version Cue and Adobe Drive failed to install, it got me thinking. Looking at the box it states that Java is needed for Version Cue Server. That being a unique requirement and the update for that particular program failing, sent me on the search. I eventually found the directory containing Version Cue Server and moved it. I next planned to restart but I didn't have to, the instant I moved the directory my CPU usage plummeted from 60% to 1%.
    I've been working on this for days and all I had to do was issue the command.
    mv /Library/Application\ Support/Adobe/Adobe\ Version\ Cue\ CS4/Server ~/Disabled
    *sigh*
    David.

    Thank you for posting this, I just spent the last 72 hours trying to figure out why my macbook was suddenly burning up my lap (and my battery) without any visible processes in Activity Monitor. This is a pretty major bug (and not the first major one by Adobe on mac) but I am just glad that I didn't need to do a full reinstall. Getting rid of Version Cue takes care of the problem straight out. Thanks for pointing me in the right direction, hopefully others will find this before trying to track down other sources.

  • Concurrent GC threads high CPU utilisation

    Hello,
    We are running Java 1.5.0_07 with "-XX:MaxPermSize=256M XX:+UseConcMarkSweepGC -XX:+OverrideDefaultLibthread" on Solaris 2.8, 4xSPARC CPU, 8GB memory box.
    We get random problems with the GC. Sometimes the 4 threads named: "Gang worker#N (Parallel GC Threads)" go into a CPU frenzy (the thread numbers are always 0, 1, 2 and 3, mapped to LWPs 2, 3, 4, 5). When this happens, each of these threads chews up 16-22% CPU. Overall, the java process uses 65-95% CPU. It lasts for about an hour, and drops back to normal, which is about 1-3% for each of these threads, and 8-15% for the whole process.
    This hits about once a week, but it is unpredictable. When it hits, it causes trouble for other system that run on the same box.
    Does anyone have a clue on this?
    Any help would be appreciated.
    Laszlo

    Thanks Ramki,
    We have found that the default space allocated in the heap for the young generation was much too small. We use a 3 gig heap, on which only 16 meg was allocated to the new generation (all of this went into the eden space - no space could be allocated to the survivor spaces).
    Increasing the size of the new generation (now 700 meg - we still may tweak this figure) vastly improved performance. Also the number of garbage collections executed decreased dramatically.

  • High CPU Usage in Word 2013

    Hi everyone,
    I'm having an issue with Word 2013 and I'm hoping someone can help. Seen as I got fleeced for my TechNet subscription this year (the price given on the invoice doubled when I called up to pay!) I’d very much like to see a speedy fix from the Microsofties
    please! :D
    Recently, I've found that Word will consistently exhibit high CPU utilisation, ~50% on a dual core second generation Core i5.
    This continues, even when I’m not providing any input to or interacting with the Word window and even when the Word window is covered by other windows. Minimising Word causes its CPU usage to drop back down to zero.
    The strange thing is that, so far, I’ve only observed this with a single document. I started writing this document at the start of this week and it’s been authored exclusively on Word 2013. In fact I’m the only one who’s editing it and I’ve only used a single
    computer (and this one single install of Word 2013) to do so…
    There’s nothing particularly unusual about this document, and a different but similar document which uses the same themes and features (in fact the theme for the second document was created from the first one) doesn’t appear to cause the same issues. Both
    these documents are stored on SkyDrive, and are being edited directly from there.
    Initially I tried the much recommended tweaks of disabling hardware graphics acceleration and sub-pixel positioning, however neither of these has had any effect.
    I investigated a bit further using Sysinternals Process Monitor, and found that while Word is using a lot of CPU it’s actually looping very, very quickly polling for the existence of the registry key “HKCU\Software\Microsoft\Avalon.Graphics”.
    This key doesn’t exist on my system, and a quick test I conducted where I created it just lead to Word looping polling for more non-existent registry keys.
    My system is a HP Folio 13 running Windows 8 Pro x64 with Office 2013 (RTM, not Beta) 32 bit. Both Windows and Office are fully patched, as is everything else on the system.
    Thanks in advance for any help anyone can offer.
    Chris

    Hello again,
    After working on my problematic document some more, I believe I’ve found the cause of the issue.
    @Microsofties – I think this one is either a bug or a dodgy OOTB template…
    I noticed that the high CPU usage only seemed to occur when the document’s footer was displayed. Experimenting with some careful scrolling and zooming to hide footer (without actually using the “hide white-space” option) seemed to confirm this so I investigated
    the footer a bit more.
    My document was using the standard Word “Facet (Odd Page)” footer template with a couple of fields added to suit my needs, namely “Publish Date” and a page number.
    The template seems to be structured such that the content of the footer is inside two nested text boxes (one inside the other). I also noticed previously that this footer template also appears to “jump” up each page and back down again as I scrolled.
    So I deleted the footer and its text boxes, and recreated the same content, in the same position (using the standard paragraph alignment tools) by hand without the nested text boxes. This has, so far, fixed the issue!
    I also noticed that the CPU usage dropped right down to idle the moment I deleted the nested text boxes.
    Altogether, this leads me to conclude that the “Facet (Odd Page)” footer template in Word 2013 is the source of my issue. Specifically, I believe that the use of nested text boxes in this template causes Word’s interactions with the .Net 4.5 WPF to go a
    bit nuts, as evidenced by the infinite loop of request for the non-existent “Avalon.Graphics” WPF registry key I mentioned previously.
    It would be nice to see some feedback on this one, Microsoft people, if only to know if this is actually a general issue or is specific to my document!
    Thanks, Chris

  • High cpu usage by query

    We have a table named "tbl_geodata" which has address and latitude and longitude values. We
    also have a "History" table which has only latitude and longitude values including other information. What we need is like following...
    We get a set of records based on a query from "History" (lat long values), say 5000 records
    Now we are using the following formula to calculate address from the "tbl_geodata" for each row
    (5000 rows).
        SELECT top 1 geo_street,geo_town,geo_country,( 3959 acos( cos( radians(History.lat) ) cos( radians(
    gps_latitude ) ) cos( radians( gps_longitude ) - radians(History.long) ) + sin( radians(History.lat) ) sin( radians( gps_latitude ) ) ) ) AS distance FROM tbl_geodata ORDER BY distance
    Currently we are seeing high cpu utilisation and performance issue. What would be the optimized way to do
    this

    We are using SQL Server Web Edition. We have a table which has around 120 million records (every second around 100 insertion). It has a AFTER INSERT trigger to update another table. There is following performance issue we are facing. 
    High CPU Usage
    Insertion failed (Connection Timeout Expired.  The timeout period elapsed during the post-login phase.  The connection could have timed out while waiting for server to complete the login process and respond; Or it could have timed out while attempting
    to create multiple active connections.  The duration spent while attempting to connect to this server was - [Pre-Login] initialization=0; handshake=10046; [Login] initialization=0; authentication=0; [Post-Login] complete=3999;)
    Insertion Failed (A severe error occurred on the current command.  The results, if any, should be discarded.)
    Insertion Failed (Some time we are getting connection pool error.. There is no limit set in the connection string)
    I ran sp_who2 command and found a lot of queries are in suspended mode..
    here is the "sys.dm_os_sys_info" result...
    cpu_count   hyperthread_ratio
    physical_memory_in_bytes virtual_memory_in_bytes
    8                   8
                                12853739520
                 8796092891136
    Can anyone please suggest the improvement steps...

  • High CPU load on 2811

    Hi,
    We have a 2811 router and we are facing high CPU load but show proc cpu does not show processes with high CPU utilisation.
    RT#show proc cpu sorted | exclude 0.00%
    CPU utilization for five seconds: 99%/94%; one minute: 99%; five minutes: 98%
     PID Runtime(ms)   Invoked      uSecs   5Sec   1Min   5Min TTY Process
      95     2157484   1814314       1189  3.29%  4.76%  4.68%   0 IP Input        
       2       14624     44123        331  0.94%  0.64%  0.63%   0 Load Meter      
       5      154552      2451      63056  0.37%  0.97%  0.91%   0 Pool Manager    
      38      126116    682626        184  0.18%  0.23%  0.24%   0 Net Background  
     185      142660    545202        261  0.09%  0.08%  0.08%   0 Crypto PAS Proc 
      88       58804   6821262          8  0.09%  0.01%  0.01%   0 ACCT Periodic Pr
     102        4548       395      11513  0.09%  0.11%  0.03% 514 SSH Process     
      42       12480    220592         56  0.09%  0.03%  0.02%   0 Per-Second Jobs 
    This does not happed outside working hours. The router is used only for VPN's (No NATing). It terminates around 25 tunnels but there is not a lot of traffic through the tunnels.
    The extremly high load started mondat morning and continues this morning. During the weekend it was fine. Earlier we would have situations when the CPU would spike but it would soon be down to normal values.
    From what I read the above output means that the CPU load is caused by interrupts. What is the best way to troubleshoot this?
    Regards.

    Disclaimer
    The Author of this posting offers the information contained within this posting without consideration and with the reader's understanding that there's no implied or expressed suitability or fitness for any purpose. Information provided is for informational purposes only and should not be construed as rendering professional advice of any kind. Usage of this posting's information is solely at reader's own risk.
    Liability Disclaimer
    In no event shall Author be liable for any damages whatsoever (including, without limitation, damages for loss of use, data or profit) arising out of the use or inability to use the posting's information even if Author has been advised of the possibility of such damage.
    Posting
    Sorry I wasn't clear, but when I was asking about "not a lot of traffic", I was wondering about actual  volume passing through the 2811.
    When I asked about "kind of tunnels", I was wondering about how they are configured.  E.g. GRE or IPSec/GRE or VTI, encryption options, other tunnel interface options.
    Perhaps the easiest way to check for fragmentation, is packet analyze the traffic passing through the 2811.

  • High CPU on CSS11503 , with one Flow

    Hi,
    I've seen some high CPU Utilisation on 2 CSS.
    I've figured out, that this Utilisation was created, by a mirroring Task from a database Server.
    There was one TCP Connection between 2 Servers, whitch transfered about 60 Mbit/s.
    But i saw a CPU-Utilisation from about 30-40% on my SCM which passed the Traffic.
    That is some thing i do not understand, because the Forwading of this Flow shoud be done in the Flow-Processor, and the CPU Utilisation which is monitored, is the Session Processor.
    The Protocol was TCP 1500, which is Used by Tivoli Storage Manager.
    Anybody has some Idea, how this makes so much CPU Utilisation?
    The Transfer was about 4 hours, and the CPU Utilisation also.
    Best regards
    Sven

    Hello Gilles,
    may i try to show you the network design
    <-> <-> <-> css <->
    So the traffic was hitting 2 css.
    One 11506 with 7.50 103
    One 11503 with 7.50.103
    The traffic was one TCP Connection between Server1 and Server2 (TCP 1500).
    There are no ACL and the traffic was not hitting a content rule.
    The 11506 handels normal about 30 mbit/s out, and 10 mbit/s in. Average CPU is about 1% over a day.
    The 11053 handles normaly about 5mit/ out and 1 mbit/s in. Average CPU is about 4% over a day.
    CPU was about 30% higher on both css, during the replicarton (~3 hours)
    I did not took any debug or show commands, because i saw it on my mrtg a day later.
    Also, the high cpu was only seen on the SCM Module, the second Module (IOM) was normal.
    Best regads
    Sven

  • High CPU Utlisation ws2003 and 2008

    I am trying to understand how to analyze High CPU Utilisation in WS 2003 and 2008 .How do I find the process causing the issue and entire days logs
    .Please help me

    Hi Ritesh,
    To troubleshoot the high CPU Usage, you can also check these tools:
    1. Task Manager (built-in to the O.S.)
    2. Process Explorer
    http://technet.microsoft.com/en-us/sysinternals/bb896653.aspx
    Make sure to enable the Microsoft public symbol to get stack information.
    3. Perfmon (built-in to the O.S.)
    4. Windows Performance Toolkit (XPerf is what replaced KernRate for Windows Server 2008 and Windows Server 2008 R2)
    http://msdn.microsoft.com/en-us/performance/default.aspx
    5. Complete memory dump.
    972110 How to generate a kernel dump file or a complete memory dump file in Windows Server 2003
    http://support.microsoft.com/?id=972110
    969028 How to generate a kernel or a complete memory dump file in Windows Server 2008
    http://support.microsoft.com/?id=969028
    And you can also refer to the troubleshooting steps in these article:
    How to troubleshoot high cpu in the System process
    The Case of the System Process CPU Spikes
    If there is anything else regarding this issue, please feel free to post back.
    Best Regards,
    Anna Wang
    Please remember to mark the replies as answers if they help and unmark them if they provide no help. If you have feedback for TechNet Support, contact [email protected]

  • EA2: high CPU usage with large packages

    EA2 + Debian GNU/Linux + jdk1.6.0_04 + Pentium IV 2,8 GHz 1 GBytes RAM
    Opening large packages (> 6.000 rows) I have high CPU usage (100%) for about 50-60 seconds. Same problem when I working with these packages
    Ciao
    Davide

    Opening works fine for me, and memory seems to be
    contained. Working inside the package on occasions
    still boosts CPU like you.Attempting to edit an 800 line type body utilises 100% CPU as soon as typing starts.
    Windows XP Professional SP2
    Oracle Server 10.2.0.3
    Java 1.5.0_06
    Intel Pentium M 1.69GHz 2GB RAM

  • High CPU consuming

    Dear All
    I have 7200 router, it was runing 124-4.XD3 IOS, the CPU was working normal avarage around 15%, its use for L2TP users, it was have avarage 2000 concurrent L2TP connection, there was no CPU consuming.
    Then for business plan we change the IOS to 12.2(33)SR and configured Broadband aggregation, after that the CPU become higher day after day even the PPPoE and L2TP session not more than 900 concurrent. the CPU reach 60%, even in non-peak hours it reach 50%
    sh ver
    Cisco IOS Software, 7200 Software (C7200P-ADVIPSERVICESK9-M), Version 12.2(33)SR
    C6, RELEASE SOFTWARE (fc1)
    Technical Support: http://www.cisco.com/techsupport
    Copyright (c) 1986-2010 by Cisco Systems, Inc.
    Compiled Mon 08-Mar-10 08:03 by prod_rel_team
    ROM: System Bootstrap, Version 12.4(12.2r)T, RELEASE SOFTWARE (fc1)
    BOOTLDR: Cisco IOS Software, 7200 Software (C7200P-KBOOT-M), Version 12.2(33)SRC
    6, RELEASE SOFTWARE (fc1)
    System returned to ROM by reload at 14:58:14 UTC Thu Jun 2 2011
    System image file is "disk2:c7200p-advipservicesk9-mz.122-33.SRC6.bin"
    Last reload reason: Reload command
    This product contains cryptographic features and is subject to United
    States and local country laws governing import, export, transfer and
    use. Delivery of Cisco cryptographic products does not imply
    third-party authority to import, export, distribute or use encryption.
    Importers, exporters, distributors and users are responsible for
    compliance with U.S. and local country laws. By using this product you
    agree to comply with applicable laws and regulations. If you are unable
    to comply with U.S. and local laws, return this product immediately.
    A summary of U.S. laws governing Cisco cryptographic products may be found at:
    http://www.cisco.com/wwl/export/crypto/tool/stqrg.html
    If you require further assistance please contact us by sending email to
    [email protected].
    Cisco 7206VXR (NPE-G2) processor (revision A) with 917504K/65536K bytes of memor
    y.
    Processor board ID 36743260
    MPC7448 CPU at 1666Mhz, Implementation 0, Rev 2.2
    6 slot VXR midplane, Version 2.11
    Last reset from power-on
    PCI bus mb1 (Slots 1, 3 and 5) has a capacity of 600 bandwidth points.
    Current configuration on bus mb1 has a total of 0 bandwidth points.
    This configuration is within the PCI bus capacity and is supported.
    PCI bus mb2 (Slots 2, 4 and 6) has a capacity of 600 bandwidth points.
    Current configuration on bus mb2 has a total of 0 bandwidth points.
    This configuration is within the PCI bus capacity and is supported.
    Please refer to the following document "Cisco 7200 Series Port Adaptor
    Hardware Configuration Guidelines" on Cisco.com <http://www.cisco.com>
    for c7200 bandwidth points oversubscription and usage guidelines.
    1 FastEthernet interface
    3 Gigabit Ethernet interfaces
    2045K bytes of NVRAM.
    250880K bytes of ATA PCMCIA card at slot 2 (Sector size 512 bytes).
    65536K bytes of Flash internal SIMM (Sector size 512K).
    Configuration register is 0x2102
    sh processes cpu
    CPU utilization for five seconds: 73%/17%; one minute: 67%; five minutes: 66%
    PID Runtime(ms)   Invoked      uSecs   5Sec   1Min   5Min TTY Process
       1         620     11401         54  0.00%  0.00%  0.00%   0 Chunk Manager   
       2        1400    238221          5  0.00%  0.00%  0.00%   0 Load Meter      
       3           0         1          0  0.00%  0.00%  0.00%   0 L2X Socket proce
       4     4594808    312806      14689  7.27%  0.99%  0.69%   0 Check heaps     
       5       36380    195739        185  0.07%  0.00%  0.00%   0 Pool Manager    
       6           0         2          0  0.00%  0.00%  0.00%   0 Timers          
       7     1789408   3141124        569  0.15%  0.16%  0.16%   0 ARP Input       
       8         408   1241861          0  0.00%  0.00%  0.00%   0 ARP Background  
       9           0         2          0  0.00%  0.00%  0.00%   0 ATM Idle Timer  
      10           0         1          0  0.00%  0.00%  0.00%   0 ATM ASYNC PROC  
      11           0        85          0  0.00%  0.00%  0.00%   0 AAA_SERVER_DEADT
      12           0         1          0  0.00%  0.00%  0.00%   0 Policy Manager  
      13           0         4          0  0.00%  0.00%  0.00%   0 Entity MIB API  
      14       87356   1933514         45  0.00%  0.00%  0.00%   0 EEM ED Syslog   
      15           8     19853          0  0.00%  0.00%  0.00%   0 IPC Dynamic Cach
      16           0         1          0  0.00%  0.00%  0.00%   0 IPC Zone Manager
      17        1276    237895          5  0.00%  0.00%  0.00%   0 IPC Service NonC
      18         896   1189354          0  0.00%  0.00%  0.00%   0 IPC Periodic Tim
      19           0         1          0  0.00%  0.00%  0.00%   0 IPC Managed Time
      20         644   1189353          0  0.00%  0.00%  0.00%   0 IPC Deferred Por
      21           0         1          0  0.00%  0.00%  0.00%   0 IPC Seat Manager
      22           0         1          0  0.00%  0.00%  0.00%   0 IPC Seat Control
      23          88    238222          0  0.00%  0.00%  0.00%   0 IPC Loadometer  
      24           0         2          0  0.00%  0.00%  0.00%   0 Serial Backgroun
      25           0         1          0  0.00%  0.00%  0.00%   0 RO Notify Timers
      26           0         1          0  0.00%  0.00%  0.00%   0 RMI RM Notify Wa
      27         472    151083          3  0.00%  0.00%  0.00%   0 DDR Timers      
      28           0         2          0  0.00%  0.00%  0.00%   0 ATM AutoVC Perio
      29           0         2          0  0.00%  0.00%  0.00%   0 ATM VC Auto Crea
      30          16    119115          0  0.00%  0.00%  0.00%   0 Compute SRP rate
      31     2400080  21414450        112  0.00%  0.01%  0.08%   0 EnvMon          
      32           0         1          0  0.00%  0.00%  0.00%   0 OIR Handler     
      33           0         1          0  0.00%  0.00%  0.00%   0 Crash writer    
      34           0         1          0  0.00%  0.00%  0.00%   0 Exception contro
      35        2200    237093          9  0.00%  0.00%  0.00%   0 ALARM_TRIGGER_SC
      36           0         1          0  0.00%  0.00%  0.00%   0 CEF MIB API     
      37         948   1189356          0  0.00%  0.00%  0.00%   0 GraphIt         
      38           0         1          0  0.00%  0.00%  0.00%   0 SERIAL A'detect 
      39           0         2          0  0.00%  0.00%  0.00%   0 Dialer event    
      40           0         2          0  0.00%  0.00%  0.00%   0 SMART           
      41           0         1          0  0.00%  0.00%  0.00%   0 RFS server proce
      42           0         1          0  0.00%  0.00%  0.00%   0 Critical Bkgnd  
      43      112568    578621        194  0.00%  0.04%  0.05%   0 Net Background  
      44           0         2          0  0.00%  0.00%  0.00%   0 IDB Work        
      45       23576    664891         35  0.00%  0.00%  0.00%   0 Logger          
      46        1564   1189352          1  0.00%  0.00%  0.00%   0 TTY Background  
      47       38788   1189374         32  0.00%  0.03%  0.02%   0 Per-Second Jobs 
      48       80568     23708       3398  0.00%  0.00%  0.00%   0 Per-minute Jobs 
      49         776         5     155200  0.00%  0.00%  0.00%   0 PrstVbl         
      50        5160    154631         33  0.00%  0.00%  0.00%   0 IF-MGR control p
      51           0        15          0  0.00%  0.00%  0.00%   0 IF-MGR event pro
      52           0         1          0  0.00%  0.00%  0.00%   0 Inode Table Dest
      53           4     13677          0  0.00%  0.00%  0.00%   0 Transport Port A
      54           0         2          0  0.00%  0.00%  0.00%   0 INTR MGR PROCESS
      55     1553620    253665       6124  0.00%  0.13%  0.12%   0 HC Counter Timer
      56           0         2          0  0.00%  0.00%  0.00%   0 CES Timer       
      57           0         1          0  0.00%  0.00%  0.00%   0 POS APS Event Pr
      58           0         1          0  0.00%  0.00%  0.00%   0 SONET alarm time
      59           0         2          0  0.00%  0.00%  0.00%   0 Hawkeye Backgrou
      60           0         1          0  0.00%  0.00%  0.00%   0 CSP Timer       
      61           0         2          0  0.00%  0.00%  0.00%   0 FPD Management P
      62           0         1          0  0.00%  0.00%  0.00%   0 FPD Action Proce
      63        1108     74393         14  0.00%  0.00%  0.00%   0 Collection proce
      64           0         1          0  0.00%  0.00%  0.00%   0 CES Line Conditi
      65           0         2          0  0.00%  0.00%  0.00%   0 Flash MIB Update
      66           4         6        666  0.00%  0.00%  0.00%   0 Flash Card Oir  
      67           0         1          0  0.00%  0.00%  0.00%   0 Async write proc
      68           0         1          0  0.00%  0.00%  0.00%   0 Async write proc
      69           0         1          0  0.00%  0.00%  0.00%   0 Async write proc
      70           0         1          0  0.00%  0.00%  0.00%   0 Async write proc
      71           8     19853          0  0.00%  0.00%  0.00%   0 DFS flush period
      72     1190120  12956856         91  0.23%  0.23%  0.22%   0 AAA Server      
      73       37948    166779        227  0.00%  0.00%  0.00%   0 AAA ACCT Proc   
      74        3796  36930272          0  0.00%  0.00%  0.00%   0 ACCT Periodic Pr
      75           0         1          0  0.00%  0.00%  0.00%   0 AAA System Acct 
      76           0         1          0  0.00%  0.00%  0.00%   0 tHUB            
      77           0         2          0  0.00%  0.00%  0.00%   0 AAA Dictionary R
      78           0         2          0  0.00%  0.00%  0.00%   0 CEF switching ba
      79           0         2          0  0.00%  0.00%  0.00%   0 IPAM/ODAP Events
      80        4792  36930277          0  0.00%  0.01%  0.00%   0 IPAM Manager    
      81           0         2          0  0.00%  0.00%  0.00%   0 IPAM Events     
      82           0        15          0  0.00%  0.00%  0.00%   0 IP ARP Adjacency
      83     4277228  78191448         54  0.95%  0.95%  0.92%   0 IP Input        
      84           0         1          0  0.00%  0.00%  0.00%   0 ICMP event handl
      85           0         5          0  0.00%  0.00%  0.00%   0 TurboACL        
      86           0         2          0  0.00%  0.00%  0.00%   0 TurboACL chunk  
      87         168      3204         52  0.00%  0.00%  0.00%   0 ADJ resolve proc
      88           0         1          0  0.00%  0.00%  0.00%   0 IPv6 Echo event 
      89           0         1          0  0.00%  0.00%  0.00%   0 LSP Tunnel FRR  
      90           0         1          0  0.00%  0.00%  0.00%   0 MPLS Auto-Tunnel
      91           0         1          0  0.00%  0.00%  0.00%   0 O-UNI Client Msg
      92           0         1          0  0.00%  0.00%  0.00%   0 AAA EPD HANDLER 
      93           0         1          0  0.00%  0.00%  0.00%   0 PM EPD API      
      94           0         5          0  0.00%  0.00%  0.00%   0 PPP SIP         
      95       14144    101572        139  0.00%  0.00%  0.00%   0 PPP Bind        
      96       33920    303835        111  0.00%  0.00%  0.00%   0 PPP IP Route    
      98     1568632  37728948         41  0.15%  0.13%  0.11%   0 SSM connection m
      99           0         2          0  0.00%  0.00%  0.00%   0 Spanning Tree   
    100           0         2          0  0.00%  0.00%  0.00%   0 ATM OAM Input   
    101           0         2          0  0.00%  0.00%  0.00%   0 ATM OAM TIMER   
    102          28      1987         14  0.00%  0.00%  0.00%   0 MOP Protocols   
    103     6304800  22987724        274  1.03%  0.88%  0.81%   0 SSS Manager     
    104           0         1          0  0.00%  0.00%  0.00%   0 SSS Policy Manag
    105       22984    468723         49  0.00%  0.00%  0.00%   0 SSS Feature Mana
    106         860   4652544          0  0.00%  0.00%  0.00%   0 SSS Feature Time
    107           0         1          0  0.00%  0.00%  0.00%   0 SG DPM          
    108           0         1          0  0.00%  0.00%  0.00%   0 Timer handler fo
    109           0         1          0  0.00%  0.00%  0.00%   0 Prepaid response
    110           0         1          0  0.00%  0.00%  0.00%   0 Timed Policy act
    111           0         1          0  0.00%  0.00%  0.00%   0 AAA response han
    112           0         2          0  0.00%  0.00%  0.00%   0 KRB5 AAA        
    113           0         1          0  0.00%  0.00%  0.00%   0 delayed evt hand
    114           0         1          0  0.00%  0.00%  0.00%   0 AC Switch       
    115           0         1          0  0.00%  0.00%  0.00%   0 SG CMD HANDLER  
    116           0         2          0  0.00%  0.00%  0.00%   0 IP PORTBUNDLE   
    117          52    290816          0  0.00%  0.00%  0.00%   0 IP SIP Process  
    118        4044  36930299          0  0.00%  0.00%  0.00%   0 RADIUS Proxy    
    119           0         1          0  0.00%  0.00%  0.00%   0 TC SIP          
    120          72     23792          3  0.00%  0.00%  0.00%   0 CEF background p
    121           0         1          0  0.00%  0.00%  0.00%   0 IP IRDP         
    122           0         2          0  0.00%  0.00%  0.00%   0 SNMP Timers     
    123       42784    919143         46  0.00%  0.00%  0.00%   0 IP Background   
    124      124252    846556        146  0.00%  0.00%  0.00%   0 IP RIB Update   
    125          32       998         32  0.00%  0.00%  0.00%   2 Virtual Exec    
    126        4900    178197         27  0.00%  0.00%  0.00%   0 VPDN failure man
    127           0         1          0  0.00%  0.00%  0.00%   0 IP Traceroute   
    128           0         1          0  0.00%  0.00%  0.00%   0 Socket Timers   
    129        1644     27024         60  0.00%  0.00%  0.00%   0 DHCPD Receive   
    130          52      2068         25  0.00%  0.00%  0.00%   0 static          
    131         240     49569          4  0.00%  0.00%  0.00%   0 TCP Timer       
    132        1696      7889        214  0.00%  0.00%  0.00%   0 TCP Protocols   
    133           0      3971          0  0.00%  0.00%  0.00%   0 HTTP CORE       
    134       10896 293774766          0  0.07%  0.07%  0.07%   0 HQF Shaper Backg
    135        6972 293774795          0  0.07%  0.06%  0.07%   0 HQF Input Shaper
    136           0         2          0  0.00%  0.00%  0.00%   0 LFDp Input Proc 
    137           0         2          0  0.00%  0.00%  0.00%   0 SCTP Main Proces
    138           0         2          0  0.00%  0.00%  0.00%   0 PPP Compress Inp
    139           0         2          0  0.00%  0.00%  0.00%   0 PPP Compress Res
    140           0         1          0  0.00%  0.00%  0.00%   0 COPS            
    141           0         2          0  0.00%  0.00%  0.00%   0 Dialer Forwarder
    142           0         3          0  0.00%  0.00%  0.00%   0 Flow Exporter Ti
    143           0         1          0  0.00%  0.00%  0.00%   0 RARP Input      
    144           0         2          0  0.00%  0.00%  0.00%   0 PPP NBF         
    145           0         1          0  0.00%  0.00%  0.00%   0 PAD InCall      
    146           0         2          0  0.00%  0.00%  0.00%   0 X.25 Background 
    147           0         1          0  0.00%  0.00%  0.00%   0 X.25 Encaps Mana
    148        2452  11862393          0  0.00%  0.00%  0.00%   0 RBSCP Background
    149           0         1          0  0.00%  0.00%  0.00%   0 VPDN call manage
    150      748224   1825397        409  0.23%  0.16%  0.16%   0 CEF: IPv4 proces
    151        3408     77652         43  0.00%  0.00%  0.00%   0 ADJ background  
    152           0         1          0  0.00%  0.00%  0.00%   0 ac_atm_state_eve
    153           0         2          0  0.00%  0.00%  0.00%   0 IP Inband Sessio
    154           0         1          0  0.00%  0.00%  0.00%   0 EM Background Pr
    155           0         1          0  0.00%  0.00%  0.00%   0 IDMGR CORE      
    156         240      4710         50  0.00%  0.00%  0.00%   0 LOCAL AAA       
    157           0         5          0  0.00%  0.00%  0.00%   0 MPLS Auto Mesh P
    158         360       219       1643  0.00%  0.20%  0.07%   3 Virtual Exec    
    159           0         2          0  0.00%  0.00%  0.00%   0 ENABLE AAA      
    160           0         2          0  0.00%  0.00%  0.00%   0 LINE AAA        
    161           0         2          0  0.00%  0.00%  0.00%   0 TPLUS           
    162           0         2          0  0.00%  0.00%  0.00%   0 Crypto Support  
    163           0        12          0  0.00%  0.00%  0.00%   0 crypto engine pr
    164           0         1          0  0.00%  0.00%  0.00%   0 Key Proc        
    165           0         6          0  0.00%  0.00%  0.00%   0 Crypto ACL      
    166           0         1          0  0.00%  0.00%  0.00%   0 Crypto INT      
    167           0         4          0  0.00%  0.00%  0.00%   0 Crypto IKMP     
    168          16     59560          0  0.00%  0.00%  0.00%   0 IPSEC key engine
    169           0         1          0  0.00%  0.00%  0.00%   0 IPSEC manual key
    170           0         1          0  0.00%  0.00%  0.00%   0 Crypto PAS Proc 
    171           0         4          0  0.00%  0.00%  0.00%   0 Crypto CA       
    172           0         1          0  0.00%  0.00%  0.00%   0 Crypto PKI-CRL  
    173           0         1          0  0.00%  0.00%  0.00%   0 Crypto SSL      
    174           0         1          0  0.00%  0.00%  0.00%   0 encrypt proc    
    175           0         1          0  0.00%  0.00%  0.00%   0 GDOI GM Process 
    176      362624   4767399         76  0.00%  0.02%  0.00%   0 AAA SEND STOP EV
    177           0         2          0  0.00%  0.00%  0.00%   0 EEM ED Resource 
    178           0         3          0  0.00%  0.00%  0.00%   0 EEM ED Track    
    179           0         1          0  0.00%  0.00%  0.00%   0 Syslog Traps    
    180          56    119107          0  0.00%  0.00%  0.00%   0 RMON Recycle Pro
    181           0         2          0  0.00%  0.00%  0.00%   0 RMON Deferred Se
    182          24       138        173  0.00%  0.01%  0.00%   4 Virtual Exec    
    183           0         1          0  0.00%  0.00%  0.00%   0 MV64 TDR Process
    184           0        37          0  0.00%  0.00%  0.00%   0 EEM Server      
    185           0         2          0  0.00%  0.00%  0.00%   0 EEM Policy Direc
    186           0         3          0  0.00%  0.00%  0.00%   0 EEM ED CLI      
    187           0         3          0  0.00%  0.00%  0.00%   0 EEM ED Config   
    188           0         3          0  0.00%  0.00%  0.00%   0 EEM ED Counter  
    189           0         3          0  0.00%  0.00%  0.00%   0 EEM ED Env      
    190           0         3          0  0.00%  0.00%  0.00%   0 EEM ED Interface
    191           0         3          0  0.00%  0.00%  0.00%   0 EEM ED IOSWD    
    192           0         3          0  0.00%  0.00%  0.00%   0 EEM ED None     
    193           0         3          0  0.00%  0.00%  0.00%   0 EEM ED OIR      
    194           0         2          0  0.00%  0.00%  0.00%   0 tENM            
    195           0         3          0  0.00%  0.00%  0.00%   0 EEM ED SNMP     
    196           0         3          0  0.00%  0.00%  0.00%   0 EEM ED Test     
    197           4     29860          0  0.00%  0.00%  0.00%   0 EEM ED Timer    
    198          12     56583          0  0.00%  0.00%  0.00%   0 IP SLA MPLSLM Pr
    199          20       305         65  0.00%  0.00%  0.00%   0 Syslog          
    200           0         1          0  0.00%  0.00%  0.00%   0 RMON Packets    
    201           0         1          0  0.00%  0.00%  0.00%   0 IP SLAs Ethernet
    202           0         1          0  0.00%  0.00%  0.00%   0 IP SLA AutoEther
    203         264     22608         11  0.00%  0.00%  0.00%   0 Net Input       
    204     5244200    388598      13495  0.47%  0.44%  0.43%   0 Compute load avg
    205           8         5       1600  0.00%  0.00%  0.00%   0 crypto sw pk pro
    207     1398812 547611831          2  5.27%  5.27%  5.24%   0 L2X Data Daemon 
    208      447680    774468        578  0.07%  0.05%  0.04%   0 L2TP mgmt daemon
    209      213684    638194        334  0.07%  0.02%  0.00%   0 L2TUN Applicatio
    210           0         1          0  0.00%  0.00%  0.00%   0 VPDN Test       
    211     1440332  27764880         51  0.07%  0.13%  0.11%   0 PPPoE Background
    212     2845100  35973988         79  0.31%  0.29%  0.28%   0 PPPoE Discovery 
    213       91560  46276259          1  0.07%  0.04%  0.05%   0 PPP manager     
    214     3801560  70682638         53  0.63%  0.61%  0.58%   0 PPP Events      
    215         116   1190837          0  0.00%  0.00%  0.00%   0 Multilink PPP   
    216           0         2          0  0.00%  0.00%  0.00%   0 Multilink event 
    218       75752   1395317         54  0.00%  0.00%  0.00%   0 IP SNMP         
    219       21072    698269         30  0.00%  0.00%  0.00%   0 PDU DISPATCHER  
    220       86136    697192        123  0.00%  0.00%  0.00%   0 SNMP ENGINE     
    221           0         2          0  0.00%  0.00%  0.00%   0 IP SNMPV6       
    222           0         1          0  0.00%  0.00%  0.00%   0 SNMP ConfCopyPro
    223           0         1          0  0.00%  0.00%  0.00%   0 SNMP Traps      
    224     1520096  73207009         20  0.31%  0.29%  0.26%   0 RADIUS          
    225           8     19906          0  0.00%  0.00%  0.00%   0 Time Range Proce
    226      695960    234558       2967  0.00%  0.11%  0.11%   0 VTEMPLATE Backgr
    227           4      9927          0  0.00%  0.00%  0.00%   0 DHCPD Timer     
    228           4     19853          0  0.00%  0.00%  0.00%   0 DHCPD Database  
    229        2120    545479          3  0.00%  0.00%  0.00%   0 EIGRP-IPv4: Rout
    230      844824  22549554         37  0.15%  0.07%  0.07%   0 EIGRP-IPv4: PDM 
    231     3497712 279146321         12 39.35% 39.22% 39.04%   0 EIGRP-IPv4: HELL
    please note that EIGRP process have high CPU
    also note that i have 2 7200 router have the same confiugration but runing 12.4(4)XD10 and same number of session the CPU working normal 6% and EIGRP normail
    Thank you
    Qaes

    Hi There,
    Just want to share a tip on ow to filter your CPU utilisation on your Cisco router. I find alias commands handy and use them all the time. From the config mode, you can type as below.
    R(config)#alias exec cpu sho process cpu sorted | exc 0.00%__0.00%__0.00%
    This will create an alias called cpu, from priviledge mode, you can then type cpu and it will filter all process' that are not consuming the CPU, this is done with the help of exc [exclude]. This excludes all process' with 0% utilisation.
    Below is my example on one of my routers.
    Thanks,
    Faraimd

  • Weblogic reaches 100% CPU utilisation

    I am using WebLogic Server Version 10.3.1.0. I got an application deployed in this server, which is developed with java1.6, ejb, hibernate. I start the server with 8GB of RAM assigned to it and after a day or two the weblogic server causes the use of the CPU to reach to 100% and as a result the server sort of doesn't response anymore. I will be creating 30-40 thread per managed server and one server has two managed server.  The volume will be around 30k per day. I dont know which is causing the 100% cpu utilisation and here below are the thread dumps which have taken at the time of high cpu in a server.  Please let me know where is the exact issue and where is the stuck thread (if any)? In which part the issue is occuring?
    10774:
    ===== FULL THREAD DUMP ===============
    Thu Jun 19 16:57:39 2014
    Oracle JRockit(R) R28.2.8-10-156881-1.6.0_51-20130611-1146-linux-x86_64
    "Main Thread" id=1 idx=0x4 tid=10776 prio=5 alive, waiting, native_blocked
        -- Waiting for notification on: weblogic/t3/srvr/T3Srvr@0xa4db4c58[fat lock]
        at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
        at java/lang/Object.wait(J)V(Native Method)
        at java/lang/Object.wait(Object.java:485)
        at weblogic/t3/srvr/T3Srvr.waitForDeath(T3Srvr.java:981)
        ^-- Lock released while waiting: weblogic/t3/srvr/T3Srvr@0xa4db4c58[fat lock]
        at weblogic/t3/srvr/T3Srvr.run(T3Srvr.java:490)
        at weblogic/Server.main(Server.java:71)
        at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
        -- end of trace
    "(Signal Handler)" id=2 idx=0x8 tid=10777 prio=5 alive, native_blocked, daemon
    "(OC Main Thread)" id=3 idx=0xc tid=10778 prio=5 alive, native_waiting, daemon
    "(GC Worker Thread 1)" id=? idx=0x10 tid=10779 prio=5 alive, daemon
    "(GC Worker Thread 2)" id=? idx=0x14 tid=10780 prio=5 alive, daemon
    "(GC Worker Thread 3)" id=? idx=0x18 tid=10781 prio=5 alive, daemon
    "(GC Worker Thread 4)" id=? idx=0x1c tid=10782 prio=5 alive, daemon
    "(Code Generation Thread 1)" id=4 idx=0x20 tid=10783 prio=5 alive, native_waiting, daemon
    "(Code Optimization Thread 1)" id=5 idx=0x24 tid=10784 prio=5 alive, native_waiting, daemon
    "(VM Periodic Task)" id=6 idx=0x28 tid=10785 prio=10 alive, native_waiting, daemon
    "Finalizer" id=7 idx=0x2c tid=10786 prio=8 alive, native_waiting, daemon
        at jrockit/memory/Finalizer.waitForFinalizees(J[Ljava/lang/Object;)I(Native Method)
        at jrockit/memory/Finalizer.access$700(Finalizer.java:12)
        at jrockit/memory/Finalizer$4.run(Finalizer.java:201)
        at java/lang/Thread.run(Thread.java:662)
        at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
        -- end of trace
    "Reference Handler" id=8 idx=0x30 tid=10787 prio=10 alive, native_waiting, daemon
        at java/lang/ref/Reference.waitForActivatedQueue(J)Ljava/lang/ref/Reference;(Native Method)
        at java/lang/ref/Reference.access$100(Reference.java:11)
        at java/lang/ref/Reference$ReferenceHandler.run(Reference.java:82)
        at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
        -- end of trace
    "(Sensor Event Thread)" id=9 idx=0x34 tid=10788 prio=5 alive, native_blocked, daemon
    "VM JFR Buffer Thread" id=10 idx=0x38 tid=10789 prio=5 alive, in native, daemon
    "Timer-0" id=13 idx=0x3c tid=10793 prio=5 alive, waiting, native_blocked, daemon
        -- Waiting for notification on: java/util/TaskQueue@0xa4d5d260[fat lock]
        at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
        at jrockit/vm/Locks.wait(Locks.java:1973)[inlined]
        at java/lang/Object.wait(Object.java:485)[optimized]
        at java/util/TimerThread.mainLoop(Timer.java:483)
        ^-- Lock released while waiting: java/util/TaskQueue@0xa4d5d260[fat lock]
        at java/util/TimerThread.run(Timer.java:462)
        at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
        -- end of trace
    "ExecuteThread: '0' for queue: 'default'" id=14 idx=0x40 tid=10839 prio=5 alive, native_blocked, daemon
        at java/util/HashMap.get(HashMap.java:385)[optimized]
        at com/appl/vsms/pi/outbound/ControllerCache.getEntry(ControllerCache.java:48)[inlined]
        at com/appl/vsms/bean/pi/MessageStatusConsumer.onMessage(MessageStatusConsumer.java:90)[optimized]
        at weblogic/ejb/container/internal/MDListener.execute(MDListener.java:574)[inlined]
        at weblogic/ejb/container/internal/MDListener.run(MDListener.java:893)[optimized]
        at weblogic/work/ExecuteRequestAdapter.execute(ExecuteRequestAdapter.java:21)[optimized]
        at weblogic/kernel/ExecuteThread.execute(ExecuteThread.java:145)[optimized]
        at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:117)
        at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
        -- end of trace
    "ExecuteThread: '1' for queue: 'default'" id=15 idx=0x44 tid=10840 prio=5 alive, waiting, native_blocked, daemon
        -- Waiting for notification on: java/util/LinkedList@0xa8cbed08[fat lock]
        at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
        at jrockit/vm/Locks.wait(Locks.java:1973)[inlined]
        at java/lang/Object.wait(Object.java:485)[inlined]
        at weblogic/ejb/container/internal/MDListener.getListener(MDListener.java:240)[optimized]
        ^-- Lock released while waiting: java/util/LinkedList@0xa8cbed08[fat lock]
        at weblogic/ejb/container/internal/MDListener.transactionalOnMessage(MDListener.java:480)[inlined]
        at weblogic/ejb/container/internal/MDListener.onMessage(MDListener.java:379)[optimized]
        at weblogic/jms/client/JMSSession.onMessage(JMSSession.java:4659)[optimized]
        at weblogic/jms/client/JMSSession.execute(JMSSession.java:4345)[optimized]
        at weblogic/jms/client/JMSSession.executeMessage(JMSSession.java:3821)[optimized]
        at weblogic/jms/client/JMSSession.access$000(JMSSession.java:115)
        at weblogic/jms/client/JMSSession$UseForRunnable.run(JMSSession.java:5170)
        at weblogic/work/ExecuteRequestAdapter.execute(ExecuteRequestAdapter.java:21)[optimized]
        at weblogic/kernel/ExecuteThread.execute(ExecuteThread.java:145)[optimized]
        at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:117)
        at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
        -- end of trace
    "ExecuteThread: '2' for queue: 'default'" id=16 idx=0x48 tid=10841 prio=5 alive, waiting, native_blocked, daemon
        -- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4d5f0d0[fat lock]
        at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
        at jrockit/vm/Locks.wait(Locks.java:1973)[inlined]
        at java/lang/Object.wait(Object.java:485)[inlined]
        at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)[optimized]
        ^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4d5f0d0[fat lock]
        at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
        at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
        -- end of trace
    "ExecuteThread: '3' for queue: 'default'" id=17 idx=0x4c tid=10842 prio=5 alive, native_blocked, daemon
        at java/util/HashMap.get(HashMap.java:385)[optimized]
        at com/appl/vsms/pi/outbound/ControllerCache.getEntry(ControllerCache.java:48)[inlined]
        at com/appl/vsms/bean/pi/MessageStatusConsumer.onMessage(MessageStatusConsumer.java:90)[optimized]
        at weblogic/ejb/container/internal/MDListener.execute(MDListener.java:574)[inlined]
        at weblogic/ejb/container/internal/MDListener.run(MDListener.java:893)[optimized]
        at weblogic/work/ExecuteRequestAdapter.execute(ExecuteRequestAdapter.java:21)[optimized]
        at weblogic/kernel/ExecuteThread.execute(ExecuteThread.java:145)[optimized]
        at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:117)
        at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
        -- end of trace
    "ExecuteThread: '4' for queue: 'default'" id=18 idx=0x50 tid=10843 prio=5 alive, waiting, native_blocked, daemon
        -- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4d5fe50[fat lock]
        at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
        at jrockit/vm/Locks.wait(Locks.java:1973)[inlined]
        at java/lang/Object.wait(Object.java:485)[inlined]
        at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)[optimized]
        ^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4d5fe50[fat lock]
        at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
        at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
        -- end of trace
    "ExecuteThread: '5' for queue: 'default'" id=19 idx=0x54 tid=10844 prio=5 alive, native_blocked, daemon
        at java/util/HashMap.get(HashMap.java:385)[optimized]
        at com/appl/vsms/pi/outbound/ControllerCache.getEntry(ControllerCache.java:48)[inlined]
        at com/appl/vsms/bean/pi/MessageStatusConsumer.onMessage(MessageStatusConsumer.java:90)[optimized]
        at weblogic/ejb/container/internal/MDListener.execute(MDListener.java:574)[inlined]
        at weblogic/ejb/container/internal/MDListener.run(MDListener.java:893)[optimized]
        at weblogic/work/ExecuteRequestAdapter.execute(ExecuteRequestAdapter.java:21)[optimized]
        at weblogic/kernel/ExecuteThread.execute(ExecuteThread.java:145)[optimized]
        at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:117)
        at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
        -- end of trace
    "ExecuteThread: '6' for queue: 'default'" id=20 idx=0x58 tid=10845 prio=5 alive, native_blocked, daemon
        at java/util/HashMap.get(HashMap.java:385)[optimized]
        at com/appl/vsms/pi/outbound/ControllerCache.getEntry(ControllerCache.java:48)[inlined]
        at com/appl/vsms/bean/pi/MessageStatusConsumer.onMessage(MessageStatusConsumer.java:90)[optimized]
        at weblogic/ejb/container/internal/MDListener.execute(MDListener.java:574)[inlined]
        at weblogic/ejb/container/internal/MDListener.run(MDListener.java:893)[optimized]
        at weblogic/work/ExecuteRequestAdapter.execute(ExecuteRequestAdapter.java:21)[optimized]
        at weblogic/kernel/ExecuteThread.execute(ExecuteThread.java:145)[optimized]
        at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:117)
        at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
        -- end of trace
    "ExecuteThread: '7' for queue: 'default'" id=21 idx=0x5c tid=10846 prio=5 alive, native_blocked, daemon
        at java/util/HashMap.get(HashMap.java:385)[optimized]
        at com/appl/vsms/pi/outbound/ControllerCache.getEntry(ControllerCache.java:48)[inlined]
        at com/appl/vsms/bean/pi/MessageStatusConsumer.onMessage(MessageStatusConsumer.java:90)[optimized]
        at weblogic/ejb/container/internal/MDListener.execute(MDListener.java:574)[inlined]
        at weblogic/ejb/container/internal/MDListener.run(MDListener.java:893)[optimized]
        at weblogic/work/ExecuteRequestAdapter.execute(ExecuteRequestAdapter.java:21)[optimized]
        at weblogic/kernel/ExecuteThread.execute(ExecuteThread.java:145)[optimized]
        at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:117)
        at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
        -- end of trace
    "ExecuteThread: '8' for queue: 'default'" id=22 idx=0x60 tid=10847 prio=5 alive, native_blocked, daemon
        at java/util/HashMap.get(HashMap.java:385)[optimized]
        at com/appl/vsms/pi/outbound/ControllerCache.getEntry(ControllerCache.java:48)[inlined]
        at com/appl/vsms/bean/pi/MessageStatusConsumer.onMessage(MessageStatusConsumer.java:90)[optimized]
        at weblogic/ejb/container/internal/MDListener.execute(MDListener.java:574)[inlined]
        at weblogic/ejb/container/internal/MDListener.run(MDListener.java:893)[optimized]
        at weblogic/work/ExecuteRequestAdapter.execute(ExecuteRequestAdapter.java:21)[optimized]
        at weblogic/kernel/ExecuteThread.execute(ExecuteThread.java:145)[optimized]
        at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:117)
        at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
        -- end of trace
    "ExecuteThread: '9' for queue: 'default'" id=23 idx=0x64 tid=10848 prio=5 alive, waiting, native_blocked, daemon
        -- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4d62010[fat lock]
        at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
        at jrockit/vm/Locks.wait(Locks.java:1973)[inlined]
        at java/lang/Object.wait(Object.java:485)[inlined]
        at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)[optimized]
        ^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4d62010[fat lock]
        at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
        at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
        -- end of trace
    "ExecuteThread: '10' for queue: 'default'" id=24 idx=0x68 tid=10849 prio=5 alive, waiting, native_blocked, daemon
        -- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4d626d0[fat lock]
        at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
        at jrockit/vm/Locks.wait(Locks.java:1973)[inlined]
        at java/lang/Object.wait(Object.java:485)[inlined]
        at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)[optimized]
        ^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4d626d0[fat lock]
        at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
        at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
        -- end of trace
    "ExecuteThread: '11' for queue: 'default'" id=25 idx=0x6c tid=10850 prio=5 alive, native_blocked, daemon
        at java/util/HashMap.get(HashMap.java:385)[optimized]
        at com/appl/vsms/pi/outbound/ControllerCache.getEntry(ControllerCache.java:48)[inlined]
        at com/appl/vsms/bean/pi/MessageStatusConsumer.onMessage(MessageStatusConsumer.java:90)[optimized]
        at weblogic/ejb/container/internal/MDListener.execute(MDListener.java:574)[inlined]
        at weblogic/ejb/container/internal/MDListener.run(MDListener.java:893)[optimized]
        at weblogic/work/ExecuteRequestAdapter.execute(ExecuteRequestAdapter.java:21)[optimized]
        at weblogic/kernel/ExecuteThread.execute(ExecuteThread.java:145)[optimized]
        at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:117)
        at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
        -- end of trace
    "ExecuteThread: '12' for queue: 'default'" id=26 idx=0x70 tid=10851 prio=5 alive, native_blocked, daemon
        at java/util/HashMap.get(HashMap.java:385)[optimized]
        at com/appl/vsms/pi/outbound/ControllerCache.getEntry(ControllerCache.java:48)[inlined]
        at com/appl/vsms/bean/pi/MessageStatusConsumer.onMessage(MessageStatusConsumer.java:90)[optimized]
        at weblogic/ejb/container/internal/MDListener.execute(MDListener.java:574)[inlined]
        at weblogic/ejb/container/internal/MDListener.run(MDListener.java:893)[optimized]
        at weblogic/work/ExecuteRequestAdapter.execute(ExecuteRequestAdapter.java:21)[optimized]
        at weblogic/kernel/ExecuteThread.execute(ExecuteThread.java:145)[optimized]
        at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:117)
        at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
        -- end of trace
    "ExecuteThread: '13' for queue: 'default'" id=27 idx=0x74 tid=10852 prio=5 alive, waiting, native_blocked, daemon
        -- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4d648e8[fat lock]
        at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
        at jrockit/vm/Locks.wait(Locks.java:1973)[inlined]
        at java/lang/Object.wait(Object.java:485)[inlined]
        at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)[optimized]
        ^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4d648e8[fat lock]
        at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
        at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
        -- end of trace
    "ExecuteThread: '14' for queue: 'default'" id=28 idx=0x78 tid=10853 prio=5 alive, waiting, native_blocked, daemon
        -- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4d65568[fat lock]
        at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
        at jrockit/vm/Locks.wait(Locks.java:1973)[inlined]
        at java/lang/Object.wait(Object.java:485)[inlined]
        at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)[optimized]
        ^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4d65568[fat lock]
        at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
        at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
        -- end of trace
    "ExecuteThread: '0' for queue: 'weblogic.kernel.Non-Blocking'" id=29 idx=0x7c tid=10854 prio=5 alive, waiting, native_blocked, daemon
        -- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4d65cc0[fat lock]
        at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
        at jrockit/vm/Locks.wait(Locks.java:1973)[inlined]
        at java/lang/Object.wait(Object.java:485)[inlined]
        at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)[optimized]
        ^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4d65cc0[fat lock]
        at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
        at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
        -- end of trace
    "ExecuteThread: '1' for queue: 'weblogic.kernel.Non-Blocking'" id=30 idx=0x80 tid=10855 prio=5 alive, waiting, native_blocked, daemon
        -- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4d663c0[fat lock]
        at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
        at jrockit/vm/Locks.wait(Locks.java:1973)[inlined]
        at java/lang/Object.wait(Object.java:485)[inlined]
        at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)[optimized]
        ^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4d663c0[fat lock]
        at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
        at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
        -- end of trace
    "ExecuteThread: '2' for queue: 'weblogic.kernel.Non-Blocking'" id=31 idx=0x84 tid=10856 prio=5 alive, waiting, native_blocked, daemon
        -- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4d66ac0[fat lock]
        at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
        at jrockit/vm/Locks.wait(Locks.java:1973)[inlined]
        at java/lang/Object.wait(Object.java:485)[inlined]
        at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)[optimized]
        ^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4d66ac0[fat lock]
        at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
        at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
        -- end of trace
    "ExecuteThread: '0' for queue: 'weblogic.kernel.System'" id=32 idx=0x88 tid=10857 prio=5 alive, waiting, native_blocked, daemon
        -- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4d671c0[fat lock]
        at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
        at jrockit/vm/Locks.wait(Locks.java:1973)[inlined]
        at java/lang/Object.wait(Object.java:485)[inlined]
        at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)[optimized]
        ^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4d671c0[fat lock]
        at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
        at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
        -- end of trace
    "ExecuteThread: '1' for queue: 'weblogic.kernel.System'" id=33 idx=0x8c tid=10858 prio=5 alive, waiting, native_blocked, daemon
        -- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4d67f48[fat lock]
        at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
        at jrockit/vm/Locks.wait(Locks.java:1973)[inlined]
        at java/lang/Object.wait(Object.java:485)[inlined]
        at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)[optimized]
        ^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4d67f48[fat lock]
        at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
        at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
        -- end of trace
    "ExecuteThread: '2' for queue: 'weblogic.kernel.System'" id=34 idx=0x90 tid=10859 prio=5 alive, waiting, native_blocked, daemon
        -- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4d68cc0[fat lock]
        at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
        at jrockit/vm/Locks.wait(Locks.java:1973)[inlined]
        at java/lang/Object.wait(Object.java:485)[inlined]
        at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)[optimized]
        ^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4d68cc0[fat lock]
        at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
        at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
        -- end of trace
    "ExecuteThread: '3' for queue: 'weblogic.kernel.System'" id=35 idx=0x94 tid=10860 prio=5 alive, waiting, native_blocked, daemon
        -- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4d69a38[fat lock]
        at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
        at jrockit/vm/Locks.wait(Locks.java:1973)[inlined]
        at java/lang/Object.wait(Object.java:485)[inlined]
        at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)[optimized]
        ^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4d69a38[fat lock]
        at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
        at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
        -- end of trace
    "ExecuteThread: '4' for queue: 'weblogic.kernel.System'" id=36 idx=0x98 tid=10861 prio=5 alive, waiting, native_blocked, daemon
        -- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4d977b8[fat lock]
        at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
        at jrockit/vm/Locks.wait(Locks.java:1973)[inlined]
        at java/lang/Object.wait(Object.java:485)[inlined]
        at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)[optimized]
        ^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4d977b8[fat lock]
        at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
        at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
        -- end of trace
    "ExecuteThread: '0' for queue: 'weblogic.Rejector'" id=37 idx=0x9c tid=10862 prio=5 alive, waiting, native_blocked, daemon
        -- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4d99570[fat lock]
        at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
        at java/lang/Object.wait(J)V(Native Method)
        at java/lang/Object.wait(Object.java:485)
        at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)
        ^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4d99570[fat lock]
        at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
        at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
        -- end of trace
    "ExecuteThread: '1' for queue: 'weblogic.Rejector'" id=38 idx=0xa0 tid=10863 prio=5 alive, waiting, native_blocked, daemon
        -- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4cf2e38[fat lock]
        at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
        at java/lang/Object.wait(J)V(Native Method)
        at java/lang/Object.wait(Object.java:485)
        at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)
        ^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4cf2e38[fat lock]
        at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
        at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
        -- end of trace
    "weblogic.time.TimeEventGenerator" id=39 idx=0xa4 tid=10864 prio=9 alive, waiting, native_blocked, daemon
        -- Waiting for notification on: weblogic/time/common/internal/TimeTable@0xa4dbed18[fat lock]
        at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
        at java/lang/Object.wait(J)V(Native Method)
        at weblogic/time/common/internal/TimeTable.snooze(TimeTable.java:286)
        ^-- Lock released while waiting: weblogic/time/common/internal/TimeTable@0xa4dbed18[fat lock]
        at weblogic/time/common/internal/TimeEventGenerator.run(TimeEventGenerator.java:117)
        at java/lang/Thread.run(Thread.java:662)
        at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
        -- end of trace
    "JMAPI event thread" id=40 idx=0xa8 tid=10865 prio=5 alive, in native, daemon
    "weblogic.timers.TimerThread" id=41 idx=0xac tid=10866 prio=9 alive, waiting, native_blocked, daemon
        -- Waiting for notification on: weblogic/timers/internal/TimerThread@0xa4dc3f70[fat lock]
        at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
        at java/lang/Object.wait(J)V(Native Method)
        at weblogic/timers/internal/TimerThread$Thread.run(TimerThread.java:262)
        ^-- Lock released while waiting: weblogic/timers/internal/TimerThread@0xa4dc3f70[fat lock]
        at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
        -- end of trace
    "ExecuteThread: '0' for queue: 'weblogic.admin.RMI'" id=42 idx=0xb0 tid=10867 prio=5 alive, waiting, native_blocked, daemon
        -- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4cf3528[fat lock]
        at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
        at java/lang/Object.wait(J)V(Native Method)
        at java/lang/Object.wait(Object.java:485)
        at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)
        ^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4cf3528[fat lock]
        at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
        at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
        -- end of trace
    "ExecuteThread: '1' for queue: 'weblogic.admin.RMI'" id=43 idx=0xb4 tid=10868 prio=5 alive, waiting, native_blocked, daemon
        -- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4cf35f0[fat lock]
        at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
        at java/lang/Object.wait(J)V(Native Method)
        at java/lang/Object.wait(Object.java:485)
        at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)
        ^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4cf35f0[fat lock]
        at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
        at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
        -- end of trace
    "ExecuteThread: '2' for queue: 'weblogic.admin.RMI'" id=44 idx=0xb8 tid=10869 prio=5 alive, waiting, native_blocked, daemon
        -- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4cf36b8[fat lock]
        at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
        at jrockit/vm/Locks.wait(Locks.java:1973)[inlined]
        at java/lang/Object.wait(Object.java:485)[inlined]
        at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)[optimized]
        ^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4cf36b8[fat lock]
        at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
        at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
        -- end of trace
    "ExecuteThread: '3' for queue: 'weblogic.admin.RMI'" id=45 idx=0xbc tid=10870 prio=5 alive, waiting, native_blocked, daemon
        -- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4cf3780[fat lock]
        at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
        at jrockit/vm/Locks.wait(Locks.java:1973)[inlined]
        at java/lang/Object.wait(Object.java:485)[inlined]
        at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)[optimized]
        ^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4cf3780[fat lock]
        at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
        at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
        -- end of trace
    "ExecuteThread: '4' for queue: 'weblogic.admin.RMI'" id=46 idx=0xc0 tid=10871 prio=5 alive, waiting, native_blocked, daemon
        -- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4cf3848[fat lock]
        at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
        at jrockit/vm/Locks.wait(Locks.java:1973)[inlined]
        at java/lang/Object.wait(Object.java:485)[inlined]
        at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)[optimized]
        ^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4cf3848[fat lock]
        at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
        at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
        -- end of trace
    "ExecuteThread: '0' for queue: 'ImageWorkManager'" id=47 idx=0xc4 tid=10896 prio=5 alive, waiting, native_blocked, daemon
        -- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4cf57e0[fat lock]
        at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
        at java/lang/Object.wait(J)V(Native Method)
        at java/lang/Object.wait(Object.java:485)
        at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)
        ^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4cf57e0[fat lock]
        at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
        at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
        -- end of trace
    "ExecuteThread: '1' for queue: 'ImageWorkManager'" id=48 idx=0xc8 tid=10899 prio=5 alive, waiting, native_blocked, daemon
        -- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4cf58a8[fat lock]
        at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
        at java/lang/Object.wait(J)V(Native Method)
        at java/lang/Object.wait(Object.java:485)
        at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)
        ^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4cf58a8[fat lock]
        at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
        at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
        -- end of trace
    "JFR request timer" id=49 idx=0xcc tid=10900 prio=5 alive, waiting, native_blocked, daemon
        -- Waiting for notification on: java/util/TaskQueue@0xa4dc69d8[fat lock]
        at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
        at java/lang/Object.wait(J)V(Native Method)
        at java/lang/Object.wait(Object.java:485)
        at java/util/TimerThread.mainLoop(Timer.java:483)
        ^-- Lock released while waiting: java/util/TaskQueue@0xa4dc69d8[fat lock]
        at java/util/TimerThread.run(Timer.java:462)
        at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
        -- end of trace
    "ExecuteThread: '0' for queue: 'weblogic.logging.DomainLogBroadcasterClient'" id=53 idx=0xd0 tid=10902 prio=5 alive, waiting, native_blocked, daemon
        -- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4cf6550[fat lock]
        at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
        at java/lang/Object.wait(J)V(Native Method)
        at java/lang/Object.wait(Object.java:485)
        at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)
        ^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4cf6550[fat lock]
        at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
        at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
        -- end of trace
    "ExecuteThread: '0' for queue: 'weblogic.logging.LogBroadcaster'" id=54 idx=0xd4 tid=10903 prio=5 alive, waiting, native_blocked, daemon
        -- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4cf6d50[fat lock]
        at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
        at java/lang/Object.wait(J)V(Native Method)
        at java/lang/Object.wait(Object.java:485)
        at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)
        ^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4cf6d50[fat lock]
        at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
        at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
        -- end of trace
    "ExecuteThread: '0' for queue: 'weblogic.socket.Muxer'" id=55 idx=0xd8 tid=10904 prio=5 alive, blocked, native_blocked, daemon
        -- Blocked trying to get lock: java/lang/String@0xa4cf78f8[fat lock]
        at jrockit/vm/Threads.waitForUnblockSignal()V(Native Method)
        at jrockit/vm/Locks.fatLockBlockOrSpin(Locks.java:1411)[optimized]
        at jrockit/vm/Locks.lockFat(Locks.java:1512)[optimized]
        at jrockit/vm/Locks.monitorEnterSecondStageHard(Locks.java:1054)[optimized]
        at jrockit/vm/Locks.monitorEnterSecondStage(Locks.java:1005)[optimized]
        at jrockit/vm/Locks.monitorEnter(Locks.java:2179)[optimized]
        at weblogic/socket/EPollSocketMuxer.processSockets(EPollSocketMuxer.java:153)
        at weblogic/socket/SocketReaderRequest.run(SocketReaderRequest.java:29)
        at weblogic/socket/SocketReaderRequest.execute(SocketReaderRequest.java:42)
        at weblogic/kernel/ExecuteThread.execute(ExecuteThread.java:145)
        at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:117)
        at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
        -- end of trace
    "ExecuteThread: '1' for queue: 'weblogic.socket.Muxer'" id=56 idx=0xdc tid=10905 prio=5 alive, blocked, native_blocked, daemon
        -- Blocked trying to get lock: java/lang/String@0xa4cf78f8[fat lock]
        at jrockit/vm/Threads.waitForUnblockSignal()V(Native Method)
        at jrockit/vm/Locks.fatLockBlockOrSpin(Locks.java:1411)[optimized]
        at jrockit/vm/Locks.lockFat(Locks.java:1512)[optimized]
        at jrockit/vm/Locks.monitorEnterSecondStageHard(Locks.java:1054)[optimized]
        at jrockit/vm/Locks.monitorEnterSecondStage(Locks.java:1005)[optimized]
        at jrockit/vm/Locks.monitorEnter(Locks.java:2179)[optimized]
        at weblogic/socket/EPollSocketMuxer.processSockets(EPollSocketMuxer.java:153)
        at weblogic/socket/SocketReaderRequest.run(SocketReaderRequest.java:29)
        at weblogic/socket/SocketReaderRequest.execute(SocketReaderRequest.java:42)
        at weblogic/kernel/ExecuteThread.execute(ExecuteThread.java:145)
        at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:117)
        at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
        -- end of trace
    "ExecuteThread: '2' for queue: 'weblogic.socket.Muxer'" id=57 idx=0xe0 tid=10906 prio=5 alive, blocked, native_blocked, daemon
        -- Blocked trying to get lock: java/lang/String@0xa4cf78f8[fat lock]
        at jrockit/vm/Threads.waitForUnblockSignal()V(Native Method)
        at jrockit/vm/Locks.fatLockBlockOrSpin(Locks.java:1411)[inlined]
        at jrockit/vm/Locks.fatLockReacquire(Locks.java:1649)[optimized]
        at jrockit/vm/Locks.lockFat(Locks.java:1517)[optimized]
        at jrockit/vm/Locks.monitorEnterSecondStageHard(Locks.java:1054)[optimized]
        at jrockit/vm/Locks.monitorEnterSecondStage(Locks.java:1005)[optimized]
        at jrockit/vm/Locks.monitorEnter(Locks.java:2179)[optimized]
        at weblogic/socket/EPollSocketMuxer.processSockets(EPollSocketMuxer.java:153)
        at weblogic/socket/SocketReaderRequest.run(SocketReaderRequest.java:29)
        at weblogic/socket/SocketReaderRequest.execute(SocketReaderRequest.java:42)
        at weblogic/kernel/ExecuteThread.execute(ExecuteThread.java:145)
        at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:117)
        at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
        -- end of trace
    "ExecuteThread: '3' for queue: 'weblogic.socket.Muxer'" id=58 idx=0xe4 tid=10907 prio=5 alive, in native, daemon
        at jrockit/ext/epoll/EPoll.epollWait0(ILjava/nio/ByteBuffer;II)I(Native Method)
        at jrockit/ext/epoll/EPoll.epollWait(EPoll.java:123)
        at weblogic/socket/EPollSocketMuxer.processSockets(EPollSocketMuxer.java:156)
        ^-- Holding lock: java/lang/String@0xa4cf78f8[fat lock]
        at weblogic/socket/SocketReaderRequest.run(SocketReaderRequest.java:29)
        at weblogic/socket/SocketReaderRequest.execute(SocketReaderRequest.java:42)
        at weblogic/kernel/ExecuteThread.execute(ExecuteThread.java:145)
        at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:117)
        at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
        -- end of trace
    "VDE Transaction Processor Thread" id=61 idx=0xe8 tid=10909 prio=2 alive, waiting, native_blocked, daemon
        -- Waiting for notification on: com/octetstring/vde/backend/standard/TransactionProcessor@0xa46c3738[fat lock]
        at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
        at java/lang/Object.wait(J)V(Native Method)
        at java/lang/Object.wait(Object.java:485)
        at com/octetstring/vde/backend/standard/TransactionProcessor.waitChange(TransactionProcessor.java:367)
        ^-- Lock released while waiting: com/octetstring/vde/backend/standard/TransactionProcessor@0xa46c3738[fat lock]
        at com/octetstring/vde/backend/standard/TransactionProcessor.run(TransactionProcessor.java:212)
        at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
        -- end of trace
    "DoSManager" id=64 idx=0xf0 tid=10911 prio=6 alive, sleeping, native_waiting, daemon
        at java/lang/Thread.sleep(J)V(Native Method)
        at com/octetstring/vde/DoSManager.run(DoSManager.java:433)
        at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
        -- end of trace
    "ExecuteThread: '0' for queue: 'UserLockout'" id=65 idx=0xf4 tid=10920 prio=5 alive, waiting, native_blocked, daemon
        -- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4cf93c8[fat lock]
        at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
        at java/lang/Object.wait(J)V(Native Method)
        at java/lang/Object.wait(Object.java:485)
        at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)
        ^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4cf93c8[fat lock]
        at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
        at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
        -- end of trace
    "ExecuteThread: '1' for queue: 'UserLockout'" id=66 idx=0xf8 tid=10921 prio=5 alive, waiting, native_blocked, daemon
        -- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4cf9490[fat lock]
        at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
        at java/lang/Object.wait(J)V(Native Method)
        at java/lang/Object.wait(Object.java:485)
        at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)
        ^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4cf9490[fat lock]
        at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
        at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
        -- end of trace
    "ExecuteThread: '2' for queue: 'UserLockout'" id=67 idx=0xfc tid=10922 prio=5 alive, waiting, native_blocked, daemon
        -- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4cf9558[fat lock]
        at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
        at java/lang/Object.wait(J)V(Native Method)
        at java/lang/Object.wait(Object.java:485)
        at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)
        ^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4cf9558[fat lock]
        at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
        at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
        -- end of trace
    "ExecuteThread: '3' for queue: 'UserLockout'" id=68 idx=0x100 tid=10923 prio=5 alive, waiting, native_blocked, daemon
        -- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4cf9620[fat lock]
        at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
        at java/lang/Object.wait(J)V(Native Method)
        at java/lang/Object.wait(Object.java:485)
        at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)
        ^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4cf9620[fat lock]
        at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
        at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
        -- end of trace
    "ExecuteThread: '4' for queue: 'UserLockout'" id=69 idx=0x104 tid=10924 prio=5 alive, waiting, native_blocked, daemon
        -- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4cf96e8[fat lock]
        at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
        at java/lang/Object.wait(J)V(Native Method)
        at java/lang/Object.wait(Object.java:485)
        at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)
        ^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4cf96e8[fat lock]
        at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
        at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
        -- end of trace
    "ExecuteThread: '0' for queue: 'weblogic.unicast.DispatchWorkManager'" id=70 idx=0x108 tid=10958 prio=5 alive, waiting, native_blocked, daemon
        -- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4cfbbb0[fat lock]
        at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
        at java/lang/Object.wait(J)V(Native Method)
        at java/lang/Object.wait(Object.java:485)
        at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)
        ^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4cfbbb0[fat lock]
        at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
        at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
        -- end of trace
    "ExecuteThread: '1' for queue: 'weblogic.unicast.DispatchWorkManager'" id=71 idx=0x10c tid=10959 prio=5 alive, waiting, native_blocked, daemon
        -- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4cfbc78[fat lock]
        at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
        at jrockit/vm/Locks.wait(Locks.java:1973)[inlined]
        at java/lang/Object.wait(Object.java:485)[inlined]
        at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)[optimized]
        ^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4cfbc78[fat lock]
        at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
        at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
        -- end of trace
    "ExecuteThread: '2' for queue: 'weblogic.unicast.DispatchWorkManager'" id=72 idx=0x110 tid=10960 prio=5 alive, waiting, native_blocked, daemon
        -- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4cfbd40[fat lock]
        at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
        at jrockit/vm/Locks.wait(Locks.java:1973)[inlined]
        at java/lang/Object.wait(Object.java:485)[inlined]
        at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)[optimized]
        ^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4cfbd40[fat lock]
        at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
        at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
        -- end of trace
    "ExecuteThread: '3' for queue: 'weblogic.unicast.DispatchWorkManager'" id=73 idx=0x114 tid=10961 prio=5 alive, waiting, native_blocked, daemon
        -- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4cfbe08[fat lock]
        at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
        at jrockit/vm/Locks.wait(Locks.java:1973)[inlined]
        at java/lang/Object.wait(Object.java:485)[inlined]
        at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)[optimized]
        ^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4cfbe08[fat lock]
        at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
        at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
        -- end of trace
    "ExecuteThread: '4' for queue: 'weblogic.unicast.DispatchWorkManager'" id=74 idx=0x118 tid=10962 prio=5 alive, waiting, native_blocked, daemon
        -- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4cfbed0[fat lock]
        at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
        at jrockit/vm/Locks.wait(Locks.java:1973)[inlined]
        at java/lang/Object.wait(Object.java:485)[inlined]
        at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)[optimized]
        ^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4cfbed0[fat lock]
        at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
        at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
        -- end of trace
    "ExecuteThread: '0' for queue: 'weblogic.unicast.ForwardingWorkManager'" id=75 idx=0x11c tid=10963 prio=5 alive, waiting, native_blocked, daemon
        -- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4c66f40[fat lock]
        at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
        at java/lang/Object.wait(J)V(Native Method)
        at java/lang/Object.wait(Object.java:485)
        at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)
        ^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4c66f40[fat lock]
        at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
        at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
        -- end of trace
    "ExecuteThread: '1' for queue: 'weblogic.unicast.ForwardingWorkManager'" id=76 idx=0x120 tid=10964 prio=5 alive, waiting, native_blocked, daemon
        -- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4c67008[fat lock]
        at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
        at java/lang/Object.wait(J)V(Native Method)
        at java/lang/Object.wait(Object.java:485)
        at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)
        ^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4c67008[fat lock]
        at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
        at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
        -- end of trace
    "ExecuteThread: '2' for queue: 'weblogic.unicast.ForwardingWorkManager'" id=77 idx=0x124 tid=10965 prio=5 alive, waiting, native_blocked, daemon
        -- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4c670d0[fat lock]
        at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
        at java/lang/Object.wait(J)V(Native Method)
        at java/lang/Object.wait(Object.java:485)
        at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)
        ^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4c670d0[fat lock]
        at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
        at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
        -- end of trace
    "ExecuteThread: '3' for queue: 'weblogic.unicast.ForwardingWorkManager'" id=78 idx=0x128 tid=10966 prio=5 alive, waiting, native_blocked, daemon
        -- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4c67198[fat lock]
        at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
        at java/lang/Object.wait(J)V(Native Method)
        at java/lang/Object.wait(Object.java:485)
        at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)
        ^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4c67198[fat lock]
        at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
        at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
        -- end of trace
    "ExecuteThread: '4' for queue: 'weblogic.unicast.ForwardingWorkManager'" id=79 idx=0x12c tid=10967 prio=5 alive, waiting, native_blocked, daemon
        -- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4c67260[fat lock]
        at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
        at java/lang/Object.wait(J)V(Native Method)
        at java/lang/Object.wait(Object.java:485)
        at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)
        ^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4c67260[fat lock]
        at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
        at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
        -- end of trace
    "ExecuteThread: '0' for queue: 'ClusterMessaging'" id=81 idx=0x134 tid=10969 prio=5 alive, waiting, native_blocked, daemon
        -- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4c692e0[fat lock]
        at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
        at jrockit/vm/Locks.wait(Locks.java:1973)[inlined]
        at java/lang/Object.wait(Object.java:485)[inlined]
        at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)[optimized]
        ^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4c692e0[fat lock]
        at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
        at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
        -- end of trace
    "ExecuteThread: '0' for queue: 'wl_oldBootStrap'" id=82 idx=0x138 tid=10972 prio=5 alive, waiting, native_blocked, daemon
        -- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4c69a90[fat lock]
        at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
        at java/lang/Object.wait(J)V(Native Method)
        at java/lang/Object.wait(Object.java:485)
        at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)
        ^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4c69a90[fat lock]
        at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
        at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
        -- end of trace
    "ExecuteThread: '1' for queue: 'wl_oldBootStrap'" id=83 idx=0x13c tid=10973 prio=5 alive, waiting, native_blocked, daemon
        -- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4c69b58[fat lock]
        at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
        at java/lang/Object.wait(J)V(Native Method)
        at java/lang/Object.wait(Object.java:485)
        at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)
        ^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4c69b58[fat lock]
        at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
        at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
        -- end of trace
    "ExecuteThread: '2' for queue: 'wl_oldBootStrap'" id=84 idx=0x140 tid=10974 prio=5 alive, waiting, native_blocked, daemon
        -- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4c69c20[fat lock]
        at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
        at java/lang/Object.wait(J)V(Native Method)
        at java/lang/Object.wait(Object.java:485)
        at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)
        ^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4c69c20[fat lock]
        at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
        at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
        -- end of trace
    "ExecuteThread: '3' for queue: 'wl_oldBootStrap'" id=85 idx=0x144 tid=10975 prio=5 alive, waiting, native_blocked, daemon
        -- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4c69ce8[fat lock]
        at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
        at java/lang/Object.wait(J)V(Native Method)
        at java/lang/Object.wait(Object.java:485)
        at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)
        ^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4c69ce8[fat lock]
        at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
        at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
        -- end of trace
    "ExecuteThread: '4' for queue: 'wl_oldBootStrap'" id=86 idx=0x148 tid=10980 prio=5 alive, waiting, native_blocked, daemon
        -- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4c69db0[fat lock]
        at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
        at java/lang/Object.wait(J)V(Native Method)
        at java/lang/Object.wait(Object.java:485)
        at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)
        ^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4c69db0[fat lock]
        at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
        at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
        -- end of trace
    "ExecuteThread: '5' for queue: 'wl_oldBootStrap'" id=87 idx=0x14c tid=10984 prio=5 alive, waiting, native_blocked, daemon
        -- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4c69e78[fat lock]
        at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
        at java/lang/Object.wait(J)V(Native Method)
        at java/lang/Object.wait(Object.java:485)
        at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)
        ^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4c69e78[fat lock]
        at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
        at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
        -- end of trace
    "ExecuteThread: '6' for queue: 'wl_oldBootStrap'" id=88 idx=0x150 tid=10987 prio=5 alive, waiting, native_blocked, daemon
        -- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4c69f40[fat lock]
        at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
        at java/lang/Object.wait(J)V(Native Method)
        at java/lang/Object.wait(Object.java:485)
        at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)
        ^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4c69f40[fat lock]
        at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
        at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
        -- end of trace
    "ExecuteThread: '7' for queue: 'wl_oldBootStrap'" id=89 idx=0x154 tid=10988 prio=5 alive, waiting, native_blocked, daemon
        -- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4c6a008[fat lock]
        at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
        at jrockit/vm/Locks.wait(Locks.java:1973)[inlined]
        at java/lang/Object.wait(Object.java:485)[inlined]
        at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)[optimized]
        ^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4c6a008[fat lock]
        at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
        at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
        -- end of trace
    "Thread-12" id=90 idx=0x158 tid=10989 prio=5 alive, parked, native_blocked
        -- Parking to wait for: java/util/concurrent/locks/AbstractQueuedSynchronizer$ConditionObject@0xa4c66b48
        at jrockit/vm/Locks.park0(J)V(Native Method)
        at jrockit/vm/Locks.park(Locks.java:2230)
        at sun/misc/Unsafe.park(ZJ)V(Native Method)
        at java/util/concurrent/locks/LockSupport.park(LockSupport.java:156)
        at java/util/concurrent/locks/AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
        at java/util/concurrent/LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
        at weblogic/utils/concurrent/JDK15ConcurrentBlockingQueue.take(JDK15ConcurrentBlockingQueue.java:89)
        at weblogic/store/internal/PersistentStoreImpl.getOutstandingWork(PersistentStoreImpl.java:672)
        at weblogic/store/internal/PersistentStoreImpl.run(PersistentStoreImpl.java:721)
        at java/lang/Thread.run(Thread.java:662)
        at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
        -- end of trace
    "ExecuteThread: '0' for queue: 'JTACoordinatorWM'" id=91 idx=0x15c tid=10990 prio=5 alive, waiting, native_blocked, daemon
        -- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4c6d128[fat lock]
        at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
        at jrockit/vm/Locks.wait(Locks.java:1973)[inlined]
        at java/lang/Object.wait(Object.java:485)[inlined]
        at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)[optimized]
        ^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4c6d128[fat lock]
        at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
        at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
        -- end of trace
    "ExecuteThread: '1' for queue: 'JTACoordinatorWM'" id=92 idx=0x160 tid=10991 prio=5 alive, waiting, native_blocked, daemon
        -- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4c6d1f0[fat lock]
        at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
        at jrockit/vm/Locks.wait(Locks.java:1973)[inlined]
        at java/lang/Object.wait(Object.java:485)[inlined]
        at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)[optimized]
        ^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4c6d1f0[fat lock]
        at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
        at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
        -- end of trace
    "ExecuteThread: '2' for queue: 'JTACoordinatorWM'" id=93 idx=0x164 tid=10992 prio=5 alive, waiting, native_blocked, daemon
        -- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4c6d2b8[fat lock]
        at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
        at jrockit/vm/Locks.wait(Locks.java:1973)[inlined]
        at java/lang/Object.wait(Object.java:485)[inlined]
        at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)[optimized]
        ^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4c6d2b8[fat lock]
        at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
        at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
        -- end of trace
    "ExecuteThread: '0' for queue: 'JmsDispatcher'" id=94 idx=0x168 tid=10995 prio=5 alive, waiting, native_blocked, daemon
        -- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4c6e688[fat lock]
        at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
        at java/lang/Object.wait(J)V(Native Method)
        at java/lang/Object.wait(Object.java:485)
        at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)
        ^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4c6e688[fat lock]
        at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
        at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
        -- end of trace
    "ExecuteThread: '1' for queue: 'JmsDispatcher'" id=95 idx=0x16c tid=10996 prio=5 alive, waiting, native_blocked, daemon
        -- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4c6e750[fat lock]
        at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
        at java/lang/Object.wait(J)V(Native Method)
        at java/lang/Object.wait(Object.java:485)
        at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)
        ^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4c6e750[fat lock]
        at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
        at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
        -- end of trace
    "ExecuteThread: '2' for queue: 'JmsDispatcher'" id=96 idx=0x170 tid=10997 prio=5 alive, waiting, native_blocked, daemon
        -- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4c6e818[fat lock]
        at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
        at java/lang/Object.wait(J)V(Native Method)
        at java/lang/Object.wait(Object.java:485)
        at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)
        ^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4c6e818[fat lock]
        at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
        at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
        -- end of trace
    "ExecuteThread: '3' for queue: 'JmsDispatcher'" id=97 idx=0x174 tid=10998 prio=5 alive, waiting, native_blocked, daemon
        -- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4c6e8e0[fat lock]
        at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
        at java/lang/Object.wait(J)V(Native Method)
        at java/lang/Object.wait(Object.java:485)
        at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:91)
        ^-- Lock released while waiting: weblogic/kernel/ServerExecuteThread@0xa4c6e8e0[fat lock]
        at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:115)
        at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
        -- end of trace
    "ExecuteThread: '4' for queue: 'JmsDispatcher'" id=98 idx=0x178 tid=10999 prio=5 alive, waiting, native_blocked, daemon
        -- Waiting for notification on: weblogic/kernel/ServerExecuteThread@0xa4c6e9a8[fat lock]
        at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
       

    Hi,
    Based on the below doc , you can narrow down the actual Thread that is causing this high CPU.
    Unexpected High CPU Usage with WebLogic Server (WLS) Support Pattern (Doc ID 779349.1)
    Thanks,
    Sharmela

Maybe you are looking for