2960 - HULC LED PROCESS

Dear All,
I have cisco WS-C2960S-48FPS-L stacked. Weekly twice, my PoE connections are dropped and when the device is restarted, everything starts working normal. This issue happens weekly once or twice.
Verified the power input and it's absolutely fine.
show power inline
Module   Available     Used     Remaining
          (Watts)     (Watts)    (Watts)
1           740.0       91.5       648.5
2           740.0       60.2       679.8
3           740.0      144.3       595.7
4           740.0      157.1       582.9
Upon taking a look to the CPU Process, device is punted with 75 - 85 % at normal usage, with "Hulc LED Process" as 18% utilisation. CPU utilisation also peaks to 90 - 100% sometimes.
CPU utilization for five seconds: 78%/48%; one minute: 58%; five minutes: 50%
100 *              *                                                     
90 * **  *  *     *                                                     
80 *###############                                                     
70 ################                                                     
60 ################                                                     
50 ################                                                     
40 ################                                                     
30 ################                                                     
20 ################                                                     
10 ################                                                     
   0....5....1....1....2....2....3....3....4....4....5....5....6....6....7.
             0    5    0    5    0    5    0    5    0    5    0    5    0
                   CPU% per hour (last 72 hours)
                  * = maximum CPU%   # = average CPU%
I can see that there is a bug id : CSCtg86211 and no work around for it.
Does anyone faced similar issue and any updates received from Cisco TAC ?
Any recommendations to overcome this issue !!
BR / Vimal.

I am also seeing an issue with two stacks of 5 x WS-C2960X-48FP-S switches similar to this case.
Code in use:  c2960x-universalk9-mz.150-2.EX5.bin
At the moment there is nothing connected to the copper 48 ports in any of the switches.
Each stack has a LACP port-channel to a 2 x WS-C3850-48F-E core stack utilising ports x/0/51 and x/0/52 on each 2960X - total 10 ports in each port-channel (8 in use 2 Hot-standby)
The uplink ports are using GLC-Ts
There are 5 vlans (including Vlan1) on the switch.
sw02#show proc cpu | excl 0.00
CPU utilization for five seconds: 42%/0%; one minute: 42%; five minutes: 42%
 PID Runtime(ms)     Invoked      uSecs   5Sec   1Min   5Min TTY Process
  91     1355993    26717170         50  0.29%  0.22%  0.19%   0 RedEarth Tx Mana
  92      886939    26742049         33  0.11%  0.12%  0.16%   0 RedEarth Rx Mana
 121      618726    15414683         40  0.11%  0.06%  0.06%   0 HLFM address lea
 127     2828875    11788935        239  0.41%  0.41%  0.42%   0 hpm main process
 132      637843     4108756        155  0.05%  0.07%  0.05%   0 HRPC pm-counters
 162   176218586    15422705      11425 25.07% 25.30% 25.31%   0 Hulc LED Process
 173     2274099      122941      18497  0.35%  0.31%  0.30%   0 HQM Stack Proces
 174     1720412     1228633       1400  0.23%  0.23%  0.23%   0 HRPC qos request
 183      844538      205684       4105  0.05%  0.10%  0.11%   0 Power RPS Proces
 207        2879       22868        125  0.11%  0.10%  0.06%   0 IP Input
 220      288284     1554997        185  0.11%  0.04%  0.05%   0 Spanning Tree
 237     1356330     6170406        219  0.11%  0.07%  0.14%   0 UDLD
 293      324727     1359786        238  0.05%  0.04%  0.05%   0 Marvell wk-a Pow
 372         120         123        975  0.05%  0.07%  0.01%   1 Virtual Exec
sw03#sh proc cpu | exc 0.00
CPU utilization for five seconds: 42%/0%; one minute: 46%; five minutes: 45%
 PID Runtime(ms)     Invoked      uSecs   5Sec   1Min   5Min TTY Process
  63         507        1733        292  0.05%  0.06%  0.03%   0 Per-Second Jobs
  91        5409       72997         74  0.35%  0.23%  0.23%   0 RedEarth Tx Mana
  92        4526       73169         61  0.11%  0.19%  0.23%   0 RedEarth Rx Mana
 129       11881       29115        408  0.54%  0.42%  0.42%   0 hpm main process
 133        1259        1674        752  0.23%  0.16%  0.12%   0 hpm counter proc
 163      494386       41535      11902 25.38% 25.35% 25.30%   0 Hulc LED Process
 174        6130         374      16390  0.29%  0.31%  0.29%   0 HQM Stack Proces
 175        4446        3249       1368  0.29%  0.25%  0.24%   0 HRPC qos request
 183        2237         559       4001  0.11%  0.16%  0.13%   0 Power RPS Proces
 206         894       41778         21  0.05%  0.03%  0.03%   0 IP ARP Retry Age
 237        5298       16703        317  0.11%  0.14%  0.26%   0 UDLD
 251        4045         452       8949  0.29%  2.68%  0.98%   1 Virtual Exec
 291        1183        3706        319  0.05%  0.05%  0.05%   0 Marvell wk-a Pow
Any advice from Cisco would be much appreciated.
Mark

Similar Messages

  • MACFLAPS seen only on 1 vlan and also has HIGH CPU HULC LED PROCESS

    Hi All, 
    Their is a weird problem we are having with MAC-FLAPS and HIGH CPU uses. The odd think is when we change the VLAN on the port to a different one then the flapping stops but when we turn it back it starts again. I removed the Vlan from a switch (3750) and the CPU usage returned back to normal even though it was only using it for a printer only (was 50%/30% cpu!). 
    However the problem now exists with TWO 2960 PCS and 1 2960 POE and they have HULC LED PROCESS at 65%/40% plus CPU usage. There are no STP or HSRP issues on that VLAN however I think it's related to that VLAN. I even backdated the IOS version and reloaded but problem still persists. 
    What else can I do or is the switch gone (memory leak?). I don't to change all 3 switches so how do I know which switch is the cause as the CPU to be High and only caused due to 1 vlan? ALso when we put something on that VLAN sometimes we get MACFLAPS; although if we change the VLAN the MACFLAPS stop. But the Macflaps are intermittent so I really am not sure what the problem is. 

    Hello.
    Regarding "Hulc led" process - it's a bug; please provide the IOS version you are running.
    It's not clear waht issue do you have with MAC flap; could you please provide command/log outputs?

  • CSCsi78581 - Hulc LED Process uses 10-15 percent CPU on Catalyst 3750/3560 - 10

    I have the same issue, but now is in 3750E with IOS
    15.0(1)SE3 does anybody see this bug in this version of IOS ?
          345433434433338833333333444433353433343334433433333533796763
          829698281199793397776998091576799099908973089099999499163527
      100                                                        * 
       90                                                        * 
       80               **                                       * *
       70               **                                      ** *
       60   *           **               *                      ****
       50   **          **         * *   *                   *  *##*
       40 *###**#******###*********###****###**#***##*##*###########
       30 ##########################################################
       20 ##########################################################
       10 ##########################################################
         0....5....1....1....2....2....3....3....4....4....5....5....6
                   0    5    0    5    0    5    0    5    0    5    0
                   CPU% per minute (last 60 minutes)
                  * = maximum CPU%   # = average CPU%
    173  1360214457   674705267       2016 19.68% 19.77% 19.76%   0 Hulc LED Process

    I see this issue on a Catalyst 3560X-48P running IOS 15.0(2)SE2.
    switch#show proc cpu sorted
    CPU utilization for five seconds: 44%/0%; one minute: 42%; five minutes: 41%
    PID Runtime(ms)     Invoked      uSecs   5Sec   1Min   5Min TTY Process
    171  3074931895   362854359       8474 20.31% 20.10% 20.00%   0 Hulc LED Process

  • CSCsi78581 - Hulc LED Process has even higher CPU (20%) running "fixed" version 15.2(2)E

    A known good fix for the CPU issue was listed as version 15.2(2)E.
    However, I have found that this version has even a higher CPU usage of 20% than the original bug indicated at 15%.
    Does anyone know a fix for this?

    Experiencing this with 12.2(55)SE9 as well.

  • Catalyst 2960 High CPU Utilization

    Hi,
    I have a WS-C2960S-48FPS-L running IOS Version 12.2(53r)SE and while monitoring CPU utilization I see I have a 70% average on the last 72 hrs.
        3333333333333333333333333333333333333333333333333333333333
        5555533333333334444422222444442222222222222233333222229999
    100                                                           
     90                                                           
     80                                                           
     70                                                           
     60                                                           
     50                                                           
     40 *****                                                 ****
     30 **********************************************************
     20 **********************************************************
     10 **********************************************************
       0....5....1....1....2....2....3....3....4....4....5....5....
                 0    5    0    5    0    5    0    5    0    5    
                   CPU% per second (last 60 seconds)
        3333346734434544477676334344333544343334434344766666666677
        9689618080091001020909970900769300918880390818198898989900
    100                                                           
     90                                                           
     80                                                           
     70       **         *###*                        *###########
     60       **         *###*                        ############
     50       #*     *   ####*         *             *############
     40 ******##*********#####*********#*************#############
     30 ##########################################################
     20 ##########################################################
     10 ##########################################################
       0....5....1....1....2....2....3....3....4....4....5....5....
                 0    5    0    5    0    5    0    5    0    5    
                   CPU% per minute (last 60 minutes)
                  * = maximum CPU%   # = average CPU%
        7777777777777777777877777777777778777777777777777777777777777777777877
        2228292592534266552024223272418464276359644452633212722634631323472084
    100                                                                       
     90                                                                       
     80    * * ** *   **** *      *   * ** ** ***   * *     *  *  *      * ** 
     70 ######################################################################
     60 ######################################################################
     50 ######################################################################
     40 ######################################################################
     30 ######################################################################
     20 ######################################################################
     10 ######################################################################
       0....5....1....1....2....2....3....3....4....4....5....5....6....6....7.
                 0    5    0    5    0    5    0    5    0    5    0    5    0 
                       CPU% per hour (last 72 hours)
                      * = maximum CPU%   # = average CPU%
    CPU utilization for five seconds: 33%/1%; one minute: 34%; five minutes: 35%
     PID Runtime(ms)   Invoked      uSecs   5Sec   1Min   5Min TTY Process 
     137  1130229138  72291065      15634 25.99% 25.54% 25.42%   0 Hulc LED Process 
    ARWWA-SW-C2960S#show processes cpu sorted 5min
    CPU utilization for five seconds: 34%/1%; one minute: 34%; five minutes: 35%
     PID Runtime(ms)   Invoked      uSecs   5Sec   1Min   5Min TTY Process 
     137  1130236489  72291533      15634 25.86% 25.52% 25.42%   0 Hulc LED Process 
    Could you please advise what this "Hulc LED Process" is? Has anyone encountered this issue before?
    Thanks,

    Version 12.2(53r)SE
    This is not your IOS version.  This is your bootstrap version. 
    What IOS are you running?  If you've got 2960S, then you've got two stable IOS versions to choose from:  12.2(55)SE9 or 15.0(2)SE4. 
    Any other version is rubbish.

  • High CPU utilization on 2960 switch

    Hi
    I have a 2960 switch connected in my network. It is showing high CPU utilization upto 80-90%. When I checked I found that the below processes are taking the highest utilization. The image in the switch is LAN base. Can someone please advice on why I am having high utilization on the switch for the below processes and how can I reduce it.
    14.69% 14.98% 15.00%   0 Hulc LED Process
    14.69% 11.49% 13.97%   0 IP Input
    29.07% 28.44% 30.06%   0 DHCPD Receive

    Duplicate post. 
    Go HERE.

  • 3750 stack - high cpu for "hulc running con" every one or two minutes

    I have a stack of 4 x 3750 :
    Switch Ports Model              SW Version            SW Image
    *    1 28    WS-C3750G-24PS     12.2(53)SE            C3750-IPSERVICES-M
         2 12    WS-C3750G-12S      12.2(53)SE            C3750-IPSERVICES-M
         3 12    WS-C3750G-12S      12.2(53)SE            C3750-IPSERVICES-M
         4 28    WS-C3750G-24PS     12.2(53)SE            C3750-IPSERVICES-M
    And every minutes or two minutes (approximately), I have high cpu for "hulc running con" without doing anything.
    coeur#sh proc cpu sort
    CPU utilization for five seconds: 86%/28%; one minute: 49%; five minutes: 49%
     PID Runtime(ms)   Invoked      uSecs   5Sec   1Min   5Min TTY Process 
     312   670849714   3362057     199535 44.22%  5.00%  4.96%   0 hulc running con 
      82   234351581 353049778        663  3.76%  5.99%  5.95%   0 HLFM address lea 
     277       11138      8537       1304  1.05%  1.70%  1.12%   1 Virtual Exec     
     142    10589757 353659360         29  0.90%  0.33%  0.22%   0 Hulc LED Process 
    coeur#sh proc cpu sort
    CPU utilization for five seconds: 95%/27%; one minute: 52%; five minutes: 49%
     PID Runtime(ms)   Invoked      uSecs   5Sec   1Min   5Min TTY Process 
     312   670832227   3361981     199545 55.59%  8.95%  5.76%   0 hulc running con 
      82   234334259 353043921        663  3.99%  5.78%  6.02%   0 HLFM address lea 
     277        6286      7531        834  0.63%  1.06%  0.61%   1 Virtual Exec     
     106    15342490  19948859        769  0.47%  0.20%  0.15%   0 hpm counter proc 
     119     2034091  13865800        146  0.31%  0.04%  0.03%   0 LLDP Protocol    
     142    10588906 353653113         29  0.31%  0.21%  0.16%   0 Hulc LED Process 
    How can I find which is doing a "show run" ? Which debug can I do ?
    Thanks

    We solved our problem it was caused because we were routing in and out the same L3 interface.
    Fixed this issue and our CPU dropped dramatically.

  • Cisco LMS 3.2.1 Causing Switch CPU Spike

    Hi,
    We have LMS 3.2.1 cauing CPU Spike on all the LAN switches. I treid to apply the commands as per the below URL but still the issue is same.
    I had attached Show version, Show stacks and Show process cpu sorted output.
    sh version
    Cisco IOS Software, C3750 Software (C3750-IPBASE-M), Version 12.2(35)SE5, RELEASE SOFTWARE (fc1)
    Copyright (c) 1986-2007 by Cisco Systems, Inc.
    Compiled Thu 19-Jul-07 19:15 by nachen
    Image text-base: 0x00003000, data-base: 0x01080000
    ROM: Bootstrap program is C3750 boot loader
    BOOTLDR: C3750 Boot Loader (C3750-HBOOT-M) Version 12.2(25r)SEC, RELEASE SOFTWARE (fc4)
    ATCBmer-OPERas08 uptime is 30 weeks, 1 day, 3 hours, 20 minutes
    System returned to ROM by power-on
    System restarted at 08:19:08 UTC Tue Jul 2 2013
    System image file is "flash:c3750-ipbase-mz.122-35.SE5/c3750-ipbase-mz.122-35.SE5.bin"
    cisco WS-C3750-48TS (PowerPC405) processor (revision Q0) with 118784K/12280K bytes of memory.
    Processor board ID FDO1339Y2FC
    Last reset from power-on
    2 Virtual Ethernet interfaces
    96 FastEthernet interfaces
    8 Gigabit Ethernet interfaces
    The password-recovery mechanism is enabled.
    512K bytes of flash-simulated non-volatile configuration memory.
    Base ethernet MAC Address       : EC:30:91:4A:CF:80
    Motherboard assembly number     : 73-9680-12
    Power supply part number        : 341-0028-02
    Motherboard serial number       : FDO133916XY
    Power supply serial number      : DTN133510WF
    Model revision number           : Q0
    Motherboard revision number     : A0
    Model number                    : WS-C3750-48TS-S
    System serial number            : FDO1339Y2FC
    SFP Module assembly part number : 73-7757-03
    SFP Module revision Number      : A0
    SFP Module serial number        : FDO13390UPX
    Top Assembly Part Number        : 800-25854-04
    Top Assembly Revision Number    : A0
    Version ID                      : V07
    CLEI Code Number                : COMD200BRA
    Hardware Board Revision Number  : 0x01
    Switch   Ports  Model              SW Version              SW Image           
         1   52     WS-C3750-48TS      12.2(35)SE5             C3750-IPBASE-M     
    *    2   52     WS-C3750-48TS      12.2(35)SE5             C3750-IPBASE-M     
    Switch 01
    Switch Uptime                   : 30 weeks, 1 day, 15 minutes
    Base ethernet MAC Address       : EC:30:91:4A:D0:80
    Motherboard assembly number     : 73-9680-12
    Power supply part number        : 341-0028-02
    Motherboard serial number       : FDO133916ZJ
    Power supply serial number      : DTN133510WC
    Model revision number           : Q0
    Motherboard revision number     : A0
    Model number                    : WS-C3750-48TS-S
    System serial number            : FDO1339Y2FB
    SFP Module assembly part number : 73-7757-03
    SFP Module revision number      : A0
    SFP Module serial number        : FDO13390UPS
    Top assembly part number        : 800-25854-04
    Top assembly revision number    : A0
    Version ID                      : V07
    CLEI Code Number                : COMD200BRA
    Configuration register is 0xF
    #sh processes cpu sorted
    CPU utilization for five seconds: 7%/1%; one minute: 7%; five minutes: 7%
    PID Runtime(ms)   Invoked      uSecs   5Sec   1Min   5Min TTY Process
    251        2051       209       9813  0.79%  0.32%  0.36%   6 Virtual Exec    
    131    61393351 248916997        246  0.63%  0.71%  0.60%   0 HL2MCM          
    123     6997778   3642657       1921  0.15%  0.03%  0.00%   0 HQM Stack Proces
    191   105112370 573529786        183  0.15%  0.29%  0.30%   0 IGMPSN          
    124    29851586  14567458       2049  0.15%  0.17%  0.15%   0 HRPC qos request
    115     4380679 434564249         10  0.15%  0.03%  0.00%   0 Hulc LED Process
       7           0         1          0  0.00%  0.00%  0.00%   0 HRPC asic-stats 
       8           0         1          0  0.00%  0.00%  0.00%   0 Crash writer    
       9       23632    348343         67  0.00%  0.00%  0.00%   0 ARP Input       
      10           0         1          0  0.00%  0.00%  0.00%   0 AAA_SERVER_DEADT
      11           0         2          0  0.00%  0.00%  0.00%   0 AAA high-capacit
      12           0         1          0  0.00%  0.00%  0.00%   0 Policy Manager  
      13          42        11       3818  0.00%  0.00%  0.00%   0 Entity MIB API  
      14           0         1          0  0.00%  0.00%  0.00%   0 IFS Agent Manage
      15         242    304021          0  0.00%  0.00%  0.00%   0 IPC Dynamic Cach
      16           0         1          0  0.00%  0.00%  0.00%   0 IPC Zone Manager
      17        5029  18121023          0  0.00%  0.00%  0.00%   0 IPC Periodic Tim
      18           0       156          0  0.00%  0.00%  0.00%   0 IPC Managed Time
      19        5258  18121026          0  0.00%  0.00%  0.00%   0 IPC Deferred Por
      20           9       157         57  0.00%  0.00%  0.00%   0 IPC Seat Manager
      21           0         1          0  0.00%  0.00%  0.00%   0 IPC Session Serv
      22      194590   5460341         35  0.00%  0.00%  0.00%   0 HC Counter Timer
      23        6925  18121021          0  0.00%  0.00%  0.00%   0 Dynamic ARP Insp
      24           0         1          0  0.00%  0.00%  0.00%   0 ARP Snoop       
      25           0         1          0  0.00%  0.00%  0.00%   0 License IPC stat
      26           0         1          0  0.00%  0.00%  0.00%   0 License IPC serv
      27        5093  18120999          0  0.00%  0.00%  0.00%   0 GraphIt         
      28           0         2          0  0.00%  0.00%  0.00%   0 XML Proxy Client
      29           0         1          0  0.00%  0.00%  0.00%   0 Critical Bkgnd  
      30      308341  44341035          6  0.00%  0.00%  0.00%   0 Net Background  
      31           0         1          0  0.00%  0.00%  0.00%   0 IDB Work        
       3           0        19          0  0.00%  0.00%  0.00%   0 SNMP Timers     
       4    18484344   2158292       8564  0.00%  0.10%  0.06%   0 Check heaps     
       6           0         2          0  0.00%  0.00%  0.00%   0 Timers          
      35     4296336    307754      13960  0.00%  0.01%  0.00%   0 Per-minute Jobs 
      36           0        33          0  0.00%  0.00%  0.00%   0 AggMgr Process  
      37           8        26        307  0.00%  0.00%  0.00%   0 Net Input       
      38     5416553   3642636       1486  0.00%  0.00%  0.00%   0 Compute load avg
      39          25        10       2500  0.00%  0.00%  0.00%   0 Collection proce
      40       20231 362215119          0  0.00%  0.00%  0.00%   0 DownWhenLooped  
      41           0         1          0  0.00%  0.00%  0.00%   0 HRPC hdwl reques
      42           0         1          0  0.00%  0.00%  0.00%   0 HRPC emac reques
      43           0         3          0  0.00%  0.00%  0.00%   0 HRPC lpip reques
      44           0         4          0  0.00%  0.00%  0.00%   0 HLPIP Sync Proce
      45           0         3          0  0.00%  0.00%  0.00%   0 HRPC Multi-FS Sy
      46           0        15          0  0.00%  0.00%  0.00%   0 HULC multifs pro
      47           0         2          0  0.00%  0.00%  0.00%   0 MIRAGE RBCP Moni
       2        1468   3642677          0  0.00%  0.00%  0.00%   0 Load Meter      
      49           0        16          0  0.00%  0.00%  0.00%   0 Stack Mgr       
      50         678        42      16142  0.00%  0.00%  0.00%   0 Stack Mgr Notifi
      51     3155448 977563359          3  0.00%  0.01%  0.00%   0 Fifo Error Detec
      52          16        11       1454  0.00%  0.00%  0.00%   0 Adjust Regions  
      53       44680  18121002          2  0.00%  0.00%  0.00%   0 hrpc -> response
      54      443728  28994073         15  0.00%  0.00%  0.00%   0 hrpc -> request 
      55     6463350  36914367        175  0.00%  0.00%  0.00%   0 hrpc <- response
      56           0         3          0  0.00%  0.00%  0.00%   0 HRPC hcomp reque
      57           0        22          0  0.00%  0.00%  0.00%   0 HULC Device Mana
      32          48      8960          5  0.00%  0.00%  0.00%   0 Logger          
      59           0         7          0  0.00%  0.00%  0.00%   0 HRPC hdm blockin
      60          25         4       6250  0.00%  0.00%  0.00%   0 HRPC cfg_backup 
      61       90371   3642630         24  0.00%  0.00%  0.00%   0 HIPC bkgrd proce
       5        8473      3810       2223  0.00%  0.00%  0.00%   0 Pool Manager    
      34       52605  18121026          2  0.00%  0.00%  0.00%   0 Per-Second Jobs 
      33        6963  18120995          0  0.00%  0.00%  0.00%   0 TTY Background  
      65        1281      7231        177  0.00%  0.00%  0.00%   0 HRPC hpsecure re
      48           0         1          0  0.00%  0.00%  0.00%   0 HRPC hsm request
      67      238024 542998290          0  0.00%  0.00%  0.00%   0 HLFM address lea
      68        6856  18120996          0  0.00%  0.00%  0.00%   0 HLFM aging proce
      69      139202 543008514          0  0.00%  0.00%  0.00%   0 HLFM address ret
      70           0         1          0  0.00%  0.00%  0.00%   0 HRPC hrcmd reque
      71       94414    693286        136  0.00%  0.00%  0.00%   0 HRPC hulc misc r
      72           0         3          0  0.00%  0.00%  0.00%   0 HRPC system mtu 
      73        3828   6074329          0  0.00%  0.00%  0.00%   0 HVLAN main bkgrd
      74     1477074   5127744        288  0.00%  0.00%  0.00%   0 IP SNMP         
      75           0         2          0  0.00%  0.00%  0.00%   0 Vlan shutdown Pr
      76       27087    170972        158  0.00%  0.00%  0.00%   0 HRPC vlan reques
      77           0         2          0  0.00%  0.00%  0.00%   0 HULC VLAN REF Ba
      78           0         5          0  0.00%  0.00%  0.00%   0 HRPC hfbm reques
      79        4132  18120976          0  0.00%  0.00%  0.00%   0 MDFS LC Download
      80           0         5          0  0.00%  0.00%  0.00%   0 HRPC ilp request
      81           0         1          0  0.00%  0.00%  0.00%   0 HPM Msg Retry Pr
      82         470     74807          6  0.00%  0.00%  0.00%   0 TACACS+         
      83      816214 201441346          4  0.00%  0.00%  0.00%   0 hpm main process
      84         313      7676         40  0.00%  0.00%  0.00%   0 HPM Stack Sync P
      85        6174     28492        216  0.00%  0.00%  0.00%   0 HRPC pm request 
      86           0         3          0  0.00%  0.00%  0.00%   0 HPM if_num mappi
      87    11970484  37978485        315  0.00%  0.07%  0.05%   0 hpm counter proc
      88     8254076  39504043        208  0.00%  0.01%  0.02%   0 HRPC pm-counters
      89           0         1          0  0.00%  0.00%  0.00%   0 hpm vp events ca
      90           0       235          0  0.00%  0.00%  0.00%   0 HRPC hcmp reques
      91           8      8695          0  0.00%  0.00%  0.00%   0 HCEF ADJ Refresh
      92           0         1          0  0.00%  0.00%  0.00%   0 HRPC hl3mm reque
      93           0         1          0  0.00%  0.00%  0.00%   0 hl3md_rpfq_thrl_
      94       78051  12727882          6  0.00%  0.00%  0.00%   0 hl3mm           
      58           0        17          0  0.00%  0.00%  0.00%   0 HRPC hdm non blo
      96           0         4          0  0.00%  0.00%  0.00%   0 HRPC acl request
      97           8        55        145  0.00%  0.00%  0.00%   0 HACL Acl Manager
      98           0         1          0  0.00%  0.00%  0.00%   0 HRPC backup inte
      99           0         1          0  0.00%  0.00%  0.00%   0 HRPC cdp request
    100        1622     13860        117  0.00%  0.00%  0.00%   0 HRPC dot1x reque
    101        3284     12837        255  0.00%  0.00%  0.00%   0 HULC DOT1X Proce
    102           0         1          0  0.00%  0.00%  0.00%   0 HRPC foxhound re
    103           0         2          0  0.00%  0.00%  0.00%   0 Hulc Foxhound Pr
    104           0         1          0  0.00%  0.00%  0.00%   0 HRPC sdm request
    105      180683  90585811          1  0.00%  0.00%  0.00%   0 Hulc Storm Contr
    106           0         2          0  0.00%  0.00%  0.00%   0 HSTP Sync Proces
    107         285     10024         28  0.00%  0.00%  0.00%   0 HRPC stp_cli req
    108          17       100        170  0.00%  0.00%  0.00%   0 HRPC stp_state_s
    109           0         2          0  0.00%  0.00%  0.00%   0 S/W Bridge Proce
    110           0        22          0  0.00%  0.00%  0.00%   0 HRPC hudld reque
    111           0         1          0  0.00%  0.00%  0.00%   0 HRPC vqpc reques
    112           0         1          0  0.00%  0.00%  0.00%   0 HRPC iec_load_ba
    113           0        14          0  0.00%  0.00%  0.00%   0 HRPC l2pt qnq rp
    114           0         3          0  0.00%  0.00%  0.00%   0 HRPC hled reques
       1         795     10877         73  0.00%  0.00%  0.00%   0 Chunk Manager   
    116       61910  13347463          4  0.00%  0.00%  0.00%   0 HL3U bkgrd proce
    117      572759   7248461         79  0.00%  0.00%  0.00%   0 HRPC hl3u reques
    118        1586   4165623          0  0.00%  0.00%  0.00%   0 HL3U PBR bkgrd p
    119         186    152011          1  0.00%  0.00%  0.00%   0 HL3U PBR n-h res
    120           0         1          0  0.00%  0.00%  0.00%   0 HRPC dtp request
    121           0         1          0  0.00%  0.00%  0.00%   0 HRPC show_forwar
    122         175        60       2916  0.00%  0.00%  0.00%   0 HRPC snmp reques
      62           0         1          0  0.00%  0.00%  0.00%   0 IPC RTTYC Messag
      63           0         1          0  0.00%  0.00%  0.00%   0 RTTYC Flush     
      64        1050      6840        153  0.00%  0.00%  0.00%   0 Hulc Port-Securi
    126           0         1          0  0.00%  0.00%  0.00%   0 HRPC obfl reques
      66       22215    141914        156  0.00%  0.00%  0.00%   0 HRPC hlfm reques
    128           0         9          0  0.00%  0.00%  0.00%   0 Hulc Reload Mana
    129           8        19        421  0.00%  0.00%  0.00%   0 HRPC hrcli-event
    130       50722     21263       2385  0.00%  0.00%  0.00%   0 HRPC rfs request
      95           0         1          0  0.00%  0.00%  0.00%   0 HACL Queue Proce
    132           8         8       1000  0.00%  0.00%  0.00%   0 HL2MCM          
    133           0         2          0  0.00%  0.00%  0.00%   0 EAPoUDP Process 
    134           0         3          0  0.00%  0.00%  0.00%   0 CEF switching ba
    135       53206  18120990          2  0.00%  0.00%  0.00%   0 PI MATM Aging Pr
    136         154      7422         20  0.00%  0.00%  0.00%   0 Switch Backup In
    137         313    304021          1  0.00%  0.00%  0.00%   0 MMN bkgrd proces
    138           0         2          0  0.00%  0.00%  0.00%   0 Dot1x Mgr Proces
    139           0         1          0  0.00%  0.00%  0.00%   0 MAB Framework   
    140         567      3467        163  0.00%  0.00%  0.00%   0 802.1x switch   
    141           0         1          0  0.00%  0.00%  0.00%   0 802.1x Webauth F
    142           0         1          0  0.00%  0.00%  0.00%   0 802.1x Critical 
    143       81511   1214105         67  0.00%  0.00%  0.00%   0 DTP Protocol    
    144           0         1          0  0.00%  0.00%  0.00%   0 EAP Framework   
    145           0         1          0  0.00%  0.00%  0.00%   0 HRPC dai request
    146           0         1          0  0.00%  0.00%  0.00%   0 HULC DAI Process
    147           0         5          0  0.00%  0.00%  0.00%   0 HRPC dhcp snoopi
    148           0         5          0  0.00%  0.00%  0.00%   0 HULC DHCP Snoopi
    149           0         1          0  0.00%  0.00%  0.00%   0 HRPC ip source g
    150           0         1          0  0.00%  0.00%  0.00%   0 HULC IP Source g
    151        7033  18242384          0  0.00%  0.00%  0.00%   0 UDLD            
    152         125    608080          0  0.00%  0.00%  0.00%   0 Port-Security   
    153           0         3          0  0.00%  0.00%  0.00%   0 OBFL Cfg Dispatc
    154           0         2          0  0.00%  0.00%  0.00%   0 Switch IP Host T
    155           0         1          0  0.00%  0.00%  0.00%   0 Link State Group
    156         367   1823271          0  0.00%  0.00%  0.00%   0 Ethchnl         
    157         303      2174        139  0.00%  0.00%  0.00%   0 VMATM Callback  
    158       23146 181145818          0  0.00%  0.00%  0.00%   0 MDFS MFIB Proces
    159           0         1          0  0.00%  0.00%  0.00%   0 IPv6 RIB Redistr
    160           9         2       4500  0.00%  0.00%  0.00%   0 AAA Server      
    161           0         1          0  0.00%  0.00%  0.00%   0 AAA ACCT Proc   
    162           0         1          0  0.00%  0.00%  0.00%   0 ACCT Periodic Pr
    163     8780828  15301673        573  0.00%  0.01%  0.00%   0 CDP Protocol    
    164       13401      6681       2005  0.00%  0.00%  0.00%   0 SNMP Traps      
    165           0         1          0  0.00%  0.00%  0.00%   0 image mgr       
    166           0         2          0  0.00%  0.00%  0.00%   0 AAA Dictionary R
    167         139    152026          0  0.00%  0.00%  0.00%   0 DHCP Snooping   
    168     4637569  47657943         97  0.00%  0.07%  0.04%   0 IP Input        
    169           0         1          0  0.00%  0.00%  0.00%   0 ICMP event handl
    170    10066119  69946340        143  0.00%  0.04%  0.05%   0 Spanning Tree   
    171          24    304027          0  0.00%  0.00%  0.00%   0 Spanning Tree St
    172       53121    364742        145  0.00%  0.00%  0.00%   0 CEF background p
    173           0         1          0  0.00%  0.00%  0.00%   0 IP IRDP         
    174          16         2       8000  0.00%  0.00%  0.00%   0 CEF RF HULC Conv
    175         162      3972         40  0.00%  0.00%  0.00%   0 XDR mcast       
    176       21665     37317        580  0.00%  0.00%  0.00%   0 AAA Accounting  
    178           0         4          0  0.00%  0.00%  0.00%   0 MDFS LC Process 
    179          41    228026          0  0.00%  0.00%  0.00%   0 Cluster L2      
    180         487   1823270          0  0.00%  0.00%  0.00%   0 Cluster RARP    
    181        1274   2279503          0  0.00%  0.00%  0.00%   0 Cluster Base    
    182        4979    582218          8  0.00%  0.00%  0.00%   0 TCP Timer       
    183        2534      4377        578  0.00%  0.00%  0.00%   0 TCP Protocols   
    184           0         1          0  0.00%  0.00%  0.00%   0 Socket Timers   
    185         370     61248          6  0.00%  0.00%  0.00%   0 HTTP CORE       
    186           0         1          0  0.00%  0.00%  0.00%   0 RARP Input      
    187         191       357        535  0.00%  0.00%  0.00%   1 Virtual Exec    
    188         312       796        391  0.00%  0.00%  0.00%   2 Virtual Exec    
    189        5986      3688       1623  0.00%  0.00%  0.00%   0 IGMPSN L2MCM    
    190     1988996  22053535         90  0.00%  0.02%  0.00%   0 IGMPSN MRD      
    125           0         1          0  0.00%  0.00%  0.00%   0 HRPC span reques
    192           0         1          0  0.00%  0.00%  0.00%   0 IGMPQR          
    193           0         1          0  0.00%  0.00%  0.00%   0 CEF MQC IPC Back
    194           0         2          0  0.00%  0.00%  0.00%   0 L2TRACE SERVER  
    195        2867      3686        777  0.00%  0.00%  0.00%   0 MLDSN L2MCM     
    196           0         1          0  0.00%  0.00%  0.00%   0 MRD             
    197           0         1          0  0.00%  0.00%  0.00%   0 MLD_SNOOP       
    198      149666  28335503          5  0.00%  0.00%  0.00%   0 CEF: IPv4 proces
    199        2752    303845          9  0.00%  0.00%  0.00%   0 IP RIB Update   
    200           0         1          0  0.00%  0.00%  0.00%   0 Auth-proxy AAA B
    201           9     60808          0  0.00%  0.00%  0.00%   0 IP Admin SM Proc
    202      189407    791688        239  0.00%  0.00%  0.00%   0 DHCPD Receive   
    203          25    152012          0  0.00%  0.00%  0.00%   0 DHCPD Timer     
    204           0         2          0  0.00%  0.00%  0.00%   0 AAA Cached Serve
    205           0         2          0  0.00%  0.00%  0.00%   0 LOCAL AAA       
    206           9         2       4500  0.00%  0.00%  0.00%   0 TPLUS           
    207           0         1          0  0.00%  0.00%  0.00%   0 HRPC x_setup req
    208           9        39        230  0.00%  0.00%  0.00%   0 VTP Trap Process
    209           0         2          0  0.00%  0.00%  0.00%   0 VTPMIB EDIT BUFF
    210           0         2          0  0.00%  0.00%  0.00%   0 DHCP Security He
    211           0         1          0  0.00%  0.00%  0.00%   0 HCD Process     
    212           0         1          0  0.00%  0.00%  0.00%   0 HRPC cable diagn
    213           0         2          0  0.00%  0.00%  0.00%   0 DiagCard2/-1    
    214      134713 218636042          0  0.00%  0.00%  0.00%   0 PM Callback     
    215           0        20          0  0.00%  0.00%  0.00%   0 VLAN Manager    
    216       97175   5167135         18  0.00%  0.00%  0.00%   0 DHCPD Database  
    217         133   1458592          0  0.00%  0.00%  0.00%   0 dhcp snooping sw
    218           0         1          0  0.00%  0.00%  0.00%   0 Licensing MIB pr
    219           0         3          0  0.00%  0.00%  0.00%   0 RADIUS TEST CMD 
    220           0         2          0  0.00%  0.00%  0.00%   0 AAA SEND STOP EV
    221        9523      2686       3545  0.00%  0.00%  0.00%   0 Syslog Traps    
    222           0         1          0  0.00%  0.00%  0.00%   0 SAA MPLSLM Proce
    223          93      2217         41  0.00%  0.00%  0.00%   0 ADJ background  
    224           0         2          0  0.00%  0.00%  0.00%   0 STP FAST TRANSIT
    225           9       135         66  0.00%  0.00%  0.00%   0 CSRT RAPID TRANS
    226       20638    168875        122  0.00%  0.00%  0.00%   0 HRPC hl2mcm igmp
    227           0        38          0  0.00%  0.00%  0.00%   0 HRPC hl2mcm mlds
    228         446       765        583  0.00%  0.00%  0.00%   3 Virtual Exec    
    229        6840      1901       3598  0.00%  0.00%  0.00%   0 SpanTree Helper 
    230           0         1          0  0.00%  0.00%  0.00%   0 DiagCard1/-1    
    231           0         1          0  0.00%  0.00%  0.00%   0 LICENSE AGENT   
    232          94       420        223  0.00%  0.00%  0.00%   0 SpanTree Flush  
    233       22092    540058         40  0.00%  0.00%  0.00%   0 IP Background   
    234      833188   2574651        323  0.00%  0.00%  0.00%   0 PDU DISPATCHER  
    235    22681449   5799556       3910  0.00%  0.00%  0.00%   0 SNMP ENGINE     
    236           0         1          0  0.00%  0.00%  0.00%   0 SNMP ConfCopyPro
    237       16028  18381176          0  0.00%  0.00%  0.00%   0 NTP             
    238          33       139        237  0.00%  0.00%  0.00%   0 hulc cfg mgr mas
    239       98617       500     197234  0.00%  0.00%  0.00%   0 hulc running con
    240           0         1          0  0.00%  0.00%  0.00%   0 RTTYS Process   
    241         335    303847          1  0.00%  0.00%  0.00%   0 802.1x MDA Aging
    242         346       513        674  0.00%  0.00%  0.00%   4 Virtual Exec    
    243         628       999        628  0.00%  0.00%  0.00%   5 Virtual Exec    
    244         321    101289          3  0.00%  0.00%  0.00%   0 HCMP sync proces
    245           0         2          0  0.00%  0.00%  0.00%   0 hulc_tb_process 
    246     1629625  18112558         89  0.00%  0.00%  0.00%   0 IPC LC Message H
    247           0         2          0  0.00%  0.00%  0.00%   0 XDR RP Ping Back
    248         181    151926          1  0.00%  0.00%  0.00%   0 XDR RP backgroun
    249           0         1          0  0.00%  0.00%  0.00%   0 XDR RP Test Back
    250           9         5       1800  0.00%  0.00%  0.00%   0 CEF RP IPC Backg
    127           0        14          0  0.00%  0.00%  0.00%   0 HRPC system post
    252         441       613        719  0.00%  0.00%  0.00%   7 Virtual Exec    
    254         626      1254        499  0.00%  0.00%  0.00%   9 Virtual Exec   
    sh stacks
    Minimum process stacks:
    Free/Size   Name
    5112/6000   hulc_flash init
    5740/6000   CDP BLOB
    8572/9000   IP Background
    5664/6000   vqpc_shim_create_addr_tbl
    5648/6000   SPAN Subsystem
    5396/6000   LICENSE AGENT DEFAULT
    5684/6000   stp_change_tree_type
    8732/9000   cdp init process
    8576/12000  Init
    5644/6000   IPC Zone Manager
    5720/6000   RTTY Slave Registration Process
    4792/6000   IPC Split Init
    5244/6000   HUDLD fiber table process
    5496/6000   bulk config process
    5496/6000   RADIUS INITCONFIG
    2600/3000   Rom Random Update Process
    5228/6000   MDFS LC IPC Init Process
    5420/6000   HVLAN Mapped Vlan Id Update Process
    8284/12000  cfg mgr new master process
    4908/6000   hpm bulk_vlan_state process
    5676/6000   HDF Stack-Mac
    5724/6000   HCMP redirect process
    5648/6000   HCMP new mac process
    5352/6000   XDR LC PRIO Background
    9652/12000  XDR LC Background
    5388/6000   CEF LC Stats
    5396/6000   CEF LC IPC Background
    5368/6000   HL3U ARP Refresh Proc
    5128/6000   RTTY Client Registration
    5152/6000   IPC LC Port Opener
    35132/36000  TCP Command
    7644/12000  Virtual Exec
    19784/24000  hulc_techsupport_process
    Interrupt level stacks:
    Level    Called Unused/Size  Name
      4  1981299888   7808/9000  NETWORK INTERFACE CHIP
      5           0   9000/9000  SUPERVISOR EXCEPTIONS
      6        6937   8932/9000  NS16550 VECTOR
    Please help me in solving the issue.
    Thanks
    Nibin Rodrigues

    Please find more EEM output
    CPU utilization for five seconds: 72%/1%; one minute: 20%; five minutes: 13%
     PID Runtime(ms)   Invoked      uSecs   5Sec   1Min   5Min TTY Process 
     325    71942425  29061721       2475 53.18%  8.25%  1.83%   0 SNMP ENGINE      
     149    70334613 922183020         76  3.98%  0.79%  0.31%   0 Hulc LED Process 
      93   209189468   3227049      64824  3.02%  0.70%  0.60%   0 HULC Tcam Memory 
      71   1093435341690591985         64  2.06%  0.71%  0.41%   0 Fifo Error Detec 
     110   291975112  66878504       4365  0.95%  0.86%  0.85%   0 hpm counter proc 
     168    81777686 292066101        279  0.63%  0.25%  0.26%   0 HL2MCM           
     158   124763719   6454160      19330  0.47%  0.35%  0.36%   0 HQM Stack Proces 
     224    42202897 212601403        198  0.31%  0.08%  0.08%   0 Spanning Tree    
     159   106423401  51600833       2062  0.31%  0.24%  0.21%   0 HRPC qos request 
     106    13220026 370664106         35  0.31%  0.07%  0.01%   0 hpm main process 
     262     3364322  67969552         49  0.15%  0.01%  0.00%   0 DHCPD Receive    
      65    52065771 720246777         72  0.15%  0.19%  0.18%   0 RedEarth Tx Mana 
     211     9167389  82882822        110  0.15%  0.15%  0.04%   0 IP Input         
      31    21262514   3938892       5398  0.15%  0.07%  0.05%   0 Net Background   
      35     3271220  32329547        101  0.15%  0.02%  0.00%   0 Per-Second Jobs  
      75    30667151 134022347        228  0.15%  0.38%  0.14%   0 hrpc <- response 
      66     53952131261367847          4  0.15%  0.02%  0.00%   0 RedEarth Rx Mana 
     118       63844  16135251          3  0.15%  0.01%  0.00%   0 hl3mm_rp         
     323     4050227  15006221        269  0.15%  0.22%  0.05%   0 IP SNMP          
     189      346264  32497199         10  0.15%  0.01%  0.00%   0 UDLD             
     329     1715198 327525422          5  0.15%  0.04%  0.00%   0 MDFS RP process  
      21      476092   8656305         54  0.00%  0.00%  0.00%   0 IPC Seat Manager 
      20       91963  32261597          2  0.00%  0.00%  0.00%   0 IPC Deferred Por 
      19           0        40          0  0.00%  0.00%  0.00%   0 IPC Managed Time 
      18      110772  32261594          3  0.00%  0.00%  0.00%   0 IPC Periodic Tim 
      22           0         1          0  0.00%  0.00%  0.00%   0 IPC Session Serv 
      23          75         4      18750  0.00%  0.00%  0.00%   0 PrstVbl          
      28       91050  32261466          2  0.00%  0.00%  0.00%   0 GraphIt          
      17           0        21          0  0.00%  0.00%  0.00%   0 ADJ background   
      30           0         1          0  0.00%  0.00%  0.00%   0 Critical Bkgnd   
      16       16573    541341         30  0.00%  0.00%  0.00%   0 IPC Dynamic Cach 
      24      117890  32261592          3  0.00%  0.00%  0.00%   0 Dynamic ARP Insp 
      15           0         1          0  0.00%  0.00%  0.00%   0 IFS Agent Manage 
      34      103038  32261458          3  0.00%  0.00%  0.00%   0 TTY Background   
      25           0         1          0  0.00%  0.00%  0.00%   0 ARP Snoop        
      26           0         1          0  0.00%  0.00%  0.00%   0 License IPC stat 
      27           0         1          0  0.00%  0.00%  0.00%   0 License IPC serv 
      38         239     10986         21  0.00%  0.00%  0.00%   0 IF-MGR event pro 
      29           0         2          0  0.00%  0.00%  0.00%   0 XML Proxy Client 
      40    25039972   6679622       3748  0.00%  0.06%  0.05%   0 Compute load avg 
      41           0         1          0  0.00%  0.00%  0.00%   0 AggMgr Process   
      42          35      5316          6  0.00%  0.00%  0.00%   0 Transport Port A 
      14          93        30       3100  0.00%  0.00%  0.00%   0 Entity MIB API   
      44           0         1          0  0.00%  0.00%  0.00%   0 SFF8472          
      45      694165 640463898          1  0.00%  0.00%  0.00%   0 DownWhenLooped   
      46           0         1          0  0.00%  0.00%  0.00%   0 HULC PBR Tcam Me 
      32           0         1          0  0.00%  0.00%  0.00%   0 IDB Work         
      48           0         1          0  0.00%  0.00%  0.00%   0 HRPC hdwl reques 
      33        3419     19356        176  0.00%  0.00%  0.00%   0 Logger           
      50           0         1          0  0.00%  0.00%  0.00%   0 HRPC actual powe 
      51           0         3          0  0.00%  0.00%  0.00%   0 HRPC lpip reques 
      52           0         2          0  0.00%  0.00%  0.00%   0 HLPIP Sync Proce 
      36    14379445    553822      25964  0.00%  0.03%  0.00%   0 Per-minute Jobs  
      54           8        18        444  0.00%  0.00%  0.00%   0 HULC multifs pro 
      55           0         1          0  0.00%  0.00%  0.00%   0 HRPC hnetwpol re 
      56           0         1          0  0.00%  0.00%  0.00%   0 Hulc Warm Upgrad 
      57           0         1          0  0.00%  0.00%  0.00%   0 HRPC ipadm reque 
      58           0         2          0  0.00%  0.00%  0.00%   0 SM RBCP Monitor  
      37          33        12       2750  0.00%  0.00%  0.00%   0 IF-MGR control p 
      60      190413   1225261        155  0.00%  0.00%  0.00%   0 EEM ED ND        
      61           0       123          0  0.00%  0.00%  0.00%   0 EEM ED Identity  
      13           0         1          0  0.00%  0.00%  0.00%   0 Policy Manager   
      63      111911   6454104         17  0.00%  0.00%  0.00%   0 Hulc LED Alchemy 
      64           0         1          0  0.00%  0.00%  0.00%   0 HRPC asic-stats  
      39           0         5          0  0.00%  0.00%  0.00%   0 IP Admission HA  
      43     2892937   6960508        415  0.00%  0.01%  0.00%   0 HC Counter Timer 
      67        2311    216633         10  0.00%  0.00%  0.00%   0 HULC Thermal Pro 
      68           0         1          0  0.00%  0.00%  0.00%   0 HRPC hsm request 
      69           0        56          0  0.00%  0.00%  0.00%   0 Stack Mgr        
      70        2510       248      10120  0.00%  0.00%  0.00%   0 Stack Mgr Notifi 
      11           0         1          0  0.00%  0.00%  0.00%   0 AAA_SERVER_DEADT 
      72           0         3          0  0.00%  0.00%  0.00%   0 Adjust Regions   
      73     1542536  32261482         47  0.00%  0.00%  0.00%   0 hrpc -> response 
      74     1646436  69466039         23  0.00%  0.00%  0.00%   0 hrpc -> request  
      47           0         1          0  0.00%  0.00%  0.00%   0 HULC ACL Tcam Me 
      76           0         7          0  0.00%  0.00%  0.00%   0 HRPC hcomp reque 
      77           0        54          0  0.00%  0.00%  0.00%   0 HULC Device Mana 
      78       10447     30047        347  0.00%  0.00%  0.00%   0 HRPC hdm non blo 
      79           0        15          0  0.00%  0.00%  0.00%   0 HRPC hdm blockin 
      80          16         6       2666  0.00%  0.00%  0.00%   0 HRPC cfg_backup  
      10           0         1          0  0.00%  0.00%  0.00%   0 CEF MIB API      
      49           0         1          0  0.00%  0.00%  0.00%   0 HRPC EnergyWise  
      83           0         1          0  0.00%  0.00%  0.00%   0 RTTYC Flush      
       9       67025    599159        111  0.00%  0.00%  0.00%   0 ARP Input        
       8           0         1          0  0.00%  0.00%  0.00%   0 Crash writer     
      86      157119    696649        225  0.00%  0.00%  0.00%   0 HRPC hlfm reques 
      53           0         3          0  0.00%  0.00%  0.00%   0 HRPC Multi-FS Sy 
      88      133411  32261466          4  0.00%  0.00%  0.00%   0 HLFM aging proce 
      89     3110261 949082564          3  0.00%  0.00%  0.00%   0 HLFM address ret 
      90           0         1          0  0.00%  0.00%  0.00%   0 HRPC hrcmd reque 
      91           0         1          0  0.00%  0.00%  0.00%   0 HRPC emac reques 
      92      267066    852763        313  0.00%  0.00%  0.00%   0 HRPC hulc misc r 
       7          27       342         78  0.00%  0.00%  0.00%   0 Net Input        
      94       71237  10801159          6  0.00%  0.00%  0.00%   0 HVLAN main bkgrd 
      59           0         1          0  0.00%  0.00%  0.00%   0 HULC QM Tcam Mem 
      96           0         2          0  0.00%  0.00%  0.00%   0 Vlan shutdown Pr 
      97       34965    206313        169  0.00%  0.00%  0.00%   0 HRPC vlan reques 
      98           0         1          0  0.00%  0.00%  0.00%   0 HULC VLAN REF Ba 
      99       30462    541338         56  0.00%  0.00%  0.00%   0 hulc_xps_process 
     100           8         9        888  0.00%  0.00%  0.00%   0 HRPC hfbm reques 
     101           0         1          0  0.00%  0.00%  0.00%   0 IPC Zone Manager 
     102           0        13          0  0.00%  0.00%  0.00%   0 HRPC ilp request 
     103           0         1          0  0.00%  0.00%  0.00%   0 HULC PM Vector P 
     104           0         1          0  0.00%  0.00%  0.00%   0 HPM Msg Retry Pr 
     105          53      1767         29  0.00%  0.00%  0.00%   0 TACACS+          
      62           0       245          0  0.00%  0.00%  0.00%   0 EEM ED MAT       
     107         425      3060        138  0.00%  0.00%  0.00%   0 HPM Stack Sync P 
     108       20145     75351        267  0.00%  0.00%  0.00%   0 HRPC pm request  
     109           0         5          0  0.00%  0.00%  0.00%   0 HPM if_num mappi 
       6           0         2          0  0.00%  0.00%  0.00%   0 Timers           
     111    46275542 207724060        222  0.00%  0.13%  0.12%   0 HRPC pm-counters 
      12           0         2          0  0.00%  0.00%  0.00%   0 AAA high-capacit 
     113           0       237          0  0.00%  0.00%  0.00%   0 HRPC hcmp reques 
     114          32     15476          2  0.00%  0.00%  0.00%   0 HCEF ADJ Refresh 
     115           0         1          0  0.00%  0.00%  0.00%   0 HRPC hl3mm reque 
     116           0         1          0  0.00%  0.00%  0.00%   0 hl3md_rpfq_thrl_ 
     117      183004  21375807          8  0.00%  0.00%  0.00%   0 hl3mm            
      87    12716516 948952078         13  0.00%  0.03%  0.00%   0 HLFM address lea 
     119           0         1          0  0.00%  0.00%  0.00%   0 HACL Queue Proce 
     120           0         4          0  0.00%  0.00%  0.00%   0 HRPC acl request 
     121           0        43          0  0.00%  0.00%  0.00%   0 HACL Acl Manager 
     122           0         1          0  0.00%  0.00%  0.00%   0 HRPC backup inte 
     123           0         1          0  0.00%  0.00%  0.00%   0 HRPC cdp request 
     124           0         3          0  0.00%  0.00%  0.00%   0 HRPC lldp reques 
     125           0         5          0  0.00%  0.00%  0.00%   0 HULC CISP Proces 
     126        4577     39096        117  0.00%  0.00%  0.00%   0 HRPC dot1x reque 
     127           0         1          0  0.00%  0.00%  0.00%   0 Dot1X Msg Retry  
     128        3417     22012        155  0.00%  0.00%  0.00%   0 HULC DOT1X Proce 
     129           0         1          0  0.00%  0.00%  0.00%   0 HRPC foxhound re 
     130           0         2          0  0.00%  0.00%  0.00%   0 Hulc Foxhound Pr 
       5       29320    892811         32  0.00%  0.00%  0.00%   0 Pool Manager     
     132           0         1          0  0.00%  0.00%  0.00%   0 SMI MSG Retry Pr 
     133           0         4          0  0.00%  0.00%  0.00%   0 HRPC Smart Insta 
     134           0         1          0  0.00%  0.00%  0.00%   0 HRPC sdm request 
     135     3266739 160267739         20  0.00%  0.00%  0.00%   0 Hulc Storm Contr 
     136           0         2          0  0.00%  0.00%  0.00%   0 HSTP Sync Proces 
     137         485      4964         97  0.00%  0.00%  0.00%   0 HRPC stp_cli req 
     138         626      4086        153  0.00%  0.00%  0.00%   0 HRPC stp_state_s 
     139           0         2          0  0.00%  0.00%  0.00%   0 S/W Bridge Proce 
     140           0        95          0  0.00%  0.00%  0.00%   0 HRPC hudld reque 
     141           0         1          0  0.00%  0.00%  0.00%   0 HRPC vqpc reques 
     142           0         1          0  0.00%  0.00%  0.00%   0 BCM cable diags  
     143           0         1          0  0.00%  0.00%  0.00%   0 HRPC iec_load_ba 
     144           9        36        250  0.00%  0.00%  0.00%   0 HRPC l2pt qnq rp 
     145           0         1          0  0.00%  0.00%  0.00%   0 HRPC aim request 
     146           0         2          0  0.00%  0.00%  0.00%   0 MACsec Process   
     147           0         1          0  0.00%  0.00%  0.00%   0 HRPC macsec requ 
     148           8        11        727  0.00%  0.00%  0.00%   0 HRPC hled reques 
       4   313471552  14477254      21652  0.00%  0.93%  0.91%   0 Check heaps      
     150      877489  23656414         37  0.00%  0.00%  0.00%   0 HL3U bkgrd proce 
     151     3374415  38698078         87  0.00%  0.00%  0.00%   0 HRPC hl3u reques 
     152       25962   7418583          3  0.00%  0.00%  0.00%   0 HL3U PBR bkgrd p 
     153        1180    270792          4  0.00%  0.00%  0.00%   0 HL3U PBR n-h res 
     154           0         1          0  0.00%  0.00%  0.00%   0 HRPC dtp request 
     155           0         1          0  0.00%  0.00%  0.00%   0 HRPC show_forwar 
     156          34       127        267  0.00%  0.00%  0.00%   0 HRPC snmp reques 
     157         686       115       5965  0.00%  0.00%  0.00%   0 HULC SNMP Proces 
       3     1984832  49726053         39  0.00%  0.00%  0.00%   0 CEF: IPv4 proces 
       2      122589   6488159         18  0.00%  0.00%  0.00%   0 Load Meter       
     160           0         1          0  0.00%  0.00%  0.00%   0 HRPC span reques 
      81      124588   6454106         19  0.00%  0.00%  0.00%   0 HIPC bkgrd proce 
      82           0         1          0  0.00%  0.00%  0.00%   0 IPC RTTYC Messag 
     163          25        25       1000  0.00%  0.00%  0.00%   0 Hulc Reload Mana 
      84         478      2192        218  0.00%  0.00%  0.00%   0 Hulc Port-Securi 
     165       26905     21811       1233  0.00%  0.00%  0.00%   0 HRPC rfs request 
     166           0         1          0  0.00%  0.00%  0.00%   0 HRFS OIR Proc    
     167       50868  10801155          4  0.00%  0.00%  0.00%   0 Power RPS Proces 
      85        2003     20399         98  0.00%  0.00%  0.00%   0 HRPC hpsecure re 
     169          24        18       1333  0.00%  0.00%  0.00%   0 HL2MCM           
      95      249189    557263        447  0.00%  0.00%  0.00%   0 IP Background    
     171           0         3          0  0.00%  0.00%  0.00%   0 Switch Backup In 
     172       18960    541337         35  0.00%  0.00%  0.00%   0 MMN bkgrd proces 
     173         192      6148         31  0.00%  0.00%  0.00%   0 Auth Manager     
     174           0         2          0  0.00%  0.00%  0.00%   0 Dot1x Mgr Proces 
     175           0         1          0  0.00%  0.00%  0.00%   0 MAB Framework    
     176           0         1          0  0.00%  0.00%  0.00%   0 AUTH POLICY Fram 
     177           0         1          0  0.00%  0.00%  0.00%   0 CMD HANDLER      
     178        1167      6162        189  0.00%  0.00%  0.00%   0 802.1x switch    
     179       25021    541339         46  0.00%  0.00%  0.00%   0 802.1x MDA Aging 
     180           0         1          0  0.00%  0.00%  0.00%   0 802.1x Webauth F 
     181      366325   2214225        165  0.00%  0.00%  0.00%   0 DTP Protocol     
     182           0         1          0  0.00%  0.00%  0.00%   0 EAP Framework    
     183           0         1          0  0.00%  0.00%  0.00%   0 EAP Test         
     184           0         1          0  0.00%  0.00%  0.00%   0 HRPC dai request 
     185           0         1          0  0.00%  0.00%  0.00%   0 HULC DAI Process 
     186           0         1          0  0.00%  0.00%  0.00%   0 HRPC ip device t 
     187           0         1          0  0.00%  0.00%  0.00%   0 HRPC ip source g 
     188           0         1          0  0.00%  0.00%  0.00%   0 HULC IP Source g 
       1      101471   1614058         62  0.00%  0.00%  0.00%   0 Chunk Manager    
     190        2331   1083241          2  0.00%  0.00%  0.00%   0 Port-Security    
     191           0         2          0  0.00%  0.00%  0.00%   0 IP Host Track Pr 
     192           0         1          0  0.00%  0.00%  0.00%   0 Link State Group 
     193       97612   3227055         30  0.00%  0.00%  0.00%   0 Ethchnl          
     194        1551      1194       1298  0.00%  0.00%  0.00%   0 VMATM Callback   
     195           0         2          0  0.00%  0.00%  0.00%   0 CDP Forward Proc 
     196      595986 321101059          1  0.00%  0.00%  0.00%   0 MDFS MFIB Proces 
     197           9         2       4500  0.00%  0.00%  0.00%   0 MKA              
     198         235      2964         79  0.00%  0.00%  0.00%   0 AAA Server       
     199         158      1645         96  0.00%  0.00%  0.00%   0 AAA ACCT Proc    
     200           0         1          0  0.00%  0.00%  0.00%   0 ACCT Periodic Pr 
     201           0         1          0  0.00%  0.00%  0.00%   0 AAA System Acct  
     202           0         1          0  0.00%  0.00%  0.00%   0 Auth-proxy AAA B 
     203         394    108323          3  0.00%  0.00%  0.00%   0 IP Admin SM Proc 
     204    32404554  40417073        801  0.00%  0.09%  0.10%   0 CDP Protocol     
     205          50        70        714  0.00%  0.00%  0.00%   0 Collection proce 
     206         851      4694        181  0.00%  0.00%  0.00%   0 HRPC hl2mcm igmp 
     207           0         2          0  0.00%  0.00%  0.00%   0 AAA Dictionary R 
     208        1247    270811          4  0.00%  0.00%  0.00%   0 DHCP Snooping    
     209           0         2          0  0.00%  0.00%  0.00%   0 EAPoUDP Process  
     210           0         3          0  0.00%  0.00%  0.00%   0 CEF switching ba 
     112           0         1          0  0.00%  0.00%  0.00%   0 hpm vp events ca 
     212           0         1          0  0.00%  0.00%  0.00%   0 ICMP event handl 
     213      127495  62409421          2  0.00%  0.00%  0.00%   0 IP ARP Track     
     214           0        17          0  0.00%  0.00%  0.00%   0 ADJ resolve proc 
     215           0         3          0  0.00%  0.00%  0.00%   0 ADJ NSF process  
     216           0         1          0  0.00%  0.00%  0.00%   0 IPv6 ping proces 
     217           0         1          0  0.00%  0.00%  0.00%   0 EM Action CNS    
     218           0         5          0  0.00%  0.00%  0.00%   0 SMI Director DB  
     219           0         1          0  0.00%  0.00%  0.00%   0 SMI CDP Update H 
     220           0         1          0  0.00%  0.00%  0.00%   0 SMI Backup Proce 
     221           0         2          0  0.00%  0.00%  0.00%   0 SMI IBC server p 
     222           0         1          0  0.00%  0.00%  0.00%   0 SMI IBC client p 
     223           0         2          0  0.00%  0.00%  0.00%   0 SMI IBC Download 
     131           0         7          0  0.00%  0.00%  0.00%   0 HRPC system mtu  
     225         667    541593          1  0.00%  0.00%  0.00%   0 Spanning Tree St 
     226           0         2          0  0.00%  0.00%  0.00%   0 KRB5 AAA         
     227          18       110        163  0.00%  0.00%  0.00%   0 SXP CORE         
     228      162382    651849        249  0.00%  0.00%  0.00%   0 CEF background p 
     229           0         1          0  0.00%  0.00%  0.00%   0 IP IRDP          
     230          34         2      17000  0.00%  0.00%  0.00%   0 CEF RF HULC Conv 
     231        2517     17418        144  0.00%  0.00%  0.00%   0 XDR mcast        
     232           8        14        571  0.00%  0.00%  0.00%   0 OBFL Cfg Dispatc 
     233          34         9       3777  0.00%  0.00%  0.00%   0 EEM Callback Thr 
     234         134       131       1022  0.00%  0.00%  0.00%   1 Virtual Exec     
     235      128857   3228265         39  0.00%  0.00%  0.00%   0 IP ACL XDR LC Ba 
     236           0         4          0  0.00%  0.00%  0.00%   0 MDFS LC Process  
     237         334     16052         20  0.00%  0.00%  0.00%   0 TCP Timer        
     238        1133      1351        838  0.00%  0.00%  0.00%   0 TCP Protocols    
     239      100612  33282916          3  0.00%  0.00%  0.00%   0 Socket Timers    
     240         466    108429          4  0.00%  0.00%  0.00%   0 HTTP CORE        
     241         917    406201          2  0.00%  0.00%  0.00%   0 Cluster L2       
     242      173885   3227054         53  0.00%  0.00%  0.00%   0 Cluster RARP     
     243       21824   4063098          5  0.00%  0.00%  0.00%   0 Cluster Base     
     244           0         2          0  0.00%  0.00%  0.00%   0 CEF LC IPC Backg 
     245      168863  39175440          4  0.00%  0.00%  0.00%   0 CEF LC Stats     
     246           0         1          0  0.00%  0.00%  0.00%   0 RARP Input       
     247           0         1          0  0.00%  0.00%  0.00%   0 Critical Auth    
     248           0         2          0  0.00%  0.00%  0.00%   0 Dot1x Supplicant 
     249           0         2          0  0.00%  0.00%  0.00%   0 Dot1x Supplicant 
     250           0         2          0  0.00%  0.00%  0.00%   0 Dot1x Supplicant 
     251           0         4          0  0.00%  0.00%  0.00%   0 HRPC dhcp snoopi 
     252           0         7          0  0.00%  0.00%  0.00%   0 HULC DHCP Snoopi 
     253           0         1          0  0.00%  0.00%  0.00%   0 HRPC stack power 
     254         728      1440        505  0.00%  0.00%  0.00%   0 IGMPSN L2MCM     
     255     6769023  58190305        116  0.00%  0.03%  0.00%   0 IGMPSN MRD       
     256    53618406 248052112        216  0.00%  0.12%  0.16%   0 IGMPSN           
     257           0         1          0  0.00%  0.00%  0.00%   0 IGMPQR           
     258           0         3          0  0.00%  0.00%  0.00%   0 L2TRACE SERVER   
     259         669      1435        466  0.00%  0.00%  0.00%   0 MLDSN L2MCM      
     260           0         1          0  0.00%  0.00%  0.00%   0 MRD              
     261           0         1          0  0.00%  0.00%  0.00%   0 MLD_SNOOP        
     161           0         1          0  0.00%  0.00%  0.00%   0 HRPC obfl reques 
     263           0         1          0  0.00%  0.00%  0.00%   0 HRPC x_setup req 
     264       43448    541183         80  0.00%  0.00%  0.00%   0 IP RIB Update    
     265          16       249         64  0.00%  0.00%  0.00%   0 HRPC hl2mcm mlds 
     266           0        20          0  0.00%  0.00%  0.00%   0 SNMP Timers      
     267           0         2          0  0.00%  0.00%  0.00%   0 EPM MAIN PROCESS 
     268           0        33          0  0.00%  0.00%  0.00%   0 LOCAL AAA        
     269          83       545        152  0.00%  0.00%  0.00%   0 ENABLE AAA       
     270           0         2          0  0.00%  0.00%  0.00%   0 LINE AAA         
     271       18756     11105       1688  0.00%  0.00%  0.00%   0 SNMP Traps       
     272       10522     22584        465  0.00%  0.00%  0.00%   0 TPLUS            
     273           8        12        666  0.00%  0.00%  0.00%   0 crypto engine pr 
     274          25         4       6250  0.00%  0.00%  0.00%   0 Crypto CA        
     275           0         1          0  0.00%  0.00%  0.00%   0 Crypto PKI-CRL   
     276           0         1          0  0.00%  0.00%  0.00%   0 Crypto SSL       
     277           0         1          0  0.00%  0.00%  0.00%   0 encrypt proc     
     278           0        41          0  0.00%  0.00%  0.00%   0 VTP Trap Process 
     279           0         2          0  0.00%  0.00%  0.00%   0 VTPMIB EDIT BUFF 
     280           0         2          0  0.00%  0.00%  0.00%   0 DHCP Security He 
     281           0         1          0  0.00%  0.00%  0.00%   0 HCD Process      
     282           0         5          0  0.00%  0.00%  0.00%   0 HRPC cable diagn 
     283           0         2          0  0.00%  0.00%  0.00%   0 DiagCard2/-1     
     284     2400870 556908702          4  0.00%  0.00%  0.00%   0 PM Callback      
     285          25        16       1562  0.00%  0.00%  0.00%   0 VLAN Manager     
     286           0         1          0  0.00%  0.00%  0.00%   0 DiagCard4/-1     
     287        6264   2600673          2  0.00%  0.00%  0.00%   0 dhcp snooping sw 
     288           0         1          0  0.00%  0.00%  0.00%   0 Licensing MIB pr 
     289           0         2          0  0.00%  0.00%  0.00%   0 EEM ED Routing   
     290         316     20942         15  0.00%  0.00%  0.00%   0 EEM ED Syslog    
     291           0         1          0  0.00%  0.00%  0.00%   0 image mgr        
     292         200      1583        126  0.00%  0.00%  0.00%   0 AAA SEND STOP EV 
     293       36339     10046       3617  0.00%  0.00%  0.00%   0 Syslog Traps     
     294        1494     11047        135  0.00%  0.00%  0.00%   0 Syslog           
     295        4313       825       5227  0.00%  0.00%  0.00%   0 SpanTree Helper  
     296         137       449        305  0.00%  0.00%  0.00%   0 EEM Server       
     297           0         2          0  0.00%  0.00%  0.00%   0 EEM Policy Direc 
     298        3474     18040        192  0.00%  0.00%  0.00%   0 SSH Event handle 
     299           0         2          0  0.00%  0.00%  0.00%   0 EEM ED CLI       
     300           0         3          0  0.00%  0.00%  0.00%   0 EEM ED Counter   
     301           0         3          0  0.00%  0.00%  0.00%   0 EEM ED Interface 
     302           0         3          0  0.00%  0.00%  0.00%   0 EEM ED IOSWD     
     303           0         3          0  0.00%  0.00%  0.00%   0 EEM ED None      
     304           0         3          0  0.00%  0.00%  0.00%   0 EEM ED OIR       
     305           0         1          0  0.00%  0.00%  0.00%   0 DiagCard3/-1     
     306         360      2582        139  0.00%  0.00%  0.00%   0 EEM ED SNMP      
     307           0         3          0  0.00%  0.00%  0.00%   0 EEM ED SNMP Obje 
     308           0         3          0  0.00%  0.00%  0.00%   0 EEM ED Ipsla     
     309           0         2          0  0.00%  0.00%  0.00%   0 EEM ED SNMP Noti 
     310        6982    939565          7  0.00%  0.00%  0.00%   0 EEM ED Timer     
     311           0         2          0  0.00%  0.00%  0.00%   0 STP FAST TRANSIT 
     312          82      1602         51  0.00%  0.00%  0.00%   0 CSRT RAPID TRANS 
     313           0         3          0  0.00%  0.00%  0.00%   0 RBM CORE         
     314    15457835    874002      17686  0.00%  0.00%  0.00%   0 crypto sw pk pro 
     316         200       638        313  0.00%  0.00%  0.00%   0 SpanTree Flush   
     317           0         1          0  0.00%  0.00%  0.00%   0 DiagCard1/-1     
     318           0         1          0  0.00%  0.00%  0.00%   0 Connection Mgr   
     319         889    279819          3  0.00%  0.00%  0.00%   0 DHCPD Timer      
     320       15192    541340         28  0.00%  0.00%  0.00%   0 DHCPD Database   
     321           0         1          0  0.00%  0.00%  0.00%   0 LICENSE AGENT    
     322       14642   2165210          6  0.00%  0.00%  0.00%   0 MDFS LC Download 
     162          17        63        269  0.00%  0.00%  0.00%   0 HRPC system post 
     324     2368689   7408400        319  0.00%  0.03%  0.00%   0 PDU DISPATCHER   
     164           8        80        100  0.00%  0.00%  0.00%   0 HRPC hrcli-event 
     326           0         2          0  0.00%  0.00%  0.00%   0 IP SNMPV6        
     327           0         1          0  0.00%  0.00%  0.00%   0 SNMP ConfCopyPro 
     328      142133  32762050          4  0.00%  0.00%  0.00%   0 NTP              
     170     2324992  32261463         72  0.00%  0.02%  0.00%   0 PI MATM Aging Pr 
     330         149       135       1103  0.00%  0.00%  0.00%   0 hulc cfg mgr mas 
     331      260212      1379     188696  0.00%  0.00%  0.00%   0 hulc running con 
     332           0         1          0  0.00%  0.00%  0.00%   0 RTTYS Process    
     334       19497    541160         36  0.00%  0.00%  0.00%   0 802.1x MDA Aging 
     337        4718    180481         26  0.00%  0.00%  0.00%   0 HCMP sync proces 
     338           0         2          0  0.00%  0.00%  0.00%   0 hulc_tb_process  
     339     2416579   9844669        245  0.00%  0.00%  0.00%   0 IPC LC Message H 
     340           0         4          0  0.00%  0.00%  0.00%   0 XDR RP Ping Back 
     341        4576    270968         16  0.00%  0.00%  0.00%   0 XDR RP backgroun 
     342           0         1          0  0.00%  0.00%  0.00%   0 XDR RP Test Back 
     343           0         7          0  0.00%  0.00%  0.00%   0 CEF RP IPC Backg
    14:58:45.422 UTC Sun Jul 13 2014
    Process 235:  IP ACL XDR LC Background
      Stack segment 0x5A69684 - 0x5A6ADF4
      FP: 0x5A6ADC0, RA: 0x294A248
      FP: 0x5A6ADE0, RA: 0x1013B38
      FP: 0x5A6ADE8, RA: 0x1BCF948
      FP: 0x0, RA: 0x1BC63C0
    CPU utilization for five seconds: 81%/0%; one minute: 21%; five minutes: 13%
     PID Runtime(ms)   Invoked      uSecs   5Sec   1Min   5Min TTY Process 
     325    71954590  29063029       2475 56.31%  8.62%  1.87%   0 SNMP ENGINE      
     149    70365005 922590436         76  5.75%  0.91%  0.32%   0 Hulc LED Process 
      93   209282090   3228477      64824  3.35%  0.71%  0.61%   0 HULC Tcam Memory 
      71   1093966061691340293         64  1.75%  0.60%  0.39%   0 Fifo Error Detec 
       4   313611314  14483676      21652  1.43%  1.04%  0.93%   0 Check heaps      
     204    32420529  40437641        801  1.27%  0.14%  0.07%   0 CDP Protocol     
     110   292104316  66908096       4365  0.79%  0.89%  0.85%   0 hpm counter proc 
      36    14385841    554067      25964  0.63%  0.07%  0.01%   0 Per-minute Jobs  
      75    30678290 134074322        228  0.63%  0.36%  0.13%   0 hrpc <- response 
     224    42221244 212687841        198  0.47%  0.13%  0.11%   0 Spanning Tree    
     111    46293754 207815887        222  0.47%  0.14%  0.11%   0 HRPC pm-counters 
     158   124818988   6457016      19330  0.47%  0.36%  0.36%   0 HQM Stack Proces 
      65    52094188 720582708         72  0.31%  0.15%  0.16%   0 RedEarth Tx Mana 
      35     3272331  32343859        101  0.31%  0.02%  0.00%   0 Per-Second Jobs  
     284     2401894 557156839          4  0.15%  0.01%  0.00%   0 PM Callback      
      87    12722467 949371888         13  0.15%  0.03%  0.00%   0 HLFM address lea 
     159   106475866  51623681       2062  0.15%  0.42%  0.43%   0 HRPC qos request 
     329     1716057 327670421          5  0.15%  0.03%  0.00%   0 MDFS RP process  
     323     4050598  15007641        269  0.15%  0.28%  0.06%   0 IP SNMP          
     196      596257 321243172          1  0.15%  0.01%  0.00%   0 MDFS MFIB Proces 
      40    25051571   6682478       3748  0.15%  0.06%  0.06%   0 Compute load avg 
      31    21272435   3941092       5397  0.15%  0.05%  0.05%   0 Net Background   
     262     3365891  67999629         49  0.15%  0.01%  0.00%   0 DHCPD Receive    
      23          75         4      18750  0.00%  0.00%  0.00%   0 PrstVbl          
      22           0         1          0  0.00%  0.00%  0.00%   0 IPC Session Serv 
      21      476260   8660138         54  0.00%  0.00%  0.00%   0 IPC Seat Manager 
      20       91963  32275874          2  0.00%  0.00%  0.00%   0 IPC Deferred Por 
      24      117939  32275869          3  0.00%  0.01%  0.00%   0 Dynamic ARP Insp 
      25           0         1          0  0.00%  0.00%  0.00%   0 ARP Snoop        
      30           0         1          0  0.00%  0.00%  0.00%   0 Critical Bkgnd   
      26           0         1          0  0.00%  0.00%  0.00%   0 License IPC stat 
      32           0         1          0  0.00%  0.00%  0.00%   0 IDB Work         
      27           0         1          0  0.00%  0.00%  0.00%   0 License IPC serv 
      19           0        40          0  0.00%  0.00%  0.00%   0 IPC Managed Time 
      15           0         1          0  0.00%  0.00%  0.00%   0 IFS Agent Manage 
      14          93        30       3100  0.00%  0.00%  0.00%   0 Entity MIB API   
      28       91067  32275743          2  0.00%  0.00%  0.00%   0 GraphIt          
      29           0         2          0  0.00%  0.00%  0.00%   0 XML Proxy Client 
      39           0         5          0  0.00%  0.00%  0.00%   0 IP Admission HA  
      18      110822  32275871          3  0.00%  0.00%  0.00%   0 IPC Periodic Tim 
      41           0         1          0  0.00%  0.00%  0.00%   0 AggMgr Process   
      42          35      5316          6  0.00%  0.00%  0.00%   0 Transport Port A 
      43     2894015   6963595        415  0.00%  0.00%  0.00%   0 HC Counter Timer 
      44           0         1          0  0.00%  0.00%  0.00%   0 SFF8472          
      45      694425 640747340          1  0.00%  0.00%  0.00%   0 DownWhenLooped   
      46           0         1          0  0.00%  0.00%  0.00%   0 HULC PBR Tcam Me 
      47           0         1          0  0.00%  0.00%  0.00%   0 HULC ACL Tcam Me 
      33        3462     19721        175  0.00%  0.00%  0.00%   0 Logger           
      34      103095  32275735          3  0.00%  0.00%  0.00%   0 TTY Background   
      50           0         1          0  0.00%  0.00%  0.00%   0 HRPC actual powe 
      37          33        12       2750  0.00%  0.00%  0.00%   0 IF-MGR control p 
      52           0         2          0  0.00%  0.00%  0.00%   0 HLPIP Sync Proce 
      53           0         3          0  0.00%  0.00%  0.00%   0 HRPC Multi-FS Sy 
      54           8        18        444  0.00%  0.00%  0.00%   0 HULC multifs pro 
      55           0         1          0  0.00%  0.00%  0.00%   0 HRPC hnetwpol re 
      56           0         1          0  0.00%  0.00%  0.00%   0 Hulc Warm Upgrad 
      38         239     11186         21  0.00%  0.00%  0.00%   0 IF-MGR event pro 
      58           0         2          0  0.00%  0.00%  0.00%   0 SM RBCP Monitor  
      48           0         1          0  0.00%  0.00%  0.00%   0 HRPC hdwl reques 
      60      190531   1226172        155  0.00%  0.00%  0.00%   0 EEM ED ND        
      61           0       123          0  0.00%  0.00%  0.00%   0 EEM ED Identity  
      62           0       245          0  0.00%  0.00%  0.00%   0 EEM ED MAT       
      63      111970   6456960         17  0.00%  0.00%  0.00%   0 Hulc LED Alchemy 
      64           0         1          0  0.00%  0.00%  0.00%   0 HRPC asic-stats  
      13           0         1          0  0.00%  0.00%  0.00%   0 Policy Manager   
      66     53976551261925972          4  0.00%  0.00%  0.00%   0 RedEarth Rx Mana 
      67        2320    216729         10  0.00%  0.00%  0.00%   0 HULC Thermal Pro 
      68           0         1          0  0.00%  0.00%  0.00%   0 HRPC hsm request 
      69           0        56          0  0.00%  0.00%  0.00%   0 Stack Mgr        
      70        2510       248      10120  0.00%  0.00%  0.00%   0 Stack Mgr Notifi 
      12           0         2          0  0.00%  0.00%  0.00%   0 AAA high-capacit 
      72           0         3          0  0.00%  0.00%  0.00%   0 Adjust Regions   
      73     1543201  32275759         47  0.00%  0.00%  0.00%   0 hrpc -> response 
      74     1647064  69492547         23  0.00%  0.00%  0.00%   0 hrpc -> request  
      11           0         1          0  0.00%  0.00%  0.00%   0 AAA_SERVER_DEADT 
      76           0         7          0  0.00%  0.00%  0.00%   0 HRPC hcomp reque 
      77           0        54          0  0.00%  0.00%  0.00%   0 HULC Device Mana 
      78       10455     30071        347  0.00%  0.00%  0.00%   0 HRPC hdm non blo 
      17           0        21          0  0.00%  0.00%  0.00%   0 ADJ background   
      10           0         1          0  0.00%  0.00%  0.00%   0 CEF MIB API      
      49           0         1          0  0.00%  0.00%  0.00%   0 HRPC EnergyWise  
      51           0         3          0  0.00%  0.00%  0.00%   0 HRPC lpip reques 
      83           0         1          0  0.00%  0.00%  0.00%   0 RTTYC Flush      
      84         478      2192        218  0.00%  0.00%  0.00%   0 Hulc Port-Securi 
       9       67328    599587        112  0.00%  0.00%  0.00%   0 ARP Input        
      86      157272    697369        225  0.00%  0.00%  0.00%   0 HRPC hlfm reques 
      57           0         1          0  0.00%  0.00%  0.00%   0 HRPC ipadm reque 
      88      133444  32275743          4  0.00%  0.00%  0.00%   0 HLFM aging proce 
      89     3111529 949502426          3  0.00%  0.01%  0.00%   0 HLFM address ret 
      90           0         1          0  0.00%  0.00%  0.00%   0 HRPC hrcmd reque 
      91           0         1          0  0.00%  0.00%  0.00%   0 HRPC emac reques 
      92      267066    852795        313  0.00%  0.00%  0.00%   0 HRPC hulc misc r 
       8           0         1          0  0.00%  0.00%  0.00%   0 Crash writer     
      94       71255  10805941          6  0.00%  0.00%  0.00%   0 HVLAN main bkgrd 
      95      249482    557783        447  0.00%  0.00%  0.00%   0 IP Background    
      96           0         2          0  0.00%  0.00%  0.00%   0 Vlan shutdown Pr 
      97       34965    206313        169  0.00%  0.00%  0.00%   0 HRPC vlan reques 
      59           0         1          0  0.00%  0.00%  0.00%   0 HULC QM Tcam Mem 
      99       30479    541578         56  0.00%  0.00%  0.00%   0 hulc_xps_process 
     100           8         9        888  0.00%  0.00%  0.00%   0 HRPC hfbm reques 
     101           0         1          0  0.00%  0.00%  0.00%   0 IPC Zone Manager 
      79           0        15          0  0.00%  0.00%  0.00%   0 HRPC hdm blockin 
     103           0         1          0  0.00%  0.00%  0.00%   0 HULC PM Vector P 
     104           0         1          0  0.00%  0.00%  0.00%   0 HPM Msg Retry Pr 
     105          53      1767         29  0.00%  0.00%  0.00%   0 TACACS+          
     106    13225682 370827741         35  0.00%  0.03%  0.01%   0 hpm main process 
     107         425      3060        138  0.00%  0.00%  0.00%   0 HPM Stack Sync P 
     108       20474     76714        266  0.00%  0.00%  0.00%   0 HRPC pm request  
     109           0         5          0  0.00%  0.00%  0.00%   0 HPM if_num mappi 
       7          27       342         78  0.00%  0.00%  0.00%   0 Net Input        
       6           0         2          0  0.00%  0.00%  0.00%   0 Timers           
       5       29320    893213         32  0.00%  0.00%  0.00%   0 Pool Manager     
     113           0       237          0  0.00%  0.00%  0.00%   0 HRPC hcmp reques 
     114          32     15483          2  0.00%  0.00%  0.00%   0 HCEF ADJ Refresh 
     115           0         1          0  0.00%  0.00%  0.00%   0 HRPC hl3mm reque 
     116           0         1          0  0.00%  0.00%  0.00%   0 hl3md_rpfq_thrl_ 
     117      183095  21385260          8  0.00%  0.00%  0.00%   0 hl3mm            
     118       63906  16142391          3  0.00%  0.00%  0.00%   0 hl3mm_rp         
     119           0         1          0  0.00%  0.00%  0.00%   0 HACL Queue Proce 
     120           0         4          0  0.00%  0.00%  0.00%   0 HRPC acl request 
     121           0        43          0  0.00%  0.00%  0.00%   0 HACL Acl Manager 
     122           0         1          0  0.00%  0.00%  0.00%   0 HRPC backup inte 
     123           0         1          0  0.00%  0.00%  0.00%   0 HRPC cdp request 
     124           0         3          0  0.00%  0.00%  0.00%   0 HRPC lldp reques 
     125           0         5          0  0.00%  0.00%  0.00%   0 HULC CISP Proces 
     126        4676     39864        117  0.00%  0.00%  0.00%   0 HRPC dot1x reque 
     127           0         1          0  0.00%  0.00%  0.00%   0 Dot1X Msg Retry  
     128        3475     22413        155  0.00%  0.00%  0.00%   0 HULC DOT1X Proce 
     129           0         1          0  0.00%  0.00%  0.00%   0 HRPC foxhound re 
     130           0         2          0  0.00%  0.00%  0.00%   0 Hulc Foxhound Pr 
       3     1985674  49748077         39  0.00%  0.00%  0.00%   0 CEF: IPv4 proces 
     132           0         1          0  0.00%  0.00%  0.00%   0 SMI MSG Retry Pr 
     133           0         4          0  0.00%  0.00%  0.00%   0 HRPC Smart Insta 
     134           0         1          0  0.00%  0.00%  0.00%   0 HRPC sdm request 
     135     3268222 160338663         20  0.00%  0.01%  0.00%   0 Hulc Storm Contr 
     136           0         2          0  0.00%  0.00%  0.00%   0 HSTP Sync Proces 
     137         485      4964         97  0.00%  0.00%  0.00%   0 HRPC stp_cli req 
     138         626      4086        153  0.00%  0.00%  0.00%   0 HRPC stp_state_s 
     139           0         2          0  0.00%  0.00%  0.00%   0 S/W Bridge Proce 
     140           0        95          0  0.00%  0.00%  0.00%   0 HRPC hudld reque 
     141           0         1          0  0.00%  0.00%  0.00%   0 HRPC vqpc reques 
      16       16573    541581         30  0.00%  0.00%  0.00%   0 IPC Dynamic Cach 
     143           0         1          0  0.00%  0.00%  0.00%   0 HRPC iec_load_ba 
     144           9        36        250  0.00%  0.00%  0.00%   0 HRPC l2pt qnq rp 
     145           0         1          0  0.00%  0.00%  0.00%   0 HRPC aim request 
     146           0         2          0  0.00%  0.00%  0.00%   0 MACsec Process   
     147           0         1          0  0.00%  0.00%  0.00%   0 HRPC macsec requ 
     148           8        11        727  0.00%  0.00%  0.00%   0 HRPC hled reques 
       2      122646   6491031         18  0.00%  0.00%  0.00%   0 Load Meter       
     150      877968  23666881         37  0.00%  0.00%  0.00%   0 HL3U bkgrd proce 
     151     3375543  38715193         87  0.00%  0.00%  0.00%   0 HRPC hl3u reques 
     152       25970   7421871          3  0.00%  0.00%  0.00%   0 HL3U PBR bkgrd p 
     153        1188    270912          4  0.00%  0.00%  0.00%   0 HL3U PBR n-h res 
     154           0         1          0  0.00%  0.00%  0.00%   0 HRPC dtp request 
     155           0         1          0  0.00%  0.00%  0.00%   0 HRPC show_forwar 
     156          34       127        267  0.00%  0.00%  0.00%   0 HRPC snmp reques 
     157         686       115       5965  0.00%  0.00%  0.00%   0 HULC SNMP Proces 
      80          16         6       2666  0.00%  0.00%  0.00%   0 HRPC cfg_backup  
       1      101670   1616162         62  0.00%  0.00%  0.00%   0 Chunk Manager    
     160           0         1          0  0.00%  0.00%  0.00%   0 HRPC span reques 
      81      124624   6456962         19  0.00%  0.00%  0.00%   0 HIPC bkgrd proce 
      82           0         1          0  0.00%  0.00%  0.00%   0 IPC RTTYC Messag 
     163          25        25       1000  0.00%  0.00%  0.00%   0 Hulc Reload Mana 
      85        2030     20798         97  0.00%  0.00%  0.00%   0 HRPC hpsecure re 
     165       26905     21811       1233  0.00%  0.00%  0.00%   0 HRPC rfs request 
     166           0         1          0  0.00%  0.00%  0.00%   0 HRFS OIR Proc    
     167       50885  10805937          4  0.00%  0.00%  0.00%   0 Power RPS Proces 
     168    81813846 292193365        279  0.00%  0.21%  0.20%   0 HL2MCM           
     169          24        18       1333  0.00%  0.00%  0.00%   0 HL2MCM           
      98           0         1          0  0.00%  0.00%  0.00%   0 HULC VLAN REF Ba 
     171           0         3          0  0.00%  0.00%  0.00%   0 Switch Backup In 
     172       18960    541577         35  0.00%  0.00%  0.00%   0 MMN bkgrd proces 
     173         192      6248         30  0.00%  0.00%  0.00%   0 Auth Manager     
     174           0         2          0  0.00%  0.00%  0.00%   0 Dot1x Mgr Proces 
     175           0         1          0  0.00%  0.00%  0.00%   0 MAB Framework    
     176           0         1          0  0.00%  0.00%  0.00%   0 AUTH POLICY Fram 
     177           0         1          0  0.00%  0.00%  0.00%   0 CMD HANDLER      
     178        1175      6262        187  0.00%  0.00%  0.00%   0 802.1x switch    
     179       25021    541579         46  0.00%  0.00%  0.00%   0 802.1x MDA Aging 
     180           0         1          0  0.00%  0.00%  0.00%   0 802.1x Webauth F 
     181      366481   2215192        165  0.00%  0.00%  0.00%   0 DTP Protocol     
     182           0         1          0  0.00%  0.00%  0.00%   0 EAP Framework    
     183           0         1          0  0.00%  0.00%  0.00%   0 EAP Test         
     184           0         1          0  0.00%  0.00%  0.00%   0 HRPC dai request 
     185           0         1          0  0.00%  0.00%  0.00%   0 HULC DAI Process 
     186           0         1          0  0.00%  0.00%  0.00%   0 HRPC ip device t 
     187           0         1          0  0.00%  0.00%  0.00%   0 HRPC ip source g 
     188           0         1          0  0.00%  0.00%  0.00%   0 HULC IP Source g 
     189      346456  32511580         10  0.00%  0.00%  0.00%   0 UDLD             
     190        2339   1083720          2  0.00%  0.00%  0.00%   0 Port-Security    
     191           0         2          0  0.00%  0.00%  0.00%   0 IP Host Track Pr 
     192           0         1          0  0.00%  0.00%  0.00%   0 Link State Group 
     193       97705   3228483         30  0.00%  0.00%  0.00%   0 Ethchnl          
     194        1551      1194       1298  0.00%  0.00%  0.00%   0 VMATM Callback   
     195           0         2          0  0.00%  0.00%  0.00%   0 CDP Forward Proc 
     102           0        13          0  0.00%  0.00%  0.00%   0 HRPC ilp request 
     197           9         2       4500  0.00%  0.00%  0.00%   0 MKA              
     198         235      2964         79  0.00%  0.00%  0.00%   0 AAA Server       
     199         158      1645         96  0.00%  0.00%  0.00%   0 AAA ACCT Proc    
     200           0         1          0  0.00%  0.00%  0.00%   0 ACCT Periodic Pr 
     201           0         1          0  0.00%  0.00%  0.00%   0 AAA System Acct  
     202           0         1          0  0.00%  0.00%  0.00%   0 Auth-proxy AAA B 
     203         394    108371          3  0.00%  0.00%  0.00%   0 IP Admin SM Proc 
     112           0         1          0  0.00%  0.00%  0.00%   0 hpm vp events ca 
     205          50        70        714  0.00%  0.00%  0.00%   0 Collection proce 
     206         851      4694        181  0.00%  0.00%  0.00%   0 HRPC hl2mcm igmp 
     207           0         2          0  0.00%  0.00%  0.00%   0 AAA Dictionary R 
     208        1247    270930          4  0.00%  0.00%  0.00%   0 DHCP Snooping    
     209           0         2          0  0.00%  0.00%  0.00%   0 EAPoUDP Process  
     210           0         3          0  0.00%  0.00%  0.00%   0 CEF switching ba 
     211     9170668  82916814        110  0.00%  0.06%  0.01%   0 IP Input         
     212           0         1          0  0.00%  0.00%  0.00%   0 ICMP event handl 
     213      127528  62437038          2  0.00%  0.00%  0.00%   0 IP ARP Track     
     214           0        17          0  0.00%  0.00%  0.00%   0 ADJ resolve proc 
     215           0         3          0  0.00%  0.00%  0.00%   0 ADJ NSF process  
     216           0         1          0  0.00%  0.00%  0.00%   0 IPv6 ping proces 
     217           0         1          0  0.00%  0.00%  0.00%   0 EM Action CNS    
     218           0         5          0  0.00%  0.00%  0.00%   0 SMI Director DB  
     219           0         1          0  0.00%  0.00%  0.00%   0 SMI CDP Update H 
     220           0         1          0  0.00%  0.00%  0.00%   0 SMI Backup Proce 
     221           0         2          0  0.00%  0.00%  0.00%   0 SMI IBC server p 
     222           0         1          0  0.00%  0.00%  0.00%   0 SMI IBC client p 
     223           0         2          0  0.00%  0.00%  0.00%   0 SMI IBC Download 
     131           0         7          0  0.00%  0.00%  0.00%   0 HRPC system mtu  
     225         667    541833          1  0.00%  0.00%  0.00%   0 Spanning Tree St 
     226           0         2          0  0.00%  0.00%  0.00%   0 KRB5 AAA         
     227          18       110        163  0.00%  0.00%  0.00%   0 SXP CORE         
     228      162442    652137        249  0.00%  0.00%  0.00%   0 CEF background p 
     229           0         1          0  0.00%  0.00%  0.00%   0 IP IRDP          
     230          34         2      17000  0.00%  0.00%  0.00%   0 CEF RF HULC Conv 
     231        2552     17690        144  0.00%  0.00%  0.00%   0 XDR mcast        
     232           8        14        571  0.00%  0.00%  0.00%   0 OBFL Cfg Dispatc 
     233          36         9       4000  0.00%  0.00%  0.00%   0 EEM Callback Thr 
     234         158       131       1206  0.00%  0.00%  0.00%   1 Virtual Exec     
     235      128933   3229695         39  0.00%  0.00%  0.00%   0 IP ACL XDR LC Ba 
     236           0         4          0  0.00%  0.00%  0.00%   0 MDFS LC Process  
     237         334     16052         20  0.00%  0.00%  0.00%   0 TCP Timer        
     238        1133      1351        838  0.00%  0.00%  0.00%   0 TCP Protocols    
     239      100661  33297644          3  0.00%  0.00%  0.00%   0 Socket Timers    
     240         466    108477          4  0.00%  0.00%  0.00%   0 HTTP CORE        
     241         917    406381          2  0.00%  0.00%  0.00%   0 Cluster L2       
     242      173986   3228482         53  0.00%  0.00%  0.00%   0 Cluster RARP     
     243       21832   4064895          5  0.00%  0.00%  0.00%   0 Cluster Base     
     244           0         2          0  0.00%  0.00%  0.00%   0 CEF LC IPC Backg 
     245      168929  39192775          4  0.00%  0.00%  0.00%   0 CEF LC Stats     
     246           0         1          0  0.00%  0.00%  0.00%   0 RARP Input       
     247           0         1          0  0.00%  0.00%  0.00%   0 Critical Auth    
     248           0         2          0  0.00%  0.00%  0.00%   0 Dot1x Supplicant 
     249           0         2          0  0.00%  0.00%  0.00%   0 Dot1x Supplicant 
     250           0         2          0  0.00%  0.00%  0.00%   0 Dot1x Supplicant 
     251           0         4          0  0.00%  0.00%  0.00%   0 HRPC dhcp snoopi 
     252           0         7          0  0.00%  0.00%  0.00%   0 HULC DHCP Snoopi 
     253           0         1          0  0.00%  0.00%  0.00%   0 HRPC stack power 
     254         728      1440        505  0.00%  0.00%  0.00%   0 IGMPSN L2MCM     
     255     6774216  58236088        116  0.00%  0.04%  0.01%   0 IGMPSN MRD       
     256    53646337 248186647        216  0.00%  0.16%  0.19%   0 IGMPSN           
     257           0         1          0  0.00%  0.00%  0.00%   0 IGMPQR           
     258           0         3          0  0.00%  0.00%  0.00%   0 L2TRACE SERVER   
     259         669      1435        466  0.00%  0.00%  0.00%   0 MLDSN L2MCM      
     260           0         1          0  0.00%  0.00%  0.00%   0 MRD              
     261           0         1          0  0.00%  0.00%  0.00%   0 MLD_SNOOP        
     142           0         1          0  0.00%  0.00%  0.00%   0 BCM cable diags  
     263           0         1          0  0.00%  0.00%  0.00%   0 HRPC x_setup req 
     264       43456    541423         80  0.00%  0.00%  0.00%   0 IP RIB Update    
     265          16       249         64  0.00%  0.00%  0.00%   0 HRPC hl2mcm mlds 
     266           0        20          0  0.00%  0.00%  0.00%   0 SNMP Timers      
     267           0         2          0  0.00%  0.00%  0.00%   0 EPM MAIN PROCESS 
     268           0        33          0  0.00%  0.00%  0.00%   0 LOCAL AAA        
     269          83       545        152  0.00%  0.00%  0.00%   0 ENABLE AAA       
     270           0         2          0  0.00%  0.00%  0.00%   0 LINE AAA         
     271       19072     11305       1687  0.00%  0.00%  0.00%   0 SNMP Traps       
     272       10522     22584        465  0.00%  0.00%  0.00%   0 TPLUS            
     273           8        12        666  0.00%  0.00%  0.00%   0 crypto engine pr 
     274          25         4       6250  0.00%  0.00%  0.00%   0 Crypto CA        
     275           0         1          0  0.00%  0.00%  0.00%   0 Crypto PKI-CRL   
     276           0         1          0  0.00%  0.00%  0.00%   0 Crypto SSL       
     277           0         1          0  0.00%  0.00%  0.00%   0 encrypt proc     
     278           0        41          0  0.00%  0.00%  0.00%   0 VTP Trap Process 
     279           0         2          0  0.00%  0.00%  0.00%   0 VTPMIB EDIT BUFF 
     280           0         2          0  0.00%  0.00%  0.00%   0 DHCP Security He 
     281           0         1          0  0.00%  0.00%  0.00%   0 HCD Process      
     282           0         5          0  0.00%  0.00%  0.00%   0 HRPC cable diagn 
     283           0         2          0  0.00%  0.00%  0.00%   0 DiagCard2/-1     
     161           0         1          0  0.00%  0.00%  0.00%   0 HRPC obfl reques 
     285          25        16       1562  0.00%  0.00%  0.00%   0 VLAN Manager     
     286           0         1          0  0.00%  0.00%  0.00%   0 DiagCard4/-1     
     287        6264   2601823          2  0.00%  0.00%  0.00%   0 dhcp snooping sw 
     288           0         1          0  0.00%  0.00%  0.00%   0 Licensing MIB pr 
     289           0         2          0  0.00%  0.00%  0.00%   0 EEM ED Routing   
     290         316     21322         14  0.00%  0.00%  0.00%   0 EEM ED Syslog    
     291           0         1          0  0.00%  0.00%  0.00%   0 image mgr        
     292         200      1583        126  0.00%  0.00%  0.0

  • High CPU usage - 3750 X stack

    Support Community
    We recently configured a stack of four 48 port 3750-x switches . We are noticing high CPU usage. "Hulc LED process" seems pretty high.
    This has coincided with VMware servers getting slow and non-responsive at times, perhaps a coincidence, not sure.
    Below I provided some outputs that might help to diagnose it
    Thanks
    John
    System image file is "flash:/c3750e-ipbasek9-mz.122-58.SE2/c3750e-ipbasek9-mz.122-58.SE2.bin"
    Show inventory output
    NAME: "1", DESCR: "WS-C3750X-48"
    PID: WS-C3750X-48T-S   , VID: V02  ,
    NAME: "Switch 1 - Power Supply 0", DESCR: "FRU Power Supply"
    PID: C3KX-PWR-350WAC   , VID: V02L ,
    NAME: "2", DESCR: "WS-C3750X-48"
    PID: WS-C3750X-48T-S   , VID: V02 
    NAME: "Switch 2 - Power Supply 0", DESCR: "FRU Power Supply"
    PID: C3KX-PWR-350WAC   , VID: V02D ,
    NAME: "3", DESCR: "WS-C3750X-48"
    PID: WS-C3750X-48T-S   , VID: V02 
    NAME: "Switch 3 - Power Supply 0", DESCR: "FRU Power Supply"
    PID: C3KX-PWR-350WAC   , VID: V02L ,
    NAME: "4", DESCR: "WS-C3750X-48"
    PID: WS-C3750X-48T-S   , VID: V02 
    NAME: "Switch 4 - Power Supply 0", DESCR: "FRU Power Supply"
    PID: C3KX-PWR-350WAC   , VID: V02L ,
    SWITCH#sh processes cpu sorted
    CPU utilization for five seconds: 61%/5%; one minute: 50%; five minutes: 49%
    PID Runtime(ms)     Invoked      uSecs   5Sec   1Min   5Min TTY Process
    168   260466386    44948517       5794 14.53% 13.98% 13.70%   0 Hulc LED Process
    231    97586088    27253906       3580  4.95%  4.73%  4.64%   0 Spanning Tree
    213    63106121   154928892        407  4.15%  3.89%  3.91%   0 IP Input
    284    70113217    34537588       2030  3.51%  3.98%  4.17%   0 RARP Input
       4     6663412      421278      15817  3.03%  0.43%  0.32%   0 Check heaps
    374     9872291    10805181        913  3.03%  0.77%  0.62%   0 IP SNMP
    376    11142951     5370604       2074  3.03%  0.73%  0.66%   0 SNMP ENGINE
      12    35389011    32152175       1100  2.87%  2.08%  2.20%   0 ARP Input
    128    34962407     3622140       9652  2.07%  1.69%  1.63%   0 hpm counter proc
      85    49034286     8536062       5744  1.91%  2.44%  2.44%   0 RedEarth Tx Mana
    107    25127806    46459053        540  1.27%  1.10%  0.93%   0 HLFM address lea
    174        2412        1714       1407  0.95%  0.39%  0.25%   1 SSH Process
    220     6423643    12634764        508  0.79%  0.70%  0.56%   0 ADJ resolve proc
    181     6913179     2890070       2392  0.63%  0.31%  0.36%   0 HRPC qos request
    375     1681949     5000777        336  0.47%  0.08%  0.07%   0 PDU DISPATCHER
      84    10180707    12623537        806  0.47%  0.30%  0.37%   0 RedEarth I2C dri
            1
          666666096996666666666666659666667666666666666666666766676666666656666666
          249363098992351145264823289455360612252332233522344115537230141392553343
      100       ** **               *
       90       ** **               *
       80       ** **               *
       70   * * *****  *   * * *    * ** ***   *       *     * ****         **
       60 **********************************************************************
       50 ######################################################################
       40 ######################################################################
       30 ######################################################################
       20 ######################################################################
       10 ######################################################################
         0....5....1....1....2....2....3....3....4....4....5....5....6....6....7..
                   0    5    0    5    0    5    0    5    0    5    0    5    0
                       CPU% per hour (last 72 hours)
                      * = maximum CPU%   # = average CPU%
          455555555554444444444555554444455555555555555555555444444444
          922222111118888866666000009999911111555554444422222444448888
      100
       90
       80
       70
       60                                     *****
       50 ***************************************************     **
       40 **********************************************************
       30 **********************************************************
       20 **********************************************************
       10 **********************************************************
         0....5....1....1....2....2....3....3....4....4....5....5....6
                   0    5    0    5    0    5    0    5    0    5    0
                   CPU% per second (last 60 seconds)
          565756555555555555555555555555555556555555555555565555565556
          518841757869248569271526666733778330496833777819929379701861
      100
       90
       80    *
       70    *
       60 **** *******  **** * * *****  ***  * ***  **** **** **** *
       50 ##########################################################
       40 ##########################################################
       30 ##########################################################
       20 ##########################################################
       10 ##########################################################
         0....5....1....1....2....2....3....3....4....4....5....5....6
                   0    5    0    5    0    5    0    5    0    5    0
                   CPU% per minute (last 60 minutes)
                  * = maximum CPU%   # = average CPU%

    Thanks to all for your replies.
    Jeff
    I was aware of the many ACLs however we used to have the same ACLs in a previous 3750G stack about 2 weeks ago and we never had this issue. I agree I need to optimize them and do somehing because it is reaching its max before the CPU starts processing them but I am not certain this is what is causing the issue.
    Nikolay,
    I am trying to understand "interrupts" with the analysis of the outputs I posted. Here is another output deom the link you provided. Please post your thoughts if you can.
    This switch also serves as a gateway(L3 role)  for many systems. Would it make sense to offload that responsability from this switch and let an actual router do it?
    Thanks
    Johnny
    show controllers cpu-interface
    ASIC    Rxbiterr   Rxunder    Fwdctfix   Txbuflos   Rxbufloc   Rxbufdrain
    ASIC0     0          0          0          0          0          0
    ASIC1     0          0          0          0          0          0
    ASIC2     0          0          0          0          0          0
    HOL Fix Counts
    No Fixes:          0 Added:          0 In Use:          0 Both:          0
    CPU Heartbeat Statistics
    Tx Success Tx Fail    1st Thr    2nd Thr    Unthr      RetryCtMax
      37139562          0          0          0          0          1
    Rx Delay
             0          1          2          3          4
      37139562          0          0          0          0
    AddlDelay AdvanceCnt
             0          0
    Rx Retries by RetryCount
             0          1          2          3          4          5          6
      37139562          0          0          0          0          0          0
             7          8          9
             0          0          0
    AddlRetry
             0
    cpu-queue-frames  retrieved  dropped    invalid    hol-block  stray
    rpc               104077409  0          0          0          0
    stp               19189469   0          0          0          0
    ipc               11093838   0          0          0          0
    routing protocol  141021559  0          0          0          0
    L2 protocol       230347     0          0          0          0
    remote console    17         0          0          0          0
    sw forwarding     257436702  0          0          0          0
    host              21146276   0          0          0          0
    broadcast         332154608  0          0          0          0
    cbt-to-spt        0          0          0          0          0
    igmp snooping     2796987    0          0          0          0
    icmp              90752156   0          0          0          0
    logging           0          0          0          0          0
    rpf-fail          0          0          0          0          0
    dstats            0          0          0          0          0
    cpu heartbeat     37139562   0          0          0          0
    cpu-queue         static inuse static added
    rpc               0            0
    stp               0            0
    ipc               0            0
    routing protocol  0            0
    L2 protocol       0            0
    remote console    0            0
    sw forwarding     0            0
    host              0            0
    broadcast         0            0
    cbt-to-spt        0            0
    igmp snooping     0            0
    icmp              0            0
    logging           0            0
    rpf-fail          0            0
    dstats            0            0
    cpu heartbeat     0            0
    Supervisor ASIC receive-queue parameters
    queue 0 maxrecevsize 7E0 pakhead 5505A88 paktail 54655A8
    queue 1 maxrecevsize 7E0 pakhead 5689164 paktail 5687F54
    queue 2 maxrecevsize 7E0 pakhead 5547AA4 paktail 554719C
    queue 3 maxrecevsize 7E0 pakhead 5DC233C paktail 5DBA4CC
    queue 4 maxrecevsize 7E0 pakhead 56A7198 paktail 56A7AA0
    queue 5 maxrecevsize 7E0 pakhead 5D61304 paktail 5D72F80
    queue 6 maxrecevsize 7E0 pakhead 5D856D4 paktail 5D989E4
    queue 7 maxrecevsize 7E0 pakhead 5BDE29C paktail 5BDC784
    queue 8 maxrecevsize 7E0 pakhead 5CC00A8 paktail 5CB3574
    queue 9 maxrecevsize 7E0 pakhead 59DD86C paktail 59DD86C
    queue A maxrecevsize 7E0 pakhead 59BF43C paktail 59C13D8
    queue B maxrecevsize 7E0 pakhead 5DD18A0 paktail 5DCE6F4
    queue C maxrecevsize 7E0 pakhead 59E9CBC paktail 5A049B8
    queue D maxrecevsize 7E0 pakhead 59D8EA0 paktail 59DD25C
    queue E maxrecevsize 0 pakhead 0 paktail 0
    queue F maxrecevsize 7E0 pakhead 59A7080 paktail 59A6BFC
    Supervisor ASIC exception status
    Receive overrun    00000000   Transmit overrun 00000000
    FrameSignatureErr  00000000   MicInitialize    00000002
    BadFrameErr        00000000   LenExceededErr   00000000
    BadJumboSegments   00000000
    Supervisor ASIC Mic Registers
    MicDirectPollInfo               80000200
    MicIndicationsReceived          00000000
    MicInterruptsReceived           00000009
    MicPcsInfo                      0000001F
    MicPlbMasterConfiguration       00000000
    MicRxFifosAvailable             00000000
    MicRxFifosReady                 0000BFFF
    MicTimeOutPeriod:       FrameTOPeriod: 00000EA6 DirectTOPeriod: 00004000
    MicTransmFramesCopied           00000003
    MicTxFifosAvailable             0000000E
    MicConfiguration:       Conf flag: 00000110     Interrupt Flag: 00000008
    MicReceiveFifoAssignmen Queue 0 - 7: 33333333   Queue 8 - 15:33333333
    MicReceiveFramesReady:  FrameAvailable: 00000181        frameAvaiMask: 00000000
    MicException:
            Exception_flag  00000000
            Message-1       00000000
            Message-2       00000000
            Message-3       00000000
    MicIntRxFifo:
            ReadPtr         000005C0        WritePtr        000005C0
            WHeadPtr        000005C0        TxFifoDepth     C0000800
    MicIntTxFifo:
            ReadPtr         00000728        WritePtr        00000728
            WHeadPtr        00000728        TxFifoDepth     C0000800
    MicDecodeInfo:
    Fifo0:  address:        03FF4000 asic_num:      00000100
    Fifo1:  address:        03FF4400 asic_num:      00000101
    MicTransmitFifoInfo:
    Fifo0:   StartPtrs:     0E2CE800        ReadPtr:        0E2CEBE8
            WritePtrs:      0E2CEBE8        Fifo_Flag:      8A800800
            Weights:        001E001E
    Fifo1:   StartPtrs:     0E02D000        ReadPtr:        0E02D138
            WritePtrs:      0E02D138        Fifo_Flag:      89800400
            Weights:        000A000A
    MicReceiveFifoInfo:
    Fifo0:  StartPtr:       0E4AF000        ReadPtr:        0E4AF2A8
            WritePtrs:      0E4AF308        Fifo_Flag:      8B000FA0
            writeHeaderPtr: 0E4AF308
    Fifo1:  StartPtr:       0E78C000        ReadPtr:        0E78C2E8
            WritePtrs:      0E78C2E8        Fifo_Flag:      89800400
            writeHeaderPtr: 0E78C2E8
    Fifo2:  StartPtr:       0E744800        ReadPtr:        0E744A70
            WritePtrs:      0E744A70        Fifo_Flag:      89800400
            writeHeaderPtr: 0E744A70
    Fifo3:  StartPtr:       0EBD1000        ReadPtr:        0EBD13B8
            WritePtrs:      0EBD13B8        Fifo_Flag:      89800400
            writeHeaderPtr: 0EBD13B8
    Fifo4:  StartPtr:       0E7D3800        ReadPtr:        0E7D3A58
            WritePtrs:      0E7D3A58        Fifo_Flag:      89800400
            writeHeaderPtr: 0E7D3A58
    Fifo5:  StartPtr:       0EB40600        ReadPtr:        0EB40688
            WritePtrs:      0EB40688        Fifo_Flag:      88800200
            writeHeaderPtr: 0EB40688
    Fifo6:  StartPtr:       0EB87400        ReadPtr:        0EB874F0
            WritePtrs:      0EB874F0        Fifo_Flag:      89800400
            writeHeaderPtr: 0EB874F0
    Fifo7:  StartPtr:       0E880000        ReadPtr:        0E880E20
            WritePtrs:      0E881520        Fifo_Flag:      8C001900
            writeHeaderPtr: 0E881520
    Fifo8:  StartPtr:       0EB1A600        ReadPtr:        0EB1A770
            WritePtrs:      0EB1A780        Fifo_Flag:      880001F0
            writeHeaderPtr: 0EB1A780
    Fifo9:  StartPtr:       0E2E0CD8        ReadPtr:        0E2E0CD8
            WritePtrs:      0E2E0CD8        Fifo_Flag:      82800008
            writeHeaderPtr: 0E2E0CD8
    Fifo10: StartPtr:       0E81D000        ReadPtr:        0E81D1D8
            WritePtrs:      0E81D1D8        Fifo_Flag:      88800200
            writeHeaderPtr: 0E81D1D8
    Fifo11: StartPtr:       0E4AEF00        ReadPtr:        0E4AEF60
            WritePtrs:      0E4AEF60        Fifo_Flag:      86800080
            writeHeaderPtr: 0E4AEF60
    Fifo12: StartPtr:       0E84A000        ReadPtr:        0E84A300
            WritePtrs:      0E84A000        Fifo_Flag:      89000100
            writeHeaderPtr: 0E84A000
    Fifo13: StartPtr:       0E4AEE00        ReadPtr:        0E4AEE00
            WritePtrs:      0E4AEE00        Fifo_Flag:      86800080
            writeHeaderPtr: 0E4AEE00
    Fifo14: StartPtr:       00000000        ReadPtr:        00000000
            WritePtrs:      00000000        Fifo_Flag:      00800000
            writeHeaderPtr: 00000000
    Fifo15: StartPtr:       0E02CEC0        ReadPtr:        0E02CED0
            WritePtrs:      0E02CED0        Fifo_Flag:      84800020
            writeHeaderPtr: 0E02CED0
    ===========================================================
    Complete Board Id:0x00B2
    ===========================================================

  • 3750x high cpu hulk and crypto

    Dear All,
    We have a stack of two 3750x switches.
    IOS 15.0.2SE02
    Looking at the CPU usage I found the following:
          578757975788577757767778668976766875799875699986676777676887766766795086
          655962145612706544336550879201862386685399919920809060899282596444499009
      100                                      **     **                     * *
       90   *   *                   **         **    ***            *        * *
       80  ***  *  ***  **    ****  **  *  ** *****  ****     *  * ** *      * *
       70  *** *** *** *** ** ********* ** ** ***** ***** ****************  ** *
       60 **************** *****************************************************
       50 **********************************************************************
       40 **********************************************************************
       30 ######################################################################
       20 ######################################################################
       10 ######################################################################
         0....5....1....1....2....2....3....3....4....4....5....5....6....6....7..
                   0    5    0    5    0    5    0    5    0    5    0    5    0
                       CPU% per hour (last 72 hours)
    In order to find the spikes origin I configured a simple event monitor and found the following:
    CPU utilization for five seconds: 75%/0%; one minute: 35%; five minutes: 33%
     PID Runtime(ms)     Invoked      uSecs   5Sec   1Min   5Min TTY Process
      40     4570503      276760      16514 45.28%  3.62%  0.75%   0 crypto sw pk pro
     178  1232406781   146765075       8397 19.20% 19.84% 19.83%   0 Hulc LED Process
    For what I know the Hulk process is normal on stacks. When I insert a simple "show run" or a "write" the CPU spikes but I read that it is normal and do not affect the switching process.
    "crypto sw pk pro" looks to be related to ssh but on our configuration there is nothing related to encryption and ssh access to vty is disabled:
    line vty 0 4
     logging synchronous
     login local
     transport input telnet
    line vty 5 15
     logging synchronous
     login local
     transport input telnet
    Have you any idea?
    thanks,
    Paolo

    I wouldn't touch 15.0(2)SE6 even if my life depended on it.  It's as buggy as he11.  
    The MOST STABLE IOS for the 3750-series of switches is 12.2(55)SE8 or SE9.  This is closely followed by 15.0(2)SE4. 
    The rest of the IOS published ... let's just say I tested them all ... only for a few minutes before I had to quickly and urgently roll back.  

  • 3560 CPU - 88%

    hello, we are using a 3560 switch and the cpu utilization is constantly showing 88 %.
    This switch is doing routing and leased lines are connected to service providers.
    Is there any trouble shooting steps that can be done?
    thanks
    VIvk

    hi Haihua, thank you for the help, I had pasted the output of the commands. Could you kindly take a look.
    TATAIDC-L3-Sw#sh proc cpu | exc 0.00
    CPU utilization for five seconds: 85%/64%; one minute: 87%; five minutes: 87%
    PID Runtime(ms)   Invoked      uSecs   5Sec   1Min   5Min TTY Process
      74  2428691633 632791155       3838 13.09% 14.72% 14.82%   0 HLFM address lea
      94    23028294  23684199        972  0.15%  0.12%  0.11%   0 hpm counter proc
    107         519        58       8948  1.27%  0.49%  0.10%   1 SSH Process     
    131    11454478 680366466         16  0.31%  0.12%  0.07%   0 Hulc LED Process
    132   129347046  17550716       7369  0.31%  0.51%  0.51%   0 HL3U bkgrd proce
    204    89892689 117800715        763  0.95%  0.42%  0.40%   0 Spanning Tree   
    309    41667912  27523089       1513  0.31%  0.28%  0.28%   0 SNMP ENGINE     
    TATAIDC-L3-Sw#
    TATAIDC-L3-Sw#sh contro cpu-int
    ASIC    Rxbiterr   Rxunder    Fwdctfix   Txbuflos   Rxbufloc   Rxbufdrain
    ASIC0     0          0          0          0          0          0        
    ASIC1     0          0          0          0          0          0        
    ASIC2     0          0          0          0          0          0        
    ASIC3     0          0          0          0          0          0        
    ASIC4     0          0          0          0          0          0        
    ASIC5     0          0          0          0          0          0        
    ASIC6     0          0          0          0          0          0        
    HOL Fix Counts
    No Fixes:          0 Added:          0 In Use:          0 Both:          0
    CPU Heartbeat Statistics
    Tx Success Tx Fail    1st Thr    2nd Thr    Unthr      RetryCtMax
    471361363          0          0          0          0          1
    Rx Delay
             0          1          2          3          4
    471361363          0          0          0          0
    AddlDelay AdvanceCnt
             0          0
    Rx Retries by RetryCount
             0          1          2          3          4          5          6
    471361363          0          0          0          0          0          0
             7          8          9
             0          0          0
    --More--                          
    AddlRetry
             0
    cpu-queue-frames  retrieved  dropped    invalid    hol-block  stray
    rpc               1          0          0          0          0        
    stp               55630558   0          0          0          0        
    ipc               0          0          0          0          0        
    routing protocol  531317387  0          0          0          0        
    L2 protocol       11709909   0          0          0          0        
    remote console    0          0          0          0          0        
    sw forwarding     1851430181 0          0          0          0        
    host              57685503   0          0          0          0        
    broadcast         402389844  0          0          0          0        
    cbt-to-spt        0          0          0          0          0        
    igmp snooping     44308332   0          0          0          0        
    icmp              615701075  0          0          0          0        
    logging           0          0          0          0          0        
    rpf-fail          0          0          0          0          0        
    dstats            0          0          0          0          0        
    cpu heartbeat     471361430  0          0          0          0        
    cpu-queue         static inuse static added
    rpc               0            0          
    stp               0            0          
    ipc               0            0          
    routing protocol  0            0          
    L2 protocol       0            0          
    remote console    0            0          
    sw forwarding     0            0          
    host              0            0          
    broadcast         0            0          
    cbt-to-spt        0            0          
    igmp snooping     0            0          
    icmp              0            0          
    --More--                          
    logging           0            0          
    rpf-fail          0            0          
    dstats            0            0          
    cpu heartbeat     0            0          
    Supervisor ASIC receive-queue parameters
    queue 0 maxrecevsize 0 pakhead 0 paktail 0
    queue 1 maxrecevsize 7E0 pakhead 525AD98 paktail 5256CA0
    queue 2 maxrecevsize 7E0 pakhead 4350598 paktail 5226E94
    queue 3 maxrecevsize 7E0 pakhead 446A804 paktail 422BB54
    queue 4 maxrecevsize 7E0 pakhead 5263324 paktail 43872E0
    queue 5 maxrecevsize 7E0 pakhead 0 paktail 0
    queue 6 maxrecevsize 7E0 pakhead 5336D68 paktail 445F248
    queue 7 maxrecevsize 7E0 pakhead 441CBA8 paktail 4426A78
    queue 8 maxrecevsize 7E0 pakhead 44305AC paktail 442C850
    queue 9 maxrecevsize 7E0 pakhead 43C87B0 paktail 43C87B0
    queue A maxrecevsize 7E0 pakhead 43BADE4 paktail 43C0F58
    queue B maxrecevsize 7E0 pakhead 534AFE8 paktail 534C1F4
    queue C maxrecevsize 7E0 pakhead 52AA8A8 paktail 43CF910
    queue D maxrecevsize 7E0 pakhead 52A2210 paktail 43C8414
    queue E maxrecevsize 0 pakhead 0 paktail 0
    queue F maxrecevsize 7E0 pakhead 43B4FA4 paktail 528D2B0
    Supervisor ASIC exception status
    Receive overrun    00000000   Transmit overrun 00000000
    FrameSignatureErr  00000000   MicInitialize    00000000
    BadFrameErr        00000000   LenExceededErr   00000000
    BadJumboSegments   00000000
    Supervisor ASIC Mic Registers
    MicDirectPollInfo                         80000200
    MicIndicationsReceived                    00000000
    MicInterruptsReceived                    00000000
    MicPcsInfo                               0000011F
    MicPlbMasterConfiguration          00000000
    MicRxFifosAvailable                      00000040
    --More--                          
    MicRxFifosReady                          0000BFBE
    MicTimeOutPeriod:          FrameTOPeriod: 00000EA6          DirectTOPeriod: 00004000
    MicTransmFramesCopied                    00000003
    MicTxFifosAvailable                      0000007F
    MicConfiguration:          Conf flag: 00000110          Interrupt Flag: 0000000A
    MicReceiveFifoAssignmen          Queue 0 - 7: 44444444          Queue 8 - 15:44444444
    MicReceiveFramesReady:          FrameAvailable: 00000041          frameAvaiMask: 00000000
    MicException:   
    Exception_flag          00000000
    Message-1          00000000
    Message-2          00000000
    Message-3          00000000
    MicIntRxFifo:
    ReadPtr                    000001B0          WritePtr          000001B0
    WHeadPtr          000001B0          TxFifoDepth          C0000800
    MicIntTxFifo:
    ReadPtr                    000004B0          WritePtr          000004B0
    WHeadPtr          000004B0          TxFifoDepth          C0000800
    MicDecodeInfo:
    Fifo0:          address:          03FF4000 asic_num:          00000100
    Fifo1:          address:          03FF4400 asic_num:          00000101
    MicTransmitFifoInfo:
    Fifo0:           StartPtrs:          0697E800          ReadPtr:          0697EE10
    WritePtrs:          0697EE10          Fifo_Flag:          8A800800
    Weights:          001E001E
    Fifo1:           StartPtrs:          06827400          ReadPtr:          068275B8
    WritePtrs:          068275B8          Fifo_Flag:          89800400
    Weights:          000A000A
    MicReceiveFifoInfo:
    Fifo0:          StartPtr:          0697E760          ReadPtr:          0697E760
    WritePtrs:          0697E768          Fifo_Flag:          04000018
    writeHeaderPtr:          0697E768
    Fifo1:          StartPtr:          06998C00          ReadPtr:          06998F10
    WritePtrs:          06998F10          Fifo_Flag:          89800400
    writeHeaderPtr:          06998F10
    Fifo2:          StartPtr:          06827300          ReadPtr:          06827360
    WritePtrs:          06827300          Fifo_Flag:          86000020
    writeHeaderPtr:          06827300
    --More--                          
    Fifo3:          StartPtr:          06B14000          ReadPtr:          06B140A0
    WritePtrs:          06B140A0          Fifo_Flag:          89800400
    writeHeaderPtr:          06B140A0
    Fifo4:          StartPtr:          069E0200          ReadPtr:          069E02A8
    WritePtrs:          069E02A8          Fifo_Flag:          88800200
    writeHeaderPtr:          069E02A8
    Fifo5:          StartPtr:          06827268          ReadPtr:          06827268
    WritePtrs:          06827268          Fifo_Flag:          82800008
    writeHeaderPtr:          06827268
    Fifo6:          StartPtr:          06ACC800          ReadPtr:          06ACC888
    WritePtrs:          06ACC888          Fifo_Flag:          89400000
    writeHeaderPtr:          06ACC888
    Fifo7:          StartPtr:          06A3C000          ReadPtr:          06A3C018
    WritePtrs:          06A3C018          Fifo_Flag:          89800400
    writeHeaderPtr:          06A3C018
    Fifo8:          StartPtr:          06AA6600          ReadPtr:          06AA66E0
    WritePtrs:          06AA66E0          Fifo_Flag:          88800200
    writeHeaderPtr:          06AA66E0
    Fifo9:          StartPtr:          06990CD8          ReadPtr:          06990CD8
    WritePtrs:          06990CD8          Fifo_Flag:          82800008
    writeHeaderPtr:          06990CD8
    Fifo10:          StartPtr:          06A06000          ReadPtr:          06A06168
    WritePtrs:          06A06168          Fifo_Flag:          88800200
    writeHeaderPtr:          06A06168
    Fifo11:          StartPtr:          06AA6500          ReadPtr:          06AA6518
    WritePtrs:          06AA6518          Fifo_Flag:          86800080
    writeHeaderPtr:          06AA6518
    Fifo12:          StartPtr:          06A33000          ReadPtr:          06A33000
    WritePtrs:          06A33000          Fifo_Flag:          86800080
    writeHeaderPtr:          06A33000
    Fifo13:          StartPtr:          069E0100          ReadPtr:          069E0100
    WritePtrs:          069E0100          Fifo_Flag:          86800080
    writeHeaderPtr:          069E0100
    Fifo14:          StartPtr:          00000000          ReadPtr:          00000000
    WritePtrs:          00000000          Fifo_Flag:          00800000
    writeHeaderPtr:          00000000
    Fifo15:          StartPtr:          0697E760          ReadPtr:          0697E778
    WritePtrs:          0697E778          Fifo_Flag:          84800020
    --More--                           
    writeHeaderPtr:          0697E778
    ===========================================================
    Complete Board Id:0x401E
    ===========================================================
    TATAIDC-L3-Sw#    

  • WS-C2960X-48TS-L cpu utilization C2960X-UNIVERSALK9-M Version 15.0(2)EX5

    WS-C2960X-48TS-L cpu utilization C2960X-UNIVERSALK9-M Version 15.0(2)EX5 
    CPU utilization for five seconds: 79%/3%; one minute: 65%; five minutes: 64%
     PID Runtime(ms)     Invoked      uSecs   5Sec   1Min   5Min TTY Process
     190   191408173    28207523       6785 34.87% 19.24% 17.42%   0 Auth Manager                                                                                                                        
     161   178000592    24299988       7325 15.12% 14.92% 14.97%   0 Hulc LED Proces

    Hey,
    Regarding high CPU issue and the two processes you have mentioned:
    1. Overview of IOS Auth Manager: The capabilities of devices connecting to a given network can be different, thus requiring that the network support different authentication methods and authorization policies. The Cisco IOS Auth Manager handles network authentication requests and enforces authorization policies, regardless of authentication method. The Auth Manager maintains operational data for all port-based network connection attempts, authentications, authorizations, and disconnections and as such, serves as a session manager. So looks like this device is managing good amount of AAA sessions.
    2. Hulc LED Process: Check this bug https://tools.cisco.com/bugsearch/bug/CSCtg86211
    HTH.
    Regards,
    RS.

  • Process used to update IOS 2960-S

    I need to upgrade Cisco 2960S Stackable IOS this week. But before I want to know what is the process used.
    HSP_TRSF_SWPB_E3#show processes cpu | include HULC
      54           0         1          0  0.00%  0.00%  0.00%   0 HULC ACL Tcam Me
      63           0         1          0  0.00%  0.00%  0.00%   0 HULC QM Tcam Mem
      72      490715   4256103        115  0.00%  0.00%  0.00%   0 HULC Thermal Pro
      81           5         5       1000  0.00%  0.00%  0.00%   0 HULC Device Mana
      99           0         1          0  0.00%  0.00%  0.00%   0 HULC VLAN REF Ba
     102           0         1          0  0.00%  0.00%  0.00%   0 HULC PM Vector P
     121           5        19        263  0.00%  0.00%  0.00%   0 HULC CISP Proces
     124      296126   2589355        114  0.00%  0.00%  0.00%   0 HULC DOT1X Proce
     148           0         1          0  0.00%  0.00%  0.00%   0 HULC SNMP Proces
     204           0         1          0  0.00%  0.00%  0.00%   0 HULC DAI Process
     208           0         1          0  0.00%  0.00%  0.00%   0 HULC IP Source g
     220           0         1          0  0.00%  0.00%  0.00%   0 CEF RF HULC Conv
     246           0         6          0  0.00%  0.00%  0.00%   0 HULC DHCP Snoopi
    Some cases saw the HULC archive dnl.
    Thanks,

    Disclaimer
    The Author of this posting offers the information contained within this posting without consideration and with the reader's understanding that there's no implied or expressed suitability or fitness for any purpose. Information provided is for informational purposes only and should not be construed as rendering professional advice of any kind. Usage of this posting's information is solely at reader's own risk.
    Liability Disclaimer
    In no event shall Author be liable for any damages whatsoever (including, without limitation, damages for loss of use, data or profit) arising out of the use or inability to use the posting's information even if Author has been advised of the possibility of such damage.
    Posting
    Generally, process is, load new IOS, insure device will chose it first when reloaded, reload device.
    Configuration guide, for your device/IOS should have all the details?
    Were you looking for something more?

  • A cisco 2960S 100% cpu

    hi
    On a Cisco 2960S-48TS-L, After a Mac flapping the CPU went to 100 %  
    700sw10_emb#sh proc cpu sort
    CPU utilization for five seconds: 99%/5%; one minute: 99%; five minutes: 99%
     PID Runtime(ms)     Invoked      uSecs   5Sec   1Min   5Min TTY Process
     155    37771767     3603617      10481 23.51% 23.68% 23.53%   0 Hulc LED Proces
    s
     225     4213295      956535       4404 23.31% 23.01% 22.98%   0 HULC DAI Proces
    s
     234     3486099     1061257       3284 20.21% 20.41% 20.48%   0 IP Host Track P
    r
     182     3761068      742168       5067 18.51% 18.40% 18.46%   0 Auth Manager
     104      244786     3413567         71  1.80%  1.57%  1.51%   0 HLFM address le
    here is the log
    Mar 13 17:43:38.076: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthe
    rnet1/0/29, changed state to up
    Mar 14 08:05:25.076: %SW_MATM-4-MACFLAP_NOTIF: Host 000b.6b22.7b72 in vlan 1 is
    flapping between port Gi1/0/48 and port Gi1/0/51
    Mar 14 08:07:52.602: %SW_MATM-4-MACFLAP_NOTIF: Host 000b.6b22.7b72 in vlan 1 is
    flapping between port Gi1/0/48 and port Gi1/0/51
    Mar 14 08:09:16.681: %SW_MATM-4-MACFLAP_NOTIF: Host 000b.6b22.7b72 in vlan 1 is
    flapping between port Gi1/0/48 and port Gi1/0/51
    .Mar 14 08:38:10.808: %SW_MATM-4-MACFLAP_NOTIF: Host 000b.6b22.79de in vlan 1 is
     flapping between port Gi1/0/48 and port Gi1/0/51
    The only solution is restart the switch
    regards

    I am not sure what IOS version you are using but here's my recommendation:  
    1.  If you are running 12.2 then use 12.2(55)SE8; or
    2.  If you are running 15.X then use 15.0(2)SE4.
    Don't even bother using the others.

  • High CPU utilisation on 2960X stack

    Hello !
    i have a lot of switches 2960-X stack in my network, but i downgraded the iOS version (15.0(2)EX4 to 15.0(2)EX3 be cause some bugs are on EX4 with SFP).
    Since I downgraded the version, i have a lot of troubles on some switches with a huge amount of CPU utilisation.
    One is "normal" and this is the famous one : Hulc LED
    but the other one is not "normal" for me :
    this line :
    190   207178513     1061976     195092  0.00% 31.06% 24.83%   0 Auth Manager
    i dont know why on some switches the "Auth Manager" CPU utilisation is too high.
    When the CPU is too high, we cannot join the switch, the ping is too high (between 500ms and 2100) and users complaining about network spikes.
    So, i dont want to upgrade the iOS to the EX5 (too recent, and i dont know if bug with SPF and CPU are fixed).
    About the Auth manager, what could raise this process to high for the CPU ?
    (i have some check with nagios and snmpwalk but every 5 min and not every minutes, and nagios works well with EX4, issues appear since EX3)
    thank for your help and advice

    15.2(2)E is the first main-line engineering software for the 2960X.  The "EX" versions you've been using are interim version and not really meant for long-term production network.  

Maybe you are looking for