DBConsole easting a CPU

Hello
just installed a 11G DB on linux RH5, simple install with silmple DB.
The problem I have is that one OMS session is constrantly querying the DB causing a high CPU usage, statement is :
SELECT EXECUTION_ID, STATUS, STATUS_DETAIL
FROM MGMT_JOB_EXEC_SUMMARY
WHERE JOB_ID = :B3 AND TARGET_LIST_INDEX = :B2 AND EXPECTED_START_TIME = :B1
Any clue ?

It seems like dbconsole works single threaded here, there is always the same oracle server process busy on the cpu, on the quad core linux box three cores are left idle, ditto on the 4 cpu e420.
Now something really strange: suddenly the cpu load on the linux box is gone, dbconsole is up and running, and doesn't consume any extra cpu time, there was no restart of dbconsole nor the whole database server. I remember clicking the link "All Metrics" in Availability->Related Links and then displaying both entries in "Database Job Status" in enterprise manager. Another followed link was "Jobs" in the same context. These operations should have been read only, no single parameter has changed. I'm really puzzled.
Have to check the rac installation in the next days, it doesn't always run due to electricity bill constraints ;-)

Similar Messages

100% CPU Usage Overhead running EM DBConsole 11g on OEL-5.2

After upgrading to OEL-5.2 and relinking all Oracle binaries, my old Oracle 11g installation, installed several months before on OEL-5.1, has been working well, including Enterprise Manager Database Console working nicely as always with respectful performance. Unfortunatelly, it lasted just several days.
Yesterday I decided to uninstall the 11g completely and perform new clean installation (software and database) with the same configuration options and settings as before, including EM dbconsole, all configured using dbca. After completing the installation (EM was started automatically by dbca), oracle continued to suck 80-85% CPU time. In further few minutes CPU utilization raised up to 99% due to only one (always the same PID) client process - "oracleorcl (LOCAL=NO)". For first ten minutes I didn't care too much since I always enable Automatic Management in dbca. But after two hours, I started to worry. The process was still running, consuming sustained 99% of CPU power. No other system activity, no database activity, no disks activity at all!
I was really puzzled since I installed and reinstalled the 11g at least 20 times on OEL-5.0 and 5.1, experimenting with ASM, raw devices, loopback devices and various combinations of installation options, but never experienced such a behaviour. It took me 3 minutes to log in to EM dbconsole as it was almost unusable performing too slow. After three hours CPU temperature was nearly 60 degrees celsius. I decided to shutdown EM and after that everything became quiet. Oracle was running normally. Started EM again, the problem was back again. Tracing enabled, it filled a 350 MB trace file in just 20 minutes. Reinstalling the software and database once again didn't help. Whenever EM is up, the CPU usage overhead of 99% persists.
Here is a cca 23 minutes session summary report taken from EM dbconsole's Performance page. The trace file is too big to list it here, but it shows the same.
        Host CPU: 100%
Active Sessions: 100%The details for the Selected 5 Minute Interval (the last 5 min interval) are shown as follow:
    TOP SESSIONS: SYSMAN, Program: OMS
        Activity: 100%
     TOP MODULES: OEM.CacheModeWaitPool, Service: orcl
        Activity: 100%
      TOP CLIENT: Unnamed
        Activity: 99.1%
     TOP ACTIONS: Unnamed (OEM.CacheModeWaitPool) (orcl)
        Activity: 100%
     TOP OBJECTS: SYSMAN.MGMT_JOB_EXEC_SUMMARY (Table)
        Activity: 100%
      TOP PL/SQL: SYSMAN.MGMT_JOB_ENGINE.INSERT_EXECUTION
   PL/SQL Source: SYSMAN.MGMT_JOB_ENGINE
     Line Number: 7135
        Activity: 100%
         TOP SQL: SELECT EXECUTION_ID, STATUS, STATUS_DETAIL FROM MGMT_JOB_EXEC_SUMMARY
WHERE JOB_ID = :B3 AND TARGET_LIST_INDEX = :B2 AND EXPECTED_START_TIME = :B1;
        Activity: 100%
                              STATISTICS SUMMARY
                            cca 23 minutes session
                        with no other system activity
                                        Per
                       Total      Execution         Per Row
Executions           105,103                 1       10,510.30
Elapsed Time (sec) 1,358.95              0.01        135.90
CPU Time (sec)      1,070.42             0.01        107.04
Buffer Gets       85,585,518 814.30 8,558,551.80
Disk Reads                 2            <0.01          0.20
Direct Writes              0              0.00          0.00
Rows                      10            <0.01             1
Fetches              105,103             1.00     10,510.30
                   ----------------------------------------Wow!!! Note: no disk, no database activity !
Has anyone experienced this or similar behaviour after clean 11g installation on OEL-5.2? If not, anyone has a clue what the hell is going on?
Thanks in advance.

Hi Tommy,
I didn't want to experiment further with already working OEL-5.2, oracle and dbconsole on this machine, specially not after googling the problem and finding out that I am not alone in this world. There are another two threads on OTN forums (Database General) showing the same problem even on 2GB machines:
DBConsole easting a CPU
11g stuck. 50-100% CPU after fresh install
So, I took another, a smaller free machine I've got at home (1GB RAM, 2.2MHz Pentium4, three 80GB disks), on which I used to experiment with new releases of software (this is the machine on which I installed 11g for the first time when it was released on OEL-5.0, and I can recall that everything was OK with EM). This is what I did:
1. I installed OEL-5.0 on the machine, adjusted linux and kernel parameters, and performed full 11g installation. Database and EM dbconsole worked nice with acceptable performance. Without activity in the database, %CPU = zero !!! The whole system was perfectly quiet.
2. Since everything was OK, I shutdown EM and oracle, and performed the full upgrade to OEL-5.2. When the upgrade finished, restarted the system, relinked all oracle binaries, and started oracle and EM dbconsole. Both worked perfectly again, just as before the upgrade. I repeated restarting the database and dbconsole several times, always with the same result - it really rocks. Without database activity, %CPU = zero%.
3. Using dbca, I dropped the database and created the new one with the same configuration options. Wow! I'm again in trouble. A half an hour after the creation of the database, %CPU raised up to 99%. That's it.
The crucial question here is: what is that in OEL-5.2, not existing in the 5.0, that causes dbca/em scripts to be embarrassed at the time of EM agent configuration?
Here are the outputs you required picked 30 minutes after starting the database and EM dbconsole (sustained 99% CPU utilization). Note that this is just a 1GB machine.
Kernel command line: ro root=LABEL=/ elevator=deadline rhgb quiet
[root@localhost ~]# cat /proc/meminfo
MemTotal:      1034576 kB
MemFree:         27356 kB
Buffers:          8388 kB
Cached:         609660 kB
SwapCached:      18628 kB
Active:         675376 kB
Inactive:       287072 kB
HighTotal:      130304 kB
HighFree:          260 kB
LowTotal:       904272 kB
LowFree:         27096 kB
SwapTotal:     3148700 kB
SwapFree:      2940636 kB
Dirty:              72 kB
Writeback:           0 kB
AnonPages:      328700 kB
Mapped:         271316 kB
Slab:            21136 kB
PageTables:      14196 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:   3665988 kB
Committed_AS: 1187464 kB
VmallocTotal:   114680 kB
VmallocUsed:      5860 kB
VmallocChunk:   108476 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     4096 kB
[root@localhost ~]# cat /proc/slabinfo
slabinfo - version: 2.1
# name            <active_objs> <num_objs> <objsize> <objperslab> <pagesperslab> : tunables <limit> <batchcount> <sharedfactor> : slabdata <active_slabs> <num_slabs> <sharedavail>
rpc_buffers            8      8   2048    2    1 : tunables   24   12    8 : slabdata      4      4      0
rpc_tasks              8     15    256   15    1 : tunables 120   60    8 : slabdata      1      1      0
rpc_inode_cache        6      7    512    7    1 : tunables   54   27    8 : slabdata      1      1      0
ip_conntrack_expect    0      0     96   40    1 : tunables 120   60    8 : slabdata      0      0      0
ip_conntrack          68     68    228   17    1 : tunables 120   60    8 : slabdata      4      4      0
ip_fib_alias           7    113     32 113    1 : tunables 120   60    8 : slabdata      1      1      0
ip_fib_hash            7    113     32 113    1 : tunables 120   60    8 : slabdata      1      1      0
fib6_nodes            22    113     32 113    1 : tunables 120   60    8 : slabdata      1      1      0
ip6_dst_cache         13     15    256   15    1 : tunables 120   60    8 : slabdata      1      1      0
ndisc_cache            1     15    256   15    1 : tunables 120   60    8 : slabdata      1      1      0
RAWv6                  4      5    768    5    1 : tunables   54   27    8 : slabdata      1      1      0
UDPv6                  9     12    640    6    1 : tunables   54   27    8 : slabdata      2      2      0
tw_sock_TCPv6          0      0    128   30    1 : tunables 120   60    8 : slabdata      0      0      0
request_sock_TCPv6     0      0    128   30    1 : tunables 120   60    8 : slabdata      0      0      0
TCPv6                  1      3   1280    3    1 : tunables   24   12    8 : slabdata      1      1      0
jbd_1k                 0      0   1024    4    1 : tunables   54   27    8 : slabdata      0      0      0
dm_mpath               0      0     28 127    1 : tunables 120   60    8 : slabdata      0      0      0
dm_uevent              0      0   2460    3    2 : tunables   24   12    8 : slabdata      0      0      0
dm_tio                 0      0     16 203    1 : tunables 120   60    8 : slabdata      0      0      0
dm_io                  0      0     20 169    1 : tunables 120   60    8 : slabdata      0      0      0
jbd_4k                 1      1   4096    1    1 : tunables   24   12    8 : slabdata      1      1      0
scsi_cmd_cache        10     10    384   10    1 : tunables   54   27    8 : slabdata      1      1      0
sgpool-128            36     36   2048    2    1 : tunables   24   12    8 : slabdata     18     18      0
sgpool-64             33     36   1024    4    1 : tunables   54   27    8 : slabdata      9      9      0
sgpool-32             34     40    512    8    1 : tunables   54   27    8 : slabdata      5      5      0
sgpool-16             35     45    256   15    1 : tunables 120   60    8 : slabdata      3      3      0
sgpool-8              60     60    128   30    1 : tunables 120   60    8 : slabdata      2      2      0
scsi_io_context        0      0    104   37    1 : tunables 120   60    8 : slabdata      0      0      0
ext3_inode_cache    4376   8216    492    8    1 : tunables   54   27    8 : slabdata   1027   1027      0
ext3_xattr           165    234     48   78    1 : tunables 120   60    8 : slabdata      3      3      0
journal_handle         8    169     20 169    1 : tunables 120   60    8 : slabdata      1      1      0
journal_head         684   1008     52   72    1 : tunables 120   60    8 : slabdata     14     14      0
revoke_table          18    254     12 254    1 : tunables 120   60    8 : slabdata      1      1      0
revoke_record          0      0     16 203    1 : tunables 120   60    8 : slabdata      0      0      0
uhci_urb_priv          0      0     28 127    1 : tunables 120   60    8 : slabdata      0      0      0
UNIX                  56    112    512    7    1 : tunables   54   27    8 : slabdata     16     16      0
flow_cache             0      0    128   30    1 : tunables 120   60    8 : slabdata      0      0      0
cfq_ioc_pool           0      0     92   42    1 : tunables 120   60    8 : slabdata      0      0      0
cfq_pool               0      0     96   40    1 : tunables 120   60    8 : slabdata      0      0      0
crq_pool               0      0     44   84    1 : tunables 120   60    8 : slabdata      0      0      0
deadline_drq         140    252     44   84    1 : tunables 120   60    8 : slabdata      3      3      0
as_arq                 0      0     56   67    1 : tunables 120   60    8 : slabdata      0      0      0
mqueue_inode_cache     1      6    640    6    1 : tunables   54   27    8 : slabdata      1      1      0
isofs_inode_cache      0      0    368   10    1 : tunables   54   27    8 : slabdata      0      0      0
hugetlbfs_inode_cache 1     11    340   11    1 : tunables   54   27    8 : slabdata      1      1      0
ext2_inode_cache       0      0    476    8    1 : tunables   54   27    8 : slabdata      0      0      0
ext2_xattr             0      0     48   78    1 : tunables 120   60    8 : slabdata      0      0      0
dnotify_cache          2    169     20 169    1 : tunables 120   60    8 : slabdata      1      1      0
dquot                  0      0    128   30    1 : tunables 120   60    8 : slabdata      0      0      0
eventpoll_pwq          1    101     36 101    1 : tunables 120   60    8 : slabdata      1      1      0
eventpoll_epi          1     30    128   30    1 : tunables 120   60    8 : slabdata      1      1      0
inotify_event_cache    1    127     28 127    1 : tunables 120   60    8 : slabdata      1      1      0
inotify_watch_cache   23     92     40   92    1 : tunables 120   60    8 : slabdata      1      1      0
kioctx               135    135    256   15    1 : tunables 120   60    8 : slabdata      9      9      0
kiocb                  0      0    128   30    1 : tunables 120   60    8 : slabdata      0      0      0
fasync_cache           0      0     16 203    1 : tunables 120   60    8 : slabdata      0      0      0
shmem_inode_cache    553    585    436    9    1 : tunables   54   27    8 : slabdata     65     65      0
posix_timers_cache     0      0     88   44    1 : tunables 120   60    8 : slabdata      0      0      0
uid_cache              5     59     64   59    1 : tunables 120   60    8 : slabdata      1      1      0
ip_mrt_cache           0      0    128   30    1 : tunables 120   60    8 : slabdata      0      0      0
tcp_bind_bucket       32    203     16 203    1 : tunables 120   60    8 : slabdata      1      1      0
inet_peer_cache        1     59     64   59    1 : tunables 120   60    8 : slabdata      1      1      0
secpath_cache          0      0     32 113    1 : tunables 120   60    8 : slabdata      0      0      0
xfrm_dst_cache         0      0    384   10    1 : tunables   54   27    8 : slabdata      0      0      0
ip_dst_cache           6     15    256   15    1 : tunables 120   60    8 : slabdata      1      1      0
arp_cache              2     15    256   15    1 : tunables 120   60    8 : slabdata      1      1      0
RAW                    2      7    512    7    1 : tunables   54   27    8 : slabdata      1      1      0
UDP                    3      7    512    7    1 : tunables   54   27    8 : slabdata      1      1      0
tw_sock_TCP            3     30    128   30    1 : tunables 120   60    8 : slabdata      1      1      0
request_sock_TCP       4     30    128   30    1 : tunables 120   60    8 : slabdata      1      1      0
TCP                   43     49   1152    7    2 : tunables   24   12    8 : slabdata      7      7      0
blkdev_ioc             3    127     28 127    1 : tunables 120   60    8 : slabdata      1      1      0
blkdev_queue          23     24    956    4    1 : tunables   54   27    8 : slabdata      6      6      0
blkdev_requests      137    161    172   23    1 : tunables 120   60    8 : slabdata      7      7      0
biovec-256             7      8   3072    2    2 : tunables   24   12    8 : slabdata      4      4      0
biovec-128             7     10   1536    5    2 : tunables   24   12    8 : slabdata      2      2      0
biovec-64              7     10    768    5    1 : tunables   54   27    8 : slabdata      2      2      0
biovec-16              7     15    256   15    1 : tunables 120   60    8 : slabdata      1      1      0
biovec-4               8     59     64   59    1 : tunables 120   60    8 : slabdata      1      1      0
biovec-1             406    406     16 203    1 : tunables 120   60    8 : slabdata      2      2    300
bio                  564    660    128   30    1 : tunables 120   60    8 : slabdata     21     22    204
utrace_engine_cache    0      0     32 113    1 : tunables 120   60    8 : slabdata      0      0      0
utrace_cache           0      0     32 113    1 : tunables 120   60    8 : slabdata      0      0      0
sock_inode_cache     149    230    384   10    1 : tunables   54   27    8 : slabdata     23     23      0
skbuff_fclone_cache   20     20    384   10    1 : tunables   54   27    8 : slabdata      2      2      0
skbuff_head_cache     86    210    256   15    1 : tunables 120   60    8 : slabdata     14     14      0
file_lock_cache       22     40     96   40    1 : tunables 120   60    8 : slabdata      1      1      0
Acpi-Operand        1147   1196     40   92    1 : tunables 120   60    8 : slabdata     13     13      0
Acpi-ParseExt          0      0     44   84    1 : tunables 120   60    8 : slabdata      0      0      0
Acpi-Parse             0      0     28 127    1 : tunables 120   60    8 : slabdata      0      0      0
Acpi-State             0      0     44   84    1 : tunables 120   60    8 : slabdata      0      0      0
Acpi-Namespace       615    676     20 169    1 : tunables 120   60    8 : slabdata      4      4      0
delayacct_cache      233    312     48   78    1 : tunables 120   60    8 : slabdata      4      4      0
taskstats_cache       12     53     72   53    1 : tunables 120   60    8 : slabdata      1      1      0
proc_inode_cache     622    693    356   11    1 : tunables   54   27    8 : slabdata     63     63      0
sigqueue               8     27    144   27    1 : tunables 120   60    8 : slabdata      1      1      0
radix_tree_node     6220   8134    276   14    1 : tunables   54   27    8 : slabdata    581    581      0
bdev_cache            37     42    512    7    1 : tunables   54   27    8 : slabdata      6      6      0
sysfs_dir_cache     4980   4992     48   78    1 : tunables 120   60    8 : slabdata     64     64      0
mnt_cache             36     60    128   30    1 : tunables 120   60    8 : slabdata      2      2      0
inode_cache         1113   1254    340   11    1 : tunables   54   27    8 : slabdata    114    114     81
dentry_cache       11442 18560    136   29    1 : tunables 120   60    8 : slabdata    640    640    180
filp                7607 10000    192   20    1 : tunables 120   60    8 : slabdata    500    500    120
names_cache           19     19   4096    1    1 : tunables   24   12    8 : slabdata     19     19      0
avc_node              14     72     52   72    1 : tunables 120   60    8 : slabdata      1      1      0
selinux_inode_security 814   1170     48   78    1 : tunables 120   60    8 : slabdata     15     15      0
key_jar               14     30    128   30    1 : tunables 120   60    8 : slabdata      1      1      0
idr_layer_cache      170    203    136   29    1 : tunables 120   60    8 : slabdata      7      7      0
buffer_head        38892 39024     52   72    1 : tunables 120   60    8 : slabdata    542    542      0
mm_struct            108    135    448    9    1 : tunables   54   27    8 : slabdata     15     15      0
vm_area_struct     11169 14904     84   46    1 : tunables 120   60    8 : slabdata    324    324    144
fs_cache              82    177     64   59    1 : tunables 120   60    8 : slabdata      3      3      0
files_cache          108    140    384   10    1 : tunables   54   27    8 : slabdata     14     14      0
signal_cache         142    171    448    9    1 : tunables   54   27    8 : slabdata     19     19      0
sighand_cache        127    135   1344    3    1 : tunables   24   12    8 : slabdata     45     45      0
task_struct          184    246   1360    3    1 : tunables   24   12    8 : slabdata     82     82      0
anon_vma            3313   5842     12 254    1 : tunables 120   60    8 : slabdata     23     23      0
pgd                   84     84   4096    1    1 : tunables   24   12    8 : slabdata     84     84      0
pid                  237    303     36 101    1 : tunables 120   60    8 : slabdata      3      3      0
size-131072(DMA)       0      0 131072    1   32 : tunables    8    4    0 : slabdata      0      0      0
size-131072            0      0 131072    1   32 : tunables    8    4    0 : slabdata      0      0      0
size-65536(DMA)        0      0 65536    1   16 : tunables    8    4    0 : slabdata      0      0      0
size-65536             2      2 65536    1   16 : tunables    8    4    0 : slabdata      2      2      0
size-32768(DMA)        0      0 32768    1    8 : tunables    8    4    0 : slabdata      0      0      0
size-32768             9      9 32768    1    8 : tunables    8    4    0 : slabdata      9      9      0
size-16384(DMA)        0      0 16384    1    4 : tunables    8    4    0 : slabdata      0      0      0
size-16384             6      6 16384    1    4 : tunables    8    4    0 : slabdata      6      6      0
size-8192(DMA)         0      0   8192    1    2 : tunables    8    4    0 : slabdata      0      0      0
size-8192              5      5   8192    1    2 : tunables    8    4    0 : slabdata      5      5      0
size-4096(DMA)         0      0   4096    1    1 : tunables   24   12    8 : slabdata      0      0      0
size-4096            205    205   4096    1    1 : tunables   24   12    8 : slabdata    205    205      0
size-2048(DMA)         0      0   2048    2    1 : tunables   24   12    8 : slabdata      0      0      0
size-2048            260    270   2048    2    1 : tunables   24   12    8 : slabdata    135    135      0
size-1024(DMA)         0      0   1024    4    1 : tunables   54   27    8 : slabdata      0      0      0
size-1024            204    204   1024    4    1 : tunables   54   27    8 : slabdata     51     51      0
size-512(DMA)          0      0    512    8    1 : tunables   54   27    8 : slabdata      0      0      0
size-512             367    464    512    8    1 : tunables   54   27    8 : slabdata     58     58      0
size-256(DMA)          0      0    256   15    1 : tunables 120   60    8 : slabdata      0      0      0
size-256             487    495    256   15    1 : tunables 120   60    8 : slabdata     33     33      0
size-128(DMA)          0      0    128   30    1 : tunables 120   60    8 : slabdata      0      0      0
size-128            2242   2490    128   30    1 : tunables 120   60    8 : slabdata     83     83      0
size-64(DMA)           0      0     64   59    1 : tunables 120   60    8 : slabdata      0      0      0
size-32(DMA)           0      0     32 113    1 : tunables 120   60    8 : slabdata      0      0      0
size-64             1409   2950     64   59    1 : tunables 120   60    8 : slabdata     50     50      0
size-32             3596   3842     32 113    1 : tunables 120   60    8 : slabdata     34     34      0
kmem_cache           145    150    256   15    1 : tunables 120   60    8 : slabdata     10     10      0
[root@localhost ~]# slabtop -d 5
Active / Total Objects (% used)    : 97257 / 113249 (85.9%)
Active / Total Slabs (% used)      : 4488 / 4488 (100.0%)
Active / Total Caches (% used)     : 101 / 146 (69.2%)
Active / Total Size (% used)       : 15076.34K / 17587.55K (85.7%)
Minimum / Average / Maximum Object : 0.01K / 0.16K / 128.00K
OBJS ACTIVE USE OBJ SIZE SLABS OBJ/SLAB CACHE SIZE NAME
25776 25764 99%    0.05K    358       72      1432K buffer_head
16146 15351 95%    0.08K    351       46      1404K vm_area_struct
15138   7779 51%    0.13K    522       29      2088K dentry_cache
9720   9106 93%    0.19K    486       20      1944K filp
7714   7032 91%    0.27K    551       14      2204K radix_tree_node
5070   5018 98%    0.05K     65       78       260K sysfs_dir_cache
4826   4766 98%    0.01K     19      254        76K anon_vma
4824   3406 70%    0.48K    603        8      2412K ext3_inode_cache
3842   3691 96%    0.03K     34      113       136K size-32
2190   2174 99%    0.12K     73       30       292K size-128
1711   1364 79%    0.06K     29       59       116K size-64
1210   1053 87%    0.33K    110       11       440K inode_cache
1196   1147 95%    0.04K     13       92        52K Acpi-Operand
1170    814 69%    0.05K     15       78        60K selinux_inode_security
   936    414 44%    0.05K     13       72        52K journal_head
   747    738 98%    0.43K     83        9       332K shmem_inode_cache
   693    617 89%    0.35K     63       11       252K proc_inode_cache
   676    615 90%    0.02K      4      169        16K Acpi-Namespace
   609    136 22%    0.02K      3      203        12K biovec-1
   495    493 99%    0.25K     33       15       132K size-256
   480    384 80%    0.12K     16       30        64K bio
   440    399 90%    0.50K     55        8       220K size-512
   312    206 66%    0.05K      4       78        16K delayacct_cache
   303    209 68%    0.04K      3      101        12K pid
   290    290 100%    0.38K     29       10       116K sock_inode_cache
[root@localhost ~]# cat /etc/sysctl.conf
# Kernel sysctl configuration file for Red Hat Linux
# Controls IP packet forwarding
net.ipv4.ip_forward=0
# Controls source route verification
net.ipv4.conf.default.rp_filter=1
# Do not accept source routing
net.ipv4.conf.default.accept_source_route=0
# Oracle
net.ipv4.ip_local_port_range=1024 65000
net.core.rmem_default=4194304
net.core.rmem_max=4194304
net.core.wmem_default=262144
net.core.wmem_max=262144
net.ipv4.tcp_rmem=4096 65536 4194304
net.ipv4.tcp_wmem=4096 65536 4194304
# Keepalive Oracle
net.ipv4.tcp_keepalive_time=3000
net.ipv4.tcp_keepalive_intvl=30
net.ipv4.tcp_keepalive_probes=15
net.ipv4.tcp_retries2=3
net.ipv4.tcp_syn_retries=2
net.ipv4.tcp_sack=0
net.ipv4.tcp_timestamps=0
net.ipv4.tcp_window_scaling=0
# Oracle
fs.file-max = 6553600
fs.aio-max-nr=3145728
kernel.shmmni=4096
kernel.sem=250 32000 100 142
kernel.shmmax=2147483648
kernel.shmall=3279547
kernel.msgmnb=65536
kernel.msgmni=2878
kernel.msgmax=8192
kernel.exec-shield=0
# Controls the System Request debugging functionality of the kernel
kernel.sysrq=1
kernel.panic=60
kernel.core_uses_pid=1
[root@localhost ~]# free | grep Swap
Swap:      3148700     319916    2828784
[root@localhost ~]# cat /etc/fstab | grep "/dev/shm"
tmpfs                   /dev/shm                tmpfs   size=1024M      0 0
[root@localhost ~]# df | grep "/dev/shm"
tmpfs                  1048576    452128    596448 44% /dev/shm
NON-DEFAULT DB PARAMETERS:
db_block_size        8192
memory_target          633339904 /* automatic memory management */
open_cursors         300
processes            256
disk_async_io        TRUE
filesystemio_options SETALL

2012 Leap Second -- 100% CPU Usage Excessive, EM DBConsole 11g on OEL6

This occurs on RHEL 6 with 11g 11.2.0.3
Enterprise Manager Management Agent or OMS CPU Use Is Excessive on or around July 1, 2012 (Doc ID 1472651.1)
Leap Second Hang - CPU Can Be Seen at 100% (Doc ID 1472421.1) Affects Java processes.
According to these notes issue should be fixed by either restarting host or running the following commands:
# /etc/init.d/ntpd stop
# date -s "`date`" (reset the system clock)
# /etc/init.d/ntpd start
example
/etc/init.d/ntpd stop
date -s "2012-07-10 14:42:42"
/etc/init.d/ntpd start
July 1st introduction of leap second for leap year update to linux
Distributions using NTP (Network Time Protocol)see cpu drop after fix. Affects java processes.

This occurs on RHEL 6 with 11g 11.2.0.3
Enterprise Manager Management Agent or OMS CPU Use Is Excessive on or around July 1, 2012 (Doc ID 1472651.1)
Leap Second Hang - CPU Can Be Seen at 100% (Doc ID 1472421.1) Affects Java processes.
According to these notes issue should be fixed by either restarting host or running the following commands:
# /etc/init.d/ntpd stop
# date -s "`date`" (reset the system clock)
# /etc/init.d/ntpd start
example
/etc/init.d/ntpd stop
date -s "2012-07-10 14:42:42"
/etc/init.d/ntpd start
July 1st introduction of leap second for leap year update to linux
Distributions using NTP (Network Time Protocol)see cpu drop after fix. Affects java processes.

CPU Jul-09 Installation - Error Cannot copy file from 'libjox10.so' to

Hi All, I received this error while I was installing the last CPU on a 10.2.0.4 Oracle Instance running on a HP-UX Itanium System:
Patch 8568398:
Copy Action: Desctination File "/u02/app/oracle/product/10.2/Db_1/lib/libjox10.so" is not writeable.
'oracle.rdbms, 10.2.0.4.0': Cannot copy file from 'libjox10.so' to '/u02/app/oracle/product/10.2/Db_1/lib/libjox10.so'
I checked metalink' site with no similar results found.
Could anybody help me?
Should I run /usr/sbin/slibclean and remove the last patch (Patch 8568398) and then try again?
This is the complete log...
$ opatch version
Invoking OPatch 10.2.0.4.7
OPatch Version: 10.2.0.4.7
OPatch succeeded.
$ sqlplus /nolog
SQL*Plus: Release 10.2.0.4.0 - Production on Fri Jul 17 16:29:29 2009
Copyright (c) 1982, 2007, Oracle. All Rights Reserved.
SQL> conn / as sysdba
Connected.
SQL> shutdown immediate
Database closed.
Database dismounted.
ORACLE instance shut down.
SQL> exit
Disconnected from Oracle Database 10g Enterprise Edition Release 10.2.0.4.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options
$ lsnrctl stop
LSNRCTL for HPUX: Version 10.2.0.4.0 - Production on 17-JUL-2009 16:32:29
Copyright (c) 1991, 2007, Oracle. All rights reserved.
Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=Hostname)(PORT=1521)))
The command completed successfully
$ emctl stop dbconsole
Oracle Enterprise Manager 10g Database Control Release 10.2.0.4.0
Copyright (c) 1996, 2007 Oracle Corporation. All rights reserved.
http://hostname:1158/em/console/aboutApplication
Stopping Oracle Enterprise Manager 10g Database Control ...
... Stopped.
$unzip p8534387_10204_HPUX-IA64.zip
<...>
$ cd 8534387
$ opatch napply -skip_subset -skip_duplicate
Invoking OPatch 10.2.0.4.7
Oracle Interim Patch Installer version 10.2.0.4.7
Copyright (c) 2009, Oracle Corporation. All rights reserved.
UTIL session
Oracle Home : /u02/app/oracle/product/10.2/Db_1
Central Inventory : /u02/app/oracle/oraInventory
from : /var/opt/oracle/oraInst.loc
OPatch version : 10.2.0.4.7
OUI version : 10.2.0.4.0
OUI location : /u02/app/oracle/product/10.2/Db_1/oui
Log file location : /u02/app/oracle/product/10.2/Db_1/cfgtoollogs/opatch/opatch2 009-07-17_16-38-05PM.log
Patch history file: /u02/app/oracle/product/10.2/Db_1/cfgtoollogs/opatch/opatch_ history.txt
Invoking utility "napply"
Checking conflict among patches...
Checking if Oracle Home has components required by patches...
Checking skip_duplicate
Checking skip_subset
Checking conflicts against Oracle Home...
OPatch continues with these patches: 7155248 7155249 7155250 7155251 7155252 7155253 7155254 7197583 7375611 7375613 7375617 7609057 7609058 8309587 8309592 8309623 8309632 8309637 8309642 8534387 8568395 8568397 8568398 8568402 8568404 8568405
Do you want to proceed? y
y
User Responded with: Y
Running prerequisite checks...
Prerequisite check "CheckApplicable" failed.
The details are:
Patch 8568398:
Copy Action: Desctination File "/u02/app/oracle/product/10.2/Db_1/lib/libjox10.so" is not writeable.
'oracle.rdbms, 10.2.0.4.0': Cannot copy file from 'libjox10.so' to '/u02/app/oracle/product/10.2/Db_1/lib/libjox10.so'
UtilSession failed: Prerequisite check "CheckApplicable" failed.
$ opatch lsinventory -detail
Invoking OPatch 10.2.0.4.7
Oracle Interim Patch Installer version 10.2.0.4.7
Copyright (c) 2009, Oracle Corporation. All rights reserved.
Oracle Home : /u02/app/oracle/product/10.2/Db_1
Central Inventory : /u02/app/oracle/oraInventory
from : /var/opt/oracle/oraInst.loc
OPatch version : 10.2.0.4.7
OUI version : 10.2.0.4.0
OUI location : /u02/app/oracle/product/10.2/Db_1/oui
Log file location : /u02/app/oracle/product/10.2/Db_1/cfgtoollogs/opatch/opatch2009-07-17_17-53-30PM.log
Patch history file: /u02/app/oracle/product/10.2/Db_1/cfgtoollogs/opatch/opatch_history.txt
Lsinventory Output file location : /u02/app/oracle/product/10.2/Db_1/cfgtoollogs/opatch/lsinv/lsinventory2009-07-17_17-53-30PM.txt
Installed Top-level Products (2):
Oracle Database 10g 10.2.0.1.0
Oracle Database 10g Release 2 Patch Set 3 10.2.0.4.0
There are 2 products installed in this Oracle Home.
Installed Products (190):
Agent Required Support Files 10.2.0.1.0
Agent Required Support Files Patch 10.2.0.4.0
Assistant Common Files 10.2.0.1.0
Assistant Common Files Patch 10.2.0.4.0
Bali Share 1.1.18.0.0
Buildtools Common Files 10.2.0.1.0
Character Set Migration Utility 10.2.0.1.0
Character Set Migration Utility Patch 10.2.0.4.0
Database Configuration and Upgrade Assistants 10.2.0.1.0
Database Configuration and Upgrade Assistants Patch 10.2.0.4.0
Database SQL Scripts 10.2.0.1.0
Database SQL Scripts Patch 10.2.0.4.0
Database Workspace Manager 10.2.0.1.0
Database Workspace Manager 10.2.0.4.0
DBJAVA Required Support Files 10.2.0.1.0
DBJAVA Required Support Files Patch 10.2.0.4.0
Enterprise Edition Options 10.2.0.1.0
Enterprise Manager Agent Core 10.2.0.1.0
Enterprise Manager Agent Core Patch 10.2.0.4.0a
Enterprise Manager Agent DB 10.2.0.1.0
Enterprise Manager Agent DB 10.2.0.4.0
Enterprise Manager Baseline 10.2.0.1.0
Enterprise Manager Baseline 10.2.0.4.0
Enterprise Manager Common Core Patch 10.2.0.4.0a
Enterprise Manager Common Files 10.2.0.1.0
Enterprise Manager Minimal Integration 10.2.0.1.0
Enterprise Manager plugin Common Files 10.2.0.1.0
Enterprise Manager plugin Common Files 10.2.0.1.0
Enterprise Manager plugin Common Files Patch 10.2.0.4.0
Enterprise Manager Repository Core 10.2.0.1.0
Enterprise Manager Repository Core patch 10.2.0.4.0a
Enterprise Manager Repository DB 10.2.0.1.0
Enterprise Manager Repository DB 10.2.0.4.0
Generic Connectivity Common Files 10.2.0.1.0
Generic Connectivity Common Files Patch 10.2.0.4.0
HAS Common Files 10.2.0.1.0
HAS Common Files Patch 10.2.0.4.0
HAS Files for DB 10.2.0.1.0
HAS Files for DB Patch 10.2.0.4.0
Installation Common Files 10.2.0.1.0
Installation Common Files Patch 10.2.0.4.0
Installer SDK Component 10.2.0.4.0
iSQL*Plus 10.2.0.1.0
iSQL*Plus 10.2.0.4.0
Java Runtime Environment 1.4.2.14.0
JDBC Common Files 10.2.0.1.0
JDBC Common Files 10.2.0.4.0
LDAP Required Support Files 10.2.0.1.0
LDAP Required Support Files Patch 10.2.0.4.0
OLAP SQL Scripts 10.2.0.1.0
OLAP SQL Scripts Patch 10.2.0.4.0
Oracle Advanced Security 10.2.0.1.0
Oracle Advanced Security Patch 10.2.0.4.0
Oracle Call Interface (OCI) 10.2.0.1.0
Oracle Call Interface (OCI) Patch 10.2.0.4.0
Oracle Clusterware RDBMS Files 10.2.0.1.0
Oracle Clusterware RDBMS Files Patch 10.2.0.4.0
Oracle Code Editor 1.2.1.0.0I
Oracle Configuration Manager 10.2.7.1.0
Oracle Containers for Java 10.2.0.1.0
Oracle Containers for Java 10.2.0.4.0
Oracle Core Required Support Files 10.2.0.1.0
Oracle Core Required Support Files Patch 10.2.0.4.0
Oracle Data Mining RDBMS Files 10.2.0.1.0
Oracle Data Mining RDBMS Files Patch 10.2.0.4.0
Oracle Database 10g 10.2.0.1.0
Oracle Database 10g 10.2.0.1.0
Oracle Database 10G 32 bit 10.1.0.2.0
Oracle Database 10g interMedia Files 10.2.0.1.0
Oracle Database 10g interMedia Files Patch 10.2.0.4.0
Oracle Database 10g Patch 10.2.0.4.0
Oracle Database 10g Patch 10.2.0.4.0
Oracle Database 10g Release 2 Patch Set 3 10.2.0.4.0
Oracle Database User Interface 2.2.13.0.0
Oracle Database Utilities 10.2.0.1.0
Oracle Database Utilities Patch 10.2.0.4.0
Oracle Display Fonts 9.0.2.0.0
Oracle Enterprise Manager Console DB 10.2.0.1.0
Oracle Enterprise Manager Console DB 10.2.0.4.0
Oracle Extended Windowing Toolkit 3.4.38.0.0
Oracle Globalization Support 10.2.0.1.0
Oracle Globalization Support Patch 10.2.0.4.0
Oracle Help For Java 4.2.6.1.0
Oracle Help for the Web 1.1.10.0.0
Oracle Ice Browser 5.2.3.6.0
Oracle interMedia 10.2.0.1.0
Oracle interMedia Annotator 10.2.0.1.0
Oracle interMedia Client Option 10.2.0.1.0
Oracle interMedia Client Option Patch 10.2.0.4.0
Oracle interMedia Java Advanced Imaging 10.2.0.1.0
Oracle interMedia Java Advanced Imaging Patch 10.2.0.4.0
Oracle interMedia Locator 10.2.0.1.0
Oracle interMedia Locator Patch 10.2.0.4.0
Oracle interMedia Locator RDBMS Files 10.2.0.1.0
Oracle interMedia Locator RDBMS Files Patch 10.2.0.4.0
Oracle interMedia Patch 10.2.0.4.0
Oracle Internet Directory Client 10.2.0.1.0
Oracle Internet Directory Client Patch 10.2.0.4.0
Oracle Java Client 10.2.0.1.0
Oracle Java Client Patch 10.2.0.4.0
Oracle JDBC Thin Driver for JDK 1.2 10.2.0.1.0
Oracle JDBC Thin Driver for JDK 1.2 Patch 10.2.0.4.0
Oracle JDBC Thin Driver for JDK 1.4 10.2.0.1.0
Oracle JDBC Thin Driver for JDK 1.4 Patch 10.2.0.4.0
Oracle JDBC/OCI Instant Client 10.2.0.1.0
Oracle JDBC/OCI Instant Client Patch 10.2.0.4.0
Oracle JFC Extended Windowing Toolkit 4.2.33.0.0
Oracle JVM 10.2.0.1.0
Oracle JVM Patch 10.2.0.4.0
Oracle LDAP administration 10.2.0.1.0
Oracle LDAP administration patch 10.2.0.4.0
Oracle Locale Builder 10.2.0.1.0
Oracle Message Gateway Common Files 10.2.0.1.0
Oracle Message Gateway Common Files Patch 10.2.0.4.0
Oracle Net 10.2.0.1.0
Oracle Net Listener 10.2.0.1.0
Oracle Net Listener Patch 10.2.0.4.0
Oracle Net Patch 10.2.0.4.0
Oracle Net Required Support Files 10.2.0.1.0
Oracle Net Required Support Files Patch 10.2.0.4.0
Oracle Net Services 10.2.0.1.0
Oracle Notification Service 9.0.4.0.0
Oracle Notification Service Patch 10.2.0.4.0
Oracle OLAP 10.2.0.1.0
Oracle OLAP API 10.2.0.1.0
Oracle OLAP API Patch 10.2.0.4.0
Oracle OLAP Patch 10.2.0.4.0
Oracle OLAP RDBMS Files 10.2.0.1.0
Oracle OLAP RDBMS Files Patch 10.2.0.4.0
Oracle One-Off Patch Installer 10.2.0.4.0
Oracle Partitioning 10.2.0.1.0
Oracle Partitioning Patch 10.2.0.4.0
Oracle Programmer 10.2.0.1.0
Oracle RAC Required Support Files-HAS 10.2.0.1.0
Oracle RAC Required Support Files-HAS Patch 10.2.0.4.0
Oracle Real Application Testing 10.2.0.4.0
Oracle Recovery Manager 10.2.0.1.0
Oracle Recovery Manager Patch 10.2.0.4.0
Oracle Required Support Files 32 bit 10.2.0.0.0
Oracle Required Support Files 32 bit Patch 10.2.0.4.0
Oracle Spatial 10.2.0.1.0
Oracle Starter Database 10.2.0.1.0
Oracle Starter Database Patch 10.2.0.4.0
Oracle Text 10.2.0.1.0
Oracle Text Patch 10.2.0.4.0
Oracle UIX 2.1.22.0.0
Oracle Universal Installer 10.2.0.4.0
Oracle Wallet Manager 10.2.0.1.0
Oracle Wallet Manager Patch 10.2.0.4.0
Oracle XML Development Kit 10.2.0.1.0
Oracle XML Development Kit Patch 10.2.0.4.0
Parser Generator Required Support Files 10.2.0.1.0
Perl Interpreter 5.8.3.0.1
Perl Interpreter Patch 5.8.3.0.1a
PL/SQL 10.2.0.1.0
PL/SQL 10.2.0.4.0
PL/SQL Embedded Gateway 10.2.0.1.0
PL/SQL Embedded Gateway Patch 10.2.0.4.0
Platform Required Support Files 10.2.0.1.0
Platform Required Support Files 10.2.0.4.0
Precompiler Common Files 10.2.0.1.0
Precompiler Common Files Patch 10.2.0.4.0
Precompiler Required Support Files 10.2.0.1.0
Precompiler Required Support Files Patch 10.2.0.4.0
RDBMS Required Support Files 10.2.0.1.0
RDBMS Required Support Files for Instant Client 10.2.0.1.0
RDBMS Required Support Files for Instant Client Patch 10.2.0.4.0
RDBMS Required Support Files Patch 10.2.0.4.0
regexp 2.1.9.0.0
Required Support Files 10.2.0.1.0
Sample Schema Data 10.2.0.1.0
Sample Schema Data Patch 10.2.0.4.0
Secure Socket Layer 10.2.0.1.0
Secure Socket Layer Patch 10.2.0.4.0
SQL*Plus 10.2.0.1.0
SQL*Plus 10.2.0.4.0
SQL*Plus Required Support Files 10.2.0.1.0
SQL*Plus Required Support Files Patch 10.2.0.4.0
SQLJ Runtime 10.2.0.1.0
SQLJ Runtime Patch 10.2.0.4.0
SSL Required Support Files for InstantClient 10.2.0.1.0
SSL Required Support Files for InstantClient Patch 10.2.0.4.0
Sun JDK 1.4.2.17.0
Sun JDK extensions 9.2.0.1.0
XDK Required Support Files 10.2.0.1.0
XDK Required Support Files Patch 10.2.0.4.0
XML Parser for Java 10.2.0.1.0
XML Parser for Java Patch 10.2.0.4.0
XML Parser for Oracle JVM 10.2.0.1.0
XML Parser for Oracle JVM Patch 10.2.0.4.0
There are 190 products installed in this Oracle Home.
There are no Interim patches installed in this Oracle Home.
OPatch succeeded.
$
$ more /u02/app/oracle/product/10.2/Db_1/cfgtoollogs/opatch/opatch_history.txt
Date & Time : Fri Jul 17 16:02:17 GMT-03:00 2009
Oracle Home : /u02/app/oracle/product/10.2/Db_1
OPatch Ver. : 10.2.0.4.7
Current Dir : /u02/app/oracle/product/10.2/Db_1
Command : -v
Log File : /u02/app/oracle/product/10.2/Db_1/cfgtoollogs/opatch/opatch2009-07-17_16-02-17PM.log
Date & Time : Fri Jul 17 16:38:05 GMT-03:00 2009
Oracle Home : /u02/app/oracle/product/10.2/Db_1
OPatch Ver. : 10.2.0.4.7
Current Dir : /home/ora10g/8534387
Command : napply -skip_subset -skip_duplicate
Log File : /u02/app/oracle/product/10.2/Db_1/cfgtoollogs/opatch/opatch2009-07-17_16-38-05PM.log
Date & Time : Fri Jul 17 17:53:30 GMT-03:00 2009
Oracle Home : /u02/app/oracle/product/10.2/Db_1
OPatch Ver. : 10.2.0.4.7
Current Dir : /home/ora10g/8534387
Command : lsinventory -detail
Log File : /u02/app/oracle/product/10.2/Db_1/cfgtoollogs/opatch/opatch2009-07-17_17-53-30PM.log
$
Please let me know if you need more info.
Regards, and thanks in advance!
Martin

You were right. It was a permissions issue:
hostname:/root# ls -ltr /u02/app/oracle/product/10.2/Db_1/lib/libjox10.so
-r-xr-xr-x 1 ora10g oinstall 20041728 Apr 23 2008 /u02/app/oracle/product/10.2/Db_1/lib/libjox10.so
Thanks.
damorgan wrote:
"is not writeable." means one of two things.
1. It does not exist.
2. Permissions are not set correctly.
Determining which it is ... that is the first step.

Is my CPU really overloaded....why all the crashes?

I'm on a quad core G5 with oodles of RAM running 10 or so tracks of pretty big multilayered Audio Instruments. My monitors say I've plenty of RAM left and plenty of CPU power to spare. In fact only 3 of the 4 processors seem to be being used. But I keep crashing and getting Core Audio overload error messages. This happens upon loading the song too as the samples load. Whatever song I create runs into the same problem at about the same stage of it's development. Any ideas?
G5 Quad Core Mac OS X (10.4.6) 16GB RAM
G5 Quad Core Mac OS X (10.4.6) 16GB RAM
G5 Quad Core Mac OS X (10.4.6) 16GB RAM
G5 Quad Core Mac OS X (10.4.6) 16GB RAM

Thanks both!
My songs are on my primary hard drive with Logic application and with the external sample players I'm using (Kompakt with East/West Symphonic Orchestra).
The samples libraries are on the second hard drive. Should the songs be there too?
Justin, what do you mean by search benchmark song 2. Sorry for being dense here! Many thanks. I'd heard of Logic only assigning itself a fixed amount of RAM...one of those small but crucial bits of infor that apple and Logic don't like to announce I imagine...
Thanks.

CPU consumption 100%: How do I see which SQL is consuming?

Hi,
I need help badly with this because I cannot get my head around the problem. I have a Windows server 2003 Enterprise 64-bit running Oracle 10.2.0.3 and last night it started consuming 100% CPU. The Windows performance monitor shows 100% CPU in any case, and it was difficult to even login via remote desktop.
It is Oracle.exe that is consuming, and there is only one database on this server. It is an Opteron 875 2.2Ghz with 8GB ram.
I want to see which SQL statement is responsible for the CPU consumption. I know exactly which user/programapplication is using it... it is useless to kick the session because the application connects immediately again from the app. server and runs the same thing. Restarting the database also has the same effect: session reconnects and CPU goes through the roof. Customer says that they are running the same stuff as always, and app. has not been updated recently. 100 % CPU to them, means something is wrong.
Incidently, I tried to CPU patch this database a couple of days beforehand, but there was not enough space on disk (needed 1.6GB) so opatch complained and would not continue. I simply restarted the database without having run opatch. I cannot imagine that this is the cause of this problem.
Anyhoo...
I have a script from Kyle Haileys web site to get the CPU for each sql_id using ash tables:
-- (c) Kyle Hailey 2007
col type for a10
select * from (
select
ash.SQL_ID , ash.SQL_PLAN_HASH_VALUE Plan_hash, aud.name type,
sum(decode(ash.session_state,'ON CPU',1,0)) "CPU",
sum(decode(ash.session_state,'WAITING',1,0)) -
sum(decode(ash.session_state,'WAITING', decode(wait_class, 'User I/O',1,0),0)) "WAIT" ,
sum(decode(ash.session_state,'WAITING', decode(wait_class, 'User I/O',1,0),0)) "IO" ,
sum(decode(ash.session_state,'ON CPU',1,1)) "TOTAL"
from v$active_session_history ash,
audit_actions aud
where SQL_ID is not NULL
and ash.sql_opcode=aud.action
and ash.sample_time > sysdate - &minutes /( 60*24)
group by sql_id, SQL_PLAN_HASH_VALUE , aud.name
order by sum(decode(session_state,'ON CPU',1,1)) desc
) where rownum < 10
This give following output:
SQL_ID PLAN_HASH TYPE CPU WAIT IO TOTAL
cxk376gut06wn 3831248992 SELECT 12 145 5855 6012
4ffvdt72n46yt 4094592372 SELECT 0 3 1131 1134
dnxgm7bg8wqg1 4025412530 SELECT 3 13 337 353
3y0dyvkc9sx69 1764840833 SELECT 18 169 56 243
abz05715h08ct 1967470119 SELECT 0 7 175 182
4gd6b1r53yt88 0 UNKNOWN 0 92 4 96
9phhuzxpqwsgc 2662599208 SELECT 0 67 0 67
275bbj5czbrwp 629013258 SELECT 0 2 64 66
1dvrrg3aah2mu 75914095 SELECT 0 6 57 63
My assumption is that this gives the average CPU usage over the last x minutes and the total cpu usage for this sql_id. There are no sql running that show a very high average CPU, but the total is up in 6000 for the longest running sql, so I get the statement for the top sql_id that has highest total, and it is this:
SQL_ID cxk376gut06wn, child number 2
SELECT NVL(MIN(ID),0) FROM TABLE_OBFUSCATED WHERE TASK = :B1 AND
STATUS='JobQueue' AND MESSAGE_ID IS NULL
Plan hash value: 3831248992
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
| 0 | SELECT STATEMENT | | | | 3231 (100)| |
| 1 | SORT AGGREGATE | | 1 | 42 | | |
|* 2 | TABLE ACCESS FULL| TABLE_OBFUSCATED | 3 | 126 | 3231 (1)| 00:00:39 |
Predicate Information (identified by operation id):
2 - filter(("TASK"=:B1 AND "STATUS"='JobQueue' AND "MESSAGE_ID" IS NULL))
It doesn't seem to fit! It's such a simple statement, but according to ash this is the sql using the most total cpu over the last 30 minutes?? So I create a temp index on the filter and check the plan for the statement again, and it uses the index, but I need to kill the session or bounce the database for the new plan to be used because the currently running session is still using the old plan. Customer was desperate to get it down so I even tried flushing the shared pool but new plan does not get used. I did not want to restart the database. Do I have to kill all sessions using that plan for the new one to take effect?
Now im stuck. I don't really know how else to get an accurate picture of which sql is currently consuming CPU. I need to know this to try to fix the SQL because stopping the application is not an option, even if it is throwing too much at the database. The only other option I can think of is setting up profile limits for cpu consumption in oracle, but then the job will likely take an unacceptable amount of time for the customer.
So my questions are:
1. How else can I check which SQL is using CPU accurately?
2. Does the SQL I show above accuratlely display current average CPU over x minutes with ASH? Why is it so low for the top consumer but my CPU is still at 100% on the OS?
Thankyou for your help.

Yeah, OK, I can see an arbitary number for CPU used since session startup (what does that refer to by the way? It can't be percent), and I can see that one user has consumed massive amounts of CPU compared to all other sessions, but he is currently not running anything, so how can I see which SQL he ran in the past day that cuased all that CPU consumption? Can I do this without AWR?
By the way, do you need a Diagnostic Pack license for simply viewing the Performance tab of dbconsole? I Can go into dbconsole and see history for 2 days back and see the massive espike that caused CPU consumption, but when dbconsole displays the list of top SQL and top sessions under the graph, is it the top listed SQL that consumed the most cpu or do you need to correlate the colours, i.e. find the SQL listed in top SQL that has the same colour as CPU usage on the graph (green) even if it is not listed as the top "consumer" (which I assume means top AAS consumer: overall consumption of all resources)?

Oracle internal queries taking more CPU time

Hi,
Following are queries taking more CPU time. I got this from awr report. Can anyone tell me why these queries are running? Is it a oracle enterprise manager query? should I use
emctl stop dbconsole to stop this?
DECLARE job BINARY_INTEGER := :job; next_date DATE := :mydate; broken BOOLEAN := FALSE; BEGIN EMD_MAINTENANCE.EXECUTE_EM_DBMS_JOB_PROCS(); :mydate := next_date; IF broken THEN :b := 1; ELSE :b := 0; END IF; END;
SELECT COUNT(*) FROM MGMT_METRIC_DEPENDENCY_DETAILS DEP, MGMT_SEVERITY SEV WHERE DEP.TARGET_GUID = :B5 AND DEP.METRIC_GUID = :B4 AND DEP.KEY_VALUE = :B3 AND DEP.EDEP_TARGET_GUID = SEV.TARGET_GUID AND DEP.EDEP_METRIC_GUID = SEV.METRIC_GUID AND DEP.DEP_KEY_VALUE = SEV.KEY_VALUE AND SEV.COLLECTION_TIMESTAMP BETWEEN :B2 AND :B1
SELECT CURRENT_STATUS FROM MGMT_CURRENT_AVAILABILITY WHERE TARGET_GUID = :B1
Thanks in advance
With Regards
boobathi.P

Hi,
maybe this document will help if you are using 10g:
SQL run by SYSMAN consuming a lot of resources on OMS with 800+ targets [ID 330383.1]
https://support.oracle.com/CSP/main/article?cmd=show&type=NOT&doctype=PROBLEM&id=330383.1
there you'll find Cause and Solution too:
SYSMAN Job and Queries are Taking Up High CPU (DB Console) [ID 1288301.1]
https://support.oracle.com/CSP/main/article?cmd=show&type=NOT&doctype=PROBLEM&id=1288301.1
For databases above version 11.1 there are paches available.
Best,
Michael T. Z.

Australia (East) hosted websites keep reporting they are not available

Hey People,
anyone else having fun with Azure websites hosted in Australia? In particular the Sydney (East) datacentre?
I've been creating and destroying Azure websites today, testing various things with the different CMS platforms. But this afternoon, my three active sites all started to fail:
Then they'd come back after 10+ minutes, but die again after about 30 seconds.
Acording to the Azure management console, I've got plenty of credit left and my CPU/etc usage is all well inside my quota (my subscription is an MSDN Premium one).
The Azure website I have hosted in Singapore (Asia South East) is fine. Just these Aussie ones have problems.
Any ideas? Are they having problems? The Portal says the service is up and the Mangaement console says my sites are up.
Am I missing something?
Thanks
Craig

Hi,
Microsoft has opened their two newest Azure regions on 27 October 2014 in Australia as detailed in their press release at New Microsoft Azure Geo opens for business in Australia, I think these two datacenter will give us more choices to improve performance,
if encounter some server side issue when use it, we can contact with the support to get the detailed reasons, here is the support channel:
http://www.windowsazure.com/en-us/support/contact/
Sorry for any inconvenience.
Best Regards,
Jambor
We are trying to better understand customer views on social support experience, so your participation in this interview project would be greatly appreciated if you have time. Thanks for helping make community forums a great place.
Click
HERE to participate the survey.

New Archived log entry in alert log after cpu patch 19 applied

Has anyone applied October critical patch for 11.1.0.7? I applied the cpu Patch 19 (8892977) for 11.1.0.7 on several windows2008 servers and after the patch the alert log displays a new message everytime a redo log transfers. This only happens on servers that have archivelog on. It looks like there is some kind of trace set. Anyone know know what the message is?
Thanks,
Kathie
Archived Log entry 10937 added for thread 1 sequence 10938 ID 0xa96e3908 dest 10:

Hi Mark!
Yes, Oracle informed me that this message is normal and is only used for formational purposes. I should just ignore it.
Did you apply patch19? I have also found issues with a high amount of "waits" of type of "other". Some jobs are running slightly slower also. Also, if you check your "database health link" on dbconsole home page you will see consistant "non-critical" alerts. I don't have these alerts on servers not yet patched. This alerts appear to be caused by wait events of "other".
Kathie

High CPU utalization

dear apps dbas,
i montior EM on daily basis and i always see the following takes high CPU dose any one know what are they.
1-WF_EVENT_OJMSTEXT_QH.DEQUEUE(:1, :2, :3); END;
2-BEGIN IEO_ICSM_AQ_PUB.DEQUEUE_REQUEST( p_api_version => 1.0 , p_node_id => :1 , p_wait => :2 , x_request_id => :3 , x_xml_data => :4 , x_return_status => :5 , x_msg_count => :6 , x_msg_data => :7); END;
3-begin JTF_FM_AQ_WRAPPER_PVT.DEQUEUE(:1,:2,:3,:4); end;
4-fndcp_tmsrv.read_message(....)
thanks for support
fadi

Fadi,
We have couple of custom packages/procedures that need to be reloaded several times during the instance life. You can automate pinning objects in Shared Pool at database startup by referring to Note: 101627.1 (How to Automate Pinning Objects in Shared Pool at Database Startup).
The note I have mentioned in my previous reply gives some details about the scripts which are used for pinning objects:
ADXCKPIN.sql: Query the shared_pool area to determine space used by PL/SQL objects and whether they have been pinned.
ADXGNPIN.sql: Creates and runs a "pin" script for all packages and functions in a give schema.
The script $AD_TOP/sql/ADXGNPIN.sql generate pinning scripts, which can be executed in SQL*Plus. Make sure you edit the script before running it. Otherwise it would try to pin all objects.
To get more details about the above two scripts, please refer to "Oracle Applications Maintenance Procedures" Manual.
http://download-east.oracle.com/docs/cd/B25284_01/current/acrobat/11iadproc.pdf

Linux Install - ulimit, dbconsole

Hello- brand new install of 11g Standard Edition on Ubuntu Linux...
I read a helpful startup script and entered most all of the configs by hand before install.. reading the logs I can see that oracle starts up and has been configured.. though the configure app did end up complaining and did not finish completely (java and oracle proceeded to keep the dual CPU box at load 5 for the next 9 hours while I slept !!)
so now I go to start the DB and a console.. two problems:
* ulimit - whether in .bashrc, or in /etc/security/limits.conf.. I cannot get these ulimits right.. I currently have, user 'oracle'
oracle soft nproc 2047
oracle hard nproc 16384
oracle soft nofile 1024
oracle hard nofile 65536
dbstart runs without error, but
when I run emctl start dbconsole , I get the msg -- ulimit: 25: Bad Number
followed by
OC4J Configuration issue. /u01/app/oracle/product/11.1.0/db_1/oc4j/j2ee/OC4J_DBConsole_hps-desktop_lin11 not found
well, I am not used to the terms yet.. my ORACLE_SID = lin11
but I know I have a database called orcl
so looking in the directory, I see
OC4J_DBConsole_hps-desktop_orcl
hmm.. dont know what to make of that mixup
clarifications for Up and Running Install welcome
thanks

I'm also using Ubuntu and get the same error message when running emctl:
ulimit: 25: bad number
I traced the problem to the file $ORACLE_HOME/bin/commonenv which is sourced by emctl.
The offending line is line 23: ulimit -S -c hard
If you type this line into bash, it works just fine, but emctl begins with the line #!/bin/sh, which on Ubuntu systems is symbolically linked to /bin/dash. Now, ulimit is a built-in shell command, not an executable file, and bash and dash seem to have a different syntax. If you type "dash" at a shell prompt to start a new dash shell, and type that particular ulimit command, it will complain about a bad number. (I'm not sure what the "25" represents, that number seems to be related to the number of lines in the shell script in which the command is found.) Be sure to type "exit" when finishing this test to return to bash or whatever your preferred shell is.
Two things to note: this ulimit command is related to core file sizes and its failure shouldn't cause any problems, so you can probably safely ignore the error messages. Secondly, Ubuntu is not an officially supported OS for Oracle, so this problem may not exist with Suse or RHEL, I don't know. I am running Oracle on Ubuntu, and it seems to work fine, although installation was a bit of a headache.

DBConsole error on startup

Hi!!
I need help in start dbconsole on linux. I try this command:
# emctl start dbconsole
So i have a error message:
TZ set to Brazil/East
OC4J Configuration issue. /u01/app/oracle/product/10.2.0.1/db_1/oc4j/j2ee/OC4J_DBConsole_localhost.localdomain_orcl not found.
What I do wrong??
Oracle 10g (10.2.0.1)
SO (Oracle Linux)
Carl

Hello,
This error is not directly related to OC4J but more a DB/EM issue. I am inviting your to post you question in:
- General Database Discussions
- Enterprise Manager
Regards
Tugdual Grall

Starting dbconsole kills database

Mac OS X 10.3.6, kernel patch applied, default install, blah, blah, blah. OK, I can start the database, and it runs fine. No problem. However, whenever I issue the 'emctl start dbconsole', CPU activity spikes to 100%, memory is consumed, and ultimately, all oracle processes just die. A crash reporter log is generated, indicating /bin/oracle threw a EXC_BAD_ACCESS (0x0001). Since the virtual memory drive gets tapped, I'm guessing that I'm just running out of VM swap space, but that's not really the error. I've noted several other things: after issuing the emctl start dbconsole, several 'perl' processes are started, as is the JVM. Issuing the command 'select s1.program, s1.OSUSER, s1.type, s.value from V$sesstat s, v$statname n, v$instance i, v$session s1 where s.statistic# = n.statistic# and n.name = 'CPU used by this session' and s1.SID = s.sid order by s.value desc' indicates that the program 'OMS' starts a session, and starts some work, then spawns several more times, ultimately taking lots of oracle time. Several additional 'oracle' processes are started, each trying to grab as much CPU as possible. Eventually, the database goes down. I'd love to figure out what is going on, rather than nuke the machine and start over. It appears that each of the 'OMS' sessions are issuing SQL commands. 'INSERT INTO MGMT_METRICS(...', 'begin dbms_application_info....', etc., so work is being done.
Does anyone have any ideas of what I should check first? The logs don't seem to be too much help.
-Darryl

I may have an answer, but I'm not certain. It appears that by default, a database is created using a dedicated server process (ie, an 'oracle' process is spawned for each connection). At some point, there are a bunch of connections that get spawned by the dbconsole startup, and ultimately, the server simply runs out of memory, both ram and swap. I've since been explicitly setting up my test boxes as shared servers, limiting the number of shared servers based on how much memory the machine has. So far, it has reduced/eliminated the problems. The annoying part of this is that I have customers that are 100% Macintosh shops, who I would really like to deploy with Oracle, but so far, Oracle support has been pretty much useless, especially on these memory consumption issues. For the price that is paid for support, you would hope that someone in there gets it.....I've mostly been gleening the forums for help, and trying to apply bits-n-pieces of that collective wisdom to the problems at hand. Such fun......
-Darryl

Dbconsole error

Hello,
I am trying to start the dbconsole service but i got an error:
vta-1000
windows xp.
How can i check if i did something wrong during the installing affetct the dbcosnole.
Thanks

enterprise manager service won't start if you are working on a dhcp based host. As soon as you get unplugged enterprise manager stops working, as it first was configured with a real ip. It could be the other way around, if you installed being unplugged it was configured to use the localhost.localdomain as its configuration parameters.
The way to solve this, if this is the scenario, is to configure a loopback adapter, rebuild enterprise manager repository and reconfigure enterprise manager.
Verify this guides for the loopback adapter issue:
http://download-west.oracle.com/docs/cd/B19306_01/install.102/b14316/reqs.htm#BABGCEAI
http://download-east.oracle.com/docs/cd/B19306_01/install.102/b14316/reqs.htm#BABGHEFB
Installing on DHCP environments:
http://download-east.oracle.com/docs/cd/B19306_01/install.102/b14316/reqs.htm#BABBDGBI
And check this procedure to reconfigure enterprise manager:
set oracle_sid=YourInstanceName
emctl stop dbconsole
emca -deconfig dbcontrol db
emca -repos recreate
emca -config dbcontrol db
~ Madrid

Dbconsole gives bogus database information

15" PB 1.25 GHz CPU 1G RAM OS 10.3.9
BANNER
Oracle Database 10g Enterprise Edition Release 10.1.0.3.0 - Prod
PL/SQL Release 10.1.0.3.0 - Production
CORE 10.1.0.3.0 Production
TNS for MacOS X Server: Version 10.1.0.3.0 - Production
NLSRTL Version 10.1.0.3.0 - Production
Hi;
I am having trouble with dbconsole. This is new, it has run correctly in the past. I must be guilty of changing something somewhere, but I can't think of what it was.
Sequence of events:
Request a static IP via System Preferences/Network
Entry in /etc/hosts for this IP to resolve to "alg4pb.local"
Set the instance via oemctl
Start the listener
Start the d/b
Register the d/b w/the listener
Start dbconsole via emctl start dbconsole
(This is where I notice the first difference, previously I would connect to dbconsole using port 5500, now it reports 5502.)
Launch Safari
Navigate to the provided address
Here I receive the message:
The database status is currently unavailable. It is possible that the database is in mount or nomount state. Click 'Startup' to obtain the current status and open the database. If the database cannot be opened, click 'Perform Recovery' to perform an appropriate recovery operation.
This is incorrect, I've just started the db.
If I attempt to start it from dbconsole there is an error reported in the dbconsole 'header' part of the page but dbconsole now recognizes that the db is actually open.
SQLException
Io exception: NL Exception was generated
Startup/Shutdown:Confirmation
Current Status     open
Operation     shutdown immediate
Are you sure you want to perform this operation?
So I play along and perform the shutdown here (this succeeds) and then do a startup (which also succeeds). But I am then back to where I started with this message in dbconsole:
Update Message
The database state has been changed successfully from shutdown to open. Note that the current status on this page gets updated when the new status information is uploaded into the repository.
View Details
Database: orcl.alg4pb.local
     The database status is currently unavailable. It is possible that the database is in mount or nomount state. Click 'Startup' to obtain the current status and open the database. If the database cannot be opened, click 'Perform Recovery' to perform an appropriate recovery operation.
Clicking on the Database link simply throws this page back up.
Trying to use the old port number gives this error in Safari:
Safari cant open the page http://alg4pb.local:5500/em because it could not connect to the server alg4pb.local.
Please let me know what I can try to understand what is wrong with this and how I can fix it.
If the answer is "rtfm", I'd love to! But I haven't found the "fm" that I need to "r" about this.
Thanks, Jack

Nope. Refreshing brings back the same incorrect page.
But I have found out something additional in the meantime.
I know why the port number changed. Each time you create a database a different port number is assigned for dbconsole. (Makes sense.)
Looiing in $ORACLE_HOME/install at portlist.ini I see:
Ultra Search HTTP port number =5620
iSQL*Plus HTTP port number =5560
Enterprise Manager Console HTTP Port (ora92) = 5500
Enterprise Manager Agent Port (ora92) = 1830
Enterprise Manager Console HTTP Port (ora101) = 5501
Enterprise Manager Agent Port (ora101) = 1831
Enterprise Manager Console HTTP Port (orcl) = 5502
Enterprise Manager Agent Port (orcl) = 1832
Enterprise Manager Console HTTP Port (orcldemo) = 5500
Enterprise Manager Agent Port (orcldemo) = 1830
I was connecting to my v 9 database with port 5500 and it's connecting to my v 10 database with 5502. What I don't here is that port 5500 has now been assigned to two databases; ora92 (which is the db I created through the v 9 install) and orcldemo which I tried to create by hand in the meantime. This can't be good.
What I wonder now is how I should go about cleaning up portlist.ini or if this file is even "functional" vs. documentary. The attempt to create a second v 9 db failed when I ran catalog.sql and there is no dbca for v 9. Or at least I haven't found it.
So, the little question is what to do with portlist.ini, or what is it's functional equivalent and the big question is how is it possible to create v 9 db's aside from the one created through the initial Oracle install.
Jack

DBConsole easting a CPU

Similar Messages

Maybe you are looking for