Performance problem, statspack report included.

Question

HI,
on my 9.2.0.8 cpu_count=12 EE I've got job which takes 5h , there is strange top sql reported:
STATSPACK report for
DB Name         DB Id    Instance     Inst Num Release     Cluster Host
E1        2020101010                E1            1 9.2.0.8.0   NO      test1
              Snap Id     Snap Time      Sessions Curs/Sess Comment
Begin Snap:     56073 22-Mar-10 15:00:04       35       5.7
  End Snap:     56078 22-Mar-10 20:00:04       38       5.6
   Elapsed:              300.00 (mins)
Cache Sizes (end)
~~~~~~~~~~~~~~~~~
               Buffer Cache:     1,024M      Std Block Size:          8K
           Shared Pool Size:       320M          Log Buffer:      2,000K
Load Profile
~~~~~~~~~~~~                            Per Second       Per Transaction
                  Redo size:            141,422.83             23,063.29
              Logical reads:             21,928.15              3,576.05
              Block changes:                695.89                113.49
             Physical reads:                424.41                 69.21
            Physical writes:                 36.35                  5.93
                 User calls:                 21.19                  3.46
                     Parses:                115.96                 18.91
                Hard parses:                  1.41                  0.23
                      Sorts:                 28.80                  4.70
                     Logons:                  0.14                  0.02
                   Executes:              5,099.90                831.69
               Transactions:                  6.13
  % Blocks changed per Read:    3.17    Recursive Call %:     99.86
Rollback per transaction %:    0.00       Rows per Sort:      3.88
Instance Efficiency Percentages (Target 100%)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
            Buffer Nowait %:  100.00       Redo NoWait %:    100.00
            Buffer  Hit   %:   98.07    In-memory Sort %:    100.00
            Library Hit   %:   99.43        Soft Parse %:     98.78
         Execute to Parse %:   97.73         Latch Hit %:    100.00
Parse CPU to Parse Elapsd %:   39.63     % Non-Parse CPU:     99.50
Shared Pool Statistics        Begin   End
             Memory Usage %:   19.64   94.21
    % SQL with executions>1:   73.59   14.19
  % Memory for SQL w/exec>1:   63.89    9.21
Top 5 Timed Events
~~~~~~~~~~~~~~~~~~                                                     % Total
Event                                               Waits    Time (s) Ela Time
CPU time                                                       10,436    78.98
db file sequential read                           231,010       1,484    11.23
db file scattered read                            333,736         733     5.55
log file sync                                      67,760         206     1.56
db file parallel write                             63,946         204     1.54
^LWait Events for DB: E1  Instance: E1  Snaps: 56073 -56078
-> s  - second
-> cs - centisecond -     100th of a second
-> ms - millisecond -    1000th of a second
-> us - microsecond - 1000000th of a second
-> ordered by wait time desc, waits desc (idle events last)
                                                                   Avg
                                                     Total Wait   wait    Waits
Event                               Waits   Timeouts   Time (s)   (ms)     /txn
db file sequential read           231,010          0      1,484      6      2.1
db file scattered read            333,736          0        733      2      3.0
log file sync                      67,760          0        206      3      0.6
db file parallel write             63,946          0        204      3      0.6
direct path write (lob)            45,642          0         82      2      0.4
control file parallel write         7,000          0         28      4      0.1
log file parallel write           205,617          0         14      0      1.9
process startup                       450          0         11     25      0.0
control file sequential read        8,068          0          8      1      0.1
log file switch completion            104          0          3     28      0.0
log buffer space                       86          0          2     22      0.0
SQL*Net break/reset to clien        3,632          0          1      0      0.0
latch free                          1,241         23          1      0      0.0
direct path read                    3,432          0          1      0      0.0
log file single write                 160          0          0      3      0.0
direct path write                   2,074          0          0      0      0.0
log file sequential read              160          0          0      2      0.0
direct path read (lob)                 76          0          0      3      0.0
SQL*Net message from dblink            38          0          0      4      0.0
buffer busy waits                      19          0          0      4      0.0
single-task message                     1          0          0     64      0.0
SQL*Net more data to client         2,493          0          0      0      0.0
LGWR wait for redo copy               369          0          0      0      0.0
library cache pin                       3          0          0      0      0.0
enqueue                                 7          0          0      0      0.0
SQL*Net message to dblink              38          0          0      0      0.0
SQL*Net message from client       240,510          0    378,158   1572      2.2
jobq slave wait                     9,559      9,475     27,883   2917      0.1
wakeup time manager                   583        583     16,960  29090      0.0
SQL*Net message to client         240,514          0          0      0      2.2
SQL*Net more data from clien        2,990          0          0      0      0.0
^LBackground Wait Events for DB: E1  Instance: E1  Snaps: 56073 -56078
-> ordered by wait time desc, waits desc (idle events last)
                                                                   Avg
                                                     Total Wait   wait    Waits
Event                               Waits   Timeouts   Time (s)   (ms)     /txn
db file parallel write             63,946          0        204      3      0.6
control file parallel write         7,000          0         28      4      0.1
log file parallel write           205,617          0         14      0      1.9
control file sequential read        3,998          0          4      1      0.0
db file scattered read                237          0          1      5      0.0
db file sequential read               161          0          1      3      0.0
log file single write                 160          0          0      3      0.0
direct path read                    3,398          0          0      0      0.0
direct path write                   2,070          0          0      0      0.0
log file sequential read              160          0          0      2      0.0
rdbms ipc reply                       259          0          0      1      0.0
log buffer space                       11          0          0     11      0.0
LGWR wait for redo copy               369          0          0      0      0.0
latch free                             34          2          0      1      0.0
buffer busy waits                       1          0          0      0      0.0
rdbms ipc message                 590,267     22,981     82,922    140      5.3
pmon timer                          6,288      6,288     17,570   2794      0.1
smon timer                             96         50     16,604 ######      0.0
^LSQL ordered by Gets for DB: E1  Instance: E1  Snaps: 56073 -56078
-> End Buffer Gets Threshold:     10000
-> Note that resources reported for PL/SQL includes the resources used by
   all SQL statements called within the PL/SQL code.  As individual SQL
   statements are also reported, it is possible and valid for the summed
   total % to exceed 100
                                                     CPU      Elapsd
  Buffer Gets    Executions  Gets per Exec  %Total Time (s)  Time (s) Hash Value
    385,750,162            1  385,750,162.0   97.7 ########  15723.50 2420406373
Module: 01@ me2xml.sql
declare v_message         VARCHAR2(2000); v_inserts         VARC
HAR2(2000); v_voucher         VARCHAR2(2000); v_vls
VARCHAR2(2000); v_posi            VARCHAR2(2000); v_posih
    VARCHAR2(2000); v_rep_inderts     VARCHAR2(2000); begin  upd
ate  PARAM_SYSTEM set SYS_WARTOSC_TEKST =  null where
    351,634,942   87,908,729            4.0   89.1  1307.36   1617.33 4265647904
Module: 01@ posi2xml.sql
select longname from javasnm$ where short = :1
...other statements are way lowerAny ideas how can I optimize 'select longname from javasnm$ where short = :1' ?
Regards.
Greg

Answer 1

Thank You , but I'm unable to reduce java calls ,because the whole code is doing xml transformations (xmlparse and xmldom).
My java pool is:
SQL> show parameters java
NAME TYPE VALUE
java_max_sessionspace_size integer 0
java_pool_size big integer 167772160
java_soft_sessionspace_limit integer 0
and my shared pool is:
SQL> show parameters shared
NAME TYPE VALUE
hi_shared_memory_address integer 0
max_shared_servers integer 20
shared_memory_address integer 0
shared_pool_reserved_size big integer 8388608
shared_pool_size big integer 335544320
shared_server_sessions integer 0
shared_servers integer 0
btw how do You know I'm using shared pool for ad-hoc queries ?
The 5h interval spans one procedure call , there is no other workload on database.
Regards.
Greg