Strange 'gc cr multi block request' during dictionary queries

Hi,
here goes the case :
I've got 4 node 10.2.0.3 RAC , DDL intensive (CTAS, DROP/TRUNCATE) database
It's like 20TB of data, milions of partitions and objects .
When I'm doing queries agains dictionary tables like select * from
dba_segments where owner = 'A' and segment_name = 'B'
I'm observing 'gc cr multi block request' , as far as I know its kind of
scattered reads but using interconnect to gather data .
Whats bothering me is profile of that 'gc cr multi block requests' , here
goes some lines from 10046 trace:
WAIT #6: nam='gc cr multi block request' ela= 861 file#=1 block#$34788
class#=1 obj#8 tim18733123446958
WAIT #6: nam='gc cr multi block request' ela= 69 file#=1 block#$34788
class#=1 obj#8 tim18733123447083
WAIT #6: nam='gc cr multi block request' ela= 60 file#=1 block#$34788
class#=1 obj#8 tim18733123447220
WAIT #6: nam='gc cr multi block request' ela= 99 file#=1 block#$34788
class#=1 obj#8 tim18733123447347
WAIT #6: nam='gc cr multi block request' ela= 111 file#=1 block#$34788
class#=1 obj#8 tim18733123447482
WAIT #6: nam='gc cr multi block request' ela= 193 file#=1 block#$34788
class#=1 obj#8 tim18733123447704
WAIT #6: nam='gc cr multi block request' ela= 84 file#=1 block#$34788
class#=1 obj#8 tim18733123447820
WAIT #6: nam='gc cr multi block request' ela= 81 file#=1 block#$34788
class#=1 obj#8 tim18733123447931
WAIT #6: nam='gc cr multi block request' ela= 108 file#=1 block#$34788
class#=1 obj#8 tim18733123448065
WAIT #6: nam='gc cr multi block request' ela= 111 file#=1 block#$34788
class#=1 obj#8 tim18733123448199
WAIT #6: nam='gc cr multi block request' ela= 105 file#=1 block#$34788
class#=1 obj#8 tim18733123448328
WAIT #6: nam='gc cr multi block request' ela= 100 file#=1 block#$34788
class#=1 obj#8 tim18733123448458
WAIT #6: nam='gc cr multi block request' ela= 151 file#=1 block#$34788
class#=1 obj#8 tim18733123448639
WAIT #6: nam='gc cr multi block request' ela= 84 file#=1 block#$34788
class#=1 obj#8 tim18733123448750
WAIT #6: nam='gc cr multi block request' ela= 90 file#=1 block#$34788
class#=1 obj#8 tim18733123448867
WAIT #6: nam='gc cr multi block request' ela= 98 file#=1 block#$34788
class#=1 obj#8 tim18733123448994
and that pattern repeats with different block#
Question is why Oracle is requesting the same block , over and over (16
times) , I thinks 16 comes from MBRC which is 16 and I've got 16kb blocksize
Looks like a bug :) isnt it ?
Generally all queries agains dictionary tables are slow (minutes) , I'm not
observing any lost block issues so its probably bad plans issue .
Any comments ?
Regards
GregG

It looks like this:
select *
from
dba_segments where owner = 'INS' and segment_name = 'T'
call     count       cpu    elapsed       disk      query    current        rows
Parse        1      0.10       0.10          0          6          0           0
Execute      1      0.00       0.00          0          0          0           0
Fetch        2     11.05      79.71      74418      96116          0           1
total        4     11.16      79.82      74418      96122          0           1
Misses in library cache during parse: 1
Optimizer mode: ALL_ROWS
Parsing user id: 842
Rows     Row Source Operation
      1  VIEW  SYS_DBA_SEGS (cr=96116 pr=74418 pw=0 time=3397685 us)
      1   UNION-ALL  (cr=96116 pr=74418 pw=0 time=3397673 us)
      1    FILTER  (cr=91749 pr=74315 pw=0 time=3397634 us)
     11     HASH JOIN RIGHT OUTER (cr=91749 pr=74315 pw=0 time=3397620 us)
   1317      TABLE ACCESS FULL USER$ (cr=34 pr=0 pw=0 time=6738 us)
     11      HASH JOIN  (cr=91715 pr=74315 pw=0 time=3392292 us)
    262       TABLE ACCESS FULL FILE$ (cr=3 pr=0 pw=0 time=1087 us)
     11       HASH JOIN  (cr=91712 pr=74315 pw=0 time=3389434 us)
     75        TABLE ACCESS FULL TS$ (cr=82 pr=0 pw=0 time=2693 us)
     11        NESTED LOOPS  (cr=91630 pr=74315 pw=0 time=3386237 us)
     12         HASH JOIN  (cr=91592 pr=74313 pw=0 time=23765169 us)
     20          TABLE ACCESS BY INDEX ROWID OBJ$ (cr=424 pr=21 pw=0 time=261735 us)
     20           INDEX SKIP SCAN I_OBJ2 (cr=406 pr=21 pw=0 time=261140 us)(object id 37)
966949          VIEW  SYS_OBJECTS (cr=91168 pr=74292 pw=0 time=52219753 us)
966949           UNION-ALL  (cr=91168 pr=74292 pw=0 time=50285852 us)
113050            TABLE ACCESS FULL TAB$ (cr=20294 pr=16672 pw=0 time=18883720 us)
627116            TABLE ACCESS FULL TABPART$ (cr=7188 pr=5576 pw=0 time=5652638 us)
     25            TABLE ACCESS FULL CLU$ (cr=20293 pr=16520 pw=0 time=1019 us)
  13835            TABLE ACCESS FULL IND$ (cr=20322 pr=16550 pw=0 time=11400216 us)
195932            TABLE ACCESS FULL INDPART$ (cr=2523 pr=2421 pw=0 time=1967714 us)
   1213            TABLE ACCESS FULL LOB$ (cr=20350 pr=16553 pw=0 time=22044772 us)
   8363            TABLE ACCESS FULL TABSUBPART$ (cr=122 pr=0 pw=0 time=9995 us)
   7160            TABLE ACCESS FULL INDSUBPART$ (cr=72 pr=0 pw=0 time=9572 us)
    255            TABLE ACCESS FULL LOBFRAG$ (cr=4 pr=0 pw=0 time=2060 us)
     11         TABLE ACCESS CLUSTER SEG$ (cr=38 pr=2 pw=0 time=32200 us)
     11          INDEX UNIQUE SCAN I_FILE#_BLOCK# (cr=26 pr=0 pw=0 time=11331 us)(object id 9)
      0    NESTED LOOPS  (cr=1 pr=1 pw=0 time=8349 us)
      0     NESTED LOOPS  (cr=1 pr=1 pw=0 time=8344 us)
      0      FILTER  (cr=1 pr=1 pw=0 time=8340 us)
      0       NESTED LOOPS OUTER (cr=1 pr=1 pw=0 time=8334 us)
      0        NESTED LOOPS  (cr=1 pr=1 pw=0 time=8327 us)
      0         TABLE ACCESS BY INDEX ROWID UNDO$ (cr=1 pr=1 pw=0 time=8322 us)
      0          INDEX RANGE SCAN I_UNDO2 (cr=1 pr=1 pw=0 time=8314 us)(object id 35)
      0         TABLE ACCESS CLUSTER SEG$ (cr=0 pr=0 pw=0 time=0 us)
      0          INDEX UNIQUE SCAN I_FILE#_BLOCK# (cr=0 pr=0 pw=0 time=0 us)(object id 9)
      0        TABLE ACCESS CLUSTER USER$ (cr=0 pr=0 pw=0 time=0 us)
      0         INDEX UNIQUE SCAN I_USER# (cr=0 pr=0 pw=0 time=0 us)(object id 11)
      0      TABLE ACCESS BY INDEX ROWID FILE$ (cr=0 pr=0 pw=0 time=0 us)
      0       INDEX UNIQUE SCAN I_FILE2 (cr=0 pr=0 pw=0 time=0 us)(object id 42)
      0     TABLE ACCESS CLUSTER TS$ (cr=0 pr=0 pw=0 time=0 us)
      0      INDEX UNIQUE SCAN I_TS# (cr=0 pr=0 pw=0 time=0 us)(object id 7)
      0    FILTER  (cr=4366 pr=102 pw=0 time=4366839 us)
      0     HASH JOIN RIGHT OUTER (cr=4366 pr=102 pw=0 time=4366833 us)
   1317      TABLE ACCESS FULL USER$ (cr=60 pr=0 pw=0 time=1397 us)
      0      HASH JOIN  (cr=4306 pr=102 pw=0 time=4361061 us)
     75       TABLE ACCESS FULL TS$ (cr=82 pr=0 pw=0 time=745 us)
      0       NESTED LOOPS  (cr=4224 pr=102 pw=0 time=4357971 us)
    262        TABLE ACCESS FULL FILE$ (cr=3 pr=0 pw=0 time=1738 us)
      0        TABLE ACCESS CLUSTER SEG$ (cr=4221 pr=102 pw=0 time=4355035 us)
      0         INDEX RANGE SCAN I_FILE#_BLOCK# (cr=4221 pr=102 pw=0 time=4351804 us)(object id 9)
Elapsed times include waiting on following events:
  Event waited on                             Times   Max. Wait  Total Waited
  ----------------------------------------   Waited  ----------  ------------
  library cache lock                             18        0.00          0.00
  SQL*Net message to client                       2        0.00          0.00
  gc current block 2-way                       2299        0.00          1.16
  gc current block 3-way                       1597        0.00          1.28
  gc cr block busy                                4        0.01          0.03
  gc cr grant 2-way                             313        0.00          0.11
  db file sequential read                       627        0.06          2.94
  gc cr block 2-way                              37        0.00          0.02
  gc cr block 3-way                              27        0.00          0.02
  gc current grant 2-way                          1        0.00          0.00
  gc cr multi block request                   23073        0.01          7.07
  db file scattered read                       1756        0.10         13.70
  db file parallel read                        4708        0.19         44.89
  latch: KCL gc element parent latch             33        0.00          0.00
  SQL*Net message from client                     2     1492.36       1492.36
  latch free                                      4        0.00          0.00
  latch: gcs resource hash                        2        0.00          0.00
  latch: cache buffers chains                     5        0.00          0.00
  gc buffer busy                                  1        0.00          0.00
  latch: cache buffers lru chain                  2        0.00          0.00
  gc cr disk read                                 2        0.00          0.00
and in raw trace:
WAIT #4: nam='gc cr multi block request' ela= 18 file#=1 block#=3396 class#=1 obj#=4 tim=1318904416456439
WAIT #4: nam='gc cr multi block request' ela= 320 file#=1 block#=3396 class#=1 obj#=4 tim=1318904416456780
WAIT #4: nam='gc cr multi block request' ela= 146 file#=1 block#=3396 class#=1 obj#=4 tim=1318904416456949
WAIT #4: nam='gc cr multi block request' ela= 14 file#=1 block#=3396 class#=1 obj#=4 tim=1318904416456979
WAIT #4: nam='gc cr multi block request' ela= 6 file#=1 block#=3396 class#=1 obj#=4 tim=1318904416457000
WAIT #4: nam='gc cr multi block request' ela= 107 file#=1 block#=3396 class#=1 obj#=4 tim=1318904416457119
WAIT #4: nam='gc cr multi block request' ela= 171 file#=1 block#=3396 class#=1 obj#=4 tim=1318904416457594
WAIT #4: nam='gc cr multi block request' ela= 1119 file#=1 block#=3387 class#=1 obj#=4 tim=1318904416458819
WAIT #4: nam='db file parallel read' ela= 9411 files=1 blocks=10 requests=10 obj#=4 tim=1318904416468429
WAIT #4: nam='gc cr multi block request' ela= 592 file#=1 block#=4244 class#=1 obj#=4 tim=1318904416470099
WAIT #4: nam='gc cr multi block request' ela= 7 file#=1 block#=4244 class#=1 obj#=4 tim=1318904416470136
WAIT #4: nam='gc cr multi block request' ela= 203 file#=1 block#=4244 class#=1 obj#=4 tim=1318904416470353
WAIT #4: nam='gc cr multi block request' ela= 69 file#=1 block#=4244 class#=1 obj#=4 tim=1318904416470487
WAIT #4: nam='gc cr multi block request' ela= 962 file#=1 block#=4244 class#=1 obj#=4 tim=1318904416471499
WAIT #4: nam='gc cr multi block request' ela= 49 file#=1 block#=4244 class#=1 obj#=4 tim=1318904416471650Any ideas ?:)
Regards
GregG

Similar Messages

  • Long waits on 'gc cr multi block request' elapsed tim anomaly - tkprof inc

    Hi,
    in my 4-node RAC 10.2.0.3 when I run query (thats part os user_segments view)
    select o.name,
    o.subname,
    so.object_type, s.type#,
    ts.ts#, ts.name, ts.blocksize,
    s.file#, s.block#,
    s.blocks * ts.blocksize, s.blocks, s.extents,
    s.iniexts * ts.blocksize,
    decode(bitand(ts.flags, 3), 1, to_number(NULL),
    s.extsize * ts.blocksize),
    s.minexts, s.maxexts,
    decode(bitand(ts.flags, 3), 1, to_number(NULL),
    s.extpct),
    decode(bitand(ts.flags, 32), 32, to_number(NULL),
    decode(s.lists, 0, 1, s.lists)),
    decode(bitand(ts.flags, 32), 32, to_number(NULL),
    decode(s.groups, 0, 1, s.groups)),
    s.cachehint, NVL(s.spare1, 0), s.hwmincr
    from sys.obj$ o, sys.ts$ ts, sys.sys_objects so, sys.seg$ s
    where s.file# = so.header_file
    and s.block# = so.header_block
    and s.ts# = so.ts_number
    and s.ts# = ts.ts#
    and o.obj# = so.object_id
    and o.owner# = userenv('SCHEMAID')
    and s.type# = so.segment_type_id
    and o.type# = so.object_type_id
    call     count       cpu    elapsed       disk      query    current        rows
    Parse        1      0.03       0.05          2          5          0           0
    Execute      1      0.00       0.00          0          0          0           0
    Fetch        5     11.59     529.83       9580     115021          0        1957
    total        7     11.62     529.88       9582     115026          0        1957
    Misses in library cache during parse: 1
    Optimizer mode: ALL_ROWS
    Parsing user id: SYS
    Elapsed times include waiting on following events:
      Event waited on                             Times   Max. Wait  Total Waited
      ----------------------------------------   Waited  ----------  ------------
      library cache lock                             13        0.00          0.00
      SQL*Net message to client                       5        0.00          0.00
      gc cr multi block request                   63408        1.22        507.30
      gc cr grant 2-way                              75        0.00          0.02
      db file sequential read                       983        0.04          6.13
      gc current block 3-way                         97        0.00          0.07
      gc current block 2-way                        111        0.00          0.06
      db file scattered read                        522        0.17          4.28
      db file parallel read                         249        0.07          2.19
      cr request retry                               81        0.00          0.02
      latch: cache buffers lru chain                  5        0.00          0.00
      latch: gcs resource hash                        5        0.00          0.00
      latch: KCL gc element parent latch             22        0.00          0.00
      latch free                                      3        0.00          0.00
      gc cr block 3-way                              10        0.00          0.00
      gc cr block 2-way                             101        0.00          0.05
      gc cr disk read                                64        0.00          0.02
      SQL*Net message from client                     4        0.04          0.14
      SQL*Net more data to client                    50        0.00          0.00
    ********************************************************************************In raw trace file I can see:
    WAIT #1: nam='gc cr multi block request' ela= 81 file#=1 block#=223460 class#=1 obj#=4 tim=1273765932709496
    WAIT #1: nam='gc cr multi block request' ela= 1221367 file#=1 block#=223456 class#=1 obj#=4 tim=1273765933930894
    WAIT #1: nam='gc cr multi block request' ela= 1221273 file#=1 block#=223456 class#=1 obj#=4 tim=1273765935152287
    WAIT #1: nam='gc cr multi block request' ela= 1223855 file#=1 block#=223456 class#=1 obj#=4 tim=1273765936376232
    WAIT #1: nam='gc cr multi block request' ela= 1180460 file#=1 block#=223456 class#=1 obj#=4 tim=1273765937556773
    WAIT #1: nam='gc cr multi block request' ela= 406 file#=1 block#=223476 class#=1 obj#=4 tim=1273765937566092
    WAIT #1: nam='gc cr multi block request' ela= 8039 file#=1 block#=263972 class#=1 obj#=4 tim=1273765937742971
    WAIT #1: nam='gc cr multi block request' ela= 1221983 file#=1 block#=263972 class#=1 obj#=4 tim=1273765938965148
    WAIT #1: nam='gc cr multi block request' ela= 1221642 file#=1 block#=263972 class#=1 obj#=4 tim=1273765940186879
    WAIT #1: nam='gc cr multi block request' ela= 1221656 file#=1 block#=263972 class#=1 obj#=4 tim=1273765941408618
    WAIT #1: nam='gc cr multi block request' ela= 1221654 file#=1 block#=263972 class#=1 obj#=4 tim=1273765942630357
    WAIT #1: nam='gc cr multi block request' ela= 1221657 file#=1 block#=263972 class#=1 obj#=4 tim=1273765943852096
    WAIT #1: nam='gc cr multi block request' ela= 578770 file#=1 block#=263972 class#=1 obj#=4 tim=1273765944430948
    WAIT #1: nam='gc cr multi block request' ela= 101 file#=1 block#=351188 class#=1 obj#=4 tim=1273765944898180
    WAIT #1: nam='gc cr multi block request' ela= 2287 file#=1 block#=351188 class#=1 obj#=4 tim=1273765944900500
    WAIT #1: nam='gc cr multi block request' ela= 1221984 file#=1 block#=351188 class#=1 obj#=4 tim=1273765946122713
    WAIT #1: nam='gc cr multi block request' ela= 1221641 file#=1 block#=351188 class#=1 obj#=4 tim=1273765947344453
    WAIT #1: nam='gc cr multi block request' ela= 1221670 file#=1 block#=351188 class#=1 obj#=4 tim=1273765948566201
    WAIT #1: nam='gc cr multi block request' ela= 1221663 file#=1 block#=351188 class#=1 obj#=4 tim=1273765949787950
    WAIT #1: nam='gc cr multi block request' ela= 1221670 file#=1 block#=351188 class#=1 obj#=4 tim=1273765951009697
    WAIT #1: nam='gc cr multi block request' ela= 182726 file#=1 block#=351188 class#=1 obj#=4 tim=1273765951192501
    WAIT #1: nam='gc cr multi block request' ela= 558 file#=1 block#=351188 class#=1 obj#=4 tim=1273765951194110
    same for obj#5
    WAIT #1: nam='gc cr multi block request' ela= 3103 file#=1 block#=35284 class#=1 obj#=5 tim=1273766101728096
    WAIT #1: nam='gc cr multi block request' ela= 1221789 file#=1 block#=35281 class#=1 obj#=5 tim=1273766102950203
    WAIT #1: nam='gc cr multi block request' ela= 1221668 file#=1 block#=35281 class#=1 obj#=5 tim=1273766104171949
    WAIT #1: nam='gc cr multi block request' ela= 1221639 file#=1 block#=35281 class#=1 obj#=5 tim=1273766105393674
    WAIT #1: nam='gc cr multi block request' ela= 1221612 file#=1 block#=35281 class#=1 obj#=5 tim=1273766106615404
    WAIT #1: nam='gc cr multi block request' ela= 838378 file#=1 block#=35281 class#=1 obj#=5 tim=1273766107453869
    WAIT #1: nam='gc cr multi block request' ela= 695 file#=1 block#=35281 class#=1 obj#=5 tim=1273766107455175Any ideas ?
    Regards
    GregG

    Hi Greg;
    You may hit a bug?What is your OS? If linux than please see:
    Bug 6268172: INCONSISTENT QUERY PERFORMANCE IN 4 NODE RAC
    https://support.oracle.com/CSP/main/article?cmd=show&type=BUG&id=6268172&productFamily=Oracle
    Also see:
    Bug 9838876: WAITING FOR 'GC CR MULTI BLOCK REQUEST'
    https://support.oracle.com/CSP/main/article?cmd=show&type=BUG&id=9838876&productFamily=Oracle
    Regard
    Helios

  • Cluster multi-block requests were consuming significant database time

    Hi,
    DB : 10.2.0.4 RAC ASM
    OS : AIX 5.2 64-bit
    We are facing too much performance issues and CPU idle time becoming 20%.Based on the AWR report , the top 5 events are showing that problem is in cluster side.I placed 1st node AWR report here for your suggestions.
    WORKLOAD REPOSITORY report for
    DB Name DB Id Instance Inst Num Release RAC Host
    PROD 1251728398 PROD1 1 10.2.0.4.0 YES msprod1
    Snap Id Snap Time Sessions Curs/Sess
    Begin Snap: 26177 26-Jul-11 14:29:02 142 37.7
    End Snap: 26178 26-Jul-11 15:29:11 159 49.1
    Elapsed: 60.15 (mins)
    DB Time: 915.85 (mins)
    Cache Sizes
    ~~~~~~~~~~~ Begin End
    Buffer Cache: 23,504M 23,504M Std Block Size: 8K
    Shared Pool Size: 27,584M 27,584M Log Buffer: 14,248K
    Load Profile
    ~~~~~~~~~~~~ Per Second Per Transaction
    Redo size: 28,126.82 2,675.18
    Logical reads: 526,807.26 50,105.44
    Block changes: 3,080.07 292.95
    Physical reads: 962.90 91.58
    Physical writes: 157.66 15.00
    User calls: 1,392.75 132.47
    Parses: 246.05 23.40
    Hard parses: 11.03 1.05
    Sorts: 42.07 4.00
    Logons: 0.68 0.07
    Executes: 930.74 88.52
    Transactions: 10.51
    % Blocks changed per Read: 0.58 Recursive Call %: 32.31
    Rollback per transaction %: 9.68 Rows per Sort: 4276.06
    Instance Efficiency Percentages (Target 100%)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    Buffer Nowait %: 99.87 Redo NoWait %: 100.00
    Buffer Hit %: 99.84 In-memory Sort %: 99.99
    Library Hit %: 98.25 Soft Parse %: 95.52
    Execute to Parse %: 73.56 Latch Hit %: 99.51
    Parse CPU to Parse Elapsd %: 9.22 % Non-Parse CPU: 99.94
    Shared Pool Statistics Begin End
    Memory Usage %: 68.11 71.55
    % SQL with executions>1: 94.54 92.31
    % Memory for SQL w/exec>1: 98.79 98.74
    Top 5 Timed Events Avg %Total
    ~~~~~~~~~~~~~~~~~~ wait Call
    Event Waits Time (s) (ms) Time Wait Class
    CPU time 18,798 34.2
    gc cr multi block request 46,184,663 18,075 0 32.9 Cluster
    gc buffer busy 2,468,308 6,897 3 12.6 Cluster
    gc current block 2-way 1,826,433 4,422 2 8.0 Cluster
    db file sequential read 142,632 366 3 0.7 User I/O
    RAC Statistics DB/Inst: PROD/PROD1 Snaps: 26177-26178
    Begin End
    Number of Instances: 2 2
    Global Cache Load Profile
    ~~~~~~~~~~~~~~~~~~~~~~~~~ Per Second Per Transaction
    Global Cache blocks received: 14,112.50 1,342.26
    Global Cache blocks served: 619.72 58.94
    GCS/GES messages received: 2,099.38 199.68
    GCS/GES messages sent: 23,341.11 2,220.01
    DBWR Fusion writes: 3.43 0.33
    Estd Interconnect traffic (KB) 122,826.57
    Global Cache Efficiency Percentages (Target local+remote 100%)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    Buffer access - local cache %: 97.16
    Buffer access - remote cache %: 2.68
    Buffer access - disk %: 0.16
    Global Cache and Enqueue Services - Workload Characteristics
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    Avg global enqueue get time (ms): 0.6
    Avg global cache cr block receive time (ms): 2.8
    Avg global cache current block receive time (ms): 3.0
    Avg global cache cr block build time (ms): 0.0
    Avg global cache cr block send time (ms): 0.0
    Global cache log flushes for cr blocks served %: 11.3
    Avg global cache cr block flush time (ms): 1.7
    Avg global cache current block pin time (ms): 0.0
    Avg global cache current block send time (ms): 0.0
    Global cache log flushes for current blocks served %: 0.0
    Avg global cache current block flush time (ms): 4.1
    Global Cache and Enqueue Services - Messaging Statistics
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    Avg message sent queue time (ms): 0.1
    Avg message sent queue time on ksxp (ms): 2.4
    Avg message received queue time (ms): 0.0
    Avg GCS message process time (ms): 0.0
    Avg GES message process time (ms): 0.0
    % of direct sent messages: 6.27
    % of indirect sent messages: 93.48
    % of flow controlled messages: 0.25
    Time Model Statistics DB/Inst: PROD/PROD1 Snaps: 26177-26178
    -> Total time in database user-calls (DB Time): 54951s
    -> Statistics including the word "background" measure background process
    time, and so do not contribute to the DB time statistic
    -> Ordered by % or DB time desc, Statistic name
    Statistic Name Time (s) % of DB Time
    sql execute elapsed time 54,618.2 99.4
    DB CPU 18,798.1 34.2
    parse time elapsed 494.3 .9
    hard parse elapsed time 397.4 .7
    PL/SQL execution elapsed time 38.6 .1
    hard parse (sharing criteria) elapsed time 27.3 .0
    sequence load elapsed time 5.0 .0
    failed parse elapsed time 3.3 .0
    PL/SQL compilation elapsed time 2.1 .0
    inbound PL/SQL rpc elapsed time 1.2 .0
    repeated bind elapsed time 0.8 .0
    connection management call elapsed time 0.6 .0
    hard parse (bind mismatch) elapsed time 0.3 .0
    DB time 54,951.0 N/A
    background elapsed time 1,027.9 N/A
    background cpu time 518.1 N/A
    Wait Class DB/Inst: PROD/PROD1 Snaps: 26177-26178
    -> s - second
    -> cs - centisecond - 100th of a second
    -> ms - millisecond - 1000th of a second
    -> us - microsecond - 1000000th of a second
    -> ordered by wait time desc, waits desc
    Avg
    %Time Total Wait wait Waits
    Wait Class Waits -outs Time (s) (ms) /txn
    Cluster 50,666,311 .0 30,236 1 1,335.4
    User I/O 419,542 .0 811 2 11.1
    Network 4,824,383 .0 242 0 127.2
    Other 797,753 88.5 208 0 21.0
    Concurrency 212,350 .1 121 1 5.6
    Commit 16,215 .0 53 3 0.4
    System I/O 60,831 .0 29 0 1.6
    Application 6,069 .0 6 1 0.2
    Configuration 763 97.0 0 0 0.0
    Second node top 5 events are as below,
    Top 5 Timed Events
              Avg %Total
    ~~~~~~~~~~~~~~~~~~ wait Call
    Event Waits Time (s) (ms) Time Wait Class
    CPU time 25,959 42.2
    db file sequential read 2,288,168 5,587 2 9.1 User I/O
    gc current block 2-way 822,985 2,232 3 3.6 Cluster
    read by other session 345,338 1,166 3 1.9 User I/O
    gc cr multi block request 991,270 831 1 1.4 Cluster
    My RAM is 95GB each node and SGA is 51 GB and PGA is 14 GB.
    Any inputs from your side are greatly helpful to me ,please.
    Thanks,
    Sunand

    Hi Forstmann,
    Thanks for your update.
    Even i have collected ADDM report, extract of Node1 report as below
    FINDING 1: 40% impact (22193 seconds)
    Cluster multi-block requests were consuming significant database time.
    RECOMMENDATION 1: SQL Tuning, 6% benefit (3313 seconds)
    ACTION: Run SQL Tuning Advisor on the SQL statement with SQL_ID
    "59qd3x0jg40h1". Look for an alternative plan that does not use
    object scans.
    SYMPTOMS THAT LED TO THE FINDING:
    SYMPTOM: Inter-instance messaging was consuming significant database
    time on this instance. (55% impact [30269 seconds])
    SYMPTOM: Wait class "Cluster" was consuming significant database
    time. (55% impact [30271 seconds])
    FINDING 3: 13% impact (7008 seconds)
    Read and write contention on database blocks was consuming significant
    database time.
    NO RECOMMENDATIONS AVAILABLE
    SYMPTOMS THAT LED TO THE FINDING:
    SYMPTOM: Inter-instance messaging was consuming significant database
    time on this instance. (55% impact [30269 seconds])
    SYMPTOM: Wait class "Cluster" was consuming significant database
    time. (55% impact [30271 seconds])
    Any help from your side , please?
    Thanks,
    Sunand

  • [질문]WAS 이용시 gc cr multi block request 발생

    OS 버젼 과 Word size : AIX 5.3 64BIT
    DB버젼과 Word size : DB VERSION은 10.2.0.3 64BIT
    RAC 환경이며 node1과 node2는 동일한 환경으로 구성되어 있습니다.
    업무상 주로 이용하는 인스턴스는 node1 이구요.
    파티션으로 구성되어 있는 테이블을 full scan하는 쿼리를 수행하는데
    WAS에서 해당 쿼리를 날리면 gc cr multi block request 이벤트가 아주 오래 점유하고 있습니다.
    동일 쿼리를 pl/sql 로 접속해서 2-tier로 쿼리하면 그런 이벤트 없이 바로 떨어지구요.
    2-tier로 쿼리하면 15초,
    WAS에서 쿼리하면 4~5분 정도가 걸리죠.
    도대체 왜 그런지 알수가 없는데, 가이드가 될 만한 힌트라도 좋으니 의견 부탁드립니다.
    감사합니다.

    "액셈이 만들어가는 오라클 백과사전"에서 내용이 잘나와있습니다.
    이 내용부터 확인하세요.
    http://wiki.ex-em.com/index.php/Gc_cr_multi_block_request

  • Gc cr multi block request.

    Hi ,
    I am sufferiung from a 40 hour long running job which normally completes in few hrs, that jus creates a table (actually create table as select,joining many tables using parallel query withe 16 sessions).
    There is only 2 session that are active at this time, one of them is the parent waiting on PX:DEQ EXECUTE REPLY.
    The other sesion is running from 24 hrs, it is actually not waiitng upon any event for a signifcant time.Most of the time it faces GC CR MULTI BLOCK REQUESTS wait event. Is this due to interconnect latency?.I guessed it because the protocol used by the interconnect is UDP which doesnot wait for an acknowledgement. Can you please suggest me how to check for the latency on the interconnect?? Is my doubt on the interconnect is reasonable???????????
    I checked the trace files,awr reports,ash reports and every thing but nothing seems to be working fine. All I see is that single sesion using up 98% of the CPU and the TOP wait event in the awr,ash reports being the CPU Time.Pleas help me.Thanks.

    You are difficult to help as you don't post
    - a four digit database version
    - information on your OS
    - information, including configuration info, of your interconnect
    RAC is a chatty protocol, and sends large packages, of even 64k and higher. This means if you don't use an 1 Gb network card, and don't have 'jumbo frames' enabled, performance will suffer.
    As a result, at least in 9i, you will see this very wait event show up all the time, at least on AIX.
    Sybrand Bakker
    Senior Oracle DBA

  • Multi User request in Access Enforcer

    Is anyone aware of a user limit in an access enforcer multi user request?
    We get errors when we submit  a multi user access enforcer request with more than 25 users.
    Thanks

    Hi
    There is no standard limit even though we advice to keep the user to max of 20 .
    The limit depends upon the email content you have configured .
    In case in your email notifications you have taken the argument USERID then mulitple user creation request causes issue and the limit gets set to anything between 20-25 , again depending on content of the email .
    Thanks

  • How to find the First block....in a multi block form

    hi
    How i can find which is the first block in a multi block form....
    ( there are n number of forms with multi blocks...so
    i need to generalise the code to find the first block in all of
    the forms)
    regards
    Kris

    If you searched in the on-line help for "First", you would find that the Get_Form_Property built-in provides two: First_Block and First_Navigation_Block.

  • How to execute each block in a multi-block canvas while select the tab?

    Hi All,
    How to execute each block in a multi-block canvas by selecting a tab? I mean to say when i select a particular tab in a tab canvas the records should execute.How can i set this?
    Arif

    Hi Arif
    Good Example Manu offered i wish it works if not pls try the following...
    Pls try in the when-tab-page-changed trigger in the form level
    DECLARE
        tp_name varchar2(30);
    BEGIN
    -- Retrieve the NAME of the top most tab page using the built-in GET_CANVAS_PROPERTY function.
    --Pass in the name of the Canvas your tabs are in and the system variable topmost_tab_page which stores a property number
    tp_name := GET_CANVAS_PROPERTY('CANVAS11',topmost_tab_page);
    -- Perform specific tasks based on the name of the top most tab
    IF tp_name = 'PAGE12' then
    GO_BLOCK('block_name');
       GO_ITEM('DATA_BLOCK_1.FIELD_1');
    EXECUTE_QUERY;
    ELSIF tp_name = 'PAGE38' then
    GO_BLOCK('block_name');
       GO_ITEM('DATA_BLOCK_2.FIELD_1');
    EXECUTE_QUERY;
    ELSIF tp_name = 'PAGE47' then
    GO_BLOCK('block_name');
       GO_ITEM('DATA_BLOCK_3.FIELD_1');
    EXECUTE_QUERY;
    END IF;
    END;
    Hope it helps...
    Note: i do agree with Craig
    Regards,
    Abdetu...
    Edited by: Abdetu on Feb 2, 2011 7:41 AM

  • Copy Multi block field value in single field

    HI!
    I have a multi block text item i want to copy its contents in single filed text item.
    1. I am using Forms 6i.
    Thanks & Regards!

    Hi,
    Try this code.
    DECLARE
         Str_Field                VARCHAR2(3000);
         Num_Total_Records NUMBER;
         Num_Loop_I                     NUMBER;
    BEGIN
         :<master_block_name>.<field_name> := '';
         FIRST_RECORD;
         LAST_RECORD;
         Num_Total_Records := TO_NUMBER(NAME_IN('SYSTEM.CURSOR_RECORD'));
         FIRST_RECORD;
         FOR Num_Loop_I IN 1..Num_Total_Records LOOP
              Str_Field := Str_Field || ',' || :<detail_block_name>.<field_name>;
              NEXT_RECORD;
         END LOOP;
         :<master_block_name>.<field_name> := SUBSTR(Str_Field, 2, LENGTH(Str_Field)-1);
    END;Regards,
    Manu.

  • Multi Block Form Navigation Problem

    Hiii
    i have got a multi block form the problem that am facing now is that when running the FORM and when am over one of the my canvases then if i tried to press any of :
    (Tab button,Down arrow ) the canvas changes to ANOTHER Datablock ...
    i dont know what is the problem ....
    ANY SUGGESTIONS
    OMD

    Make the datablock you want to show on top as the first block in the object navigator.
    HTH.
    Regards,
    Arun

  • Formatting a multi-block report

    I am having trouble with formatting a multi-block report. For some reason, when I get to the second page - the builder automatically starts putting in extra white space at the top of each page thereafter which really throws off my formatting.also, for one of the blocks, the builder puts each record onto its own page. does anyone have any suggestions?

    i was unable to resolve the issue -
    it is as follows
    i am creating a complex report that includes several different reports and graphs - so the report that i am referring to had to be created manually -
    it is a manually created group above report -
    the group above works nicely - but between each group there is a page break -
    although there is no page break between the last group and the next report block.

  • Crash using block from mutable dictionary

    What can people tell me about why this program crashes:
    #import <Foundation/Foundation.h>
    @interface Action : NSObject
    + (void)execute;
    @end
    @implementation Action
    + (void)execute
        NSLog(@"The action was executed.");
    @end
    @interface ActionPerformer : NSObject {
    @private
        NSMutableDictionary* _namedActions;
    + (ActionPerformer*) performer;
    - (void)addAction:(NSString*)action withClass:(Class)class andSelector:(SEL)selector;
    - (void)performAction:(NSString*)action;
    @end
    @implementation ActionPerformer
    typedef void(^ActionBlock)();
    + (ActionPerformer*) performer
        ActionPerformer* performer = [[ActionPerformer alloc] init];
        performer->_namedActions = [NSMutableDictionary dictionary];
        return performer;
    - (void)addAction:(NSString*)actionName withClass:(Class)class andSelector:(SEL)selector
        ActionBlock block = ^(){ [class performSelector:selector]; };
        // Verify things work
        block();
        [_namedActions setObject:block forKey:actionName];
    - (void)performAction:(NSString*)actionName
        ActionBlock block = [_namedActions objectForKey:actionName];
        // Crash happens here!!!
        block();
    @end
    int main()
        @autoreleasepool {
            ActionPerformer* performer = [ActionPerformer performer];
            [performer addAction:@"action" withClass:[Action class]
                andSelector:@selector(execute)];
            [performer performAction:@"action"];       
        return 0;
    I am using Mac OS X 10.6.8 with Xcode 4.2 and Apple LLVM compiler 3.0

    Thanks etersoft,
    I tried exactly what you suggested and I was able to call the block from the dictionary.
    I assume that if someone is using garbage collection they don't need to worry about calling release on the block held in the dictionary.
    Also, if someone is using automatic reference counting, I'm wondering if that would also take care of calling the release as well.

  • Sending multi part request

    hai all,
    i have to send an image file as multi part request.how to do this. ihave to upload it using forms.cn any one help it out
    thanking u chandru

    sending it is easy, parsing the results with handrolled code is not; I would suggest some API, you can find information here:
    http://www.oop-reserch.com/mime_save.html

  • How to block popups during downloading

    how to block popups during downloading

    What kind of pop-ups do you get?
    Do you mean that you get some kind of alert or do additional Firefox windows open with content?
    Do a malware check with a few malware scan programs.<br />
    You need to use all programs because each detects different malware.<br />
    Make sure that you update each program to get the latest version of the database before doing a scan.
    * http://www.malwarebytes.org/mbam.php - Malwarebytes' Anti-Malware
    * http://www.superantispyware.com/ - SuperAntispyware
    * http://www.safer-networking.org/en/index.html - Spybot Search & Destroy
    * http://www.lavasoft.com/products/ad_aware_free.php - Ad-Aware Free
    * http://www.microsoft.com/windows/products/winfamily/defender/default.mspx - Windows Defender: Home Page
    See also "Spyware on Windows": http://kb.mozillazine.org/Popups_not_blocked and [[Searches are redirected to another site]]

  • Blocking Objects during development phase

    Dear All,
    I would like to block objects during the development phase.
    I mean if for example an user is currently modifying one specific table and a second user wants to do some modification on the same table (object) the second developer will be notified that is not possible because the table is being used.
    Is this something that I can control / manage from the authorization perspective? How could I do that?
    I will appreciate any idea.
    Thanks,
    FedeX

    Hi,
    as it was mentioned two users can't modify one object at same time. But I guess your question is little bit different. All modified objects (ignore local objects) are assigned to a transport after first change until the transport is released. After that they can be assigned to next transport after new change. When second user wants to change object already assigned to transport then he can't assign it to another transport. He needs to create a subtask which is assigned to his user account. After that he will be able to modify same object. So if you remove authorizations to create sub task then it will block them. Don't forget to remove authorization to change task assignment to another user. 
    Cheers

Maybe you are looking for

  • Subsidized Palm Pre 2 and future web OS phones

    I am currently a verizon cutomer using the motorola droid. My contract is almost up, and while the droid has held up well over the last 1.5 years I have been very interested in switching to a web OS device. The only reason I chose the droid over the

  • Is it possible to change BP Customer in Sales Order ?

    PLease advise, In sales Order create a order for  a BP customer and ADD. Is it possible to change the BP customer  added ? As it is the wrong BP customer ? I do not think so , but I would like confirmation ? Thanks D

  • 3D shapes duplicated in same document using CS2

    In a new blank document I used the pen tool to created a simple shape and then used the  Effect, 3D to make a 3D shape from it. When I draw a second new shape on the same page, say a rectangle, it assumes the 3D properties of the first shape instead

  • Resizing widgets in Belle?

    According to Nokia sources and articles for example on Thenokiablog.com, widgets in Belle are resizable. But I have Belle now on my C6-01 and all I can do is move widgets around on my screen but not resize them? For a widget like the calendar that is

  • How to sync apps from phone to comp

    I just down loaded the remote app works great. then my display blew out on my phone. I think a drop of sweat went into the ear speaker nice HUH!! It's going in tomorrow for repair. I don't want to loose the settings for remote and other apps that are