Gc current request waits for concurrent DL inserts

Question

I see 8 different concurrent session runing following insert statement ( DL load ) on 2 different node of a 4 node RAC cluster (10.2.0.4). Table T1_TEMP is on ASSM tablespace. This DL insert from a ETL tool.
INSERT /*+ SYS_DL_CURSOR */ INTO T1_TEMP
("C1","C2","C3","C4","C5","C6"
) VALUES
(NULL,NULL,NULL,NULL,NULL,NULL)and they all observing 'gc current request' or "gc buffer busy" waits.. for several files with Block# 2. Wondeing why it's having so many waits on Block#2 ( I guess, t's file header block ).. Since it's a direct level Insert( Yes I have confirmed it's indeed DL insert), all it should aquire is TEMP segement above HWM and start inserting directly into DL. So just wondering why it's experincing gc related waits...
I tried to find object name by querying dba_extents but it does not return any thing for file# 717, block# =2 or for any of the file#/block#2 mentioned below.
See bellow..
   SID Wait State    EVENT                               P1                         P2              P3                    SEQ# % TotTime TEventTime(ms)  DistEvnts Avgtime(ms)/Evnt
   1758 WAITING       gc current request                  file#= 717                 block#= 2       id#= 33554445                   5.99         86.227          2           43.114
   1758 WAITING       gc buffer busy                      file#= 1058                block#= 2       id#= 65549                      5.33         76.694          2           38.347
   1758 WAITING       gc current request                  file#= 738                 block#= 2       id#= 33554445                   5.29         76.147          2           38.074
   1758 WAITING       gc current request                  file#= 710                 block#= 2       id#= 33554445                   5.18         74.650          2           37.325
   1758 WAITING       gc current request                  file#= 719                 block#= 2       id#= 33554445                   5.13         73.901          2           36.950
   1758 WAITING       gc current request                  file#= 716                 block#= 2       id#= 33554445                   4.82         69.408          2           34.704
   1758 WAITING       gc current request                  file#= 13                  block#= 2       id#= 33554445                   4.80         69.091          2           34.546
   1758 WAITING       gc current request                  file#= 707                 block#= 2       id#= 33554445                   4.73         68.141          2           34.070
   1758 WAITING       gc current request                  file#= 726                 block#= 2       id#= 33554445                   4.45         64.138          2           32.069
   1758 WAITING       gc current request                  file#= 702                 block#= 2       id#= 33554445                   4.43         63.734          2           31.867
   1758 WAITING       gc current request                  file#= 1550                block#= 2       id#= 33554445                   4.34         62.496          2           31.248
   1758 WAITING       gc current request                  file#= 739                 block#= 2       id#= 33554445                   3.49         50.198          2           25.099
   1758 WAITING       gc current request                  file#= 1                   block#= 73575   id#= 33554433                   3.25         46.829          1           46.829
   1758 WAITING       gc current request                  file#= 900                 block#= 2       id#= 33554445                   3.01         43.315          1           43.315
   1758 WAITING       gc buffer busy                      file#= 750                 block#= 2       id#= 65549                      2.88         41.501          1           41.501
   1758 WAITING       gc buffer busy                      file#= 901                 block#= 2       id#= 65549                      2.58         37.094          1           37.094
   1758 WAITING       gc buffer busy                      file#= 744                 block#= 2       id#= 65549                      2.41         34.762          1           34.762
   1758 WAITING       gc buffer busy                      file#= 745                 block#= 2       id#= 65549                      2.41         34.733          1           34.733
   1758 WAITING       gc buffer busy                      file#= 742                 block#= 2       id#= 65549                      2.39         34.474          1           34.474
   1758 WAITING       gc current request                  file#= 744                 block#= 2       id#= 33554445                   2.37         34.070          1           34.070
   1758 WAITING       gc current request                  file#= 749                 block#= 2       id#= 33554445                   2.36         34.042          1           34.042
   1758 WAITING       gc current request                  file#= 898                 block#= 2       id#= 33554445                   2.36         34.013          1           34.013
   1758 WAITING       gc current request                  file#= 742                 block#= 2       id#= 33554445                   2.31         33.235          1           33.235
   1758 WAITING       gc buffer busy                      file#= 749                 block#= 2       id#= 65549                      2.30         33.149          1           33.149
   1758 WAITING       gc current request                  file#= 750                 block#= 2       id#= 33554445                   2.29         32.918          1           32.918
   1758 WAITING       gc current request                  file#= 745                 block#= 2       id#= 33554445                   2.25         32.400          1           32.400
   1758 WAITING       gc current request                  file#= 901                 block#= 2       id#= 33554445                   2.18         31.392          1           31.392
   1758 WAITING       gc buffer busy                      file#= 739                 block#= 2       id#= 65549                      1.74         24.998          1           24.998
   1758 WAITING       SQL*Net message from client         driver id= 1413697536      #bytes= 1       = 0                             1.58         22.694          5            4.539
   1758 WORKING       On CPU / runqueue                                                                                       .73         10.512         64             .164
   1758 WAITING       gc cr request                       file#= 4                   block#= 825     class#= 177                      .08          1.210          1            1.210
   1758 WAITING       gc cr request                       file#= 1                   block#= 73575   class#= 1                        .08          1.152          1            1.152
   1758 WAITING       gc buffer busy                      file#= 1055                block#= 2       id#= 65549                       .06           .922          2             .461
   1758 WAITING       SQL*Net more data from client       driver id= 1413697536      #bytes= 40149   = 0                              .06           .806          1             .806Can anyone explain this??

Answer 1

Hi Tanel,
From the OP's Inputs
Buffer buy wait is with respect to : id#= 65549 (P3) in block#= 2
gc current request with respect to : id#= 33554445 in block#= 2
As you stated from first post,
Block #2 is where the file level space usage bitmap header lives in locally managed tablespaces. This manages the file level space, not segment level.
If you're doing heavy parallel insertions with small fixed LMT extent size then when segments are extending frequently you can end up with contention on that block. First of all you way for troubling shooting this problem is pretty good..!!
My doubt, what is the reason that it would "end up with contention", since as user is performing "parallel insertions" small fixed LMT Extent size. In this case does row extending would not happen..??
From the second post
The ID# is a partially a bitfield and partially the class# of block involved in the wait.
The lower two bytes indicate the block class. Which in your case is mostly 13 (if you look only at the lower 2 bytes of the values, e.g. 65549 - power(2,16) = 13.How about the "33554445" - (33554432 ->2^25) = 13
I would be thankful to if you could added some inputs.
- Pavan Kumar N