Question on fragmentation and ALTER INDEX REBUILD/REORGANIZE not effecting it

Question

The problem I ran into was troubleshooting a sporadically slow singleton lookup on a Clustered Index in a table with about 8 million rows, which is a separate issue I may need to submit for help. That aside, during that troubleshooting I noticed fragmentation
on the Unique Clustered Index (it's a VARCHAR(20)), and then noticed the fragmentation in other indexes on this table. See sys.dm_db_index_physical_stats and DBCC SHOWCONTIG results below.
SELECT
 substring(OBJECT_NAME(i.object_id),1,30) AS TableName,
 substring(i.name,1,40) AS TableIndexName,
 i.index_id, phystat.index_level,
 phystat.avg_fragmentation_in_percent 
FROM
 sys.dm_db_index_physical_stats(DB_ID(), NULL, NULL, NULL, 'DETAILED') phystat inner JOIN sys.indexes i 
ON i.object_id = phystat.object_id 
AND i.index_id = phystat.index_id WHERE
OBJECT_NAME(i.object_id) = 'CONSUMERS'
TableName                      TableIndexName                           index_id    index_level avg_fragmentation_in_percent
CONSUMERS                      UNI2K_CONSUMERS                          1           0      
    0.154827346202469
CONSUMERS                      UNI2K_CONSUMERS                          1           1      
    35.2941176470588
CONSUMERS                      UNI2K_CONSUMERS                          1           2      
    0
CONSUMERS                      PK__CONSUMER__7F6B0B8B286302EC           2           0           0.336078590685822
CONSUMERS                      PK__CONSUMER__7F6B0B8B286302EC           2           1           100
CONSUMERS                      PK__CONSUMER__7F6B0B8B286302EC           2           2           0
CONSUMERS                      UNI1K_CONSUMERS                          3           0      
    0.156451316031658
CONSUMERS                      UNI1K_CONSUMERS                          3           1      
    61.1510791366906
CONSUMERS                      UNI1K_CONSUMERS                          3           2      
    0
CONSUMERS                      IDX1_CONSUMERS                           4           0      
    0.215271389144434
CONSUMERS                      IDX1_CONSUMERS                           4           1      
    40
CONSUMERS                      IDX1_CONSUMERS                           4           2      
    100
CONSUMERS                      IDX1_CONSUMERS                           4           3      
    0
CONSUMERS                      IDX2_CONSUMERS                           5           0      
    0.222614710968834
CONSUMERS                      IDX2_CONSUMERS                           5           1      
    38.6281588447653
CONSUMERS                      IDX2_CONSUMERS                           5           2      
    75
CONSUMERS                      IDX2_CONSUMERS                           5           3      
    0
(17 row(s) affected)
DBCC SHOWCONTIG('CONSUMERS') WITH ALL_INDEXES
DBCC SHOWCONTIG scanning 'CONSUMERS' table...
Table: 'CONSUMERS' (645577338); index ID: 1, database ID: 5
TABLE level scan performed.
- Pages Scanned................................: 70401
- Extents Scanned..............................: 8827
- Extent Switches..............................: 8843
- Avg. Pages per Extent........................: 8.0
- Scan Density [Best Count:Actual Count].......: 99.51% [8801:8844]
- Logical Scan Fragmentation ..................: 0.15%
- Extent Scan Fragmentation ...................: 23.76%
- Avg. Bytes Free per Page.....................: 47.2
- Avg. Page Density (full).....................: 99.42%
DBCC SHOWCONTIG scanning 'CONSUMERS' table...
Table: 'CONSUMERS' (645577338); index ID: 2, database ID: 5
LEAF level scan performed.
- Pages Scanned................................: 27077
- Extents Scanned..............................: 3402
- Extent Switches..............................: 3402
- Avg. Pages per Extent........................: 8.0
- Scan Density [Best Count:Actual Count].......: 99.47% [3385:3403]
- Logical Scan Fragmentation ..................: 0.34%
- Extent Scan Fragmentation ...................: 11.88%
- Avg. Bytes Free per Page.....................: 24.1
- Avg. Page Density (full).....................: 99.70%
DBCC SHOWCONTIG scanning 'CONSUMERS' table...
Table: 'CONSUMERS' (645577338); index ID: 3, database ID: 5
LEAF level scan performed.
- Pages Scanned................................: 54330
- Extents Scanned..............................: 6803
- Extent Switches..............................: 6805
- Avg. Pages per Extent........................: 8.0
- Scan Density [Best Count:Actual Count].......: 99.79% [6792:6806]
- Logical Scan Fragmentation ..................: 0.16%
- Extent Scan Fragmentation ...................: 7.03%
- Avg. Bytes Free per Page.....................: 50.3
- Avg. Page Density (full).....................: 99.38%
DBCC SHOWCONTIG scanning 'CONSUMERS' table...
Table: 'CONSUMERS' (645577338); index ID: 4, database ID: 5
LEAF level scan performed.
- Pages Scanned................................: 54350
- Extents Scanned..............................: 6808
- Extent Switches..............................: 6837
- Avg. Pages per Extent........................: 8.0
- Scan Density [Best Count:Actual Count].......: 99.36% [6794:6838]
- Logical Scan Fragmentation ..................: 0.22%
- Extent Scan Fragmentation ...................: 7.17%
- Avg. Bytes Free per Page.....................: 53.2
- Avg. Page Density (full).....................: 99.34%
DBCC SHOWCONTIG scanning 'CONSUMERS' table...
Table: 'CONSUMERS' (645577338); index ID: 5, database ID: 5
LEAF level scan performed.
- Pages Scanned................................: 54354
- Extents Scanned..............................: 6804
- Extent Switches..............................: 6846
- Avg. Pages per Extent........................: 8.0
- Scan Density [Best Count:Actual Count].......: 99.24% [6795:6847]
- Logical Scan Fragmentation ..................: 0.22%
- Extent Scan Fragmentation ...................: 7.13%
- Avg. Bytes Free per Page.....................: 53.8
- Avg. Page Density (full).....................: 99.33%
DBCC execution completed. If DBCC printed error messages, contact your system administrator.
This fragmentation I found shocking because I reorg nightly and have a weekly rebuild task running that I set up through the Maintenance Plan wizard (which I've verified has been running). So I attempted to reorg these manually (especially index ID: 1)
and to my shock the fragmentation % did not change at all. I then took the SQL provided by the Maintenance Plan for rebuilding the indexes and found that after running that it didn't change the fragementation % at all either (commands run shown below).
ALTER INDEX [IDX1_CONSUMERS] ON [dbo].[CONSUMERS] REBUILD PARTITION = ALL WITH ( PAD_INDEX  = OFF, STATISTICS_NORECOMPUTE  = OFF, ALLOW_ROW_LOCKS  = ON, ALLOW_PAGE_LOCKS  = ON, ONLINE = ON, SORT_IN_TEMPDB = ON )
GO
ALTER INDEX [IDX2_CONSUMERS] ON [dbo].[CONSUMERS] REBUILD PARTITION = ALL WITH ( PAD_INDEX  = OFF, STATISTICS_NORECOMPUTE  = OFF, ALLOW_ROW_LOCKS  = ON, ALLOW_PAGE_LOCKS  = ON, ONLINE = ON, SORT_IN_TEMPDB = ON )
GO
ALTER INDEX [UNI1K_CONSUMERS] ON [dbo].[CONSUMERS] REBUILD PARTITION = ALL WITH ( PAD_INDEX  = OFF, STATISTICS_NORECOMPUTE  = OFF, ALLOW_ROW_LOCKS  = ON, ALLOW_PAGE_LOCKS  = ON, IGNORE_DUP_KEY  = OFF, ONLINE = ON, SORT_IN_TEMPDB
= ON )
GO
ALTER INDEX [UNI2K_CONSUMERS] ON [dbo].[CONSUMERS] REBUILD PARTITION = ALL WITH ( PAD_INDEX  = OFF, STATISTICS_NORECOMPUTE  = OFF, ALLOW_ROW_LOCKS  = ON, ALLOW_PAGE_LOCKS  = ON, IGNORE_DUP_KEY  = OFF, ONLINE = ON, SORT_IN_TEMPDB
= ON )
GO
Fragmentation did not change until I performed a "CREATE ... DROP_EXISTING = ON" on the 4 non PK indexes and a manual rebuild of Primary Key offline not specifying any other parameters, which all seemed completely overkill to ensure the defragmentation
actually got done. Final sys.dm_db_index_physical_stats and DBCC SHOWCONTIG results below.
SELECT
 substring(OBJECT_NAME(i.object_id),1,30) AS TableName,
 substring(i.name,1,40) AS TableIndexName,
 i.index_id, phystat.index_level,
 phystat.avg_fragmentation_in_percent 
FROM
 sys.dm_db_index_physical_stats(DB_ID(), NULL, NULL, NULL, 'DETAILED') phystat inner JOIN sys.indexes i 
ON i.object_id = phystat.object_id 
AND i.index_id = phystat.index_id WHERE
OBJECT_NAME(i.object_id) = 'CONSUMERS'
TableName                      TableIndexName                           index_id    index_level avg_fragmentation_in_percent
CONSUMERS                      UNI2K_CONSUMERS                          1           0      
    0.0213458562356583
CONSUMERS                      UNI2K_CONSUMERS                          1           1      
    11.2426035502959
CONSUMERS                      UNI2K_CONSUMERS                          1           2      
    0
CONSUMERS                      PK__CONSUMER__7F6B0B8B286302EC           2           0           0.0460971112476951
CONSUMERS                      PK__CONSUMER__7F6B0B8B286302EC           2           1           14.2857142857143
CONSUMERS                      PK__CONSUMER__7F6B0B8B286302EC           2           2           0
CONSUMERS                      UNI1K_CONSUMERS                          3           0      
    0.0225314031431307
CONSUMERS                      UNI1K_CONSUMERS                          3           1      
    10.6194690265487
CONSUMERS                      UNI1K_CONSUMERS                          3           2      
    0
CONSUMERS                      IDX1_CONSUMERS                           4           0      
    0.0225318262045139
CONSUMERS                      IDX1_CONSUMERS                           4           1      
    10.7296137339056
CONSUMERS                      IDX1_CONSUMERS                           4           2      
    0
CONSUMERS                      IDX1_CONSUMERS                           4           3      
    0
CONSUMERS                      IDX2_CONSUMERS                           5           0      
    0.0225314031431307
CONSUMERS                      IDX2_CONSUMERS                           5           1      
    12.0171673819742
CONSUMERS                      IDX2_CONSUMERS                           5           2      
    0
CONSUMERS                      IDX2_CONSUMERS                           5           3      
    0
(17 row(s) affected)
DBCC SHOWCONTIG('CONSUMERS') WITH ALL_INDEXES
DBCC SHOWCONTIG scanning 'CONSUMERS' table...
Table: 'CONSUMERS' (645577338); index ID: 1, database ID: 5
TABLE level scan performed.
- Pages Scanned................................: 56217
- Extents Scanned..............................: 7029
- Extent Switches..............................: 7028
- Avg. Pages per Extent........................: 8.0
- Scan Density [Best Count:Actual Count].......: 99.99% [7028:7029]
- Logical Scan Fragmentation ..................: 0.02%
- Extent Scan Fragmentation ...................: 0.44%
- Avg. Bytes Free per Page.....................: 32.4
- Avg. Page Density (full).....................: 99.60%
DBCC SHOWCONTIG scanning 'CONSUMERS' table...
Table: 'CONSUMERS' (645577338); index ID: 2, database ID: 5
LEAF level scan performed.
- Pages Scanned................................: 26032
- Extents Scanned..............................: 3256
- Extent Switches..............................: 3255
- Avg. Pages per Extent........................: 8.0
- Scan Density [Best Count:Actual Count].......: 99.94% [3254:3256]
- Logical Scan Fragmentation ..................: 0.05%
- Extent Scan Fragmentation ...................: 0.31%
- Avg. Bytes Free per Page.....................: 11.1
- Avg. Page Density (full).....................: 99.86%
DBCC SHOWCONTIG scanning 'CONSUMERS' table...
Table: 'CONSUMERS' (645577338); index ID: 3, database ID: 5
LEAF level scan performed.
- Pages Scanned................................: 53259
- Extents Scanned..............................: 6659
- Extent Switches..............................: 6658
- Avg. Pages per Extent........................: 8.0
- Scan Density [Best Count:Actual Count].......: 99.98% [6658:6659]
- Logical Scan Fragmentation ..................: 0.02%
- Extent Scan Fragmentation ...................: 0.35%
- Avg. Bytes Free per Page.....................: 40.5
- Avg. Page Density (full).....................: 99.50%
DBCC SHOWCONTIG scanning 'CONSUMERS' table...
Table: 'CONSUMERS' (645577338); index ID: 4, database ID: 5
LEAF level scan performed.
- Pages Scanned................................: 53258
- Extents Scanned..............................: 6659
- Extent Switches..............................: 6658
- Avg. Pages per Extent........................: 8.0
- Scan Density [Best Count:Actual Count].......: 99.98% [6658:6659]
- Logical Scan Fragmentation ..................: 0.02%
- Extent Scan Fragmentation ...................: 0.53%
- Avg. Bytes Free per Page.....................: 40.3
- Avg. Page Density (full).....................: 99.50%
DBCC SHOWCONTIG scanning 'CONSUMERS' table...
Table: 'CONSUMERS' (645577338); index ID: 5, database ID: 5
LEAF level scan performed.
- Pages Scanned................................: 53259
- Extents Scanned..............................: 6659
- Extent Switches..............................: 6658
- Avg. Pages per Extent........................: 8.0
- Scan Density [Best Count:Actual Count].......: 99.98% [6658:6659]
- Logical Scan Fragmentation ..................: 0.02%
- Extent Scan Fragmentation ...................: 0.59%
- Avg. Bytes Free per Page.....................: 40.5
- Avg. Page Density (full).....................: 99.50%
DBCC execution completed. If DBCC printed error messages, contact your system administrator.
For the record, here's the version I'm running:
select @@VERSION
Microsoft SQL Server 2008 R2 (SP1) - 10.50.2500.0 (X64) 
 Jun 17 2011 00:54:03 
 Copyright (c) Microsoft Corporation
 Enterprise Edition (64-bit) on Windows NT 6.1 <X64> (Build 7601: Service Pack 1)
In summary my question is - why didn't ALTER INDEX ... REBUILD/REORGANIZE modify the index_level 1 fragmentation as reported by sys.dm_db_index_physical_stats, nor would it correct the Extent Scan Fragmentation as reported by DBCC SHOWCONTIG?

Answer 1

Hi Brian.cs ,
SQL Server will not rebuild indexes that are not large enough. Could you please have a look a the
fragment_count which is  one of the fields in the
sys.dm_db_index_physical_stats view to check whether it is low or not ?
Best Regards,
Peja
Please remember to click "Mark as Answer" on the post that helps you, and to click "Unmark as Answer" if a marked post does not actually answer your question. This can be beneficial to other community members reading the thread.
Peja, here's the information you requested, and of course this was after I dropped/recreated because rebuild didn't seem to actually address the fragmentation - the index I was most concerned about at index_level = 0 showed a fragment_count = 8922. FYI this
table has over 8 million rows. And apologies for the delayed response, was on vacation over the near year.
SELECT
 substring(OBJECT_NAME(i.object_id),1,30) AS TableName,
 substring(i.name,1,40) AS TableIndexName,
 i.index_id, phystat.index_level,
 phystat.avg_fragmentation_in_percent, fragment_count 
FROM
 sys.dm_db_index_physical_stats(DB_ID(), NULL, NULL, NULL, 'DETAILED') phystat inner JOIN sys.indexes i 
ON i.object_id = phystat.object_id 
AND i.index_id = phystat.index_id WHERE
OBJECT_NAME(i.object_id) = 'CONSUMERS'
TableName                      TableIndexName                           index_id    index_level avg_fragmentation_in_percent
fragment_count
CONSUMERS                      UNI2K_CONSUMERS                          1           0        
  0.259780818806428            8922
CONSUMERS                      UNI2K_CONSUMERS                          1           1        
  39.4190871369295             86
CONSUMERS                      UNI2K_CONSUMERS                          1           2        
  0                            1
CONSUMERS                      PK__CONSUMER__7F6B0B8B286302EC           2           0           0.240887634434766    
       5182
CONSUMERS                      PK__CONSUMER__7F6B0B8B286302EC           2           1           80.3738317757009    
        84
CONSUMERS                      PK__CONSUMER__7F6B0B8B286302EC           2           2           0          
                 1
CONSUMERS                      UNI1K_CONSUMERS                          3           0        
  0.0661472879611936           8532
CONSUMERS                      UNI1K_CONSUMERS                          3           1        
  53.4883720930233             86
CONSUMERS                      UNI1K_CONSUMERS                          3           2        
  0                            1
CONSUMERS                      IDX1_CONSUMERS                           4           0        
  0.192426334498663            8598
CONSUMERS                      IDX1_CONSUMERS                           4           1        
  31.5315315315315             95
CONSUMERS                      IDX1_CONSUMERS                           4           2        
  85.7142857142857             7
CONSUMERS                      IDX1_CONSUMERS                           4           3        
  0                            1
CONSUMERS                      IDX2_CONSUMERS                           5           0        
  0.189494094835184            8613
CONSUMERS                      IDX2_CONSUMERS                           5           1        
  31.8840579710145             97
CONSUMERS                      IDX2_CONSUMERS                           5           2        
  85.7142857142857             7
CONSUMERS                      IDX2_CONSUMERS                           5           3        
  0                            1