Bit Map Join Index Creation

Hi,
I have a db of version 10.2.0.3 on a Unix box. This is used for DW.
I have a dbms job which is basically creates 6 bit map join indexes between two tables parallelly. One of the table (Fact table) has almost 170 million data and the other one (Dimension table) has 1.2 million data. Fact table is partioned and the indexes are local index.
For the last couple of days, this job is taking almost 70+ minutes whereas previously, it used to take around 44 to 48 minutes.
My pga_aggregate_size is 50GB and temp tablespace size is 33GB.
Per day 1 million data gets populated in the Fact table.
It used to run fine earlier. One fine morning we found this performance issue.
It will be really helpful, if we get some suggesstion about the tuning options.
Regards,
Soumyajit

Jeet wrote:
I have a db of version 10.2.0.3 on a Unix box. This is used for DW.
I have a dbms job which is basically creates 6 bit map join indexes between two tables parallelly. One of the table (Fact table) has almost 170 million data and the other one (Dimension table) has 1.2 million data. Fact table is partioned and the indexes are local index.
For the last couple of days, this job is taking almost 70+ minutes whereas previously, it used to take around 44 to 48 minutes.
My pga_aggregate_size is 50GB and temp tablespace size is 33GB.
Per day 1 million data gets populated in the Fact table.
It used to run fine earlier. One fine morning we found this performance issue.
It will be really helpful, if we get some suggesstion about the tuning options.Soumyajit,
since you probably don't have suitable data available from those runs (by "data" I mean e.g. SQL traces with row source operation statistics, wait event information and number of I/Os etc.) where it used to be faster it's going to be hard to tell where the difference comes from since there are many possibilities.
If really nothing else has changed apart from the data volume then one possible explanation could be that the join operation that is required to populate the bitmap join index switched from e.g. single pass to multi-pass due to the increased data volume.
Another explanation could be that one of the tables used to be cached in the buffer cache (small table threshold) but has now exceeded the default 2% limit for cached table scans and therefore requires more physical I/O than previously.
But there is a multitude of other possibilities starting from changes to the software/hardware (patches to Oracle, O/S, storage etc.), concurrent processes running at the same time, changes to the logging attributes of the tablespace/database, different execution plan due to statistics change, ASMM automatic cache buffer shrinkage etc. etc.
It definitely makes sense to trace the execution now to have that information available, furthermore you could try to check if the usage of the TEMP tablespace has increased in comparison to previous runs which could indicate above mentioned switch to a less efficient pass mode of the join.
Can you pinpoint already if only particular bitmap join indexes take longer than before or all six are slower than before?
Do I understand this correctly, that you run six jobs in parallel to create the indexes, or is it one job that runs the create index statements one after the other?
And do you use the PARALLEL DDL option to create each individual index? What about LOGGING/NOLOGGING?
By the way, what is reason that you rebuild the indexes every day? Is this after a load into a particular partition to rebuild the indexes of that partition after the data load? Or is it a rebuild of all partitions?
Regards,
Randolf
Oracle related stuff blog:
http://oracle-randolf.blogspot.com/
SQLTools++ for Oracle (Open source Oracle GUI for Windows):
http://www.sqltools-plusplus.org:7676/
http://sourceforge.net/projects/sqlt-pp/

Similar Messages

Does BI 7.0 use bit map join indexes?

I have seen several threads on SDN which show that SAP BI is using bit map indexes with Oracle database. However, it is not clear whether BI utilizes bit map join indexes or just bit map index. Bit map index is on a single table whereas bitmap join index is on a join of two or more tables.
It will make sense for SAP BI to use bit map join indexes with star transformations where a join is done among fact and dimension tables.

> Thanks. Another question, are bit map indexes used both for F and E fact tables or just for F. I am not using real-time cubes (where SAP BI uses B-Tree indexes for F table to avoid dead locks).
Why don't you simply check this in your BI system yourself?
And, yes, of course it uses bitmap indexes for both fact table types.
E-fact tables are the same as F-Facttables except that they contain the compressed/rolled up data.
regards,
Lars

BIT MAP & BTREE INDEX

Hi Friends,
How to create the bitmap index? Pls given the syntax
When should we create the Btree index?
Given one example btree index
regs
renga

Hi Vinas,
generally bitmap is not good for data on which u are making dml operations. Since there are lock issues and index size issues.
1) lock issues bitmap index has different structre than b-tree one. There are flag, lock byte, value, start rowid, end rowid and bitmap in your index entry. So when you change one bitmap you have to lock all rows in your row id's range. In btree you lock just one row.
2) when you update index u mark one entry as deleted and create new one with new value. When you look on structure described in point 1) you get know that index entry could be quite big (generally bigger than index entry for btree index). So this is the way how your bitmap index can growth.
Both these points are really dangerous from the point of view of performance and it's reason why it's not good idea use bitmap index on columns with dml activity.
Jakub.

Creating a bit map index on a partitioned table

Dear friends,
I am trying to create a bitmap index on a partitioned table but am receiving the following ORA error. Can you please let me know on how to create a local bit map index as the message suggests?
ERROR at line 1:
ORA-25122: Only LOCAL bitmap indexes are permitted on partitioned tables
Trying to use the keyword local in front leads to wrong syntax.
Thanks in advance !!
Somnath

ORA-25122 Only LOCAL bitmap indexes are permitted on partitioned tables
Cause: An attempt was made to create a global bitmap index on a partitioned table.
Action: Create a local bitmap index instead
Example of a Local Index Creation
CREATE INDEX employees_local_idx ON employees (employee_id) LOCAL;
Example is about btree and I think it will work for bitmap also.

Creation of logical index terminated. Join index create error.

Hi to all,
After running for a few weeks fine. See my earlier message Delete and recreate BWA index of a cube in process chain.
Chains are "terminating" again. I get the following error messages:
Creation of logical index '[Technical Cube_name]' terminated. Error text:
Join index create error;index=bwp_[technical cube_name],location
=nlscli63:30203 2433
Creation of BIA index for InfoCube '[Technical Cube_name]' terminated during activation
A manual deletion by the program RSDDTREX_INDEX_DELETE and a recreation in RSDDV is necessarry to get the index active.
Has anyone a clue on beforehand what is happening or same experiences, before i put in a SAP OSS message.
Regards,
René

Hi
Firstly please ensure that no error exists for the cube when you are
trying to create the index. You can check that in RSRV. Also you need
to check if no change run or loading is occuring.
Please also check that you are not using the backup mode mutual
as this can cause the problem, please refer to the SAP note 1383228.
Best Regards,
Des Gallagher

How to set the option Bit-mapped indexes to true ?

Hi,
I a not a an Oracle Administrator. Usually, I just use Oracle as developper. I have got a Oracle 10g database. I can not import data in my database because the Bit-mapped options is not set.
When I type
select banner from v$version
I get this result :
BANNER
Oracle Database 10g Release 10.2.0.1.0 - Production
PL/SQL Release 10.2.0.1.0 - Production
CORE 10.2.0.1.0 Production
TNS for Linux: Version 10.2.0.1.0 - Production
NLSRTL Version 10.2.0.1.0 - Production
I run the import and I get this error :
IMP-00017: following statement failed with ORACLE error 439:
"CREATE BITMAP INDEX "ID_ACC_DO" ON "ES_AUDIT_CONNEXION_ARCH" ("DATE_OUVERTU"
"RE" ) PCTFREE 10 INITRANS 2 MAXTRANS 255 STORAGE(INITIAL 33554432 FREELIST"
"S 1 FREELIST GROUPS 1 BUFFER_POOL DEFAULT) TABLESPACE "IND_XL" LOGGING"
IMP-00003: ORACLE error 439 encountered
ORA-00439: feature not enabled: Bit-mapped indexes
I typed the command :
SELECT * FROM V$OPTION WHERE PARAMETER LIKE 'Bit-mapped%';
I get the result :
PARAMETER
VALUE
Bit-mapped indexes
FALSE
How to set this parameter to true ?
Best regards,
Xavier

I beleiev that you are on Standard Edition.Bitmap indexes can be used only when you are working with Enterprise edition.
SQL> select * from V$option where parameter like '%Bit%';
PARAMETER
VALUE
Bit-mapped indexes
TRUE
SQL> select * from V$version;
BANNER
Oracle Database 11g *Enterprise Edition* Release 11.1.0.6.0 - Production
PL/SQL Release 11.1.0.6.0 - Production
CORE 11.1.0.6.0 Production
TNS for 32-bit Windows: Version 11.1.0.6.0 - Production
NLSRTL Version 11.1.0.6.0 - Production
SQL>I guess you see the entry coming of Enterprise Edition in my banner.This is missing in your V$version, so you are using Standard Edition install.
You need to move to Enterprise edition to use Bitmap Indexes.
HTH
Aman....
Edited by: Aman.... on Dec 11, 2008 3:09 PM
added a little more.

Table name where I can see the Secondary bit map Index for Cube Loads in BI

Hello Friends I want to know the table name where I can see the Secondary bit map Index that gets created with each request that get loaded to Cube so each 1 request will be multiplied by 7 dimension and 7x1 = 7 local secondary bit map index will be created where is that table which show me that .
Under Oracle every uncompressed request corresponds to one partition in the
F facttable of the corresponding infocube. This partition also brings along
a full set of secondary local bitmap index partitions.
If you have a cube with 10 uncompressd requests and 10 dimensions (every
dimension brings with it a local bitmap index) you have 100 partitions for
the table itself and 10 x 10 index partitions which makes a total of
100 database objects. where is that table in BI WHICH STORE THIS INFORMATION
Also if I have E Table with no entries for future 2009 dentries will that be an issue with blank partition ion E Fact table if yes please suggest how to resolve this issue

Hi Soniya
Information about BI objects is stored in the RSD* tables. For example:
Infocubes: RSDCUBE / RSDCUBET
Aggregates: RSDDAGGR_V / RSDDAGGRT
You can see information on abap dictionary object as well in transactions SE11 / SE14. You will have to know the real table name of the cube, should be something like /BIC/F* for the uncompressed and /BIC/E* for the compressed table.
Best regards, Michael

How bit map index works?

I am not getting an example for bitmap index mechanism.
Supposing their is a table T1 with column c1 having records
Y
N
N
Y
Y
N
and i query the table like
select * from t1 where c1 = 'Y'
How bit map index functions?
thanks,
Vinodh

I think you should check the google's new feature to explore -> [How bitmap Index Works|http://www.google.co.in/search?hl=en&rlz=1G1GGLQ_ENIN333&q=how+bitmap+index+works&meta=&aq=0&oq=how+bitma]
Regards.
Satyaki De.

Bit Map Indexing

Hi All
i have a table with 5.5 million records and i have a fields (project id and owner id) with 8400 and 507 distinct values. For reporting reasons i want to create a bit map index on the project id. My question is even though it makes the query faster will it take more space? How can i calculate how much space it takes? Secondly if the ROI is not good for creating index on the project id can i creat index on the second field (owner id) and can still make the query faster?
Thanks

897837 wrote:
i have a table with 5.5 million records and i have a fields (project id and owner id) with 8400 and 507 distinct values. For reporting reasons i want to create a bit map index on the project id. My question is even though it makes the query faster will it take more space? Strange question - indexes are an overhead. Space is needed for an index structure. The structure needs to be maintained when data is added, changed or deleted.
The question is whether these overheads are justified by the increase in performance when reading the table structure for selected rows and using the index structure as part of process to get to those selected rows.
So it is not only space that is the coin of payment for indexes. There's also extra CPU to burn in order to maintain those indexes.
How can i calculate how much space it takes? Well, today I would say that space is much less of a consideration. Disk space is cheap. Overheads to maintain additional data structures are not cheap. So I would first look at the performance impact. If you need to maintain a 100 rows/sec insert rate for example, you can ill afford overheads that do not pay back with time gained when reading data.
I would also want to measure just how much the index reduces the workload for reading data. Does it decrease the read I/O load by 1% or 50% ?
You need to quantify the impact of that index to determine whether it is worthwhile. And it does not mean merely calculating the cost ito space used and then see how fast it makes a query by using a stop watch or timer (the wrong metric for measuring SQL performance).

Bit Map index in DW application

Hi
I have a DW database having a table contains 9 million records. Normally for DW application bit map index is preffered if the cardinality is low.
I want to create an index on one of the column having the cardinality is about millions, so which index is preferrable?. If I go with normal indexing any performance degradation will be happen for the application? or if I used bit map indexing any improvement will be there?
Many thanks

I would go for normal index, there is a certain additional load when inserting, updating, deleting, but in that case a bitmapped index wouldn't be good idea anyway.

Bit-mapped Index

Hi,
I run this
CREATE BITMAP INDEX bx_time_period ON time_period (time_period_cd)
and i got this error:
ORA-00439: feature not enabled: Bit-mapped
I am using a Oracle 9i SE.
How can i do to fix this error ?
Do i need to install any patch
Thanks.
Leo

About differences between normal an bitmap indexes you can see [url http://download-west.oracle.com/docs/cd/B14117_01/server.101/b10743/schema.htm#sthref883]The documentation
In general, bitmap indexes are useful in application with huge data, low cardinality keys and low level of concurrent transactions (i.e. DataWarehouse), while normal indexes are tipically suitable for OLTP applications.

ORA-00439: feature not enabled: Bit-mapped indexes

Do I need to change any parameter to enable this feature or I just can't use it in Oracle XE?
Thanks,
Allwyn

Hi Allwyn,
the feature list of XE says that Bitmapped index and bitmapped join index are not supported ( http://download-east.oracle.com/docs/cd/B25329_01/doc/relnotes.102/b25327/toc.htm#BEIDGBCD ) . I guess it won't work then.
We get the function based index, though.
Regards,
~Dietmar.

Bitmap join Index

I'm using oracle9i.2.
When I execute select * from v$option;
Bit-mapped indexes FALSE
How can I enable Bitmap Join Index ?
Is bitmap join index feature enable in Standard Edition ?

Nanda Kishore,
Unfortunately you cannot currently define bitmap join indexes in Warehouse Builder. We are planning to include those in an upcoming release.
Mark.

Index creation online - performance impact on database

hi,
I have oracle 11.1.0.7 database running on Linux as 3 node RAC.
I have a huge table which has more than 255 columns and is about 400GB in size which is also highly fragmented because of constant DML activities.
Questions:
1. For now i am trying to create an index Online while the business applications are running.
Will there be any performance impact on the database to create index Online on a single column of a table 'TBL' while applications are active against the same table? So basically my question will index creation on a object during DML operations on the same object have performance impact on the database? is there a major performance impact difference in the database in creating index online and not online?
2. I tried to build an index on a column which has NULL value on this same table 'TBL' which has more than 255 columns and is about 400GB in size highly fragmented and has about 140 million rows.
I requested the applications to be shutdown, but the index creation with parallel of 4 a least took more than 6 hours to complete.
We have a Pre-Prod database which has the exported and imported copy of the Prod data. So the pre-Prod is a highly de-fragmented copy of the Prod.
When i created the same index on the same column with NULL, it only took 15 minutes to complete.
Not sure why on a highly fragmented copy of Prod it took more than 6 hours compared to highly defragmented copy of Pre-Prod where the index creation took only 15 minutes.
Any thoughts would be helpful.
Thanks.
Phil.

How are you measuring the "fragmentation" of the table ?
Is the pre-prod database running single instance or RAC ?
Did you collect any workload stats (AWR / Statspack) on the pre-prod and production systems while creating (or failing to create) the index ?
Did you check whether the index creation ended up in-memory, single pass or multi pass in in the two environments ?
The commonest explanation for this type of difference is two-fold:
a) the older data needs a lot of delayed block cleanout, which results in a lot of random I/O to the undo tablespace - slowing down I/O generally
b) the newer end of the table is subject to lots of change, so needs a lot of work relating to read-consistency - which also means I/O on the undo system
-- UPDATED: but you did say that you had stopped the application so this bit wouldn't have been relevant.
On top of this, an online (re)build has to lock the table briefly at the start and end of the build, and in a busy system you can wait a long time for the locks to be acquired - and if the system has been busy while the build has been going on it can take quite a long time to apply the journal file to finish the index build.
Regards
Jonathan Lewis

Parallel Index creation takes more time...!!

OS - Windows 2008 Server R2
Oracle - 10.2.0.3.0
My table size is - 400gb
Number of records - 657,45,95,123
my column definition first_col varchar2(22) ; -> I am creating index on this column
first_col -> actual average size of column value is 10
I started to create index on this column by following command
CREATE INDEX CALL_GROUP1_ANO ON CALL_GROUP1(A_NO) LOCAL PARALLEL 8 NOLOGGING COMPRESS ;
-> In my first attempt after three hours I got an error :
ORA-01652: unable to extend temp segment by 128 in tablespace TEMP
So I increased the size of temp tablespace to 380GB ,Because i expect the size of first_col index this much.
-> In my second attempt Index creation is keep going even after 17 hours...!!
Now the usage of temp space is 162 GB ... still it is growing..
-> I checked EM Advisor Central ADDM :
it says - The PGA was inadequately sized, causing additional I/O to temporary tablespaces to consume significant database time.
1. why this takes this much of Temp space..?
2. It is required this much of time to CREATE INDEX in parallel processing...? more than 17 hrs
3. How to calculate and set the size of PGA..?

OraFighter wrote:
Oracle - 10.2.0.3.0
My table size is - 400gb
Number of records - 657,45,95,123
my column definition first_col varchar2(22) ; -> I am creating index on this column
first_col -> actual average size of column value is 10
I started to create index on this column by following command
CREATE INDEX CALL_GROUP1_ANO ON CALL_GROUP1(A_NO) LOCAL PARALLEL 8 NOLOGGING COMPRESS ;
Now the usage of temp space is 162 GB ... still it is growing..The entire data set has to be sorted - and the space needed doesn't really vary with degree of parallelism.
6,574,595,123 index entries with a key size of 10 bytes each (assuming that in your choice of character set one character = one byte) requires per row approximately
4 bytes row overhead 10 bytes data, 2 bytes column overhead for data, 6 bytes rowid, 2 bytes column overhead for rowid = 24 bytes.
For the sorting overheads, using the version 2 sort, you need approximately 1 pointer per row, which is 8 bytes (I assumed you're on 64 bit Oracle on this platform) - giving a total of 32 bytes per row.
32 * 6,574,595,123 / 1073741824 = 196 GB
You haven't said how many partitions you have, but you might want to consider creating the index unusable, then issuing a rebuild command on each partition in turn. From "Practical Oracle 8i":
<blockquote>
In the absence of partitioned tables, what would you do if you needed to create a new index on a massive data set to address a new user requirement? Can you imagine the time it would take to create an index on a 450M row table, not to mention the amount of space needed in the temporary segment. It's the sort of job that you schedule for Christmas or Easter and buy a couple of extra discs to add to the temporary tablespace.
With suitably partitioned tables, and perhaps a suitably friendly application, the scale of the problems isn't really that great, because you can build the index on each partition in turn. This trick depends on a little SQL feature that appears to be legal even though I haven't managed to find it in the SQL reference manual:
     create index big_new_index on partitioned_table (colX)
     local
     UNUSABLE
     tablespace scratchpad
The key word is UNUSABLE. Although the manual states that you can 'alter' an index to be unusable, it does not suggest that you can create it as initially unusable, nevertheless this statement works. The effect is to put the definition of the index into the data dictionary, and allocate all the necessary segments and partitions for the index - but it does not do any of the real work that would normally be involved in building an index on 450M rows.
</blockquote>
(The trick was eventually documented a couple of years after I wrote the book.)
Regards
Jonathan Lewis

Bit Map Join Index Creation

Similar Messages

Maybe you are looking for