Table and Index compression in data warehouse - thoughts?

Hi,
We have a data warehouse with large fact tables and materialized views of this data.
Approx 3 million inserts per day week-ends about 12 million.
The fact tables we have expected to have 200 million, and couple with 1-3 billion.
Tables partitioned and have bitmap indexes.
Just wondered what thoughts were about compressing large fact tables and mviews both from point of view of ETL into them and reporting from them afterwards.
I take it, can compress/uncompress accordingly without any problem?
Many Thanks

After compression, most SELECT statements would not get slower. Actually, many can get faster due to reduced IO and buffer needs.
The situation with DMLs is more complex. It depends on the exact compression options (basic or advanced) and the DML (INSERT,UPDATE, direct load,..),but generally DML are negatively affected by compression.
In a Data Warehouses (DWs), it is usually quite beneficial to compress partitions or tables that contain data that is not supposed to be modified (read only or read mostly). Please note that in many cases you do not have to compress while you are loading the data – you can do that later.
You can also consider compressing some of your B-tree indexes (if you use them in your DW system).
Iordan Iotzov
http://iiotzov.wordpress.com/

Similar Messages

Table and index compression

Hello all,
is there any command to take all table and/or all indexes in one tablespace to compress yes???
Regards
Olaf

Hi Olaf,
strange.
I think it is related to the different kind of the quotation marks ' " ...
I modified the command a little bit by putting the double quotation marks " into single ones ' ' , the following runs on my environment:
coe6p001:db2x72 17> db2 "SELECT 'ALTER TABLE '||RTRIM(TABSCHEMA)||'.''"'||RTRIM(TABNAME)||'"'' COMPRESS YES ; ' from    syscat.tables where tabschema='SAPX72' and TYPE='T' and VOLATILE NOT LIKE 'C' and TBSPACEID=14"
1
ALTER TABLE SAPX72.'||RTRIM(TABNAME)||' COMPRESS YES ;
ALTER TABLE SAPX72.'||RTRIM(TABNAME)||' COMPRESS YES ;
ALTER TABLE SAPX72.'||RTRIM(TABNAME)||' COMPRESS YES ;
Edit: Sorry, I just saw, that it does produce the same thing:
So, i put the following into a script compr.sql
SELECT 'ALTER TABLE '||RTRIM(TABSCHEMA)||'."'||RTRIM(TABNAME)||'" COMPRESS YES;'
        from    syscat.tables
        where   tabschema='SAPX72'
        and     TYPE='T'
        and VOLATILE NOT LIKE 'C'
        and TBSPACEID=14
(do not forget the termination character ; at the end )
When I run it with db2 -tvf compr.sql, it produces the following output:
coe6p001:db2x72 21> db2 -tvf compr.sql
SELECT 'ALTER TABLE '||RTRIM(TABSCHEMA)||'."'||RTRIM(TABNAME)||'" COMPRESS YES;' from    syscat.tables where   tabschema='SAPX72' and     TYPE='T' and VOLATILE NOT LIKE 'C' and TBSPACEID=14
1
ALTER TABLE SAPX72."DYNPTXTLD" COMPRESS YES;
ALTER TABLE SAPX72."CO2MAP" COMPRESS YES;
ALTER TABLE SAPX72."CO2MAPINF" COMPRESS YES;
ALTER TABLE SAPX72."D344L" COMPRESS YES;
ALTER TABLE SAPX72."O2PAGINC" COMPRESS YES;
ALTER TABLE SAPX72."O2PAGRT" COMPRESS YES;
ALTER TABLE SAPX72."DDFTX" COMPRESS YES;
ALTER TABLE SAPX72."D342L" COMPRESS YES;
ALTER TABLE SAPX72."D346T" COMPRESS YES;
9 record(s) selected.
Edited by: Hans-Juergen Moldowan on Jul 8, 2011 8:21 AM

Separate table and index data in RAC database

Hi Experts,
Our database is Oracle11g RAC database. I need your expertise on this
Do we need to retain the table and index data in two different tablespaces for performance perspective in RAC database too?
Please share your practical experience…Thanks in advance.
Regards
Richard

g777 wrote:
In my opinion, if there is striping implemented then performance shouldn't degrade even if the index and table blocks are in one tablespace. Exactly.. striping is NOT a good idea at tablespace level as a tablespace is a logical storage device. It is very difficult to stripe comprehensively/correctly at that level, if not impossible.
Striping is a function of the actual storage system and need to happen at physical level. A proper RAID0 implementation.
So the question about multiple tablespaces for a performance increase should not be about striping - but about issues such as data management, block sizes, transportable tablespaces and so on.
Thus my question (at the OP) - what performance problems are expected and are these relevant to the number of tablespaces?

Compression for oracle database and index compression during import of data

Hi All,
I have a query , in order to import into oracle database and also have compression and index compression , do we have some kind of load args for r3load and also do we have to change the tpl file ?

Hello guy,
I did this kind of compression within migration project before.
I performed index compress first and then export -> import with table compress.
One thing you should take care, delete nocompress flag from TARGET.SQL (created by program SMIGR_CREATE_DDL, program SMIGR_CREATE_DDL created pure non-compression objects for these considered non-standard tables). For table columns > 255, we should not delete this flag.
Regarding to the index compress in source system, please check the following notes:
Note 1464156 - Support for index compression in BRSPACE 7.20
Note 1109743 - Use of Index Key Compression for Oracle Databases
Note 682926 - Composite SAP note: Problems with "create/rebuild index"
Best Regards,
Ning Tong

DB02 view is empty on Table and Index analyses DB2 9.7 after system copy

Dear All,
I did the Quality refresh by System copy export/import method. ECC6 on HP-UX DB29.7.
After Import Runstats status n Db02 for Table and Index analysis was empty and all value showing '-1'. Eventhough
a) all standard backgrnd job scheduled in sm36
b) Automatic runstats are enabled in db2 parameters
c) Reorgchk all scheduled periodically from db13 and already ran twice.
4) 'reorgchk update statistics on table all' was also ran on db2 level.
but Run stats staus in db02 was not getting updated. Its empty.
Please suggest.
Regards
Vinay

Hi Deepak,
Yes, that is possible (but only offline backup). But for the new features like reclaimable tablespace (to lower the high watermark)
it's better to export/import with systemcopy.
Also with systemcopy you can use index compression.
After backup and restore you can have also reclaimable tablespace, but you have to create new tablespaces
and then work with db6conv and online table move to move one tablespace online to the new one.
Best regards,
Joachim

Report to find all table and index sizes

Hi all,
Good day..
Is there any report.sql or so to find out the sizes of all the tables and indexes in a database.
thanks,
baskar.l

1.To get table size
What will be the table size if?
<or>
break on report
set line 200
COMPUTE SUM LABEL "Total Reclaimable Space" OF "KB Free Space" ON REPORT
column "Table Size" Format a20
column "Actual Data Size" Format a20
column "KB Free Space" Format "9,99,999.99"
select table_name,
round((blocks*8),2)||'kb' "Table size",
round((num_rows*avg_row_len/1024),2)||'kb' "Actual Data size",
pct_free,
round((blocks*8),2) - (round((blocks*8),2)*pct_free/100) - (round((num_rows*avg_row_len/1024),2)) "KB Free Space"
from user_tables
where round((blocks*8),2) - (round((blocks*8),2)*pct_free/100) - (round((num_rows*avg_row_len/1024),2)) > 0
order by round((blocks*8),2) - (round((blocks*8),2)*pct_free/100) - (round((num_rows*avg_row_len/1024),2)) desc
2.To get index size
How to size the Index
Hth
Girish Sharma

Import dumpfile with seperate tablespaces for table and index

Hi,
We have a schema for which the its tables are stored in seperate tablespace and indexes are stored in different tablespace. Now we have take full schema export. Now we want to import it on another schema. Now I want to know if the we have difference in the tablespace name we use REMAP_TABLESPACE clause of the impdp command but what about the seperate tablespace for table and indexes. How would Oracle handle this.
Regards,
Abbasi

Hi,
I hope you created the same tablespace structure on the target side if not so remap_tablespace option you have to use for specifying different tablespaces.Oracle will take care of putting data and index.Any how if a index is moved from one tablespace to other you have to rebuild them,once you rebuild them than only stattistics are gathered otherwise you
might face some performance issue.
Better option is to keep same tablespace structures in source and target environment.
Best regards,
Rafi.
http://rafioracledba.blogspot.com
Edited by: Rafi (Oracle DBA) on May 9, 2011 7:07 AM

Transaction DB02 = Missing Tables and Indexes shows under Secondary indexes from package $tmp

Hi SAP Gurus,
I noticed on our SAP business warehouse (BW) system (NetWeaver 7.0 based on AIX/Oracle 11g) in transaction DB02 => Diagnostics => Missing Tables and Indexes under Secondary indexes, 26 indexes, in package $tmp:
The indexes are defined, but they don't have been created on the database.
When I try to create them via the database tool, I am getting the warning:
Does the warning raise up, because they where defined in the $tmp package?
I was looking in SAP notes for a solution for this "warning" concerning the /BIC/F-Indexes but I didn't found yet anything helpful ...
Thanks for your help and ideas ...
Best regards
Carlos

Hi Carlos,
These are secondary indexes created during process chains in BI to improvise performance.
You may refer below SCN blogs on creation of these indices in BI.
Secondary Index - Improving DSO insert/read performance
How to Create Secondary Index on DSO in SAP BW
Hope this helps.
Regards,
Deepak Kori

Cannot INSERT records into Partitioned Spatial Table and Index

I am trying to tune our Spatial Storage by creating partitioning our spatial_entity table and index. I used the World Geographic Reference System (GEOREF) creating a partition for each 15 x 15 degree grid square assigning a partition key of decimal longitude, decimal_latitude. The build went OK, however when trying to insert a data record I receive an ORA-14400: Inserted partition key does not map to any partition.
I validated the CREATE(s), and all appears correct, but obviously something is not correct, which is prompting for expert help in this forum.
I would be very grateful for your help.
Below are the code snippets for the table and index, and an insert statement.
CREATE TABLE spatial_entity
     geoloc_type VARCHAR2 (60 BYTE) NOT NULL
,entity_id NUMBER NOT NULL
,metadata_xml_uuid VARCHAR2 (40 BYTE) NOT NULL
,geoloc MDSYS.sdo_geometry NOT NULL
,nee_method CHAR (1 BYTE) NOT NULL
,nee_status CHAR (1 BYTE) NOT NULL
,decimal_latitude NUMBER (15, 6) NOT NULL
,decimal_longitude NUMBER (15, 6) NOT NULL
PARTITION BY RANGE (decimal_longitude, decimal_latitude)
     PARTITION p_lt_0_90s
          VALUES LESS THAN (1, -90)
     ,PARTITION p_lt_0_75s
          VALUES LESS THAN (1, -75)
     ,PARTITION p_lt_0_60s
          VALUES LESS THAN (1, -60)
     ,PARTITION p_lt_0_45s
          VALUES LESS THAN (1, -45)
     ,PARTITION p_lt_0_30s
          VALUES LESS THAN (1, -30)
     ,PARTITION p_lt_0_15s
          VALUES LESS THAN (1, -15)
     ,PARTITION p_lt_0_0
          VALUES LESS THAN (1, 0)
     ,PARTITION p_lt_0_15n
          VALUES LESS THAN (1, 15)
     ,PARTITION p_lt_0_30n
          VALUES LESS THAN (1, 30)
     ,PARTITION p_lt_0_45n
          VALUES LESS THAN (1, 45)
     ,PARTITION p_lt_0_60n
          VALUES LESS THAN (1, 60)
     ,PARTITION p_lt_0_75n
          VALUES LESS THAN (1, 75)
     ,PARTITION p_lt_0_90n
          VALUES LESS THAN (1, maxvalue)
CREATE INDEX geo_spatial_ind ON spatial_entity (geoloc)
INDEXTYPE IS mdsys.spatial_index
PARAMETERS ('layer_gtype=MULTIPOINT TABLESPACE=GEO_SPATIAL_IND') LOCAL
(PARTITION p_lt_0_90s,
PARTITION p_lt_0_75s,
PARTITION p_lt_0_60s,
PARTITION p_lt_0_45s,
PARTITION p_lt_0_30s,
PARTITION p_lt_0_15s,
PARTITION p_lt_0_0,
PARTITION p_lt_0_15n,
PARTITION p_lt_0_30n,
PARTITION p_lt_0_45n,
PARTITION p_lt_0_60n,
PARTITION p_lt_0_75n,
PARTITION p_lt_0_90n,
INSERT INTO spatial_entity
     geoloc_type
     ,entity_id
     ,metadata_xml_uuid
     ,geoloc
     ,nee_method
     ,nee_status
     ,decimal_latitude
     ,decimal_longitude
VALUES
               'BATCH'
               ,0
               ,'6EC25B76-8482-4F95-E0440003BAD57EDF'
               ,"MDSYS"."SDO_GEOMETRY"
                    2001
                    ,8307
                    ,"MDSYS"."SDO_POINT_TYPE" (32.915286, 44.337902, NULL)
                    ,NULL
                    ,NULL
               ,'M'
               ,'U'
               ,32.915286
               ,44.337902
Thank you for you help.
Dave

Thank you for your quick reply. I did not post the entire CREATE script as it is quite long. The portion of the script that is applicable to the INSERT is:
,PARTITION p_lt_45e_90s
          VALUES LESS THAN (23, -90)
     ,PARTITION p_lt_45e_75s
          VALUES LESS THAN (23, -75)
     ,PARTITION p_lt_45e_60s
          VALUES LESS THAN (23, -60)
     ,PARTITION p_lt_45e_45s
          VALUES LESS THAN (23, -45)
     ,PARTITION p_lt_45e_30s
          VALUES LESS THAN (23, -30)
     ,PARTITION p_lt_45e_15s
          VALUES LESS THAN (23, -15)
     ,PARTITION p_lt_45e_0
          VALUES LESS THAN (23, 0)
     ,PARTITION p_lt_45e_15n
          VALUES LESS THAN (23, 15)
     ,PARTITION p_lt_45e_30n
          VALUES LESS THAN (23, 30)
     ,PARTITION p_lt_45e_45n
          VALUES LESS THAN (23, 45)
     ,PARTITION p_lt_45e_60n
          VALUES LESS THAN (23, 60)
     ,PARTITION p_lt_45e_75n
          VALUES LESS THAN (23, 75)
     ,PARTITION p_lt_45e_90n
          VALUES LESS THAN (23, maxvalue)
Or, I do not fully understand. Are you indicating that I must explcitly state the longitude in each clause,
e.g ,PARTITION p_lt_45e_45n
          VALUES LESS THAN (45, 45)
,PARTITION p_lt_45w_45n
          VALUES LESS THAN (-45, 45)
If so, that answers the question of why it cannot find a partition, however an Oracle White Paper "Oracle Spatial Partitioning Best Practices" Sept 2004, discusses multi column partitioning such as represented by this problem, and gives an INSERT statement example of :
CREATE TABLE multi_partn_table (in_date DATE,
geom SDO_GEOMETRY, x_value NUMBER, y_value NUMBER)
PARTITION BY RANGE (X_VALUE,Y_VALUE)
PARTITION P_LT_90W_45S VALUES LESS THAN (1,-45),
PARTITION P_LT_90W_0 VALUES LESS THAN (1,0),
PARTITION P_LT_90W_45N VALUES LESS THAN (1,45),
PARTITION P_LT_90W_90N VALUES LESS THAN (1,MAXVALUE
and as I am writing this I am seeing that I failed to include the longitude and latitude in the SDO_GEOMETRY clause, so it does appear tht I need to explicitly state the longitude valuues.
What is your judgement sir?
Dave

Fragmentation in tables and indexes with 9.2.0.5

Hi guys ,
I need of query to identified fragmentation in tables and indexes using Oracle 9i,someone could help me send that's querys
tks
spaulonci

Fragmentation by itself at the tablespace level doesn't mean any thing from the performance point of view, only from the storage capacity perspective, and this just in case fragments are not necessarily the same size as in the case of dictionary manged tablespaces.
You may be concerned about block density, which actually could lead to performance degradation. I suggest you to consider using coalesce instead of rebuilding in the case of indexes, and in the case of tables you should perform a segment reorganization. In the case of 9iR2 the coalesce command for table segments is not available so you will have to perform another segment reorganization strategy. If there are meaningful differences with the high water mark, average number of rows per block and chained rows then you will notice performance differences.
I suggest you to take metrics for block usage prior to perform an index coalesce operation, as well as prior to perform a data segment reorganization to assess if the reorganization effort is useful.
~ Madrid
http://hrivera99.blogspot.com/

Generate the tablespace clause for tables and indexes

How can I make Designer generate the tablespace clause for the create table and create index statements?
I assigned tables and indexes to Tablespaces objects in designer but they don't seem to have any effect on the generator.
I am using the latest version of Oracle Designer.
Thanks
Message was edited by:
bikerc

Guess I am not real clear what you want.
In the DB admin tab you will need to create the tablespace with the data file.
Then you need to assign the table space to the table.
You will need to generate from the DB Admin tab.
Hope this helps.
Michael

How Find space Used by Tables and Indexes

Dear All(s)
How i check space used (and number of rows) by each table and index in schema. ( can i check current space utilization and sysdate-date past in time)
How i can check used, and free space for each database and tablespace.
Thanks

You can always use the search feature
anyway
how to calculate the percentage of free space for a table in Oracle
Re: incorrect free space in dba_free_space
These links could give you all the necessary info

How to partition tables and indexes in this scenario?

So our situation is pretty simple. We have 3 tables.
A, B and C
the model is A->>B->>C
Currently A, B and C are range partitioned on a key created_date however it's typical that only C is every qualfied with created date. There is a foreign key from B -> A and C -> B
we have many queries where the data is identified by state that is indexed currently non partitioned on columns in A ... there are also indexes on the foreign keys that get from C -> B -> A. Again these are non partitioned indexes at this time.
It is typical that we qualify A on either account or user or both. There are indexes (non partitioned) on these
We have a problem now because many of the queries use leading wildcards ie. account like '%ACCOUNT' etc. This often results in large full table scans. Our solution has been to remove the leading wildcard but this isn't always possible.
We are wondering how we can benefit from partitioning and or sub partitioning table A. since it's partitioned on created_date but rarely qualify by that.
We are also wondering where and how we can benefit from either global partitioned index or local partitioned indexes on tables A. We suspect that the index on the foreign key from C to B could be a local partitioned index.
I am also wondering what impact pushing the state from A that's used to qualify A down to C would have any advantage.
C is the table that currently we qualify with the partition key so I figure if you also pushed down the state from A that's used to qualify the set of C's we want based on the set of B's we want based on the set of A thru qualfying on columns within A.
If we push down some of those values to C and simply use C when filtering I'm wondering what the plans will look like compared to having to work all the way up from the bottom to the top before it begins qualifying A.
Edited by: steffi on Jan 14, 2011 11:36 PM

We are wondering how we can benefit from partitioning and or sub partitioning table A. since it's partitioned on >created_date but rarely qualify by that. Very good question. Why did you partition on it? You will never have INSERTS on these partitions, but maybe deletes and updates? The only advantage (I can think of) would be to set these partitions in a read only tablespace to ease backup... but that's a weired reason.
we have many queries where the data is identified by state that is indexed currently non partitioned on columns in >A ... there are also indexes on the foreign keys that get from C -> B -> A. Again these are non partitioned indexes at >this time.Of course. Why should they be partitioned by Create_date?
It is typical that we qualify A on either account or user or both. There are indexes (non partitioned) on these
We have a problem now because many of the queries use leading wildcards ie. account like '%ACCOUNT' etc. This >often results in large full table scans. Our solution has been to remove the leading wildcard but this isn't always possible.I would suspect full index scan. Isn't it?
We are also wondering where and how we can benefit from either global partitioned index or local partitioned >indexes on tables A. We suspect that the index on the foreign key from C to B could be a local partitioned index.As A is not accessed by any partition, why should C and B profit? You should look to partition by the key you are using to access. But, you are looking to tune your SQLs where the access is like '%ACCOUNT' on A. Then when there is a match. ORACLE joins via your index and nested loop (right?) to B and C.
I am also wondering what impact pushing the state from A that's used to qualify A down to C would have any >advantage.Why should it. It just makes the table and indexes larger => more IO.
C is the table that currently we qualify with the partition key so I figure if you also pushed down the state from A >that's used to qualify the set of C's we want based on the set of B's we want based on the set of A thru qualfying >on columns within A.If the access from A to C would be .. AND A.CREATE_DATE =C.CREATE_DATE and c.key like '%what I want%' which does not qualifify for a FK ;-) then, as that could be resulting in a partition scan, you could "profit". But, I'm sure that's not your model.
If we push down some of those values to C and simply use C when filtering I'm wondering what the plans will look >like compared to having to work all the way up from the bottom to the top before it begins qualifying A.So you want to denormalize A,B,C and into one table? With the same access is like '%ACCOUNT' you would get a full scan on an index as before, just the objects would be larger due to redundance and harder to maintain. In the end you would have a bad and slower design.
Maybe you explain what the problem is.
Full index scan can not be avoided, but that can be made faster by e.g. parallel query, and then the join to B and C should be a "snip" if you just identify a small subset of rows in these tables.

Script to find out table and index candidates to keep in the buffer pool

I am looking for a script to find out tables and indexes to keep in the buffer pool.
Could you help me on this ?
thanks...
Markus

this is more of a open question. As you know ur data well. We do not know whats ur data. cachin tables in buffer pool is okay, but it might age out after not being used...instead you can use the KEEP POOL...to cache small tables/popular tables into the keep pool...as keep pool guarantees full caching .....
here are some links on keep pool cacheing
http://download.oracle.com/docs/cd/B19306_01/server.102/b14211/memory.htm#sthref410
http://www.dba-oracle.com/oracle_tips_cache_small_fts.htm
http://www.dba-oracle.com/t_script_automate_keep_pool_tables_indexes.htm
http://www.dba-oracle.com/oracle_news/news_caching_keep_pool_large_objects_clob_blob.htm
Edited by: user630084 on Apr 8, 2009 5:48 AM

Getting a list of all table and index names

Hi,
I have just create a set of tables and indexs, but I have misspelt on of the table names and am now trying to find out what it is. I am using sql*plus as a command line interface but can only find desc as a way of finding any data on a table. What is the command to list all the tables within a table space?
Cheers
Simon

Hi,
there are almost always three ways to find things in Oracle's dictionary :
USER_xxx = thing that you own
ALL_xxx = thing that you own and have access to
DBA_xxx = all things in hole database
If you want to find tables/indexes that you own :
SELECT table_name FROM user_tables
WHERE tablespace_name = 'APP_TBS';
SELECT index_name FROM user_indexes
WHERE tablespace_name = 'APP_TBS';
/Uffe

Table and Index compression in data warehouse - thoughts?

Similar Messages

Maybe you are looking for