Scope for data warehousing ETL Tool

Hi all
can anybody explain scope for data warehousing ETL Tool
for oracle developer in future this is ok or..
regards
Message was edited by:
174313

What exactly is your question?
The scope of using an ETL tool would be setting up and maintaining datawarehouses and building ETL processes to populate these datawarehouses.
A tool is generally preferred over hand coding because tools allow better maintenance, shorter development cycles etc.
Oracle has a pretty good ETL tool, called Oracle Warehouse Builder. It's not the best tool available, but if you compare price/ functionality I would say in most cases it will do.
If your question is if it's a wise pick to master OWB or any ETL tool my answer would be a clear YES! Datawarehouses and BI are becoming more important every day, their use gets broader everyday.
If your question is if an investment in OWB is a wise investment for the future I would answer a clear YES again. It's increddilbe to see what progress Oracle made with the tool, coming from a 'laughing stock' postition, regarded completely immature good for nothing tool, to where they are now, with 10.2. Regarded as one of the leaders by Gartner.
Oracle has recognized (a long while ago) that ETL is the bread and butter of the future and invests in a good quality tool for accomplishing this.
I hope this answes your question, if not, please try to specify more clearly.
Regards,
Toin.

Similar Messages

Best Course for Data Warehousing

Hi,
I am planning to join data warehousing course .I heard there is lot courses in data warehousing .
Data warehousing with ETL tools or
Data warehousing with Crystal Reports or
Data warehousing with Business object or
Data warehousing with Informatica or
Data warehousing with Bo-Webel or
Data warehousing with Cognos or
Data warehousing with Data Stage or
Data warehousing with MSTR or
Data warehousing with Erwin or
Data warehousing with oracle.
Please suggest me which best to choose and which have more scope because I don't know the ABC of data warehousing but I have some experience in oracle.
Is it must that I need work experience in data warehousing then only can get a job ?Please tell me which is the best book for data warehousing which should start from scratch. Please give your suggestions about to my queries.
Thanks & Regards,
Raji

Hi,
Basically Datawarehouse is a concept.To develop DW , we need two tools mainly. One is ETL tool and other one is Reporting tool .
The few famous ETL tools are
Informatica
Data Stage
Few famous Reporting tools are
Crystal Reports
Cognos
Business object
As a DW developer you should aware of atleat one ETL tool and atleat one Reporting tool.The combination is your choice.It better to finout the best combination in point of job market , and then learn them.
Erwin is Datamodel tool. It can aslo be used in DW implementation. You have already have experience on ORacle,So my adivce is go for Data warehousing with oracle or Data warehousing with Informatica .And learn one reporting tool.I donot is there any reporting tool available from ORACLE.
My suggestion on books.
Fundamentals of Datawarehouse by PaulRaj Ponnai and
Datawarehouse toolkit.
http://www.inmoncif.com/about.html is one of the best site for Datawarehouse.
With rgds,
Anil Kumar Sharma .P
Assigning points is the way to say thanks in SDN site.

Webcast : Sun Oracle Database Machine for Data Warehousing -Sep 30 noon ET

Sun Oracle Database Machine for Data Warehousing
Jean Pierre Dijcks - Data Warehousing Product Mgmt, Oracle
https://conference.oracle.com/imtapp/app/cmn_jm_hub.uix?mID=158101510
On September 15 Oracle announced the second generation of its Database Machine, making an already strong data warehousing product significantly stronger. The new version runs on Sun hardware and offers important new features. Available in full rack, half rack, quarter rack, and basic unit configurations, the Sun Oracle Database Machine can add value at many data warehouse size levels.
The Sun Oracle Database Machine runs on Oracle Database 11g Release 2 and has new features such as:
Smart Flash Cache memory for ultra-fast IO - Reaches 50GB/second on a full rack system (not even counting gains from compression)
Exadata Hybrid Columnar Compression - Maximizes data capacity and reduces scan times: think 500GB/second IO
Offloaded Data Mining Scoring - Moves CPU-intensive operations from database servers to Exadata storage servers
In-Memory Parallel Execution - Caches full tables in memory across nodes: foundation of new TPC-H world record
There is plenty more we have not listed above, so come to this TechCast and learn about this major new product!
Audio Dial-In: 888 967 2253 Audio Meeting ID: 572994 Audio Meeting Passcode: 334451
Web Conference: https://conference.oracle.com/imtapp/app/cmn_jm_hub.uix?mID=158101510
Compatibility Check: If you have not used Oracle's web conference system before, please ensure your system
compatibility by going to https://conference.oracle.com/imtapp/app/nuf_sys.uix

Is there any way, one could get this webcast to watch it offline?
regards

Migration of data using ETL tools.

Hi,
We want to tranfer data from Sap to Oracle.Could you please suggest which ETL tool we should choose
for fulfilling our requirement.
Would somebody will be able to access it from outside SAP? Is it possible to extract data at certain frequency e.g. every week, every month using ETL tools.
Regards,
Divya.

Hi,
A feature in BI called Open Hub Service is used for transporting SAP data to non-SAP datamarts, etc. This data can be transferred to database tables or flat files. To know the detials, please refer:
http://help.sap.com/saphelp_nw70/helpdata/en/bf/50453c01f4f75fe10000000a11402f/frameset.htm
Rgds,
Sonal

RFP for Data Warehousing Project

My manager contracted a consultant to come up with an RFP for us. But we are running way behind schedule :(
Does anyone have a sample RFP for a Data Warehousing project for me to have a reference on what is required?
Thanks,
Tamara

Tamara,
I assume this question is not for members of Oracle Warehouse Builder team. Otherwise it'd be a bit awkward - you are asking us for a sample of what to request from us.
Nikolai

How to extract data from RR cluster n B2 cluster for data warehousing.

Hi,
Our company wants to make backup of RR cluster and B2 cluster data of all employees. its basically moving all the employee master data to data warehousing.
Your suggestions are appreciated.
Cheers,
Senthil

Hi,
Our company wants to make backup of RR cluster and B2 cluster data of all employees. its basically moving all the employee master data to data warehousing.
Your suggestions are appreciated.
Cheers,
Senthil

Query tuning for data-warehousing application in Oracle 8i.

We have to pick up 24 months old data. Each month data is kept in a different partition.
2007-May month data is kept in PRESC200705 partition
SELECT r.account_id,
p.presc_num,
spm.product_id,
p.month,
t.best_call_state,
sum(p.trx_count)
FROM rlup_assigned_account r,
temp_presc_num_TEST t,
retail.prescrip_retail partition (PRESC200705) p,
sherlock.sherlock_product_mapping spm
WHERE spm.product_id like '056%'
and spm.mds6 = p.product_id
and t.CLIENT_ID = p.presc_num
and r.ndc_pyr_id = p.payer_plan
and t.best_call_state = r.ST
GROUP BY r.account_id,
p.presc_num,
spm.product_id,
p.month,
t.best_call_state
Q This Query is to be tuned
SQL> SELECT table_name,
2 partition_name,
3 high_value,
4 num_rows
5 FROM user_tab_partitions
6 ;
no rows selected

I have the following task:
Requirement:
According to the client, new partitions are created every month.
So the query should contain only the 24 most recent partions leaving one old partition every month.
So, the query becomes dynamic.
The query will have to leave one old partion every month and move ahead with the new partion created.
The total partitions accessed should not cross 24.
Is this possible?
Partition# for OCt 2007
1
2
3
4
5
6
7
24
Partition# for Nov 2007
Old|New
1
2..1
3..2
4..3
5..4
6..5
7..6
24..23
..24
Secondly, with one month data (in a partitioned table), the query takes about one hour to run.
with 24 month data accessed by the query, the query will take 24 hours to run.
I am sure that Oracle can be tuned to run on such huge data with ease and the query output can
come within seconds.. Otherwise, nobody would use oracle for datawarehousing applicaions.
Q. How do I write a dynamic query that references 24 recent partitions, using the query provided:
abc>SELECT r.account_id,
2 p.presc_num,
3 spm.product_id,
4 p.month,
5 t.best_call_state,
6 sum(p.trx_count)
7 FROM rlup_assigned_account r,
8 temp_presc_num_TEST t,
9 retail.prescrip_retail partition (PRESC200705) p,
10 sherlock.sherlock_product_mapping spm
11 WHERE spm.product_id like '056%'
12 and t.CLIENT_ID='934759'
13 and spm.mds6 = p.product_id
14 and t.CLIENT_ID = p.presc_num
15 and r.ndc_pyr_id = p.payer_plan
16 and t.best_call_state = r.ST
17 GROUP BY r.account_id,
18 p.presc_num,
19 spm.product_id,
20 p.month,
21 t.best_call_state
22 ;
retail.prescrip_retail partition (PRESC200705) p,
Partition name, PRESC200705 cannot be hardcoded into the sql.
Sql should take a range of 24 recent partitions.
And the query should execute fast too.
Now, is that what is called a challenge?
+++++++++++++++++++++++++++++++++++++++++++++++
Here are the index/constraints/explain_plan output on prescrip_retail table (which is partitioned)
as well as other tables to which prescrip_retail table is joined
This is what prescrip_retail looks like. This is the table having partitions.
It does not seem to have a primary key!
SQL> desc prescrip_retail
Name Null? Type
PRESC_NUM NUMBER
PFIER_NUM CHAR(8)
RELID NOT NULL CHAR(9)
ME_NUM NOT NULL CHAR(10)
PRODUCT_ID NOT NULL CHAR(6)
PRODUCT_FRMSTR NOT NULL CHAR(1)
PAYER_PLAN NOT NULL CHAR(6)
MONTH NOT NULL DATE
PYMT_CODE NOT NULL CHAR(1)
NRX_COUNT NOT NULL NUMBER(7)
NRX_QUANTITY NOT NULL NUMBER(9)
NRX_DOLLARS NOT NULL NUMBER(13,2)
TRX_COUNT NOT NULL NUMBER(7)
TRX_QUANTITY NOT NULL NUMBER(9)
TRX_DOLLARS NOT NULL NUMBER(13,2)
Table Size of Prescrip_Retail...
1 select table_name,tablespace_name,pct_free,pct_used,num_rows,avg_space
2 from all_tables
3* where table_name='PRESCRIP_RETAIL'
SQL> /
TABLE_NAME TABLESPACE_NAME PCT_FREE PCT_USED NUM_ROWS AVG_SPACE
PRESCRIP_RETAIL 2806673860 360
Explain Plan for the query to be tuned...
22:32:31 SQL> explain plan set statement_id='vista_query'
22:43:33 2 for
22:43:35 3 SELECT r.pfier_account_id,
22:43:41 4 p.presc_num,
22:43:41 5 spm.product_id,
22:43:41 6 p.month,
22:43:41 7 t.best_call_state,
22:43:41 8 sum(p.trx_count)
22:43:41 9 FROM rlup_assigned_account r,
22:43:41 10 temp_presc_num_TEST t,
22:43:41 11 retail.prescrip_retail partition (PRESC200705) p,
22:43:41 12 sherlock.sherlock_product_mapping spm
22:43:41 13 WHERE spm.product_id like '056%'
22:43:41 14 and spm.mds6 = p.product_id
22:43:41 15 and t.CLIENT_ID = p.presc_num
22:43:41 16 and r.ndc_pyr_id = p.payer_plan
22:43:41 17 and t.best_call_state = r.ST
22:43:41 18 GROUP BY r.pfier_account_id,
22:43:41 19 p.presc_num,
22:43:41 20 spm.product_id,
22:43:41 21 p.month,
22:43:41 22 t.best_call_state;
Explained.
SQL> select statement_id,operation,options,object_name
2 from plan_table
3 where statement_id='vista_query';
22:46:03 SQL> /
STATEMENT_ID OPERATION OPTIONS OBJECT_NAME
vista_query SELECT STATEMENT
vista_query SORT GROUP BY
vista_query HASH JOIN
vista_query TABLE ACCESS FULL TEMP_PRESC_NUM_TEST
vista_query HASH JOIN
vista_query TABLE ACCESS FULL RLUP_ASSIGNED_ACCOUNT
vista_query HASH JOIN
vista_query TABLE ACCESS FULL SHERLOCK_PRODUCT_MAPPING
vista_query TABLE ACCESS FULL PRESCRIP_RETAIL
9 rows selected.
Partition Pruning: This is supposed to provide an insight to the partitions oracle
vists internally...
I guess we can use "month>= add_months(sysdate,-24)" instead of partions too.
I don't think Oracle is visiting any partitions.
I'll also search into all_tab_partitions to verify this.
Explain_Plan for what partitions oracle visits internally (partition pruning):
SQL> ed
Wrote file afiedt.buf
1 explain plan set statement_id='vista'
2 for select * from retail.prescrip_retail
3* where month>= add_months(sysdate,-24)
SQL> /
Explained.
Elapsed: 00:00:00.05
22:13:56 SQL> select statement_id,operation,options,object_name
22:14:28 2 from plan_table
22:14:30 3 where statement_id='vista';
STATEMENT_ID OPERATION
OPTIONS OBJECT_NAME
vista SELECT STATEMENT
vista PARTITION RANGE
ITERATOR
vista TABLE ACCESS
FULL PRESCRIP_RETAIL
Elapsed: 00:00:01.00
Indexes/Constraints on PRESCRIP_RETAIL table:
SQL> ED
Wrote file afiedt.buf
1 SELECT TABLE_NAME,TABLE_TYPE,INDEX_NAME,INDEX_TYPE,PCT_FREE,STATUS,PARTITIONED
2 FROM ALL_INDEXES
3* WHERE TABLE_NAME IN ('PRESCRIP_RETAIL')
SQL> /
TABLE_NAME TABLE INDEX_NAME INDEX_TYPE PCT_FREE STATUS PAR
PRESCRIP_RETAIL TABLE BX6_PRESC_RELID BITMAP N/A YES
PRESCRIP_RETAIL TABLE BX7_PRESC_ME BITMAP N/A YES
PRESCRIP_RETAIL TABLE BX1_PRESC_PROD BITMAP N/A YES
PRESCRIP_RETAIL TABLE BX2_PRESC_PAYER BITMAP N/A YES
PRESCRIP_RETAIL TABLE BX3_PRESC_PAYERCD BITMAP N/A YES
PRESCRIP_RETAIL TABLE BX4_PRESC_PRESC BITMAP N/A YES
PRESCRIP_RETAIL TABLE BX5_PRESC_PFIER BITMAP N/A YES
7 rows selected.
SQL> ed
Wrote file afiedt.buf
1 SELECT TABLE_NAME,CONSTRAINT_NAME,CONSTRAINT_TYPE,STATUS,DEFERRABLE
2 FROM ALL_CONSTRAINTS
3* WHERE TABLE_NAME IN ('PRESCRIP_RETAIL')
SQL> /
TABLE_NAME CONSTRAINT_NAME C STATUS DEFERRABLE
PRESCRIP_RETAIL SYS_C001219 C ENABLED NOT DEFERRABLE
PRESCRIP_RETAIL SYS_C001220 C ENABLED NOT DEFERRABLE
PRESCRIP_RETAIL SYS_C001221 C ENABLED NOT DEFERRABLE
PRESCRIP_RETAIL SYS_C001222 C ENABLED NOT DEFERRABLE
PRESCRIP_RETAIL SYS_C001223 C ENABLED NOT DEFERRABLE
PRESCRIP_RETAIL SYS_C001224 C ENABLED NOT DEFERRABLE
PRESCRIP_RETAIL SYS_C001225 C ENABLED NOT DEFERRABLE
PRESCRIP_RETAIL SYS_C001226 C ENABLED NOT DEFERRABLE
PRESCRIP_RETAIL SYS_C001227 C ENABLED NOT DEFERRABLE
PRESCRIP_RETAIL SYS_C001228 C ENABLED NOT DEFERRABLE
PRESCRIP_RETAIL SYS_C001229 C ENABLED NOT DEFERRABLE
PRESCRIP_RETAIL SYS_C001230 C ENABLED NOT DEFERRABLE
PRESCRIP_RETAIL SYS_C001231 C ENABLED NOT DEFERRABLE
13 rows selected.
In all_tables:
NUM_ROWS:2806673860
AVG_SPACE:360
Here is the data size in the table.
SQL> select count(*) from PRESCRIP_RETAIL;
COUNT(*)
4602980312
Again, here is the partition information and the amount of data in each partition:
SQL> ed
Wrote file afiedt.buf
1 select
2 partition_name,SUBPARTITION_COUNT,PARTITION_POSITION,TABLESPACE_NAME,
3 NUM_ROWS
4 from all_tab_partitions
5 where table_name='PRESCRIP_RETAIL'
6* order by partition_name desc
SQL> /
PARTITION_NAME SUBPARTITION_COUNT PARTITION_POSITION TABLESPACE_NAME NUM_ROWS
PRESC200705 0 36 PRESC_PARTITION_29 141147085
PRESC200704 0 35 PRESC_PARTITION_28 140299317
PRESC200703 0 34 PRESC_PARTITION_27 140703128
PRESC200702 0 33 PRESC_PARTITION_26 132592733
PRESC200701 0 32 PRESC_PARTITION_25 145832356
PRESC200612 0 31 PRESC_PARTITION_24 136702837
PRESC200611 0 30 PRESC_PARTITION_23 137421767
PRESC200610 0 29 PRESC_PARTITION_22 140836119
PRESC200609 0 28 PRESC_PARTITION_21 131273578
PRESC200608 0 27 PRESC_PARTITION_20 134967317
PRESC200607 0 26 PRESC_PARTITION_19 130785504
PRESC200606 0 25 PRESC_PARTITION_18 131472696
PRESC200605 0 24 PRESC_PARTITION_17 138590581
PRESC200604 0 23 PRESC_PARTITION_16 126849798
PRESC200603 0 22 PRESC_PARTITION_15 137164667
PRESC200602 0 21 PRESC_PARTITION_14 126938544
PRESC200601 0 20 PRESC_PARTITION_13 135408324
PRESC200512 0 19 PRESC_PARTITION_12 123285100
PRESC200511 0 18 PRESC_PARTITION_11 121245764
PRESC200510 0 17 PRESC_PARTITION_10 122112932
PRESC200509 0 16 PRESC_PARTITION_09 119137399
PRESC200508 0 15 PRESC_PARTITION_08 123372311
PRESC200507 0 14 PRESC_PARTITION_07 112905435
PRESC200506 0 13 PRESC_PARTITION_06 119581406
PRESC200505 0 12 PRESC_PARTITION_05 123977315
PRESC200504 0 11 PRESC_PARTITION_04 118975597
PRESC200503 0 10 PRESC_PARTITION_03 125782688
PRESC200502 0 9 PRESC_PARTITION_02 117448839
PRESC200501 0 8 PRESC_PARTITION_01 122214436
PRESC200412 0 7 PRESC_PARTITION_36 124799998
PRESC200411 0 6 PRESC_PARTITION_35 125471042
PRESC200410 0 5 PRESC_PARTITION_34 118457422
PRESC200409 0 4 PRESC_PARTITION_33 119537488
PRESC200408 0 3 PRESC_PARTITION_32 121319137
PRESC200407 0 2 PRESC_PARTITION_31 115226621
PRESC200406 0 1 PRESC_PARTITION_30 119143031
36 rows selected.
Data in induvidual partition of PRESCRIP_RETAIL.
SQL> SELECT COUNT(*) FROM PRESCRIP_RETAIL PARTITION(PRESC200704);
COUNT(*)
140299317
SQL> SELECT COUNT(*) FROM PRESCRIP_RETAIL PARTITION(PRESC200703);
COUNT(*)
140703128
SQL> SELECT COUNT(*) FROM PRESCRIP_RETAIL PARTITION(PRESC200702);
COUNT(*)
132592733
SQL> SELECT COUNT(*) FROM PRESCRIP_RETAIL PARTITION(PRESC200701);
COUNT(*)
145832356
SQL> SELECT COUNT(*) FROM PRESCRIP_RETAIL PARTITION(PRESC200606);
COUNT(*)
131472696
SQL> SELECT COUNT(*) FROM PRESCRIP_RETAIL PARTITION(PRESC200605);
COUNT(*)
138590581
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Other tables info:
Index of other tables related to PRESCRIP_REATIL:
SQL> SELECT TABLE_NAME,TABLE_TYPE,INDEX_NAME,INDEX_TYPE,PCT_FREE,STATUS,PARTITIONED
2 FROM ALL_INDEXES
3 WHERE TABLE_NAME IN ('RLUP_ASSIGNED_ACCOUNT','TEMP_PRESC_NUM_TEST','SHERLOCK_PRODUCT_MAPPING');
SQL> /
TABLE_NAME TABLE INDEX_NAME INDEX_TYPE PCT_FREE STATUS PAR
SHERLOCK_PRODUCT_MAPPING TABLE SHERLOCK_PRODUCT_MAPPING_PK NORMAL 10 VALID NO
SHERLOCK_PRODUCT_MAPPING TABLE SHERLOCK_PRODUCT_MAPPING_X1 NORMAL 0 VALID NO
SHERLOCK_PRODUCT_MAPPING TABLE SHERLOCK_PRODUCT_MAPPING_BX1 BITMAP 0 VALID NO
SHERLOCK_PRODUCT_MAPPING TABLE SHERLOCK_PRODUCT_MAPPING_BX2 BITMAP 0 VALID NO
SHERLOCK_PRODUCT_MAPPING TABLE SHERLOCK_PRODUCT_MAPPING_BX3 BITMAP 0 VALID NO
RLUP_ASSIGNED_ACCOUNT TABLE BX1_RLUP_ASSIGNED_ACCT_PYR BITMAP 10 VALID NO
RLUP_ASSIGNED_ACCOUNT TABLE BX2_RLUP_ASSIGNED_ACCT_TOPLVL BITMAP 10 VALID NO
RLUP_ASSIGNED_ACCOUNT TABLE BX3_RLUP_ASSIGNED_ACCT_PBM BITMAP 10 VALID NO
RLUP_ASSIGNED_ACCOUNT TABLE BX4_RLUP_ASSIGNED_ACCT_AA_FLAG BITMAP 10 VALID NO
RLUP_ASSIGNED_ACCOUNT TABLE BX5_RLUP_ASSIGNED_ACCT_AA_CHD BITMAP 10 VALID NO
RLUP_ASSIGNED_ACCOUNT TABLE BX6_RLUP_ASSIGNED_ACCT_PBM_FLG BITMAP 10 VALID NO
RLUP_ASSIGNED_ACCOUNT TABLE BX7_RLUP_ASSIGNED_ACCT_ACCTID BITMAP 10 VALID NO
RLUP_ASSIGNED_ACCOUNT TABLE PK_RLUP_ASSIGNED_ACCOUNT NORMAL 10 VALID NO
13 rows selected.
Constraints in other tables:
SQL> SELECT TABLE_NAME,CONSTRAINT_NAME,CONSTRAINT_TYPE,STATUS,DEFERRABLE
2 FROM ALL_CONSTRAINTS
3 WHERE TABLE_NAME IN ('RLUP_ASSIGNED_ACCOUNT','TEMP_PRESC_NUM_TEST','SHERLOCK_PRODUCT_MAPPING');
TABLE_NAME CONSTRAINT_NAME C STATUS DEFERRABLE
RLUP_ASSIGNED_ACCOUNT SYS_C00637753 C ENABLED NOT DEFERRABLE
RLUP_ASSIGNED_ACCOUNT SYS_C00637754 C ENABLED NOT DEFERRABLE
RLUP_ASSIGNED_ACCOUNT SYS_C00637755 C ENABLED NOT DEFERRABLE
RLUP_ASSIGNED_ACCOUNT SYS_C00637756 C ENABLED NOT DEFERRABLE
RLUP_ASSIGNED_ACCOUNT SYS_C00637757 C ENABLED NOT DEFERRABLE
RLUP_ASSIGNED_ACCOUNT SYS_C00637758 C ENABLED NOT DEFERRABLE
RLUP_ASSIGNED_ACCOUNT SYS_C00637759 C ENABLED NOT DEFERRABLE
RLUP_ASSIGNED_ACCOUNT SYS_C00637760 C ENABLED NOT DEFERRABLE
RLUP_ASSIGNED_ACCOUNT SYS_C00637761 C ENABLED NOT DEFERRABLE
RLUP_ASSIGNED_ACCOUNT SYS_C00637762 C ENABLED NOT DEFERRABLE
RLUP_ASSIGNED_ACCOUNT PK_RLUP_ASSIGNED_ACCOUNT P ENABLED NOT DEFERRABLE
TEMP_PRESC_NUM_TEST SYS_C00640536 C ENABLED NOT DEFERRABLE
12 rows selected.
TEMP_PRESC_NUM_TEST does not contain any constraints or index.

Replication for Data Warehousing

Hi all,
I need a suggestion. We have few Data Warehouse Applications, which supposed to share the same summaries in their Staging Area. We don't want to keep all of them in one database, since our goal is to have a data warehouse environment where down time in any one application does not impact the other applications. What are the options? Currently we see to ways how to implement it. The first is - to use Orale advanced replication. Second - to create individual database for summaries. Both approaches are require the use of db links. Comments? Also, how the db links will impact performance? The Oracle Parallel Server it is not an option.
Thanks

I would suggest the biggest issue here is complexity, rather than the dblinks.
Speed of dblinks depends almost entirely on the speed of the network and will
need testing at peak network usage to get a proper idea.
Advanced replication is can be complex to setup and administer. Individual database's are
maybe less elegant but simpler to administer.
To make the choice I would read up on the replication manuals to get a picture of the work involved. Certainly the replication route is more interesting...

In general how is T series performance for data warehousing databases ~1 TB

Hi,
I am planning to use a Sun T3-2 for a data warehouse app. Oracle 11g around a 1 TB worth of data and batch jobs and ~ 200 concurrent users.
We are using HP UX Superdome server now ( node partition of 8 physical CPU's 750 Mhz ).
I know that the T series servers are good for highly multi threaded application. Can I consider the database 11g as a highly multithreaded app ?
How about the batch jobs which run long SQL jobs, I don't think those can be considered multithreaded.
Anyone is using these T series srevers for databases around a TB size ?
Please suggest.
Thanks ..

The T3-2 can do much more work then the old Superdome, but the T3-2 won't run single threaded queires significantly faster then the old Superdome, it will just be able to run about 100x more of them.
If you care most about price/performance look at the X4800 or X4450 servers.
If you care most about availablity and performance look at the M4000/M5000 servers
My experience with coolthreads servers for Oracle DB is be 100% sure what you're doing isn't single thread dependent before deploying.

Creating A Database For Data Warehousing

I am trying to create a DAD. I type in all of the informatin correctly, but when I type in the password for SYS and hit OK, I get an error. The error I get is "Message 12514 not found; product=RDBMS; facility=ORA". I have modified my tnsnames.ora and sqlnet.ora, but am not sure if I did it correctly. I someone could please help and tell me what the error means it would be appreciated.
null

This error message is right after other listener error messages, so I think maybe there is a problem with listener.

Where to find best practices for tuning data warehouse ETL queries?

Hi Everybody,
Where can I find some good educational material on tuning ETL procedures for a data warehouse environment? Everything I've found on the web regarding query tuning seems to be geared only toward OLTP systems. (For example, most of our ETL
queries don't use a WHERE statement, so the vast majority of searches are table scans and index scans, whereas most index tuning sites are striving for index seeks.)
I have read Microsoft's "Best Practices for Data Warehousing with SQL Server 2008R2," but I was only able to glean a few helpful hints that don't also apply to OLTP systems:
often better to recompile stored procedure query plans in order to eliminate variances introduced by parameter sniffing (i.e., better to use the right plan than to save a few seconds and use a cached plan SOMETIMES);
partition tables that are larger than 50 GB;
use minimal logging to load data precisely where you want it as fast as possible;
often better to disable non-clustered indexes before inserting a large number of rows and then rebuild them immdiately afterward (sometimes even for clustered indexes, but test first);
rebuild statistics after every load of a table.
But I still feel like I'm missing some very crucial concepts for performant ETL development.
BTW, our office uses SSIS, but only as a glorified stored procedure execution manager, so I'm not looking for SSIS ETL best practices. Except for a few packages that pull from source systems, the majority of our SSIS packages consist of numerous "Execute
SQL" tasks.
Thanks, and any best practices you could include here would be greatly appreciated.
-Eric

Online ETL Solutions are really one of the biggest challenging solutions and to do that efficiently , you can read my blogs for online DWH solutions to know at the end how you can configure online DWH Solution for ETL using Merge command of SQL Server
2008 and also to know some important concepts related to any DWH solutions such as indexing , de-normalization..etc
http://www.sqlserver-performance-tuning.com/apps/blog/show/12927061-data-warehousing-workshop-1-4-
http://www.sqlserver-performance-tuning.com/apps/blog/show/12927103-data-warehousing-workshop-2-4-
http://www.sqlserver-performance-tuning.com/apps/blog/show/12927173-data-warehousing-workshop-3-4-
http://www.sqlserver-performance-tuning.com/apps/blog/show/12927061-data-warehousing-workshop-1-4-
Kindly let me know if any further help is needed
Shehap (DB Consultant/DB Architect) Think More deeply of DB Stress Stabilities

Database VS Data warehousing

pls help me in knowing the differences between database and data warehousing in detail
send me links and PDF if u have
thanks

Hi,
A data base is a structured collection of records or data. A computer database relies upon software to organize the storage of data. The software models the database structure in what are known as database models.
A data warehouse is a repository of an organization's electronically stored data. Data warehouses are designed to facilitate reporting and analysis.This classic definition of the data warehouse focuses on data storage. However, the means to retrieve and analyze data, to extract, transform and load data, and to manage the dictionary data are also considered essential components of a data warehousing system. Many references to data warehousing use this broader context. An expanded definition for data warehousing includes business intelligence tools, tools to extract, transform, and load data into the repository, and tools to manage and retrieve metadata.
Regards.

What about data warehousing?

I have attended the 10g launch, perused the CD I got there and listened to a couple of online seminars. I can easily envision how 10g benefits OLTP applications, but data warehousing seems to be ignored in the presentations.
Does 10g offer anything over 9i for data warehousing, which has quite different resource demands? Is there any published information I could read?

DATA WAREHOUSING
Oracle Database 10g has also enhanced its data warehouse and business intelligence capabilities, which results in
further reduction of the total cost of ownership while enabling customers to derive more value from their data and
supporting real time data feeds.
Consolidation and integration of traditionally disparate business intelligence systems into a single integrated engine is
further enhanced in Oracle Database 10g. Database size limits have been raised to millions of terabytes. Business
Intelligence applications can be consolidated alongside transactional applications using Real Application Clusters
automatic service provisioning to manage resource allocation. This consolidation means analysis can be performed
directly against operational data and resource utilization can be maximized by reallocating servers to workloads as the
business needs change. The value of data is increased with the ability to perform even more diverse analytic
operations against core data with enhanced OLAP analytics, a data mining GUI and a new SQL model feature. The
SQL model cause allows query results to be treated as sets of multidimensional arrays upon which sophisticated
interdependent formulas are built. These formulas can be used in complex number-crunching applications such as
budgets and forecasts without the need to extract the data to a spreadsheet or perform complex joins and unions.
Real Time Warehousing is enabled either by consolidating business intelligence with operational applications, or by
new change data capture capabilities based on Oracle Streams which produce low or zero latency trickle feeds with
integrated ETL processing.
Joel P�rez

Is RAC suited for Data Warehouse

Hi,
I have to provide a best architecture for our data warehouse project, as i googled , i found documents about Oracle RAC for Data Warehouse and features that Oracle RAC provide for a warehouse environment, like performane, availability ...
But i see some document that says , Oracle RAC is not best choice for data warehousing.
I want to provide flexible design that don't need to change after years. but our resources are limited, we have two IBM Power7 servers with 32cores and 62GB ram, we estimate about 1 to 2TB data yearly that should upload into warehouse, because each server just have 62GB ram, what should we do?
Implementing Oracle RAC or just running warehouse on sigle-instance database on one server and just increase the resources of that server?

VahidS wrote:
Hi,
I have to provide a best architecture for our data warehouse project, as i googled , i found documents about Oracle RAC for Data Warehouse and features that Oracle RAC provide for a warehouse environment, like performane, availability ...
But i see some document that says , Oracle RAC is not best choice for data warehousing.
I want to provide flexible design that don't need to change after years. but our resources are limited, we have two IBM Power7 servers with 32cores and 62GB ram, we estimate about 1 to 2TB data yearly that should upload into warehouse, because each server just have 62GB ram, what should we do?
Implementing Oracle RAC or just running warehouse on sigle-instance database on one server and just increase the resources of that server?
I am not sure that which documents you have read based on which you have got the impression that RAC is for DWH. RAC is a part of the Oracle's High Availability solution architecture. It's not designed to cater any specific kind of workload but to provide HA in the case of a node or instance crash. And also, RAC itself doesn't give any performance benefit. If you have a database which is poorly performing, it would remain the same or may be , will become even worse after implementing RAC due to too many things being involved in the architecture now e.g storage, network etc. So if you do want to implement RAC, give it a careful thought, do some testing and then only project it. If you are determined to go for RAC and you are a pure DWH shop, I would suggest to go fro Exadata running RAC . Smart Scan, Compression are just few of those benefits which would come with Exadata along with the RAC's HA solution that would be serving better for your requirements.
HTH
Aman....

Data Warehouse and ETL tools for data verification ?

Data Warehouse and ETL tools for data verification ?
How need to to data verification using ETL tool ? Also how to relate this thing to datawaehouse ?
Thanks in Advance

Hi Shyamal Kumar,
1) BW it self facilitates to do the ETL (Extraction Transformation Loading) steps:
     example:
                 Extraction - from SAP or other data bases
                 Transformation - using transfer rules, Updates rules
                 Loading - Loading into ODS, Cube, master data
2) Typically used ETL tools in the industry are:
     a)   datastage from Ascential (owned by IBM)
     b)   Informatica
     c)   Mercator
Regards, BB

Scope for data warehousing ETL Tool

Similar Messages

Maybe you are looking for