View in Datawarehouse environment

Question

Hi there,
Need your inputs to decide on the approach to move more than two billion rows from one table to another. The problem statement is:
1. Source and target table structures are different. So can't use Insert into ... Select statement.
2. For every N records read from source table, 1 record goes into the target, where N can be any number less than 24. So, its basically denormalization and aggregation. Yes, both.
3. Data verification by looking up column values is required.
Approach 1:
Using ETL tool (Informatica) move the records. Perform aggregation in the Informatica. Use 'order by' in the source query. I have doubts whether the tool will be able to handle large amount of data.
Approach 2:
Create a view to perform aggregation. (Not materialized view, due to space constraints). Use ETL tool to move data.
Question is, does view help in performance improvement? As in, because Oracle will append the SQL used to create view to that used to read from view, aggregation will be done on the Database in Approach 2 as opposed to aggregation done by the tool in Approach 1.
Thanks for the help.
-N

Answer 1

Are both the source and target tables in Oracle? If so, I suggest Approach 3: do everything in the database. The major drawback I see with most ETL tools like Informatica, DataStage, etc., is that they usually do row-by-row processing, which can be very slow. It is possible to do very complex transformations using only SQL.
In your post you state that you can't use "Insert Into...select" because the table structures are different; in many cases, you can formulate a select statement which can produce a result set consistent with structure of the target statement. While the SQL may be somewhat complex, you will almost alwas get better performance using a set-based approach (processing many records at once) than a row-by-row approach.
I strongly suggest looking at the Data Warehousing Guide