Best practise to detect changes between two tables

Question

Hi,
I try to write a query, that shows me the differences between a table in my DWH and the table in the source system. It should show me new, deleted and updated rows.
My approach is to do a full outer join based on the key and then check if any of the columns changed (source.A!=DWH.A or Source.B!=DWH.B, etc.) to get the updated rows.
My problem is now that my table has millions of rows und more than 100 columns (number, nvarchar, etc.). So the query takes hours.
Is there any best practise solution to optimize that query, by rewriting it, setting indexes or using hash code? I played around with hash code, but it wasn't really faster.
(BTW: CDC, etc are not allowed)
Thanks for any ideas!

Answer 1

890408 wrote:
So i guess I can't use the merge statement, as it is just for SCD1.
Yes you can:
create table products(
                      name varchar2(20),
                      price number,
                      effective_from date,
                      effective_to date,
                      active number
insert
  into products
  values(
         'Samuel Adams, 6-pack',
         6.99,
         null,
         sysdate - 51,
         0
insert
  into products
  values(
         'Samuel Adams, 6-pack',
         7.29,
         sysdate - 50,
         null,
         1
create table product_updates(
                             name varchar2(20),
                             price number
insert
  into product_updates
  values(
         'Samuel Adams, 6-pack',
         7.49
insert
  into product_updates
  values(
         'Corona, 6-pack',
         6.49
select  *
  from  products
NAME                      PRICE EFFECTIVE EFFECTIVE     ACTIVE
Samuel Adams, 6-pack       6.99           13-OCT-11          0
Samuel Adams, 6-pack       7.29 14-OCT-11                    1
select  *
  from  product_updates
NAME                      PRICE
Samuel Adams, 6-pack       7.49
Corona, 6-pack             6.49
merge
  into products p
  using (
          select  name,
                  price,
                  'update' flag
            from  product_updates
         union all
          select  chr(0) || name name,
                  price,
                  'insert' flag
            from  product_updates
        ) u
  on (
      p.name = u.name
  when matched
    then update
             set effective_to = sysdate,
                 active = 0
           where active = 1
  when not matched
    then insert
           values(
                  substr(u.name,2),
                  u.price,
                  sysdate,
                  null,
                  1
           where flag = 'insert'
3 rows merged.
select  *
  from  products
NAME                      PRICE EFFECTIVE EFFECTIVE     ACTIVE
Samuel Adams, 6-pack       6.99           13-OCT-11          0
Samuel Adams, 6-pack       7.29 14-OCT-11 03-DEC-11          0
Samuel Adams, 6-pack       7.49 03-DEC-11                    1
Corona, 6-pack             6.49 03-DEC-11                    1
SQL> SY.
SY.