Trigger based approach to capture changes in OLTP DB

Question

I have this scenario where we need to capture changes (updates/deletes) to transaction tables in our OLTP database. The changes will be populated to temp tables and then moved to Hadoop for analysis. The OLTP table record huge volume of data on a daily basis, some tables having thousands of updates every day and some having a few million.
We have thought of using a trigger-based approach, where a trigger in the OLTP tables will take care of inserting records in the temp tables. It will be a "After Update and Delete" trigger as:
IF updating THEN
      BEGIN
       insert into temp table // insert statement here
      EXCEPTION
         WHEN others THEN
           null;
      END;
  ELSIF DELETING THEN
     BEGIN
       insert into temp table //insert statement here
      EXCEPTION
         WHEN others THEN
           null;
      END;
   END IF;
END;Please let me know if this is an acceptable approach. Are there any better ways of doing the same?
I am using Oracle 11g r1.

Answer 1

>
So, as a test, I created a table called test, with a PK on col1.
Then, I created a MV on it,
Next, I inserted 2 records, committed, and deleted 1 record, and did commit.
The RUPD$_TEST is empty, but the MLOG$_TEST has data.
>
The RUPD$_TEST is a temporary table that Oracle creates and uses with primary key based MV logs. As a temporary table you won't see data in it unless your session puts data there.
>
But how do I read this data?
>
What data? You didn't log any data to read.
Here is an example that doesn't even use a primary key - just a plain old table with familiar data.
create table emp_copy as select * from emp
create materialized view log on emp_copy
with rowid, sequence (empno, ename, job, mgr, hiredate, sal, comm, deptno)
including new values;
insert into emp_copy (empno, ename) values (9999, 'test')
delete from  emp_copy where empno = 7369
update emp_copy set job = 'NEW JOB' where empno = 7499
select * from mlog$_emp_copy
EMPNO     ENAME     JOB     MGR     HIREDATE     SAL     COMM     DEPTNO     M_ROW$$     SEQUENCE$$     SNAPTIME$$     DMLTYPE$$     OLD_NEW$$     CHANGE_VECTOR$$     XID$$
9999     test                                   AAAUayAAEAAAAJVAAO     1     1/1/4000     I     N     FEFF     844454994905658
7369     SMITH     CLERK     7902     12/17/1980     30          20     AAAUayAAEAAAAJVAAA     2     1/1/4000     D     O     0000     844454994905658
7499     ALLEN     SALESMAN     7698     2/20/1981     1600     300     30     AAAUayAAEAAAAJVAAB     3     1/1/4000     U     U     0800     844454994905658
7499     ALLEN     NEW JOB     7698     2/20/1981     1600     300     30     AAAUayAAEAAAAJVAAB     4     1/1/4000     U     N     0800     844454994905658SNAPTIME$$ shows the SEQUENCE of events.
XID$$ shows the transaction id
DMLTYPE$$ I/D/U for Insert/Update/Delete
For the Insert you can see the new EMPNO and ENAME
For the Delete you can see the values that were in the deleted row
For the Update you can see the old job 'SALESMAN' in sequence #3 and the new job 'NEW JOB' in sequence #4
Just pull the data that you haven't mined/processed yet based on SNAPTIME$$ and then delete those rows when you are done with them.
It took a few minutes to create that example and start collecting change data. Oracle will collect the changes automatically. All the user needs to do is manage the MV Log to keep it from growing indefinitely.
For some uses you may not need the actual column values; maybe you just want to know what ROWS in the table changed. This modified example uses ROWID but you could use primary key instead.
create materialized view log on emp_copy
with rowid, sequence;
insert into emp_copy (empno, ename) values (9999, 'test')
delete from  emp_copy where empno = 7369
update emp_copy set job = 'NEW JOB' where empno = 7499
select * from mlog$_emp_copy
M_ROW$$     SEQUENCE$$     SNAPTIME$$     DMLTYPE$$     OLD_NEW$$     CHANGE_VECTOR$$     XID$$
AAAUa3AAEAAAR+mAAA     5     1/1/4000     I     N     FEFF     1.68898729922029E15
AAAUa3AAEAAAR+jAAA     6     1/1/4000     D     O     0000     1.68898729922029E15
AAAUa3AAEAAAR+jAAB     7     1/1/4000     U     U     0800     1.68898729922029E15There are the ROWIDs for the rows that have changed. The row for the delete MAY not be in the table anymore. The original row with that ROWID was deleted (SNAPTIME$$ = 6) but there could be a later INSERT whose row now has that ROWID. And if ROW MOVEMENT is enabled the rowid of a row could change.
For that last example you would access the MASTER table (emp_copy) and pull rows based on ROWID. Those are the rows that have 'changed' since the last time you mined the MV log.
I don't know how it can get any simpler than that! ;)