Capture change on certain columns only in ODI,

Question

Hi experts:
Need you help to suggest the proper way to meet the following requirements.
Requirements”
(a) Source– Oracle 11g, need to track changes to 2 columns only - [Table A, Column A1] & [Table B, Column B1]. These 2 columns may be changed many times during a day.
(b) Every 4 hours, based on what are captured from (a), find out the distinct rows changed (i.e. based on primary keys of Table A, Table B) , and only ship the most current row images for these rows to target
(c) Target – Sybase, meet to perform transformations on the 2 column values (i.e. Column A1, Column B1) to map to target table [Table C, Column C1]
From my limited understanding, I think trigger-based CDC is to be used. However, not sure how to do this , especially (a) and (b). For example, where do we configure trigger logic to achieve (a) from the ODI studio ?
Your help is much appreciated.
Regards

Answer 1

Hi,
887999 wrote:
Hi experts:
Need you help to suggest the proper way to meet the following requirements.
Requirements”
(a) Source– Oracle 11g, need to track changes to 2 columns only - [Table A, Column A1] & [Table B, Column B1]. These 2 columns may be changed many times during a day.
(b) Every 4 hours, based on what are captured from (a), find out the distinct rows changed (i.e. based on primary keys of Table A, Table B) , and only ship the most current row images for these rows to target-- Check out the JV$ and JVD$ views that are created when you start the ODI Journal, they do exactly what you have described in (b) - You get the latest update based on the SCN number when they occured.
(c) Target – Sybase, meet to perform transformations on the 2 column values (i.e. Column A1, Column B1) to map to target table [Table C, Column C1]To be honest the target is irrelvant, you just need to decide if you want to do the transformation on the source on the way out of oracle (set staging area different to target and choose your Source logical schema).
I would design an interface that uses Table A and Table B as the sources, do the join , transformation on Oracle and map to Target Table C, choose your staging area based on where you want the joins / transformation to take place, pick a Knowlege module based on Technology and how you want to update the target.
>
From my limited understanding, I think trigger-based CDC is to be used. However, not sure how to do this , especially (a) and (b). For example, where do we configure trigger logic to achieve (a) from the ODI studio ?You can use Synchronous (Trigger based) or Asynchronous (Logminer / Streams based) to perform what you want, see this nice guide on setting up CDC and consuming the changes :
http://soainfrastructure.blogspot.ie/2009/02/setting-up-oracle-data-integrator-odi.html
If your friendly with your source system DBA then I prefer Asynchronous CDC , its less intrusive than triggers, it does however need a bit of knowledge on how to monitor it, Metalink / Support has plenty of info.
>
Your help is much appreciated.You welcome, have a play with it in a demo environment and get a feel for how you consume the captured changes (Lock Subscriber, Extend Window, Consume, Purge + Unlock, Loop) etc.
The guide I've linked to uses an ODI WAit for Data to trigger the consumption of changes, you have stated every 4 hours so I would skip the ODI wait for data and simply schedule your package to run every 4 hours.