Is this Possible using ODI / ODQ ?

Question

Hi ALL,
Is the below requirement possible with ODI / ODQ?
(right now I am doing it mannually, and its like a monthly/weekly process)
If we can do the same using ODQ how I need to start and proceed?
My requirement is some thing like this. I have two columns
Address1 (street address)
Address2 (additional Information)
In some cases the address is not in the proper format. So I need to do some data cleansing (data scrubbing) before loading using ODI
Use Address2 if Address1 is PO Box, building name, or any info aside from a street address. Only one address column is needed in the output (Address1)
For example :
INPUT
ADDRESS1--------------------------------------------------ADDRESS2
HIGHWAY # ------------------------------------------------5010944 MARSH RD
PO BOX 40964---------------------------------------------1100 AA CORP PARKWAY
DBA VANTAGE PP---------------------------------------11031 PERRY HIGHWAY
HOME INFUSION THERAPY SERVICES--------11031 PERRY HIGHWAY
7275 BUFFALO ROAD----------------------------------(WAS 6855 BUFFALO ROAD)
7601 N FEDERAL HWY---------------------------------#2506
MCAS ---------------------------------------------------------(MARINE CORPS AIR STATION)
212 BRADDOCK AVENUE------------------------------
212 NINTH STREET----------------------------------------
2138 JONATHON DRIVE---------------------------------
OUTPUT (Note : we don’t need the Address2 column in the output)
ADDRESS1----------------------------------ADDRESS2
10944 MARSH RD------------------------10944 MARSH RD
1100 AA CORP PARKWAY-----------1100 AA CORP PARKWAY
11031 PERRY HIGHWAY-------------11031 PERRY HIGHWAY
11031 PERRY HIGHWAY-------------11031 PERRY HIGHWAY
7275 BUFFALO ROAD------------------(WAS 6855 BUFFALO ROAD)
7601 N FEDERAL HWY----------------#2506
MCAS-----------------------------------------(MARINE CORPS AIR STATION)
212 BRADDOCK AVENUE------------
212 NINTH STREET----------------------
2138 JONATHON DRIVE---------------
I have posted a similar kind of question in DATA Quality (ODQ) forum
Forum Home » Master Data Management (MDM) » Data Quality
ADDRESS CLEANING(US) USING ODQ
Thread: ADDRESS CLEANING(US) USING ODQ
if any body is working on address cleansing using ODQ, Please guide me to start working on it.
IS there any specific documents for ADDRESS CLEANING which gives more information about the process flows such as
Transformer, Customer Data Parser, Sort for Postal Matcher, Postal Matcher, Window Key Generator,Sort for Linking, Relationship Linker, Commonizer, and Data Reconstructor.
Thanks in Advance,
Rathish

Answer 1

Hi Rathish,
I was building the java code but I got other idea to make more simple.
1) load the values into a temp table with one column for each address (means 2 columns per line) plus 2 other columns to control.
Something like:
table temp_data
columns: add_1 , control_1, add_2, control_2
You can do that thru an interface. Let the fields "control" with no value when loaded
2) create a table to storage the values you sent to me (Avenue, street...)
table street_names
columns: street_name, country (to your future use)
3) Create an ODI procedure (or you can customize a KM with the same technique)
3.1) Step 1 at target Tab, technology javaScript
code:
<@String vU=" ";@>
3.2.1) step 2 - Tab Source - technology "your database" and the Logical Schema where the "street_names" table as created
select street_name
from street_names
where country = 'US'
3.2.2) step 3 - Tab Target - tecnology javaScript
<@vU+="or instr(upper(add_1),upper('#street_name'))>0 ";@>
3.3) step 3 - Target Tab - technology "your database" and the Logical Schema where the "temp_data" table as created
<@vU=vU.substring(3);@>
update temp_data
set control_1 = 'Y'
where <@=vU@>
3.4) step 4 step 3 - Target Tab - technology "your database" and the Logical Schema where the "temp_data" table as created
<@vU=vU.replaceAll("add_1","add_2")@>
update temp_data
set control_2 = 'Y'
where <@=vU@>
4) Now you have a table (temp_data) with a flag "Y" where there is a street name and "null" where doesn't exist. Just generate your file by a interface or odiSqlUnload Toll.
Put all together into a package and you will have your process.
I create in this way to minimize the access to the temp_table, just 2 updates.
can be used some alternatives to tunning the hole process if the time doesn't match what you need.
That is a fast solution. Something more elaborated can be done if necessary.
Does it help you?
Cezar Santos