FILE model support for CSV and fixed-length format!!

Question

Hi all,
As I'm working on a data conversation project, we are in the middel to design whether the extraction for the data source should be in CSV or fixed-length. Here is the pros and cons:
CSV:
1. There are a lot of remarks field in the data which are all free text. It's really hard to define the record delimiter.
2. Extracted file size for CSV is relatively small compare with fixed-length.
fixed-length:
Even for fixed length, we can either
1. Extract the file without an separator/delimiter (1 row) with fixed column length
2. Extract the file with fixed column length per record and define the record delimiter to separarter the record.
I'd try all the above formats and it seems ODI is more prefer on CSV or combined mode (fixed column length) as:
1. The LKM (file to external table) need to be customzised to handle the file format which without any separator/delimiter
2. It is very hard to reverse the column definition for fixed-length (no delimiter)
Any suggestion?
Tao

Answer 1

You can transform your files to an xml format using respective tags:
eg:
<?xml version="1.0" encoding="UTF-8"?>
<MONEY>
<MoneyID>MoneyID0</MoneyID>
<ContractID>ContractID0</ContractID>
<EffectiveDate>2006-05-04T18:13:51.0Z</EffectiveDate>
<MessageDate>2006-05-04T18:13:51.0Z</MessageDate>
<ReversalIndicator>false</ReversalIndicator>
<PriorMoneyID>PriorMoney, something "ID0" for life;</PriorMoneyID>
<Amount>0</Amount>
<MoneyType>0</MoneyType>
<ExchangeDetails>
<CostBasis>0</CostBasis>
<ExchangeType>0</ExchangeType>
<MEC>false</MEC>
<LoanAmount>0</LoanAmount>
</ExchangeDetails>
</MONEY>
So, as you see, you can embed any symbol in a tag.
After creating this xml file, you can use XML technology in Topology Manager to map to this file and then create a data store based on this technology.
And you will be able to retrieve the data in this file under respective column headers without worrying about delimiters existing in the actual data.