Error loading from csv - how to integrate valid rows and drop invalid?

Question

Hello,
Log from execution (step: load csv data):
ODI-1228: Task SrcSet0 (Loading) fails on the target ORACLE connection ODI_STAGE.
Caused By: java.sql.BatchUpdateException: ORA-12899: wartość zbyt duża dla kolumny "ODI_STAGE"."C$_0URZADZENIA"."C1_ZWIAZKI" (obecna: 17, maksymalna: 6)
So, it's clear. There are too long fields in csv, maximum is 6, current: 17. But, some rows are good (with ZWIAZKI lenght >=6). I need to load valid rows, integrate them, sent to destination table. Invalid rows should be isolated in external csv - it happens. But how can I integrate valid rows and prevent entire interface fail?

Answer 1

I don't think something exists to really be prepared for every situation (invalid length, invalid format, ...).
Here is an idea but it's not beautifull and I'm not sure it's the good way to do that :
-First you should have a unique key (easier if it is on only one column).
-Then Develop an new IKM to insert on a row-by-row insert/commit basis with an ORDER BY clause on the UK in the select. Before the insert step, the value of the UK for the current row should be stored in an ODI variable.
-Create your interface and put a filter on your UK : UK_FIELD > :CURRENT_UK
-Create a package, set CURRENT_UK to a value you know will be less than the possible min and refresh a new variable MAX_UK with a query to find the MAX(UK_FIELD). Then put your interface in the package and do a loop around it so it ends only when CURRENT_UK = MAX_UK. You should redirect both OK and KO arrows from the interface to the testing condition.
@Phil : Sorry for "stealing" your reply :P. Nice to see you agree with me.