Commit per row

Question

Hi guys,
Need your suggestions. The problem I'm having is with primary keys. I have two columns (ID and Version Number) in my target data store. ID is not a unique identifier, so we added a version number for uniqueness. For example, records in the target datastore could be like this:
ID | Version
1 | 1
1 | 2
1 | 3
2 | 1
2 | 2
In the ODI mapping, I pick up the max(version) and add + 1 to the new record to be inserted for a particular ID. However, there could be multiple rows per ID in the source. And since these records are not in the datastore not after the commit, all the records for that ID gets the same version number (from max), therefore those rows could not be inserted because the primary key constraint will be violated.
Can anyone suggest a good approach on how to handle this?
Thanks,
Jason

Answer 1

jaysen,
I am not sure why would you do that. I am quoting what I wanted to do:
----QUOTE--------------------
INSERT INTO I$_TGT_TABLE (ID, VERSION)
SELECT ID,
(SELECT MAX(A.version) + 1 FROM TGT_TABLE A WHERE A.id = C$_0TGT_TABLE.id)
FROM C$_0TGT_TABLE C$_0TGT_TABLE
-----QUOTE ENDS--------------
Please notice that I wanted to do max from C$ table, not I$ table. Put the expression there:
(SELECT NVL(MAX(A.version),0) + 1 FROM <%=odiRef.getTargetTable("RES_NAME")%> A WHERE A.id = <%=odiRef.getTable("L","COLL_NAME","D")%>.id)
Execute the mapping at source.
Now this would not solve the problem if there are multiple rows with same id in the source table (go back to my post and see Rathish's reply to my post for an excellent case scenario). If you find the method of KM modification suggested in my earlier post too complicated, then here is another way remove the problem. Break the entire interface into three interfaces. The target resultset of the first and second interfaces should be used as a source in the third interface.
The first interface should take the data out of the source table and "version" column in the target table (in this case a temporary data store or landing table, let us call it T1) must be mapped as
ROW_NUMBER() OVER (PARTITION BY id ORDER BY id)
Use LKM and IKM as provided with ODI.
In the second interface take table TGT_TABLE as source and populate a second temp data store T2. This temp table should have "id" and "version" columns. Put max value of "version" column for each "id" value in this temp data store.
Use LKM and IKM as provided with ODI.
The third interface should use T1 and T2 as source sets (use left outer join as a join method with id as the common joining column). Use TGT_TABLE as target and mapping for "version" column should be "T1.version + NVL(T2.version, 0)".
Use LKM and IKM as provided with ODI.
That should take care of it. I am sure a far more "elegant" or "least-maintenance" solution exists, but I do not see any simpler solution than this. If you try to accomplish this same task through plain SQL, it is difficult to do this without using two temporary tables or two CTES (or derived tables).
If I were you, I would be using the target temporary data store that ODI can build instead of creating and maintaining two pairs of temporary tables in the database. But that is a decision you should be making.