Groupby condition in output table

Question

Hello,
I have to implement a groupby condition and remove the duplicate records from output table. I could do that in sql and my query looks like this:
select col1,col2,col3, COUNT(col3) from TARGET_TABLE group by col1, col2, col3 having count(col3) >=2
I get 99 records when i run this query in sql. I need that 99 records to go in as error table output in DI and remaining records to TARGET_TABLE.
Where can i insert the sql and get two outputs (final output) and (error table).
Appreciate your ideas.
Arun

Answer 1

Built-in transforms inside DS will let you achieve this. Connect the source table to a Query transform - say Query1
Inside Query1:
Map all the source columns col1,col2,col3 to target as it is. Add a column - say CNT_COL3 with datatype int.
In the mapping tab, use Count (col3). Add col1,col2 and col3 to the Group By tab of Query1
Connect a validation transform next to Query1.
Inside Validation transform:
Enable validation for CNT_COL3. Provide the condition as < 2 (Since this your the pass condition)
In the Action on Failure tab, check 'Send to Fail'
Connect the Pass path from Validation to Target table and Fail path to Error table. Values will be populated as required.
Regards,
Suneer