Table transform weird behaviour

Question

Hi,
I am really fed up with the weird behaivour of table transform, i have used the same table comparsion transform in two sequential dataflows and the output is different.
Below are the details.
1. First data flow is inserting records into our target table.
2. Target table is used as comparison table in table transform of second dataflow, the target table itself behaves a my destination table also, my requirement is to only update the changed columns rather than all columns.
3. I have specified only the changed columns in the compare columns section, even though the my whole record being changed.
Table :
Primary key     col1       col2       col3         col4
Dataflow 1 values :  PK1,10,20,30,40
Dataflow 2 values :  PK1, null,50,60,null
My output should look like this : PK1,10,50,60,40
Could you please guide me how can i achieve this.

Answer 1

P1300941063 wrote:
> Table :
> Primary key     col1       col2       col3         col4
>
> Dataflow 1 values :  PK1,10,20,30,40
> Dataflow 2 values :  PK1, null,50,60,null
>
> My output should look like this : PK1,10,50,60,40
>
> Could you please guide me how can i achieve this.
What I think you mean is that you want DS to figure out that an incoming NULL in COL1 and COL4 is not to be treated as "the new information" -- that you want to treat nulls in your incoming data as though the column didn't exist at all.  I also assume that you don't know beforehand which incoming columns may hold nulls -- that, on any given job run, you may see incoming data like this:
PK1, null, 50, null, 80
PK1, 5, null, null, null
PK1, null, 40, 100, 90
In each case, you only want to apply the non-null values to the target -- you don't want to "null-out" anything in the target. Is that right?
If so, there are a number of approaches:
1) Don't use a Table Comparison transform at all. Instead, join your target table to your incoming source. Let's call the target table "TGT" and the source information "SRC". In a Query transform, your WHERE clause might look like this:
TGT.PK1 = SRC.PK1
and (
  nvl(SRC.COL1,TGT.COL1) != nvl(TGT.COL1)
  or nvl(SRC.COL2,TGT.COL2) != nvl(TGT.COL2)
  or nvl(SRC.COL3,TGT.COL2) != nvl(TGT.COL3)
  or nvl(SRC.COL4,TGT.COL2) != nvl(TGT.COL4)
Each column mapping looks like this, for, for instance, COL1:
ifthenelse(nvl(SRC.COL1,'') != TGT.COL1,SRC.COL1,TGT.COL1)
Then, just sent the records into a Map Operation transform, changing all of them from Normal to Update, and send them into your target table.  (Might need to run them through a Data Transfer transform or do something else that forces the read to end before sending records into the tgt if you have problems using the same table as source & tgt in the same dataflow.)
2) You could also split the incoming records into four different queries, each holding the PK and just one attribute column:
Q1 = PK1, COL1
Q2 = PK1, COL2
Q3 = PK1, COL3
Q4 = PK1, COL4
Filter-out any records where the attribute column holds nulls, and then run them into four Table Comparison transforms; assume you can handle the rest.
3) Or, what amounts to the same thing (and would likely be faster, vs. #2), building on your original approach: downstream from the existing TC transform in which, presumably, you're using all four attribute columns as comparison columns, map the output into a Map Operation transform in which you map updates to normals, discarding everything else. (For inserts, use another Map Operation transform in which you map inserts to inserts and discard all else, and send the records in (if you already have a PK).) From the Map Operation transform for your updates, multiplex the output into four query transforms as in solution #2, shrinking the schemas down to only one attribute column, and filtering-out nulls (on that column). Map each to four more Map Operation transforms in which you change normals to inserts, and send 'em in.
Hope I've understood your problem correctly, and, if so, that this proves helpful.
Best wishes,
Jeff Prenevost
Data Services Practice Manager
itelligence