Delta load from netezza to sql server

Question

i have 20 million rows in a table and 100 million rows in another table in netezza
Very first time, i got all the rows from netezza to sql server....
Say in a week, we have new rows of data and updated rows in those tables in Netezza,
how would i go about in doing the delta to sql server??
should i use
1) SCD
2) Merge Statement using T-sql
3) Lookup
Not able to decide
we also need to think about performance as well right
Can somebody point me to the right resource
Thanks

Answer 1

So this is what you load, you require to have 2 tables to make the incremental logic on your side
a) Load_Log: Here you would capture the load start time/ End time/ max date(modified date) from the Nettiza tables.
b) As ione suggested load this data to a staging table( this table would be used for comparison against the main table), this step is required as SSIS merge wouldn't be suited for million records, so the comparison is best served in a SP via a merge or classic
UPSERT 
Next time the load is run fetch the max date from Load_Log table and pass this as parameter on the source query something like where modified date> param.
if its a weekly load you might want to run it as daily on source or modify the source query to be like modified date between max date and mas date+1 doing some iterations, though best would be to run on daily basis. a million records would be fine to be
processed.
Abhinav http://bishtabhinav.wordpress.com/