Anybody got SCD Type 2's to perform quickly using dimension operator
Hi there,
Hitting major performance problems running mappings to populate SCD Type 2's when they have large amounts of pre-existing data.
Anybody got this performing acceptably? Tried indexing but to no avail.
Many Thanks
Hi there,
Thanks for getting back to me - found the patch and this patch hasd already been applied.
An example of the sql being generated in a really simple mapping with the dimension operator for small tables is as follows
MERGE
/*+ APPEND PARALLEL("NS_0") */
INTO
"RETAILER_PUBLISHER_NS"
USING
(SELECT
"MERGE_DELTA_ROW_0"."NS_OUTLET_SRC_ID$1" "NS_OUTLET_SRC_ID",
"MERGE_DELTA_ROW_0"."NS_PUBLISHER_CODE$1" "NS_PUBLISHER_CODE",
"MERGE_DELTA_ROW_0"."NS_TITLE_CLASSIFICATION_CODE$1" "NS_TITLE_CLASSIFICATION_CODE",
"MERGE_DELTA_ROW_0"."NS_SUPPLY_FLAG$1" "NS_SUPPLY_FLAG",
"MERGE_DELTA_ROW_0"."NS_EFF_DATE$1" "NS_EFF_DATE",
"MERGE_DELTA_ROW_0"."NS_EXP_DATE$1" "NS_EXP_DATE",
"MERGE_DELTA_ROW_0"."NS_ID$1" "NS_ID"
FROM
(SELECT
"NS_ID" "NS_ID$1",
"NS_OUTLET_SRC_ID" "NS_OUTLET_SRC_ID$1",
"NS_PUBLISHER_CODE" "NS_PUBLISHER_CODE$1",
"NS_TITLE_CLASSIFICATION_CODE" "NS_TITLE_CLASSIFICATION_CODE$1",
"NS_SUPPLY_FLAG" "NS_SUPPLY_FLAG$1",
"NS_EFF_DATE" "NS_EFF_DATE$1",
"NS_EXP_DATE" "NS_EXP_DATE$1"
FROM
(SELECT
(Case When (("SPLITTER_INPUT_SUBQUERY"."NS_ID_0_0" IS NULL) OR ((("SPLITTER_INPUT_SUBQUERY"."NS_EXP_DATE_0_0" IS NULL AND TO_CHAR("SPLITTER_INPUT_SUBQUERY"."NS_EFF_DATE_0_0", 'J.HH24.MI.SS') <= TO_CHAR("SPLITTER_INPUT_SUBQUERY"."NS_EFF_DATE_1", 'J.HH24.MI.SS')) OR ("SPLITTER_INPUT_SUBQUERY"."NS_EXP_DATE_0_0" IS NOT NULL AND TO_CHAR("SPLITTER_INPUT_SUBQUERY"."NS_EFF_DATE_0_0", 'J.HH24.MI.SS') <= TO_CHAR("SPLITTER_INPUT_SUBQUERY"."NS_EFF_DATE_1", 'J.HH24.MI.SS') AND TO_CHAR("SPLITTER_INPUT_SUBQUERY"."NS_EXP_DATE_0_0", 'J.HH24.MI.SS') >= TO_CHAR("SPLITTER_INPUT_SUBQUERY"."NS_EFF_DATE_1", 'J.HH24.MI.SS'))) AND (("SPLITTER_INPUT_SUBQUERY"."NS_TITLE_CLASSIFICATION_CO_1" IS NULL AND "SPLITTER_INPUT_SUBQUERY"."NS_TITLE_CLASSIFICATION_CO_2" IS NOT NULL) OR ("SPLITTER_INPUT_SUBQUERY"."NS_TITLE_CLASSIFICATION_CO_1" IS NOT NULL AND "SPLITTER_INPUT_SUBQUERY"."NS_TITLE_CLASSIFICATION_CO_2" IS NULL) OR ("SPLITTER_INPUT_SUBQUERY"."NS_TITLE_CLASSIFICATION_CO_1" != "SPLITTER_INPUT_SUBQUERY"."NS_TITLE_CLASSIFICATION_CO_2") OR ("SPLITTER_INPUT_SUBQUERY"."NS_SUPPLY_FLAG_1" IS NULL AND "SPLITTER_INPUT_SUBQUERY"."NS_SUPPLY_FLAG_0_0" IS NOT NULL) OR ("SPLITTER_INPUT_SUBQUERY"."NS_SUPPLY_FLAG_1" IS NOT NULL AND "SPLITTER_INPUT_SUBQUERY"."NS_SUPPLY_FLAG_0_0" IS NULL) OR ("SPLITTER_INPUT_SUBQUERY"."NS_SUPPLY_FLAG_1" != "SPLITTER_INPUT_SUBQUERY"."NS_SUPPLY_FLAG_0_0")))) then ("SPLITTER_INPUT_SUBQUERY"."NS_ID_1") else ("SPLITTER_INPUT_SUBQUERY"."NS_ID_0_0") end)/* MERGE_DELTA_ROW.OUTGRP1.NS_ID */ "NS_ID",
"SPLITTER_INPUT_SUBQUERY"."NS_OUTLET_SRC_ID_1"/* MERGE_DELTA_ROW.OUTGRP1.NS_OUTLET_SRC_ID */ "NS_OUTLET_SRC_ID",
"SPLITTER_INPUT_SUBQUERY"."NS_PUBLISHER_CODE_1"/* MERGE_DELTA_ROW.OUTGRP1.NS_PUBLISHER_CODE */ "NS_PUBLISHER_CODE",
"SPLITTER_INPUT_SUBQUERY"."NS_TITLE_CLASSIFICATION_CO_1"/* MERGE_DELTA_ROW.OUTGRP1.NS_TITLE_CLASSIFICATION_CODE */ "NS_TITLE_CLASSIFICATION_CODE",
"SPLITTER_INPUT_SUBQUERY"."NS_SUPPLY_FLAG_1"/* MERGE_DELTA_ROW.OUTGRP1.NS_SUPPLY_FLAG */ "NS_SUPPLY_FLAG",
(Case When (("SPLITTER_INPUT_SUBQUERY"."NS_ID_0_0" IS NULL)) then ((case when ("SPLITTER_INPUT_SUBQUERY"."NS_EFF_DATE_1" < SYSDATE ) then ("SPLITTER_INPUT_SUBQUERY"."NS_EFF_DATE_1") else ( SYSDATE ) end)) when ((("SPLITTER_INPUT_SUBQUERY"."NS_EXP_DATE_0_0" IS NULL AND TO_CHAR("SPLITTER_INPUT_SUBQUERY"."NS_EFF_DATE_0_0", 'J.HH24.MI.SS') <= TO_CHAR("SPLITTER_INPUT_SUBQUERY"."NS_EFF_DATE_1", 'J.HH24.MI.SS')) OR ("SPLITTER_INPUT_SUBQUERY"."NS_EXP_DATE_0_0" IS NOT NULL AND TO_CHAR("SPLITTER_INPUT_SUBQUERY"."NS_EFF_DATE_0_0", 'J.HH24.MI.SS') <= TO_CHAR("SPLITTER_INPUT_SUBQUERY"."NS_EFF_DATE_1", 'J.HH24.MI.SS') AND TO_CHAR("SPLITTER_INPUT_SUBQUERY"."NS_EXP_DATE_0_0", 'J.HH24.MI.SS') >= TO_CHAR("SPLITTER_INPUT_SUBQUERY"."NS_EFF_DATE_1", 'J.HH24.MI.SS'))) AND (("SPLITTER_INPUT_SUBQUERY"."NS_TITLE_CLASSIFICATION_CO_1" IS NULL AND "SPLITTER_INPUT_SUBQUERY"."NS_TITLE_CLASSIFICATION_CO_2" IS NOT NULL) OR ("SPLITTER_INPUT_SUBQUERY"."NS_TITLE_CLASSIFICATION_CO_1" IS NOT NULL AND "SPLITTER_INPUT_SUBQUERY"."NS_TITLE_CLASSIFICATION_CO_2" IS NULL) OR ("SPLITTER_INPUT_SUBQUERY"."NS_TITLE_CLASSIFICATION_CO_1" != "SPLITTER_INPUT_SUBQUERY"."NS_TITLE_CLASSIFICATION_CO_2") OR ("SPLITTER_INPUT_SUBQUERY"."NS_SUPPLY_FLAG_1" IS NULL AND "SPLITTER_INPUT_SUBQUERY"."NS_SUPPLY_FLAG_0_0" IS NOT NULL) OR ("SPLITTER_INPUT_SUBQUERY"."NS_SUPPLY_FLAG_1" IS NOT NULL AND "SPLITTER_INPUT_SUBQUERY"."NS_SUPPLY_FLAG_0_0" IS NULL) OR ("SPLITTER_INPUT_SUBQUERY"."NS_SUPPLY_FLAG_1" != "SPLITTER_INPUT_SUBQUERY"."NS_SUPPLY_FLAG_0_0"))) then ("SPLITTER_INPUT_SUBQUERY"."NS_EFF_DATE_1") else ("SPLITTER_INPUT_SUBQUERY"."NS_EFF_DATE_0_0") end)/* MERGE_DELTA_ROW.OUTGRP1.NS_EFF_DATE */ "NS_EFF_DATE",
(Case When ((ROW_NUMBER() OVER (PARTITION BY "SPLITTER_INPUT_SUBQUERY"."NS_OUTLET_SRC_ID_1","SPLITTER_INPUT_SUBQUERY"."NS_PUBLISHER_CODE_1" ORDER BY "SPLITTER_INPUT_SUBQUERY"."NS_EFF_DATE_1" DESC)) = 1) then (Case When (("SPLITTER_INPUT_SUBQUERY"."NS_ID_0_0" IS NULL) OR ((("SPLITTER_INPUT_SUBQUERY"."NS_EXP_DATE_0_0" IS NULL AND TO_CHAR("SPLITTER_INPUT_SUBQUERY"."NS_EFF_DATE_0_0", 'J.HH24.MI.SS') <= TO_CHAR("SPLITTER_INPUT_SUBQUERY"."NS_EFF_DATE_1", 'J.HH24.MI.SS')) OR ("SPLITTER_INPUT_SUBQUERY"."NS_EXP_DATE_0_0" IS NOT NULL AND TO_CHAR("SPLITTER_INPUT_SUBQUERY"."NS_EFF_DATE_0_0", 'J.HH24.MI.SS') <= TO_CHAR("SPLITTER_INPUT_SUBQUERY"."NS_EFF_DATE_1", 'J.HH24.MI.SS') AND TO_CHAR("SPLITTER_INPUT_SUBQUERY"."NS_EXP_DATE_0_0", 'J.HH24.MI.SS') >= TO_CHAR("SPLITTER_INPUT_SUBQUERY"."NS_EFF_DATE_1", 'J.HH24.MI.SS'))) AND (("SPLITTER_INPUT_SUBQUERY"."NS_TITLE_CLASSIFICATION_CO_1" IS NULL AND "SPLITTER_INPUT_SUBQUERY"."NS_TITLE_CLASSIFICATION_CO_2" IS NOT NULL) OR ("SPLITTER_INPUT_SUBQUERY"."NS_TITLE_CLASSIFICATION_CO_1" IS NOT NULL AND "SPLITTER_INPUT_SUBQUERY"."NS_TITLE_CLASSIFICATION_CO_2" IS NULL) OR ("SPLITTER_INPUT_SUBQUERY"."NS_TITLE_CLASSIFICATION_CO_1" != "SPLITTER_INPUT_SUBQUERY"."NS_TITLE_CLASSIFICATION_CO_2") OR ("SPLITTER_INPUT_SUBQUERY"."NS_SUPPLY_FLAG_1" IS NULL AND "SPLITTER_INPUT_SUBQUERY"."NS_SUPPLY_FLAG_0_0" IS NOT NULL) OR ("SPLITTER_INPUT_SUBQUERY"."NS_SUPPLY_FLAG_1" IS NOT NULL AND "SPLITTER_INPUT_SUBQUERY"."NS_SUPPLY_FLAG_0_0" IS NULL) OR ("SPLITTER_INPUT_SUBQUERY"."NS_SUPPLY_FLAG_1" != "SPLITTER_INPUT_SUBQUERY"."NS_SUPPLY_FLAG_0_0")))) then ( TO_DATE('31-DEC-4000') ) else ("SPLITTER_INPUT_SUBQUERY"."NS_EXP_DATE_0_0") end) else (("SPLITTER_INPUT_SUBQUERY"."NS_EFF_DATE_1" - INTERVAL '1' SECOND)) end)/* MERGE_DELTA_ROW.OUTGRP1.NS_EXP_DATE */ "NS_EXP_DATE"
FROM
(SELECT
"INGRP1"."NS_ID" "NS_ID_1",
"INGRP1"."NS_OUTLET_SRC_ID" "NS_OUTLET_SRC_ID_1",
"INGRP1"."NS_PUBLISHER_CODE" "NS_PUBLISHER_CODE_1",
"INGRP1"."NS_TITLE_CLASSIFICATION_CODE" "NS_TITLE_CLASSIFICATION_CO_1",
"INGRP1"."NS_SUPPLY_FLAG" "NS_SUPPLY_FLAG_1",
"INGRP1"."NS_EFF_DATE" "NS_EFF_DATE_1",
"INGRP1"."NS_EXP_DATE" "NS_EXP_DATE_1",
"INGRP2"."NS_ID" "NS_ID_0_0",
"INGRP2"."NS_OUTLET_SRC_ID" "NS_OUTLET_SRC_ID_0_0",
"INGRP2"."NS_PUBLISHER_CODE" "NS_PUBLISHER_CODE_0_0",
"INGRP2"."NS_TITLE_CLASSIFICATION_CODE" "NS_TITLE_CLASSIFICATION_CO_2",
"INGRP2"."NS_SUPPLY_FLAG" "NS_SUPPLY_FLAG_0_0",
"INGRP2"."NS_EFF_DATE" "NS_EFF_DATE_0_0",
"INGRP2"."NS_EXP_DATE" "NS_EXP_DATE_0_0",
"INGRP2"."DIMENSION_KEY" "DIMENSION_KEY_0"
FROM
( SELECT
"RETAILER_PUBLISHER_NS"."NS_ID" "NS_ID",
"RETAILER_PUBLISHER_NS"."NS_OUTLET_SRC_ID" "NS_OUTLET_SRC_ID",
"RETAILER_PUBLISHER_NS"."NS_PUBLISHER_CODE" "NS_PUBLISHER_CODE",
"RETAILER_PUBLISHER_NS"."NS_TITLE_CLASSIFICATION_CODE" "NS_TITLE_CLASSIFICATION_CODE",
"RETAILER_PUBLISHER_NS"."NS_SUPPLY_FLAG" "NS_SUPPLY_FLAG",
"RETAILER_PUBLISHER_NS"."NS_EFF_DATE" "NS_EFF_DATE",
"RETAILER_PUBLISHER_NS"."NS_EXP_DATE" "NS_EXP_DATE",
"RETAILER_PUBLISHER_NS"."DIMENSION_KEY" "DIMENSION_KEY"
FROM
"RETAILER_PUBLISHER_NS" "RETAILER_PUBLISHER_NS"
WHERE
( "RETAILER_PUBLISHER_NS"."DIMENSION_KEY" = "RETAILER_PUBLISHER_NS"."NS_ID" ) AND
( "RETAILER_PUBLISHER_NS"."NS_ID" IS NOT NULL ) ) "INGRP2"
RIGHT OUTER JOIN ( SELECT
NULL "NS_ID",
"LOOKUP_INPUT_SUBQUERY"."NS_OUTLET_SRC_ID$2" "NS_OUTLET_SRC_ID",
"LOOKUP_INPUT_SUBQUERY"."NS_PUBLISHER_CODE$2" "NS_PUBLISHER_CODE",
"LOOKUP_INPUT_SUBQUERY"."NS_TITLE_CLASSIFICATION_CODE$2" "NS_TITLE_CLASSIFICATION_CODE",
"LOOKUP_INPUT_SUBQUERY"."NS_SUPPLY_FLAG$2" "NS_SUPPLY_FLAG",
"LOOKUP_INPUT_SUBQUERY"."NS_EFF_DATE$2" "NS_EFF_DATE",
"LOOKUP_INPUT_SUBQUERY"."NS_EXP_DATE$2" "NS_EXP_DATE"
FROM
(SELECT
"DEDUP_SRC"."NS_ID$3" "NS_ID$2",
"DEDUP_SRC"."NS_OUTLET_SRC_ID$3" "NS_OUTLET_SRC_ID$2",
"DEDUP_SRC"."NS_PUBLISHER_CODE$3" "NS_PUBLISHER_CODE$2",
"DEDUP_SRC"."NS_TITLE_CLASSIFICATION_CODE$3" "NS_TITLE_CLASSIFICATION_CODE$2",
"DEDUP_SRC"."NS_SUPPLY_FLAG$3" "NS_SUPPLY_FLAG$2",
"DEDUP_SRC"."NS_EFF_DATE$3" "NS_EFF_DATE$2",
"DEDUP_SRC"."NS_EXP_DATE$3" "NS_EXP_DATE$2"
FROM
(SELECT
NULL/* DEDUP_SRC.OUTGRP1.NS_ID */ "NS_ID$3",
("PUB_AGENT_MATRIX_CC"."PAM_CUSTOMER_ID"/* EXPR_SRC.OUTGRP1.NS_OUTLET_SRC_ID */)/* DEDUP_SRC.OUTGRP1.NS_OUTLET_SRC_ID */ "NS_OUTLET_SRC_ID$3",
((to_char("PUB_AGENT_MATRIX_CC"."PAM_PUBLISHER_CODE")/* EXP.OUTGRP1.PAM_PUBLISHER_CODE */)/* EXPR_SRC.OUTGRP1.NS_PUBLISHER_CODE */)/* DEDUP_SRC.OUTGRP1.NS_PUBLISHER_CODE */ "NS_PUBLISHER_CODE$3",
("PUB_AGENT_MATRIX_CC"."PAM_TITLCLAS_CODE"/* EXPR_SRC.OUTGRP1.NS_TITLE_CLASSIFICATION_CODE */)/* DEDUP_SRC.OUTGRP1.NS_TITLE_CLASSIFICATION_CODE */ "NS_TITLE_CLASSIFICATION_CODE$3",
("PUB_AGENT_MATRIX_CC"."PAM_SUPPLY_FLAG"/* EXPR_SRC.OUTGRP1.NS_SUPPLY_FLAG */)/* DEDUP_SRC.OUTGRP1.NS_SUPPLY_FLAG */ "NS_SUPPLY_FLAG$3",
MIN(("PUB_AGENT_MATRIX_CC"."PAM_EFFECTIVE_DATE"/* EXPR_SRC.OUTGRP1.NS_EFF_DATE */)) KEEP (DENSE_RANK FIRST ORDER BY NULL/* EXPR_SRC.OUTGRP1.NS_ID */)/* DEDUP_SRC.OUTGRP1.NS_EFF_DATE */ "NS_EFF_DATE$3",
NULL/* DEDUP_SRC.OUTGRP1.NS_EXP_DATE */ "NS_EXP_DATE$3"
FROM
"REFSTG"."PUB_AGENT_MATRIX_CC" "PUB_AGENT_MATRIX_CC"
WHERE
( "PUB_AGENT_MATRIX_CC"."PAM_ADD_REMOVE_FLAG" = 'A' )
GROUP BY
("PUB_AGENT_MATRIX_CC"."PAM_CUSTOMER_ID"/* EXPR_SRC.OUTGRP1.NS_OUTLET_SRC_ID */), ((to_char("PUB_AGENT_MATRIX_CC"."PAM_PUBLISHER_CODE")/* EXP.OUTGRP1.PAM_PUBLISHER_CODE */)/* EXPR_SRC.OUTGRP1.NS_PUBLISHER_CODE */), ("PUB_AGENT_MATRIX_CC"."PAM_TITLCLAS_CODE"/* EXPR_SRC.OUTGRP1.NS_TITLE_CLASSIFICATION_CODE */), ("PUB_AGENT_MATRIX_CC"."PAM_SUPPLY_FLAG"/* EXPR_SRC.OUTGRP1.NS_SUPPLY_FLAG */),NULL,NULL/* RETAILER_PUBLISHER_NS.DEDUP_SRC */) "DEDUP_SRC") "LOOKUP_INPUT_SUBQUERY"
WHERE
( NOT ( "LOOKUP_INPUT_SUBQUERY"."NS_OUTLET_SRC_ID$2" IS NULL AND "LOOKUP_INPUT_SUBQUERY"."NS_PUBLISHER_CODE$2" IS NULL ) ) ) "INGRP1" ON ( ( ( "INGRP2"."NS_EFF_DATE" IS NULL OR ( ( "INGRP2"."NS_EXP_DATE" IS NULL AND TO_CHAR ( "INGRP2"."NS_EFF_DATE" , 'J.HH24.MI.SS' ) <= TO_CHAR ( "INGRP1"."NS_EFF_DATE" , 'J.HH24.MI.SS' ) ) OR ( "INGRP2"."NS_EXP_DATE" IS NOT NULL AND TO_CHAR ( "INGRP2"."NS_EFF_DATE" , 'J.HH24.MI.SS' ) <= TO_CHAR ( "INGRP1"."NS_EFF_DATE" , 'J.HH24.MI.SS' ) AND TO_CHAR ( "INGRP2"."NS_EXP_DATE" , 'J.HH24.MI.SS' ) >= TO_CHAR ( "INGRP1"."NS_EFF_DATE" , 'J.HH24.MI.SS' ) ) ) ) ) AND ( ( "INGRP2"."NS_PUBLISHER_CODE" = "INGRP1"."NS_PUBLISHER_CODE" ) ) AND ( ( "INGRP2"."NS_OUTLET_SRC_ID" = "INGRP1"."NS_OUTLET_SRC_ID" ) ) )) "SPLITTER_INPUT_SUBQUERY"
WHERE
( ( ( "SPLITTER_INPUT_SUBQUERY"."NS_OUTLET_SRC_ID_1" = "SPLITTER_INPUT_SUBQUERY"."NS_OUTLET_SRC_ID_0_0" AND "SPLITTER_INPUT_SUBQUERY"."NS_PUBLISHER_CODE_1" = "SPLITTER_INPUT_SUBQUERY"."NS_PUBLISHER_CODE_0_0" ) ) OR ( "SPLITTER_INPUT_SUBQUERY"."NS_OUTLET_SRC_ID_0_0" IS NULL AND "SPLITTER_INPUT_SUBQUERY"."NS_PUBLISHER_CODE_0_0" IS NULL ) )
UNION
SELECT
"DEDUP_SCD_SRC"."NS_ID$4" "NS_ID",
"DEDUP_SCD_SRC"."NS_OUTLET_SRC_ID$4" "NS_OUTLET_SRC_ID",
"DEDUP_SCD_SRC"."NS_PUBLISHER_CODE$4" "NS_PUBLISHER_CODE",
"DEDUP_SCD_SRC"."NS_TITLE_CLASSIFICATION_CODE$4" "NS_TITLE_CLASSIFICATION_CODE",
"DEDUP_SCD_SRC"."NS_SUPPLY_FLAG$4" "NS_SUPPLY_FLAG",
"DEDUP_SCD_SRC"."NS_EFF_DATE$4" "NS_EFF_DATE",
"DEDUP_SCD_SRC"."NS_EXP_DATE$4" "NS_EXP_DATE"
FROM
(SELECT
"AGG_INPUT"."NS_ID$5"/* DEDUP_SCD_SRC.OUTGRP1.NS_ID */ "NS_ID$4",
"AGG_INPUT"."NS_OUTLET_SRC_ID$5"/* DEDUP_SCD_SRC.OUTGRP1.NS_OUTLET_SRC_ID */ "NS_OUTLET_SRC_ID$4",
"AGG_INPUT"."NS_PUBLISHER_CODE$5"/* DEDUP_SCD_SRC.OUTGRP1.NS_PUBLISHER_CODE */ "NS_PUBLISHER_CODE$4",
MIN("AGG_INPUT"."NS_TITLE_CLASSIFICATION_CODE$5") KEEP (DENSE_RANK FIRST ORDER BY "AGG_INPUT"."NS_TITLE_CLASSIFICATION_CODE$5" NULLS LAST)/* DEDUP_SCD_SRC.OUTGRP1.NS_TITLE_CLASSIFICATION_CODE */ "NS_TITLE_CLASSIFICATION_CODE$4",
MIN("AGG_INPUT"."NS_SUPPLY_FLAG$5") KEEP (DENSE_RANK FIRST ORDER BY "AGG_INPUT"."NS_SUPPLY_FLAG$5" NULLS LAST)/* DEDUP_SCD_SRC.OUTGRP1.NS_SUPPLY_FLAG */ "NS_SUPPLY_FLAG$4",
MIN("AGG_INPUT"."NS_EFF_DATE$5") KEEP (DENSE_RANK FIRST ORDER BY "AGG_INPUT"."NS_EFF_DATE$5" NULLS LAST)/* DEDUP_SCD_SRC.OUTGRP1.NS_EFF_DATE */ "NS_EFF_DATE$4",
MIN("AGG_INPUT"."NS_EXP_DATE$5") KEEP (DENSE_RANK FIRST ORDER BY "AGG_INPUT"."NS_EXP_DATE$5" NULLS LAST)/* DEDUP_SCD_SRC.OUTGRP1.NS_EXP_DATE */ "NS_EXP_DATE$4"
FROM
(SELECT
"SPLITTER_INPUT_SUBQUERY$1"."NS_ID_0_0$1"/* UPDATE_DELTA_ROW.OUTGRP1.NS_ID */ "NS_ID$5",
"SPLITTER_INPUT_SUBQUERY$1"."NS_OUTLET_SRC_ID_1$1"/* UPDATE_DELTA_ROW.OUTGRP1.NS_OUTLET_SRC_ID */ "NS_OUTLET_SRC_ID$5",
"SPLITTER_INPUT_SUBQUERY$1"."NS_PUBLISHER_CODE_1$1"/* UPDATE_DELTA_ROW.OUTGRP1.NS_PUBLISHER_CODE */ "NS_PUBLISHER_CODE$5",
"SPLITTER_INPUT_SUBQUERY$1"."NS_TITLE_CLASSIFICATION_CO_2$1"/* UPDATE_DELTA_ROW.OUTGRP1.NS_TITLE_CLASSIFICATION_CODE */ "NS_TITLE_CLASSIFICATION_CODE$5",
"SPLITTER_INPUT_SUBQUERY$1"."NS_SUPPLY_FLAG_0_0$1"/* UPDATE_DELTA_ROW.OUTGRP1.NS_SUPPLY_FLAG */ "NS_SUPPLY_FLAG$5",
"SPLITTER_INPUT_SUBQUERY$1"."NS_EFF_DATE_0_0$1"/* UPDATE_DELTA_ROW.OUTGRP1.NS_EFF_DATE */ "NS_EFF_DATE$5",
("SPLITTER_INPUT_SUBQUERY$1"."NS_EFF_DATE_1$1" - INTERVAL '1' SECOND)/* UPDATE_DELTA_ROW.OUTGRP1.NS_EXP_DATE */ "NS_EXP_DATE$5"
FROM
(SELECT
"INGRP1"."NS_ID" "NS_ID_1$1",
"INGRP1"."NS_OUTLET_SRC_ID" "NS_OUTLET_SRC_ID_1$1",
"INGRP1"."NS_PUBLISHER_CODE" "NS_PUBLISHER_CODE_1$1",
"INGRP1"."NS_TITLE_CLASSIFICATION_CODE" "NS_TITLE_CLASSIFICATION_CO_1$1",
"INGRP1"."NS_SUPPLY_FLAG" "NS_SUPPLY_FLAG_1$1",
"INGRP1"."NS_EFF_DATE" "NS_EFF_DATE_1$1",
"INGRP1"."NS_EXP_DATE" "NS_EXP_DATE_1$1",
"INGRP2"."NS_ID" "NS_ID_0_0$1",
"INGRP2"."NS_OUTLET_SRC_ID" "NS_OUTLET_SRC_ID_0_0$1",
"INGRP2"."NS_PUBLISHER_CODE" "NS_PUBLISHER_CODE_0_0$1",
"INGRP2"."NS_TITLE_CLASSIFICATION_CODE" "NS_TITLE_CLASSIFICATION_CO_2$1",
"INGRP2"."NS_SUPPLY_FLAG" "NS_SUPPLY_FLAG_0_0$1",
"INGRP2"."NS_EFF_DATE" "NS_EFF_DATE_0_0$1",
"INGRP2"."NS_EXP_DATE" "NS_EXP_DATE_0_0$1",
"INGRP2"."DIMENSION_KEY" "DIMENSION_KEY_0$1"
FROM
( SELECT
"RETAILER_PUBLISHER_NS"."NS_ID" "NS_ID",
"RETAILER_PUBLISHER_NS"."NS_OUTLET_SRC_ID" "NS_OUTLET_SRC_ID",
"RETAILER_PUBLISHER_NS"."NS_PUBLISHER_CODE" "NS_PUBLISHER_CODE",
"RETAILER_PUBLISHER_NS"."NS_TITLE_CLASSIFICATION_CODE" "NS_TITLE_CLASSIFICATION_CODE",
"RETAILER_PUBLISHER_NS"."NS_SUPPLY_FLAG" "NS_SUPPLY_FLAG",
"RETAILER_PUBLISHER_NS"."NS_EFF_DATE" "NS_EFF_DATE",
"RETAILER_PUBLISHER_NS"."NS_EXP_DATE" "NS_EXP_DATE",
"RETAILER_PUBLISHER_NS"."DIMENSION_KEY" "DIMENSION_KEY"
FROM
"RETAILER_PUBLISHER_NS" "RETAILER_PUBLISHER_NS"
WHERE
( "RETAILER_PUBLISHER_NS"."DIMENSION_KEY" = "RETAILER_PUBLISHER_NS"."NS_ID" ) AND
( "RETAILER_PUBLISHER_NS"."NS_ID" IS NOT NULL ) ) "INGRP2"
RIGHT OUTER JOIN ( SELECT
NULL "NS_ID",
"LOOKUP_INPUT_SUBQUERY"."NS_OUTLET_SRC_ID$2" "NS_OUTLET_SRC_ID",
"LOOKUP_INPUT_SUBQUERY"."NS_PUBLISHER_CODE$2" "NS_PUBLISHER_CODE",
"LOOKUP_INPUT_SUBQUERY"."NS_TITLE_CLASSIFICATION_CODE$2" "NS_TITLE_CLASSIFICATION_CODE",
"LOOKUP_INPUT_SUBQUERY"."NS_SUPPLY_FLAG$2" "NS_SUPPLY_FLAG",
"LOOKUP_INPUT_SUBQUERY"."NS_EFF_DATE$2" "NS_EFF_DATE",
"LOOKUP_INPUT_SUBQUERY"."NS_EXP_DATE$2" "NS_EXP_DATE"
FROM
(SELECT
"DEDUP_SRC"."NS_ID$3" "NS_ID$2",
"DEDUP_SRC"."NS_OUTLET_SRC_ID$3" "NS_OUTLET_SRC_ID$2",
"DEDUP_SRC"."NS_PUBLISHER_CODE$3" "NS_PUBLISHER_CODE$2",
"DEDUP_SRC"."NS_TITLE_CLASSIFICATION_CODE$3" "NS_TITLE_CLASSIFICATION_CODE$2",
"DEDUP_SRC"."NS_SUPPLY_FLAG$3" "NS_SUPPLY_FLAG$2",
"DEDUP_SRC"."NS_EFF_DATE$3" "NS_EFF_DATE$2",
"DEDUP_SRC"."NS_EXP_DATE$3" "NS_EXP_DATE$2"
FROM
(SELECT
NULL/* DEDUP_SRC.OUTGRP1.NS_ID */ "NS_ID$3",
("PUB_AGENT_MATRIX_CC"."PAM_CUSTOMER_ID"/* EXPR_SRC.OUTGRP1.NS_OUTLET_SRC_ID */)/* DEDUP_SRC.OUTGRP1.NS_OUTLET_SRC_ID */ "NS_OUTLET_SRC_ID$3",
((to_char("PUB_AGENT_MATRIX_CC"."PAM_PUBLISHER_CODE")/* EXP.OUTGRP1.PAM_PUBLISHER_CODE */)/* EXPR_SRC.OUTGRP1.NS_PUBLISHER_CODE */)/* DEDUP_SRC.OUTGRP1.NS_PUBLISHER_CODE */ "NS_PUBLISHER_CODE$3",
("PUB_AGENT_MATRIX_CC"."PAM_TITLCLAS_CODE"/* EXPR_SRC.OUTGRP1.NS_TITLE_CLASSIFICATION_CODE */)/* DEDUP_SRC.OUTGRP1.NS_TITLE_CLASSIFICATION_CODE */ "NS_TITLE_CLASSIFICATION_CODE$3",
("PUB_AGENT_MATRIX_CC"."PAM_SUPPLY_FLAG"/* EXPR_SRC.OUTGRP1.NS_SUPPLY_FLAG */)/* DEDUP_SRC.OUTGRP1.NS_SUPPLY_FLAG */ "NS_SUPPLY_FLAG$3",
MIN(("PUB_AGENT_MATRIX_CC"."PAM_EFFECTIVE_DATE"/* EXPR_SRC.OUTGRP1.NS_EFF_DATE */)) KEEP (DENSE_RANK FIRST ORDER BY NULL/* EXPR_SRC.OUTGRP1.NS_ID */)/* DEDUP_SRC.OUTGRP1.NS_EFF_DATE */ "NS_EFF_DATE$3",
NULL/* DEDUP_SRC.OUTGRP1.NS_EXP_DATE */ "NS_EXP_DATE$3"
FROM
"REFSTG"."PUB_AGENT_MATRIX_CC" "PUB_AGENT_MATRIX_CC"
WHERE
( "PUB_AGENT_MATRIX_CC"."PAM_ADD_REMOVE_FLAG" = 'A' )
GROUP BY
("PUB_AGENT_MATRIX_CC"."PAM_CUSTOMER_ID"/* EXPR_SRC.OUTGRP1.NS_OUTLET_SRC_ID */), ((to_char("PUB_AGENT_MATRIX_CC"."PAM_PUBLISHER_CODE")/* EXP.OUTGRP1.PAM_PUBLISHER_CODE */)/* EXPR_SRC.OUTGRP1.NS_PUBLISHER_CODE */), ("PUB_AGENT_MATRIX_CC"."PAM_TITLCLAS_CODE"/* EXPR_SRC.OUTGRP1.NS_TITLE_CLASSIFICATION_CODE */), ("PUB_AGENT_MATRIX_CC"."PAM_SUPPLY_FLAG"/* EXPR_SRC.OUTGRP1.NS_SUPPLY_FLAG */),NULL,NULL/* RETAILER_PUBLISHER_NS.DEDUP_SRC */) "DEDUP_SRC") "LOOKUP_INPUT_SUBQUERY"
WHERE
( NOT ( "LOOKUP_INPUT_SUBQUERY"."NS_OUTLET_SRC_ID$2" IS NULL AND "LOOKUP_INPUT_SUBQUERY"."NS_PUBLISHER_CODE$2" IS NULL ) ) ) "INGRP1" ON ( ( ( "INGRP2"."NS_EFF_DATE" IS NULL OR ( ( "INGRP2"."NS_EXP_DATE" IS NULL AND TO_CHAR ( "INGRP2"."NS_EFF_DATE" , 'J.HH24.MI.SS' ) <= TO_CHAR ( "INGRP1"."NS_EFF_DATE" , 'J.HH24.MI.SS' ) ) OR ( "INGRP2"."NS_EXP_DATE" IS NOT NULL AND TO_CHAR ( "INGRP2"."NS_EFF_DATE" , 'J.HH24.MI.SS' ) <= TO_CHAR ( "INGRP1"."NS_EFF_DATE" , 'J.HH24.MI.SS' ) AND TO_CHAR ( "INGRP2"."NS_EXP_DATE" , 'J.HH24.MI.SS' ) >= TO_CHAR ( "INGRP1"."NS_EFF_DATE" , 'J.HH24.MI.SS' ) ) ) ) ) AND ( ( "INGRP2"."NS_PUBLISHER_CODE" = "INGRP1"."NS_PUBLISHER_CODE" ) ) AND ( ( "INGRP2"."NS_OUTLET_SRC_ID" = "INGRP1"."NS_OUTLET_SRC_ID" ) ) )) "SPLITTER_INPUT_SUBQUERY$1"
WHERE
( "SPLITTER_INPUT_SUBQUERY$1"."NS_OUTLET_SRC_ID_1$1" = "SPLITTER_INPUT_SUBQUERY$1"."NS_OUTLET_SRC_ID_0_0$1" AND "SPLITTER_INPUT_SUBQUERY$1"."NS_PUBLISHER_CODE_1$1" = "SPLITTER_INPUT_SUBQUERY$1"."NS_PUBLISHER_CODE_0_0$1" ) AND
( ( "SPLITTER_INPUT_SUBQUERY$1"."NS_EXP_DATE_0_0$1" IS NULL AND TO_CHAR ( "SPLITTER_INPUT_SUBQUERY$1"."NS_EFF_DATE_0_0$1" , 'J.HH24.MI.SS' ) <= TO_CHAR ( "SPLITTER_INPUT_SUBQUERY$1"."NS_EFF_DATE_1$1" , 'J.HH24.MI.SS' ) ) OR ( "SPLITTER_INPUT_SUBQUERY$1"."NS_EXP_DATE_0_0$1" IS NOT NULL AND TO_CHAR ( "SPLITTER_INPUT_SUBQUERY$1"."NS_EFF_DATE_0_0$1" , 'J.HH24.MI.SS' ) <= TO_CHAR ( "SPLITTER_INPUT_SUBQUERY$1"."NS_EFF_DATE_1$1" , 'J.HH24.MI.SS' ) AND TO_CHAR ( "SPLITTER_INPUT_SUBQUERY$1"."NS_EXP_DATE_0_0$1" , 'J.HH24.MI.SS' ) >= TO_CHAR ( "SPLITTER_INPUT_SUBQUERY$1"."NS_EFF_DATE_1$1" , 'J.HH24.MI.SS' ) ) ) AND
( ( "SPLITTER_INPUT_SUBQUERY$1"."NS_TITLE_CLASSIFICATION_CO_1$1" IS NULL AND "SPLITTER_INPUT_SUBQUERY$1"."NS_TITLE_CLASSIFICATION_CO_2$1" IS NOT NULL ) OR ( "SPLITTER_INPUT_SUBQUERY$1"."NS_TITLE_CLASSIFICATION_CO_1$1" IS NOT NULL AND "SPLITTER_INPUT_SUBQUERY$1"."NS_TITLE_CLASSIFICATION_CO_2$1" IS NULL ) OR ( "SPLITTER_INPUT_SUBQUERY$1"."NS_TITLE_CLASSIFICATION_CO_1$1" != "SPLITTER_INPUT_SUBQUERY$1"."NS_TITLE_CLASSIFICATION_CO_2$1" ) OR ( "SPLITTER_INPUT_SUBQUERY$1"."NS_SUPPLY_FLAG_1$1" IS NULL AND "SPLITTER_INPUT_SUBQUERY$1"."NS_SUPPLY_FLAG_0_0$1" IS NOT NULL ) OR ( "SPLITTER_INPUT_SUBQUERY$1"."NS_SUPPLY_FLAG_1$1" IS NOT NULL AND "SPLITTER_INPUT_SUBQUERY$1"."NS_SUPPLY_FLAG_0_0$1" IS NULL ) OR ( "SPLITTER_INPUT_SUBQUERY$1"."NS_SUPPLY_FLAG_1$1" != "SPLITTER_INPUT_SUBQUERY$1"."NS_SUPPLY_FLAG_0_0$1" ) )) "AGG_INPUT"
GROUP BY
"AGG_INPUT"."NS_ID$5", "AGG_INPUT"."NS_OUTLET_SRC_ID$5", "AGG_INPUT"."NS_PUBLISHER_CODE$5"/* RETAILER_PUBLISHER_NS.DEDUP_SCD_SRC */) "DEDUP_SCD_SRC") ) "MERGE_DELTA_ROW_0"
MERGE_SUBQUERY
ON (
"RETAILER_PUBLISHER_NS"."NS_OUTLET_SRC_ID" = "MERGE_SUBQUERY"."NS_OUTLET_SRC_ID" AND
"RETAILER_PUBLISHER_NS"."NS_PUBLISHER_CODE" = "MERGE_SUBQUERY"."NS_PUBLISHER_CODE" AND
"RETAILER_PUBLISHER_NS"."NS_EFF_DATE" = "MERGE_SUBQUERY"."NS_EFF_DATE" AND
"RETAILER_PUBLISHER_NS"."NS_ID" = "MERGE_SUBQUERY"."NS_ID"
WHEN MATCHED THEN
UPDATE
SET
"NS_TITLE_CLASSIFICATION_CODE" = "MERGE_SUBQUERY"."NS_TITLE_CLASSIFICATION_CODE",
"NS_SUPPLY_FLAG" = "MERGE_SUBQUERY"."NS_SUPPLY_FLAG",
"NS_EXP_DATE" = "MERGE_SUBQUERY"."NS_EXP_DATE"
WHEN NOT MATCHED THEN
INSERT
("RETAILER_PUBLISHER_NS"."NS_ID",
"RETAILER_PUBLISHER_NS"."NS_OUTLET_SRC_ID",
"RETAILER_PUBLISHER_NS"."NS_PUBLISHER_CODE",
"RETAILER_PUBLISHER_NS"."NS_TITLE_CLASSIFICATION_CODE",
"RETAILER_PUBLISHER_NS"."NS_SUPPLY_FLAG",
"RETAILER_PUBLISHER_NS"."NS_EFF_DATE",
"RETAILER_PUBLISHER_NS"."NS_EXP_DATE",
"RETAILER_PUBLISHER_NS"."DIMENSION_KEY")
VALUES
("RETAILER_PUBLISHER_NS_SEQ".NEXTVAL,
"MERGE_SUBQUERY"."NS_OUTLET_SRC_ID",
"MERGE_SUBQUERY"."NS_PUBLISHER_CODE",
"MERGE_SUBQUERY"."NS_TITLE_CLASSIFICATION_CODE",
"MERGE_SUBQUERY"."NS_SUPPLY_FLAG",
"MERGE_SUBQUERY"."NS_EFF_DATE",
"MERGE_SUBQUERY"."NS_EXP_DATE",
"RETAILER_PUBLISHER_NS_SEQ".CURRVAL)
Explain plan:
MERGE STATEMENT, GOAL = ALL_ROWS 1412 2 286
MERGE DW RETAILER_PUBLISHER_NS
VIEW DW
SEQUENCE DW RETAILER_PUBLISHER_NS_SEQ
HASH JOIN OUTER 1412 2 256
VIEW DW 940 2 170
SORT UNIQUE 940 2 218
UNION-ALL
WINDOW SORT 470 1 133
FILTER
NESTED LOOPS OUTER 468 1 133
VIEW DW 4 1 65
SORT GROUP BY 4 1 25
TABLE ACCESS FULL REFSTG PUB_AGENT_MATRIX_CC 3 1 25
VIEW SYS 464 1 68
VIEW DW 464 1 68
TABLE ACCESS FULL DW RETAILER_PUBLISHER_NS 464 1 43
VIEW DW 469 1 85
SORT GROUP BY 469 1 90
NESTED LOOPS 468 1 90
VIEW DW 4 1 37
SORT GROUP BY 4 1 25
TABLE ACCESS FULL REFSTG PUB_AGENT_MATRIX_CC 3 1 25
VIEW SYS 464 1 53
VIEW DW 464 1 68
TABLE ACCESS FULL DW RETAILER_PUBLISHER_NS 464 1 43
TABLE ACCESS FULL DW RETAILER_PUBLISHER_NS 467 337417 14508931
Is this similar to the sql generated at your end? Do you use special loading hints, anything specail with indexing - we have tried standard indexing.
Does this look untoward - have you any other suggestions?
Thanks for your interest.
Similar Messages
-
Creating an SCD Type 2 in T SQL without using MERGE
I am attempting to create an SCD type 2 using T-SQL without MERGE (I'm not allowed to use it as a condition of the development work I am doing). I can use a CTE, but I have not tried that yet.
I have a temp table that contains ten records that I am testing with. The following is one variant of the code I have used to try and make this work:
declare
@System_User nchar(50)
,@CurrentDate datetime
,@MaxCheckDate datetime
set @System_User = system_user
set @CurrentDate = getdate()
--INSERT
insert dim.slot
Source_Slot_ID
,Slot_Start_DateTime
,Patients_PerSlot
,IsSlotSearchable
,IsSlotVisible
,[Created_Date]
,[Created_By]
select
src.IdSlot
,src.SlotDateTime
,src.PatientsPerSlot
,src.IsSlotSearchable
,src.IsSlotVisible
,@CurrentDate
,@System_User
from #TmepSlot src
left join dim.Slot dest
on src.IdSlot = dest.Source_Slot_ID
left join (select source_slot_id, max(created_date) as created_date from dim.slot group by Source_Slot_ID) MaxRecord
on dest.Source_Slot_ID = MaxRecord.Source_Slot_ID
and dest.Created_Date = MaxRecord.created_date
where dest.Source_Slot_ID is null
or
src.PatientsPerSlot
<> dest.Patients_PerSlot
or
src.IsSlotSearchable <> dest.IsSlotSearchable
or
src.IsSlotVisible
<> dest.IsSlotVisible
The problem with this variation is that when I change a value in the source like src.Patients_PerSlot, and then run the query, I get the new record i expect, but when I run the query again, a duplicate record gets created.
How do I correctly isolate the correct latest record and add the changed record without inserting that changed record more than once?
Thank you for your help.
cdun2Hi,
shouldn't you use an inner join between dest and maxrecord like so:
from #TmepSlot src
left join (dim.Slot dest
inner join (select source_slot_id, max(created_date) as created_date from dim.slot group by Source_Slot_ID) MaxRecord
on dest.Source_Slot_ID = MaxRecord.Source_Slot_ID
and dest.Created_Date = MaxRecord.created_date)
on src.IdSlot = dest.Source_Slot_ID
where dest.Source_Slot_ID is null
regards,
Rudolf
Rudolf Swiers
Thanks! I don't remember when I've done a join that way, but it makes sense.
cdun2 -
Has anyone ever implemented SCD Type-4 using SSIS??
Hello Experts!!
I have been trying to implement SCD TYPE-4 using SSIS and really got stuck and searched on-line for help. for my surprise, there isn't anything up on this topic.
I know the theory behind SCD Type-4 is to maintain history in seperate tables in a rapid changing dimensions.
please help if any of you ever implemented scd type-4 using SSIS.Hi,
The stock Slowly Changing Dimension Transformation of SSIS only supports SCD Type 1 and Type 2. For SCD Type 4, it maintains two tables: one to keep the current data, and the other one to keep the historical data. As a workaround, you can also implement
SCD Type 1 via SCD Transformation, and implement Change Data Capture at the same time. SSIS also provides CDC Control Task and related Data Flow components.
References:
http://www.bidn.com/blogs/TomLannen/bidn-blog/2606/slowly-changing-dimension-type-1-changes-using-ssis-scd-component
http://www.mattmasson.com/2011/12/cdc-in-ssis-for-sql-server-2012-2/
Regards,
Mike Yin
TechNet Community Support -
Hello
I am writing a pl-sql which will perform SCD type 1.
The Type 1 methodology overwrites old data with new data, and therefore does not track historical data at all. This is most appropriate when correcting certain types of data errors, such as the spelling of a name. (Assuming you won't ever need to know how it used to be misspelled in the past.)
Another example would be of a database table that keeps supplier information.
Supplier_key Supplier_Name Supplier_State
001 Phlogistical Supply Company CA
so hence I created two tables
create table ssn_load1
( ssn number(10,0),
credit_score number(6,0));
create table ssn_load2
( ssn number(10,0),
credit_score number(6,0));
and the target table
create table ssn_target
( sq_id number(8,0) primary key,
ssn number(10,0),
credit_score number(6,0));
since I want sq_id as auto incremented,I have created a following trigger
CREATE SEQUENCE test_sequence
START WITH 1
INCREMENT BY 1;
CREATE OR REPLACE TRIGGER test_trigger
BEFORE INSERT
ON ssn_target
REFERENCING NEW AS NEW
FOR EACH ROW
BEGIN
SELECT test_sequence.nextval INTO :NEW.SQ_ID FROM dual;
END;
Now inoder to perform type 1 I have followed a following thing
YOu have source table
tbl1(col1, col2, col3) - where col1 is the key and (col2, col3) are the attributes
target table
tbl2(col21, col22) - where (col21, col21) are the attributes
do this join for change data capture
select <col list>,
case when col22 is null and col23 is null then 'NEW'
when col2 = col22 and col3 = col23 then 'NO CHANGE'
else 'MODIFIED' end
from
tbl1 LEFT OUTER JOIN tbl2
ON tbl1.col1 = tbl2.col21
following is my pl-sql ,but I dont know how to write insert update statement in the switch case.
declare
cursor ssn1
is select from ssn_load1;*
for row in ssn1 loop
select s2.ssn,s2.credit_score from ssn_load2 s2 left outer join ssn_load1 s1 on s1.ssn=s2.ssn;
* SWITCH (ssn)*
* DO*
CASE 'Insert':if s2.ssn!=s1.ssn
Insert into ssn_target(ssn,credit_score)values();
CASE 'No Change':
CASE 'Modify':Update ssn_target set
DEFAULT:
DO END;
end loop;
END;
please help me how to proceed from here
will be waiting for reply
Thank You!!Isn't this a MERGE?
In the outer join approach, it might be simpler to do the join in the cursor and keep the PL/SQL simple. I didn't really understand your three-table scenario though. -
Hi,
has anybody ever designed a mapping to fill a scd type 2 with more than one level? I know the example from the transformation guide and the whitepaper and it works fine with one level. But you can't use it with dimensions with more levels in my opinion:
The problem is how to fill the id from the not base levels.
1) If I use a sequence for that, Oracle can't use query rewrite on this dimension for aggregating higher levels, can it?
2) If I use let's say the id from the enterprise model for the higher levels, that dimension becomes invalid as soon as a higher level attribute changes (check this with dbms_dimension.validate_dimension). So query rewrite will fail again, won't it?
All examples found in oracle docs are using just one level except for one example, and this doesn't fill the id from the second level.
A solution? Comments?
Regards,
TorstenHi Patrick!
You've problably defined a number of trigger columns
which determines the loading action, so
NUMBER_OF_CITIZENS would be one of them. As soon as
this changes, on the next load all persons that live
in Köln would be SCD'd (i.e. have their current
record closed and a new one inserted with the new
number of citizens).Correct. Let's use this as an example.
You are making it sound like you have two completely
different source sets which you offer 'independently'
to the dimension (since you want to make use of the
key on the city-level). No. I don't want to use the city-level-key. But when defining a dimension, there is one key (called id in the owb) and some more dependend attributes.
CREATE DIMENSION dim_person
LEVEL person IS (dim_person.pers_id)
LEVEL city IS (dim_person.city_id)
HIERARCHY std (
person CHILD OF
city
JOIN KEY (dim_person.city_id) REFERENCES city
ATTRIBUTE city DETERMINES (dim_person.population)
But in the end it does not matter whether you have a
unique identification of the city level, since
records are inserted on person-level, and that is
where you have defined that extra MATCHING column on,
yes?No. The unique identification on city level is for validating the dimension. Of course the table which holds the dimension data is always valid whatever the data is. But the data has to be well formed for a valid dimension. If I use the example from above with dbms_dimension.validate_dimension(...) the result will be an invalid dimension.
If I'm completely missing the point here, I guess I'm
not understanding your problem and its approach
then.The approach would be building your changed/unchanged transformation for each level. But I am asking here if I am the only person who sees this problem? Or maybe here is no problem.
My bottom line is: when you're loading a dimension
you have only one source set that is compared to the
current dimension, and your point of view is the
lowest level, unregarding any of its higher level
attributes unless they are part of the natural key.
Clarifying this using your example, there are two
source tables:
-ODS_PERSON with attributes PERSON_ID (surrogate
key), PERSON_NAME, PERSON_TAX_NUMBER (natural key),
MARITAL_STATUS, CITY_ID (FK to ODS_CITY)
-ODS_CITY with attributes CITY_ID (SK), NAME (NK),
POPULATION.
The dimension would have the following attributes:
ID, PERSON_NAME, PERSON_TAX_NUMBER,
PERSON_MARITAL_STATUS, CITY_NAME, CITY_POPULATION
(divided over 2 levels).
The JOIN would then compare values from joined
ODS_PERSON-ODS_CITY with the existing dimension
values.
Trigger columns would be PERSON_MARITAL_STATUS,
CITY_NAME, CITY_POPULATION and the overall natural
key (for matching purposes) for the dimension would
be PERSON_TAX_NUMBER.
As you can see, in this example there is no need for
any unique identification on any other level than the
lowest.But you have to give the dimension the unique key. This is what you are doing when renaming the id of the city level to the city_name.
LEVEL city IS (dim_person.city_name)
ATTRIBUTE city DETERMINES (dim_person.population)
Cheers, PatrickI hope I could point to the problem a little bit more.
Regards,
Torsten -
Hi All,
Can anyone tell me the procedure to do a SCD type 2 in ODI. Please list me the steps. I tried doing it myself but it seems to be not working. Especially the Sequence generator. The surrogate keys are not getting generated and nor the values getting updated. I tried to look for the solution or examples in the internet but without success.
Thanks in advanceFor a better performance, you can use a Query transform before Table Comparison, use the primary key fields (that you use inside Table Comparison) in the Order By tab of the Query transform and select the option Sorted input in Table Comparison transform.
-
When I try to sync my iPod with iTunes I can't because it says it's over capacity. It's not. It's a 16 GB iPod, it's only 105 songs on there. No movies or pictures or such. Anybody got a word of advise? I've tried resetting the iPod but get the message that it's not possible cause i'm not connected to internet. I am though. Tried managing music manually. I've tried all the usual stuff.
Go to Settings > General > Usage and see how much is shown available and how much is in "Other".
-
ODI - SCD Type 2 - Insert new row error
Hi All,
For Dimension I have a surrogate key, a natural key, and a column with "overwrite on change", start_date, end_date, current_record_ind. When I run the interface with the default SCD Type 2 for SQl server, this runs fine. But when I change that one column from "overwrite on change" to "insert new row", it fails on the update step. What should I be looking for anf fixing.
Thanks for your time and help.DB: SQL Server 2008.
IKM = IKM MSSQL Slowly Changing Dimension
Error Message: ODI-1228 - Incorrect syntax near the keyword from.
Code: update T
set
from database.dbo.Dim_type as T,
database.dbo.I$_Dim_type as S
where T.Cd = S.Cd
and T.Current_rec_ind = 1
and IND_UPDATE = 'U'
To overcome the issue, I have commented the update code in the Knowledge module and the insert works, but for this one it is ok, but I have requirements where one column needs to be overwritten and other column changes will require to add a new record. How to handle both?
Thanks for your time. -
My music files etc were all located on my old laptop which was stolen. I have just plugged my iphone in to my new computer and it has deleted all my files. Has anybody got any ideas on how to retrieve the files?
WINDOWS?
Connect the iPod to your PC. If iTunes starts syncing (ie erasing) your music automatically, hit the X in the upper right hand corner of iTunes display, to the left of the search box, to stop it.
In Control Panel, Portable Media Devices, double-click your iPod.
In the Tools menu -> Options, in the View Tab, check "Show hidden files and folders."
Navigate to the Music folder. On my 'pod, the full path is
Portable Media Devices\NAME of IPOD (F:)\iPod_Control\Music
Select all the music folders, and drag and drop them into a folder on your hard drive, or directly into iTunes.
And you're done! The iPod music folder structure is strange and inexplicable, but once you move your files into iTunes you can set it to automatically organize your folder by artist and album to clean that up. (To do this, in iTunes Edit menu, choose Preferences and in the Advanced tab, check "Keep iTunes Music Folder organized."
might be out of date worth a try -
Error in merge statement when trying to impliment SCD type 2 using merge...
Hi ,
I'm trying to impliment SCD type 2 using Merge using below blog as reference but sime how it is erroring out with error
http://www.made2mentor.com/2013/08/how-to-load-slowly-changing-dimensions-using-t-sql-merge/
Msg 207, Level 16, State 1, Line 40
Invalid column name 'Current'.
Msg 207, Level 16, State 1, Line 38
Invalid column name 'Current'.
Msg 207, Level 16, State 1, Line 47
Invalid column name 'Current'.
Here is the code below...
--Create Temporaty table to hold dimension records
IF OBJECT_ID('tempdb..#DimVirtualQueue') IS NOT NULL
DROP TABLE #DimVirtualQueue;
CREATE TABLE #DimVirtualQueue
( [VQ_name] [varchar](50) NULL,
[contact_type] [varchar](50) NULL,
[center_node_id] [int] NULL,
[sed_id] [datetime] NULL,
[eed_id] [datetime] NULL,
[insert_date] [datetime] NULL,
[Current] [char](1) NOT NULL
INSERT INTO #DimVirtualQueue(VQ_name, contact_type, center_node_id, sed_id, eed_id, insert_date,[Current] )
SELECT VQ_name, contact_type, center_node_id, sed_id , eed_id,GETDATE(),'Y'
FROM
( --Declare Source and Target tables.
MERGE dbo.tblSwDM_dim_VQ_test AS TARGET
--Source
USING (SELECT
RTRIM(LTRIM(Stage.RESOURCE_NAME)) AS VQ_name,
'Unknown' AS contact_type,
0 AS center_node_id,
CONVERT(INT,CONVERT(VARCHAR(8),GMT_START_TIME,112)) AS sed_id,
CONVERT(INT,CONVERT(VARCHAR(8),ISNULL(GMT_END_TIME,'2070-01-01'),112)) AS eed_id,
GETDATE() AS insert_date
FROM dbo.tblGenesys_stg_RESOURCE_ Stage
WHERE resource_type = 'queue'
AND resource_subtype = 'VirtualQueue'
AND NOT EXISTS (SELECT 1 FROM dbo.tblSwDM_dim_VQ AS dim
WHERE RTRIM(LTRIM(stage.RESOURCE_NAME)) = RTRIM(LTRIM(dim.vq_name))) ) SOURCE
ON TARGET.VQ_name = SOURCE.VQ_name
WHEN NOT MATCHED BY TARGET
THEN
INSERT ( VQ_name, contact_type, center_node_id, sed_id, eed_id, insert_date,[Current] )
VALUES (SOURCE.VQ_name,SOURCE.contact_type,SOURCE.center_node_id,SOURCE.sed_id,SOURCE.eed_id,SOURCE.insert_date,'Y')
WHEN MATCHED AND TARGET.[Current] = 'Y'
AND EXISTS (
SELECT SOURCE.VQ_name
EXCEPT
SELECT TARGET.VQ_name
--Expire the records in target if exist in source.
THEN UPDATE SET TARGET.[Current] = 'N',
TARGET.[eed_id] = SOURCE.eed_id
OUTPUT $Action ActionOut, SOURCE.VQ_name,SOURCE.contact_type,SOURCE.center_node_id,SOURCE.sed_id,SOURCE.eed_id) AS MergeOut
WHERE MergeOut.ActionOut = 'UPDATE';
--Insert data into dimension
INSERT tblSwDM_dim_VQ_test
SELECT VQ_name,contact_type,center_node_id,sed_id,eed_id,insert_date,[Current] FROM #DimVirtualQueue
Any help to resolve issue is appreciated...
Thanks,
Vishal..You need to show the DDL of your target table: dbo.tblSwDM_dim_VQ_test.
Do you have a column named [Current] in this table? -
i have updated my ipad yestarday sicne then i am not able to open any application on my ipad2. some of the application able to open in the page and not able to open all the applications in the second page. Does any one got this type of issue? any suggestions?
This problem has been reported a few times. The solution is to install any new App say a free one. This seems to fix whatever has gone wrong.
-
Does anybody know why type might render strangely when viewing a PDF in Chrome, Firefox and IE?
I have a client who I create a report for every year. This year there seems to be a problem with the PDFs that I have exported when they are viewed in various web browsers. To elaborate, the type is rendering inconsistently – in Chrome the weight of the type is inconsistent (ranging from light to something more heavy) and in IE the type appears more condensed, and the x-height is variable.
To my knowledge Chrome, Firefox and IE are all affected in one way or another. The problem does not occur in Safari.
We always use Helvetica Neue and have never had problems like this before. I am using the same version of Helvetica Neue as I always use, but even when I change the font to Helvetica the problem still persists.
On a deadline so would appreciate some support here. Box of beer to the person who can provide me with a working solution!
Thanks kindly
Simon WinterHello Ellis. Thanks for your efforts in helping narrow down this issue, and for offering workarounds for Simon. Here are the steps I sent to Simon, enhanced by the info you provided above:
1. I typed some text in Id CC 2014.0 (Mac OS 10.9.4) using the system font Helvetica Neue Light at 12 pt. All defaults were on, including ligatures.
1.a. I used the text in Simon's example, “An Annual Report marks the end of a year’s work, but it is also an opportunity to reflect on the principles that” and reduced the text frame width so that it wrapped across 4 lines.
1.b. I also tried creating an all-page text frame and filling it with placeholder text (same font, same point size).
2. Export > Interactive PDF (or Static PDF: doesn’t make a difference), using all the default PDF export settings.
3. View resulting PDF in the latest Chrome (36.0.1985.143) on that same Mac, or on Windows, at various zoom levels, on a retina display as well as on a lower-dpi monitor. It looks fine to me: all words and lines have the same even "color" or weight.
In all cases, the fonts embedded in the PDF were listed as:
HelveticaNeue-Light (Embedded Subset)
Type: TrueType
Encoding: Ansi
HelveticaNeue-Light (Embedded Subset)
Type: TrueType (CID)
Encoding: Identity-H
If you follow the above steps, can you repro the problem? -
User domain\SPFservice is not authorized to perform request using authentication type Negotiate
Hi,
I have installed WAP/SPF in the same domain via express installation. The SPF domain service account is sysadmin on the SPF database. The SPF domain service account is running as identity in IIS application pool. I have registered SPF in
WAP via SPFcomputeraccount\LocalSPFaccount. The LocalSPFaccount is member of the 4 local groups created by the SPF setup. The domain SPF service account is member of the VMM administrators.
When a new tenant want to subscribe to a hosting plan I get an error "One or more errors occurred while contacting the underlying resource providers. The operation may be partially completed. Details: Failed to create subscription".
When I look in the eventviewer of the SPF server in ManagementODataService, I can see "User domain\SPFservice is not authorized to perform request using authentication type Negotiate".
SPF/VMM are both on the latest update rollup. The VMM console is also updated on the SPF server.
I can successfully reproduce the troubleshooting steps from http://blogs.technet.com/b/privatecloud/archive/2013/11/08/troubleshooting-windows-azure-pack-spf-amp-vmm.aspx.Hi,
During the install it is also asking you to specify groups during the installation (4 x) Is the user you specified als spf runas account also member of those group in the AD?
So you have 4 groups created on the local box by the installation. But also 4 specified during the installation. Check if the account is member of those group(s) as well, reboot the spf and you should be up and running.
Best regards, Mark Scholman. Please remember to click “Mark as Answer” on the post that helps you, and to click “Unmark as Answer” if a marked post does not actually answer your question. This can be beneficial to other community members reading the thread. -
Hi,
I'm relatively new to OBIEE and trying to implement slowly changing dimension type 2.-- i.e. to look up the correct record in the customer dimension (A_FICC_ACCOUNT) based on the transaction date in the fact table (A_FICC_PROFITABILITY). The customer dimension has two time stamps (start_effective_date, end_effective_date), where end_effective_date will be set to NULL for the most recent record.
So far I've set this up as a complex join in the physical layer, as follows:
A_FICC_ACCOUNT.ACCOUNT_ID = A_FICC_PROFITABILITY.ACCOUNT_ID
AND
A_FICC_PROFITABILITY.DAYID >= A_FICC_ACCOUNT.START_EFFECTIVE_DATE AND A_FICC_PROFITABILITY.DAYID < A_FICC_ACCOUNT.END_EFFECTIVE_DATE
OR
A_FICC_PROFITABILITY.DAYID >= A_FICC_ACCOUNT.START_EFFECTIVE_DATE AND A_FICC_ACCOUNT.END_EFFECTIVE_DATE IS NULL
When i run a report in Answers, the generated SQL is as follows:
select distinct T2327.YEARCAPTION as c1,
T13994.SHORT_NAME as c2,
T13994.RANK_W12 as c3,
T13994.RANK_W52 as c4,
T2327.YEARID as c5
from
DIM_FICC_ACCOUNT T13994 /* A_FICC_ACCOUNT */ ,
DIM_TIME T2327 /* A_FICC_TIME */ ,
FACT_FICC_PROFITABILITY T13406 /* A_FICC_PROFITABILITY */
where ( T2327.SKEY = T13406.TIME_ID and T13406.ACCOUNT_ID = T13994.ACCOUNT_ID and T13994.SHORT_NAME = 'PETEROLA' and (T13406.DAYID < T13994.END_EFFECTIVE_DATE or T13994.END_EFFECTIVE_DATE is null) and T13406.DAYID >= T13994.START_EFFECTIVE_DATE )
order by c5, c2, c3, c4
Ran directly against the (Oracle) database using an SQL client, it gives meaningful results according to the recorded historical changes in the dimension. (see attached screenshot). In Answers, however, the results are not correct (see attached screenshot).
http://dl.dropbox.com/u/3345113/output.jpg
Any tips as to what might be wrong would be greatly appreciated.
best regards
MagnusMagnus,
As mentioned customer dimension (A_FICC_ACCOUNT) is a SCD type 2, what is the primary key on this table? I don't think it is ACCOUNT_ID alone.
Usually there exists a surrogate key to keep track of any changes, thus it is on this surrogate key a join should exist with a fact table and applying a filter end_effective date is null should produce the correct result.
If the above scenario is not true in your case, do let me know the structure of the customer dimension and fact table (only columns associated with customer dimension), with relation ship information please.
J
-bifacts
http://www.obinotes.com -
Hello,
I am trying out the SCD type 2 in OWB 10g R2. I have no hierarchies and hence created a dimension with one level. Apart from three business attributes I have 2 mandatory attributes as "EFFECTIVE_DATE" and "EXPIRY_DATE". I have set one of the business attributes to "Trigger History".
I have 3000 rows in the source table. There is no transformation. the data is loaded from source to this dimension directly. One business attribute is loaded using a constant.
When I executed the mapping, all 3000 rows are populated in the target with the expiry date column having null values and current date in the effective date, which is absolutely fine. When I execute the mapping again without changing anything in the source, rows are inserted in to the target with the effective date set to the second run and the expiry date set to previous effective date. As I understand new rows should be inserted only if there is change in the data.
Please correct me if I am wrong. Please clarify and if my understanding is right where am I doing wrong with OWB?
Your help is greatly appreciated
Thanks a lot on advance!
Regards,
MaruthiHi Roelant,
I think it is important to be aware that although Paris - 10gR2 - is not actually buggy (in this respect!), it is really quite idiosyncratic in exactly how it processes SCDs.
I followed up on your and Mark's comments, and did an in depth analysis of this topic. It is at http://www.donnapkelly.pwp.blueyonder.co.uk/documents/OWB_10gR2_SCD.pdf
My conclusions are perhaps of interest to anyone considering doing SCD processing with Paris.
I'll be doing a follow-up this weekend, and publishing a sort of 'how-to-do-it' guide.
Cheers,
Donna
Message was edited to add the words: "in this respect"
Maybe you are looking for
-
Why won't iTunes open when I plug in my iPod?
I plugged in my iPod, and if I already have iTunes open, I need to close it because it freezes. However, when I try to re-open iTunes, it will not open. I recently updated iTunes to 7.2 and I also updated my iPod software to the most current one. Wha
-
HP Deskjet 3056A Printing issue
I have recently uninstalledf and reinstalled my all in one printer due to scanning and printing issues. I have fixed the scanning issue BUT the printing side of things STILL NOTHING. I have cleaned, aligned, ect... I HAVE INK lol but NOTHING is prin
-
Automatically set "volumes for work"
For every project I want to keep all relevant files together. Thus for each project I want to set the path for "working volumes" ( Final Cut Pro --> system settings --> volumes for work) and waveform cache, pictogramm cache and auto save automaticall
-
Indesign CC: Alternate Layout
Does anyone know the correct workflow to add a new page to an alternate layout? I have my vertical document p. 1-50 and created an alternate horizontal layout from it. Currently all my text links across so when I update my vertical, the horizontal wa
-
The sound bar on my music app has gone missing and there is no sound coming from the speakers. The ringtone is still working so this has definitely got to be a glitch. Are there any solutions?