Owb 9.0.4 with SQL Server source

Question

Hello,
I'm trying to use a SQL Server table as my source (approx. 2M rows) but I only need to pull the current period (approx. 5K rows). My performance is HORRIBLE across the transparent gateway/dblink but I'm not sure if this is the gateway itself or the code.
I've placed a filter in my mapping, but it would seem this is not effective on the initial recordset at the source, but it get's applied once the recordset has been brought over to my target warehouse. Is there a way to force this to occur at the source with some configuration setting changes?
The difference is seconds for a generic ODBC query to pull this recordset down vs. over an hour for my mapping to execute. (Even still, this seems excessive for 2 M rows to me). Anyone have any advice for me?
Regards,
Paul

Answer 1

My dblink used the tranparent gateway not the generic ODBC setup. On further digging, It looks like I need to use 'target only' based runtime 'modes' exclusively, which is not the default. These are the config settings, runtime parameters, where your choices are 'set based', 'row based', 'set based (target only)', 'set based failover to row based', and 'set based failover to row based(target only)'.
Based on the program flow, if I choose one of the target only methods, the package fetches rows from my source using a where clause (using Cursor FLTR_c), but otherwise the where clause is not generated (using cursor HC_tblActiveDaily_c) see cursor declarations below.
However, it seems that this doesn't buy me very much time... My dba told me the cursors declaration itself causes the rows to be populated from my source, which in my case means either way, I'm pulling every single record from the database, even though I'm only using the one with the where clause in it.
Is there a patch for this behavior? If I'm understanding the implications of this cursor declaration, it seems highly inefficent to me.
-- "HC_tblActiveDaily_c" Cursor declaration
CURSOR "HC_tblActiveDaily_c" IS
SELECT
/*+ NO_MERGE */
"HC_tblActiveDaily_JXNSQL01"."AcctgUnit" "AcctgUnit",
"HC_tblActiveDaily_JXNSQL01"."CreateDate" "CreateDate",
"HC_tblActiveDaily_JXNSQL01"."DateAdded" "DateAdded",
"HC_tblActiveDaily_JXNSQL01"."Department" "Department",
"HC_tblActiveDaily_JXNSQL01"."DeptName" "DeptName",
"HC_tblActiveDaily_JXNSQL01"."Division" "Division",
"HC_tblActiveDaily_JXNSQL01"."Employee" "Employee",
"HC_tblActiveDaily_JXNSQL01"."FicaNbr" "FicaNbr",
"HC_tblActiveDaily_JXNSQL01"."First_Name" "First_Name",
"HC_tblActiveDaily_JXNSQL01"."Hire_Date" "Hire_Date",
"HC_tblActiveDaily_JXNSQL01"."JobCodeDesc" "JobCodeDesc",
"HC_tblActiveDaily_JXNSQL01"."Job_Code" "Job_Code",
"HC_tblActiveDaily_JXNSQL01"."Last_Name" "Last_Name",
"HC_tblActiveDaily_JXNSQL01"."Process_Level" "Process_Level",
"HC_tblActiveDaily_JXNSQL01"."Status" "Status",
"HC_tblActiveDaily_JXNSQL01"."SubAccount" "SubAccount",
"HC_tblActiveDaily_JXNSQL01"."WorkState" "WorkState",
"HC_tblActiveDaily_JXNSQL01"."term_date" "term_date",
ROWNUM "ROWNUM_ALIAS$$$"
FROM "dbo"."HC_tblActiveDaily"@"INTERGRATION"@"JXNSQL01" "HC_tblActiveDaily_JXNSQL01" ;
-- "FLTR_c" Cursor declaration
CURSOR "FLTR_c" IS
SELECT
/*+ DRIVING_SITE("HC_tblActiveDaily_JXNSQL01") */
"HC_tblActiveDaily_JXNSQL01"."AcctgUnit" "ACCTGUNIT",
"HC_tblActiveDaily_JXNSQL01"."CreateDate" "CREATEDATE",
"HC_tblActiveDaily_JXNSQL01"."DateAdded" "DATEADDED",
"HC_tblActiveDaily_JXNSQL01"."Department" "DEPARTMENT",
"HC_tblActiveDaily_JXNSQL01"."DeptName" "DEPTNAME",
"HC_tblActiveDaily_JXNSQL01"."Division" "DIVISION",
"HC_tblActiveDaily_JXNSQL01"."Employee" "EMPLOYEE",
"HC_tblActiveDaily_JXNSQL01"."FicaNbr" "FICANBR",
"HC_tblActiveDaily_JXNSQL01"."First_Name" "FIRST_NAME",
"HC_tblActiveDaily_JXNSQL01"."Hire_Date" "HIRE_DATE",
"HC_tblActiveDaily_JXNSQL01"."JobCodeDesc" "JOBCODEDESC",
"HC_tblActiveDaily_JXNSQL01"."Job_Code" "JOB_CODE",
"HC_tblActiveDaily_JXNSQL01"."Last_Name" "LAST_NAME",
"HC_tblActiveDaily_JXNSQL01"."Process_Level" "PROCESS_LEVEL",
"HC_tblActiveDaily_JXNSQL01"."Status" "STATUS",
"HC_tblActiveDaily_JXNSQL01"."SubAccount" "SUBACCOUNT",
"HC_tblActiveDaily_JXNSQL01"."WorkState" "WORKSTATE",
"HC_tblActiveDaily_JXNSQL01"."term_date" "TERM_DATE",
ROWNUM "ROWNUM_ALIAS$$$"
FROM "dbo"."HC_tblActiveDaily"@"INTERGRATION"@"JXNSQL01" "HC_tblActiveDaily_JXNSQL01" WHERE( TRUNC ( "HC_tblActiveDaily_JXNSQL01"."DateAdded" ) = TRUNC ( (HC_CYC_ACTIVES_MAP."CONST_ACT_AS_OF_DATE") ) );