Using INSERT INTO for FDM Memory Issue

Question

All -
We configured FDM to run an integration script into our EBS instance upon the import step. It worked fine for a time, then we began running out of memory on the FDM server. Upon Oracle's suggestion, we stopped running through a record set to append and update the values to strWorkTableName and instead began using INSERT INTO. I admit, I have never needed to use the INSERT INTO way of doing things so I may be missing something bery basic.
Our integration script:
Function SQLIntegration2(strLoc, lngCatKey, dblPerKey, strWorkTableName)
'Oracle Hyperion FDM IMPORT Integration Script:
'Created By:       karks
'Date Created:       2011-10-26 13:05:18
'Purpose:
Dim strSQL     'SQL string
Dim lngPartitionKey
Dim strConn 'Connection string to the source data
Set cnSS = CreateObject("ADODB.Connection")
lngPartitionKey = RES.PlngLocKey
strConn= "File Name=C:\Users\karksadm\Desktop\NewConnection.udl;"
cnSS.open strConn
strSQL = "Insert Into " & strWorkTableName & " (PartitionKey, CatKey, PeriodKey, DataView, Amount , Account, Entity, ICP, UD1, UD2, UD3, UD4) "
strSQL = strSQL & "SELECT " & lngPartitionKey & ", " & lngCatKey & ", TO_DATE (TO_DATE ('30/12/1899','dd/mm/yyyy')+" & dblPerKey & "), 'YTD', EBS.YTD_BALANCE, EBS.ACCOUNT, EBS.SEGMENT1, EBS.SEGMENT5, EBS.SEGMENT4, EBS.SEGMENT5, '[None]', EBS.CURRENCY_CODE FROM "
strSQL = strSQL & "(Select D.NAME, A.CODE_COMBINATION_ID, D.NAME LEDGER_NAME, A.ACTUAL_FLAG, C.PERIOD_YEAR, TO_CHAR (C.START_DATE, 'MON-YY') AS PERIOD, B.SEGMENT1, (B.SEGMENT2 || B.SEGMENT3) As ACCOUNT, B.SEGMENT4, B.SEGMENT5, A.CURRENCY_CODE, "
strSQL = strSQL & "(SUM (A.BEGIN_BALANCE_DR) + SUM (A.PERIOD_NET_DR) - SUM (A.BEGIN_BALANCE_CR) - SUM (A.PERIOD_NET_CR)) As YTD_BALANCE "
strSQL = strSQL & "FROM GL.GL_BALANCES A, GL.GL_CODE_COMBINATIONS B, GL.GL_PERIODS C, GL.GL_LEDGERS D "
strSQL = strSQL & "WHERE 1 = 1 And A.LEDGER_ID = D.LEDGER_ID And A.PERIOD_NUM = C.PERIOD_NUM And C.PERIOD_YEAR = A.PERIOD_YEAR "
strSQL = strSQL & "And A.CODE_COMBINATION_ID = B.CODE_COMBINATION_ID And B.SUMMARY_FLAG = 'N' AND C.PERIOD_SET_NAME = D.PERIOD_SET_NAME "
strSQL = strSQL & "And (B.SEGMENT1 = 001 And A.CURRENCY_CODE = 'USD' And D.LEDGER_ID = 2022) "
strSQL = strSQL & "And C.END_DATE = TO_DATE (TO_DATE ('30/12/1899','dd/mm/yyyy')+" & dblPerKey & ") "
strSQL = strSQL & "And B.CHART_OF_ACCOUNTS_ID = D.CHART_OF_ACCOUNTS_ID And A.ACTUAL_FLAG = 'A' "
strSQL = strSQL & "And ((A.BEGIN_BALANCE_DR) + (A.PERIOD_NET_DR) - ((A.BEGIN_BALANCE_CR) + (A.PERIOD_NET_CR))) <> 0 "
strSQL = strSQL & "GROUP BY A.CODE_COMBINATION_ID, D.NAME, A.CURRENCY_CODE, TO_CHAR (C.START_DATE,'MON-YY'), C.PERIOD_YEAR, A.ACTUAL_FLAG, B.SEGMENT1, (B.SEGMENT2 || B.SEGMENT3), B.SEGMENT4, B.SEGMENT5 "
strSQL = strSQL & "ORDER BY B.SEGMENT4) EBS"
DW.DataManipulation.fExecuteDML(strSQL)
'Give success message
RES.PlngActionType = 2
RES.PstrActionValue = "SQL Import successful!"
'Assign Return value
SQLIntegration2 = True
cnSS.close
Set cnSS = Nothing
End FunctionI can run the SQL minus the Insert line in my SQL Developer and it works fine. When I run the script in its entirety in FDM, we receive the following:
** Begin FDM Runtime Error Log Entry [2012-01-23 11:37:30] **
ERROR:
Code............................................. -2147217865
Description...................................... ORA-00942: table or view does not exist
Insert Into tWibison72564424799 (PartitionKey, CatKey, PeriodKey, DataView, Amount , Account, Entity, ICP, UD1, UD2, UD3, UD4) SELECT 772, 28, TO_DATE (TO_DATE (N'30/12/1899',N'dd/mm/yyyy')+40724), N'YTD', EBS.YTD_BALANCE, EBS.ACCOUNT, EBS.SEGMENT1, EBS.SEGMENT5, EBS.SEGMENT4, EBS.SEGMENT5, N'[None]', EBS.CURRENCY_CODE FROM (Select D.NAME, A.CODE_COMBINATION_ID, D.NAME LEDGER_NAME, A.ACTUAL_FLAG, C.PERIOD_YEAR, TO_CHAR (C.START_DATE, N'MON-YY') AS PERIOD, B.SEGMENT1, (B.SEGMENT2 || B.SEGMENT3) As ACCOUNT, B.SEGMENT4, B.SEGMENT5, A.CURRENCY_CODE, (SUM (A.BEGIN_BALANCE_DR) + SUM (A.PERIOD_NET_DR) - SUM (A.BEGIN_BALANCE_CR) - SUM (A.PERIOD_NET_CR)) As YTD_BALANCE FROM GL.GL_BALANCES A, GL.GL_CODE_COMBINATIONS B, GL.GL_PERIODS C, GL.GL_LEDGERS D WHERE 1 = 1 And A.LEDGER_ID = D.LEDGER_ID And A.PERIOD_NUM = C.PERIOD_NUM And C.PERIOD_YEAR = A.PERIOD_YEAR And A.CODE_COMBINATION_ID = B.CODE_COMBINATION_ID And B.SUMMARY_FLAG = N'N' AND C.PERIOD_SET_NAME = D.PERIOD_SET_NAME And (B.SEGMENT1 = 001 And A.CURRENCY_CODE = N'USD' And D.LEDGER_ID = 2022) And C.END_DATE = TO_DATE (TO_DATE (N'30/12/1899',N'dd/mm/yyyy')+40724) And B.CHART_OF_ACCOUNTS_ID = D.CHART_OF_ACCOUNTS_ID And A.ACTUAL_FLAG = N'A' And ((A.BEGIN_BALANCE_DR) + (A.PERIOD_NET_DR) - ((A.BEGIN_BALANCE_CR) + (A.PERIOD_NET_CR))) <> 0 GROUP BY A.CODE_COMBINATION_ID, D.NAME, A.CURRENCY_CODE, TO_CHAR (C.START_DATE,N'MON-YY'), C.PERIOD_YEAR, A.ACTUAL_FLAG, B.SEGMENT1, (B.SEGMENT2 || B.SEGMENT3), B.SEGMENT4, B.SEGMENT5 ORDER BY B.SEGMENT4) EBS
Procedure........................................ clsDataManipulation.fExecuteDML
Component........................................ upsWDataWindowDM
Version.......................................... 1112
Thread........................................... 4424
IDENTIFICATION:
User............................................. ibisons
Computer Name.................................... HOU-HYSDEV02
App Name......................................... SWNFDMRC
Client App....................................... WebClient
CONNECTION:
Provider......................................... ORAOLEDB.ORACLE
Data Server......................................
Database Name.................................... FDMDEV
Trusted Connect.................................. False
Connect Status.. Connection Open
GLOBALS:
Location......................................... EBSINTEGRATION
Location ID...................................... 772
Location Seg..................................... 25
Category......................................... EBS2
Category ID...................................... 28
Period........................................... Jun - 2011
Period ID........................................ 6/30/2011
POV Local........................................ False
Language......................................... 1033
User Level....................................... 1
All Partitions................................... True
Is Auditor....................................... False
** Begin FDM Runtime Error Log Entry [2012-01-23 11:37:31] **
ERROR:
Code............................................. -2147217865
Description...................................... Data access error.
At line: 33
Procedure........................................ clsImpProcessMgr.fExecuteImpScript
Component........................................ upsWObjectsDM
Version.......................................... 1112
Thread........................................... 4424
IDENTIFICATION:
User............................................. ibisons
Computer Name.................................... HOU-HYSDEV02
App Name......................................... SWNFDMRC
Client App....................................... WebClient
CONNECTION:
Provider......................................... ORAOLEDB.ORACLE
Data Server......................................
Database Name.................................... FDMDEV
Trusted Connect.................................. False
Connect Status.. Connection Open
GLOBALS:
Location......................................... EBSINTEGRATION
Location ID...................................... 772
Location Seg..................................... 25
Category......................................... EBS2
Category ID...................................... 28
Period........................................... Jun - 2011
Period ID........................................ 6/30/2011
POV Local........................................ False
Language......................................... 1033
User Level....................................... 1
All Partitions................................... True
Is Auditor....................................... False
** Begin FDM Runtime Error Log Entry [2012-01-23 11:37:31] **
ERROR:
Code............................................. -2147217865
Description...................................... Data access error.
At line: 33
Procedure........................................ clsImpProcessMgr.fLoadAndProcessFile
Component........................................ upsWObjectsDM
Version.......................................... 1112
Thread........................................... 4424
IDENTIFICATION:
User............................................. ibisons
Computer Name.................................... HOU-HYSDEV02
App Name......................................... SWNFDMRC
Client App....................................... WebClient
CONNECTION:
Provider......................................... ORAOLEDB.ORACLE
Data Server......................................
Database Name.................................... FDMDEV
Trusted Connect.................................. False
Connect Status.. Connection Open
GLOBALS:
Location......................................... EBSINTEGRATION
Location ID...................................... 772
Location Seg..................................... 25
Category......................................... EBS2
Category ID...................................... 28
Period........................................... Jun - 2011
Period ID........................................ 6/30/2011
POV Local........................................ False
Language......................................... 1033
User Level....................................... 1
All Partitions................................... True
Is Auditor....................................... False
I really think the issue is that I am not telling the script where to find the Oracle tables at. I left the code in where we were using the UDL to call in the opening of the record set, but since we are doing away with the record set how do I let FDM know where to find the Oracle tables?
Thanks in advance - I've been wrestling with this for several days.
Thanks you,
Sarah

Answer 1

Here's a script with INSERT INTO that works fine if FDM back-end is SQL DB and it's pulling from FDMHarris data warehouse which is on SQL Server.
Problem is when FDM back-end is Oracle DB and it's pulling from FDMHarris data warehouse which is on SQL Server.
I'm assuming the INSERT INTO statement needs to be written differently, syntax wise?
Here's the error message displayed.
** Begin FDM Runtime Error Log Entry [2012-06-03 21:18:15] **
ERROR:
Code............................................. -2147217900
Description...................................... ORA-00933: SQL command not properly ended
INSERT INTO tWadmin476032843931 (PartitionKey, CatKey, PeriodKey, DataView, CalcAcctType, Entity, Account, UD1, UD2, UD4, Amount) SELECT 752, 12, N'30-Apr-2012', N'YTD', 9, Entity, Account, UD1, UD2, UD4, Amount FROM FDMHarris.dbo.tdataseg4;
Procedure........................................ clsDataManipulation.fExecuteDML
Component........................................ upsWDataWindowDM
Version.......................................... 1112
Thread........................................... 3284
Function INSERTINTO(strLoc, lngCatKey, dblPerKey, strWorkTableName)
'Oracle Hyperion FDM IMPORT Integration Script:
'Created By:      admin
'Date Created:      2012-06-03 11:31:39
'Purpose:
Dim objSS 'ADODB.Connection
Dim strSQL 'SQL String
Dim rs 'Recordset
Dim rsAppend 'tTB table append rs Object
Dim strPeriod
Dim strYear
'Period
'strPeriod=MonthName(Month(RES.PdtePerKey))
a=CStr(FormatDateTime(RES.PdtePerKey,1))
'Tuesday,January 30, 2012
b=Right(a,(Len(a)-Len(DW.Utilities.fParseString(a,1,1,",")))) '7
c=DW.Utilities.fParseString(b,2,2,",")
strPeriod=Left(c,3)
'Year
'strYear=Year(RES.PdtePerKey)
strYear=Right(b,4)
DW.DBTools.mLogError 1, CStr(strPeriod), CStr(strYear), Nothing
'Initialize objects
Set cnSS = CreateObject("ADODB.Connection")
'Connect To SQL Server database
cnss.open "Provider=SQLOLEDB.1;Password=datafusion;Persist Security Info=True;User ID=sa;Initial Catalog=FDMHarris;Data Source=dfv11122"
'DW.DBTools.mLogError 1, CStr(strSQL), CStr(strSQL), Nothing
'Initialize common SQL statement
strSQL = "INSERT INTO " & _
strWorkTableName & " " & _
"(PartitionKey, CatKey, PeriodKey, DataView, CalcAcctType, Entity, Account, UD1, UD2, UD4, Amount) " & _
"SELECT " & RES.PlngLocKey & ", " & RES.PlngCatKey & ", " & _
"'" & Day(RES.PdtePerKey) & "-" & MonthName(Month(RES.PdtePerKey), True) & "-" & Year(RES.PdtePerKey) & "', " & _
"'YTD', 9, Entity, Account, UD1, UD2, UD4, Amount " & _
"FROM FDMHarris.dbo.tdataseg4;" '& _
' "WHERE Month = '" & strPeriod & "' And CalYear = '" & strYear & "'"
DW.DBTools.mLogError 1, CStr(strSQL), CStr(strWorkTableName), Nothing
DW.DataManipulation.fExecuteDML(strSQL)
'cnss.Execute strSQL
'Records loaded
RES.PlngActionType = 6
RES.PstrActionValue = "SQL Import successful!"
'Assign Return value
INSERTINTO = True
cnss.Close
Set cnss = Nothing
End Function
Edited by: user12152138 on Jun 3, 2012 6:43 PM