ETL Failure Due to Failure of a Task

Question

Hi All,
I am getting ETL failure due to failure of a task, sporadically. Also, when i re-run the ETL, the ETL completed successfully.
I have copied the informatica session log file.
DIRECTOR> VAR_27027 Use default value [0] for user-defined workflow/worklet variable:[$$passInStatus].
DIRECTOR> VAR_27028 Use override value [DataWarehouse] for session parameter:[$DBConnection_OLAP].
DIRECTOR> VAR_27028 Use override value [SEBL_80] for session parameter:[$DBConnection_OLTP].
DIRECTOR> VAR_27028 Use override value [1] for mapping parameter:[$$DATASOURCE_NUM_ID].
DIRECTOR> VAR_27028 Use override value [] for mapping parameter:[$$Hint1].
DIRECTOR> VAR_27028 Use override value [] for mapping parameter:[$$Hint2].
DIRECTOR> TM_6014 Initializing session [VisibilityActivityParty_Full] at [Wed Aug 17 00:31:31 2011]
DIRECTOR> TM_6683 Repository Name: [Oracle_BI_DW_Base]
DIRECTOR> TM_6684 Server Name: [Oracle_BI_DW_Base_Integration_Service]
DIRECTOR> TM_6686 Folder: [SDE_SBL_80_Adaptor]
DIRECTOR> TM_6685 Workflow: [VisibilityActivityParty_Full]
DIRECTOR> TM_6101 Mapping name: VisibilityActivityParty [version 1]
DIRECTOR> TM_6827 [D:\Informatica\PowerCenter8.1.1\server\infa_shared\Storage] will be used as storage directory for session [VisibilityActivityParty_Full].
DIRECTOR> CMN_1802 Session recovery cache initialization is complete.
DIRECTOR> TM_6703 Session [VisibilityActivityParty_Full] is run by 32-bit Integration Service [node01_sb8anaprd], version [8.1.1 SP5], build [0822].
MANAGER> PETL_24058 Running Partition Group [1].
MANAGER> PETL_24000 Parallel Pipeline Engine initializing.
MANAGER> PETL_24001 Parallel Pipeline Engine running.
MANAGER> PETL_24003 Initializing session run.
MAPPING> CMN_1569 Server Mode: [ASCII]
MAPPING> CMN_1570 Server Code page: [MS Windows Latin 1 (ANSI), superset of Latin1]
MAPPING> TM_6151 Session Sort Order: [Binary]
MAPPING> TM_6156 Using LOW precision decimal arithmetic
MAPPING> TM_6180 Deadlock retry logic will not be implemented.
MAPPING> TM_6307 DTM Error Log Disabled.
MAPPING> TE_7022 TShmWriter: Initialized
MAPPING> DBG_21075 Connecting to database [obieeprd], user [OLAP]
MAPPING> DBG_21474 Warning: Update Strategy transform [UPDTRANS] is disabled. Set session type to [Data Driven] to enable it.
MAPPING> TM_6007 DTM initialized successfully for session [VisibilityActivityParty_Full]
DIRECTOR> PETL_24033 All DTM Connection Info: [<NONE>].
MANAGER> PETL_24004 Starting pre-session tasks. : (Wed Aug 17 00:31:31 2011)
MANAGER> PETL_24027 Pre-session task completed successfully. : (Wed Aug 17 00:31:31 2011)
DIRECTOR> PETL_24006 Starting data movement.
MAPPING> TM_6660 Total Buffer Pool size is 36000000 bytes and Block size is 65536 bytes.
READER_1_1_1> DBG_21438 Reader: Source is [SB8PRD], user [SIEBEL]
READER_1_1_1> BLKR_16003 Initialization completed successfully.
WRITER_1_*_1> WRT_8146 Writer: Target is database [obieeprd], user [OLAP], bulk mode [ON]
WRITER_1_*_1> WRT_8106 Warning! Bulk Mode session - recovery is not guaranteed.
WRITER_1_*_1> WRT_8397 Appending output data to file [D:\Informatica\PowerCenter8.1.1\server\infa_shared\BadFiles\w_act_party.bad] (initial size [717]).
WRITER_1_*_1> WRT_8124 Target Table W_ACT_PARTY :SQL INSERT statement:
INSERT INTO W_ACT_PARTY(INTEGRATION_ID,DATASOURCE_NUM_ID,ETL_PROC_WID,TABLE_WID,ACTIVITY_ID,PARTY_ID) VALUES ( ?, ?, ?, ?, ?, ?)
WRITER_1_*_1> WRT_8124 Target Table W_ACT_PARTY :SQL UPDATE statement:
UPDATE W_ACT_PARTY SET ETL_PROC_WID = ?, ACTIVITY_ID = ?, PARTY_ID = ? WHERE INTEGRATION_ID = ? AND DATASOURCE_NUM_ID = ? AND TABLE_WID = ?
WRITER_1_*_1> WRT_8124 Target Table W_ACT_PARTY :SQL DELETE statement:
DELETE FROM W_ACT_PARTY WHERE INTEGRATION_ID = ? AND DATASOURCE_NUM_ID = ? AND TABLE_WID = ?
WRITER_1_*_1> WRT_8270 Target connection group #1 consists of target(s) [W_ACT_PARTY]
WRITER_1_*_1> WRT_8003 Writer initialization complete.
READER_1_1_1> BLKR_16007 Reader run started.
READER_1_1_1> RR_4029 SQ Instance [SQ_S_ACT_EMP] User specified SQL Query [SELECT
     AEMP.ROW_ID,
     AEMP.ACTIVITY_ID,
     U.PAR_ROW_ID
FROM
     V_ACT_EMP AEMP,
     S_USER U
WHERE
     AEMP.EMP_ID = U.ROW_ID]
READER_1_1_1> RR_4049 SQL Query issued to database : (Wed Aug 17 00:31:32 2011)
WRITER_1_*_1> WRT_8005 Writer run started.
WRITER_1_*_1> WRT_8158
*****START LOAD SESSION*****
Load Start Time: Wed Aug 17 00:31:32 2011
Target tables:
W_ACT_PARTY
READER_1_1_1> RR_4050 First row returned from database to reader : (Wed Aug 17 00:31:32 2011)
LKPDP_1> DBG_21097 Lookup Transformation [LKP_W_ACT_PARTY]: Default sql to create lookup cache: SELECT ETL_PROC_WID,INTEGRATION_ID,TABLE_WID,DATASOURCE_NUM_ID FROM W_ACT_PARTY ORDER BY INTEGRATION_ID,TABLE_WID,DATASOURCE_NUM_ID,ETL_PROC_WID
LKPDP_1> TE_7212 Increasing [Index Cache] size for transformation [LKP_W_ACT_PARTY] from [1048576] to [1050000].
LKPDP_1> TM_6660 Total Buffer Pool size is 609824 bytes and Block size is 65536 bytes.
LKPDP_1:READER_1_1> DBG_21438 Reader: Source is [obieeprd], user [OLAP]
LKPDP_1:READER_1_1> BLKR_16003 Initialization completed successfully.
LKPDP_1:READER_1_1> BLKR_16007 Reader run started.
LKPDP_1:READER_1_1> RR_4049 SQL Query issued to database : (Wed Aug 17 00:31:32 2011)
LKPDP_1:READER_1_1> BLKR_16019 Read [0] rows, read [0] error rows for source table [LKP_W_ACT_PARTY{{DSQ}}] instance name [LKP_W_ACT_PARTY{{SRC}}]
LKPDP_1:READER_1_1> BLKR_16008 Reader run completed.
LKPDP_1:TRANSF_1_1> DBG_21682 Lookup table row count : 0
LKPDP_1:TRANSF_1_1> DBG_21297 Lookup cache row count : 0
LKPDP_1:TRANSF_1_1> DBG_21294 Lookup cache creation completed : (Wed Aug 17 00:31:32 2011)
LKPDP_1:TRANSF_1_1> CMN_1671 Created new cache files PMLKUP12190_9W32.[dat/idx] in directory D:\Informatica\PowerCenter8.1.1\server\infa_shared\Cache for Lookup [LKP_W_ACT_PARTY].
LKPDP_1:TRANSF_1_1> DBG_21641 LKP_W_ACT_PARTY: Index cache size = [1050000], Data cache size = [2097152]
LKPDP_1:TRANSF_1_1> DBG_21216 Finished transformations for Source Qualifier [LKP_W_ACT_PARTY{{DSQ}}]. Total errors [0]
TRANSF_1_1_1> CMN_1079 WARNING: Lookup table contains no data.
TRANSF_1_1_1> DBG_21524 Transform : LKP_W_ACT_PARTY
TRANSF_1_1_1> DBG_21313 Lookup table : W_ACT_PARTY
TRANSF_1_1_1> DBG_21562 WARNING : Output rows from LKP_W_ACT_PARTY will be the default port value
LKPDP_2> DBG_21097 Lookup Transformation [LKP_ETL_PROC_WID]: Default sql to create lookup cache: SELECT ETL_PROC_WID,ROW_WID FROM W_PARAM_G ORDER BY ROW_WID,ETL_PROC_WID
LKPDP_2> TE_7212 Increasing [Index Cache] size for transformation [LKP_ETL_PROC_WID] from [1048576] to [1050000].
LKPDP_2> TM_6660 Total Buffer Pool size is 609824 bytes and Block size is 65536 bytes.
LKPDP_2:READER_1_1> DBG_21438 Reader: Source is [obieeprd], user [OLAP]
LKPDP_2:READER_1_1> BLKR_16003 Initialization completed successfully.
LKPDP_2:READER_1_1> BLKR_16007 Reader run started.
LKPDP_2:READER_1_1> RR_4049 SQL Query issued to database : (Wed Aug 17 00:31:32 2011)
LKPDP_2:READER_1_1> BLKR_16019 Read [0] rows, read [0] error rows for source table [LKP_ETL_PROC_WID{{DSQ}}] instance name [LKP_ETL_PROC_WID{{SRC}}]
LKPDP_2:READER_1_1> BLKR_16008 Reader run completed.
LKPDP_2:TRANSF_1_1> DBG_21682 Lookup table row count : 0
LKPDP_2:TRANSF_1_1> DBG_21297 Lookup cache row count : 0
LKPDP_2:TRANSF_1_1> DBG_21294 Lookup cache creation completed : (Wed Aug 17 00:31:32 2011)
LKPDP_2:TRANSF_1_1> CMN_1671 Created new cache files PMLKUP12190_10W32.[dat/idx] in directory D:\Informatica\PowerCenter8.1.1\server\infa_shared\Cache for Lookup [LKP_ETL_PROC_WID].
LKPDP_2:TRANSF_1_1> DBG_21641 LKP_ETL_PROC_WID: Index cache size = [1050000], Data cache size = [2097152]
LKPDP_2:TRANSF_1_1> DBG_21216 Finished transformations for Source Qualifier [LKP_ETL_PROC_WID{{DSQ}}]. Total errors [0]
TRANSF_1_1_1> CMN_1079 WARNING: Lookup table contains no data.
TRANSF_1_1_1> DBG_21524 Transform : LKP_ETL_PROC_WID1
TRANSF_1_1_1> DBG_21313 Lookup table : W_PARAM_G
TRANSF_1_1_1> DBG_21562 WARNING : Output rows from LKP_ETL_PROC_WID1 will be the default port value
WRITER_1_*_1> WRT_8167 Start loading table [W_ACT_PARTY] at: Wed Aug 17 00:31:32 2011
WRITER_1_*_1> CMN_1761 Timestamp Event: [Wed Aug 17 00:31:32 2011]
WRITER_1_*_1> WRT_8229 Database errors occurred:
ORA-01400: cannot insert NULL into (ETL_PROC_WID)
Database driver error...
Function Name : Execute Multiple
SQL Stmt : INSERT INTO W_ACT_PARTY(INTEGRATION_ID,DATASOURCE_NUM_ID,ETL_PROC_WID,TABLE_WID,ACTIVITY_ID,PARTY_ID) VALUES ( ?, ?, ?, ?, ?, ?)
WRITER_1_*_1> CMN_1761 Timestamp Event: [Wed Aug 17 00:31:32 2011]
WRITER_1_*_1> WRT_8425 ERROR: Writer execution failed.
WRITER_1_*_1> CMN_1761 Timestamp Event: [Wed Aug 17 00:31:32 2011]
WRITER_1_*_1> WRT_8114
Row # [1] in bad file
WRITER_1_*_1> CMN_1053 : Rowdata: ( RowType=0(insert) Src Rowid=1 Targ Rowid=1
INTEGRATION_ID (INTEGRATION_ID:Char.30:): "1-B8THY"
DATASOURCE_NUM_ID (DATASOURCE_NUM_ID:Double:): "1.000000000000000"
ETL_PROC_WID (ETL_PROC_WID:Double:): "(NULL)"
TABLE_WID (TABLE_WID:Double:): "68.00000000000000"
ACTIVITY_ID (ACTIVITY_ID:Char.15:): "1-N8FBX"
PARTY_ID (PARTY_ID:Char.15:): "0-1"
WRITER_1_*_1> WRT_8164 Error loading into target [W_ACT_PARTY] : Bad rows exceeded Session Threshold [1]
WRITER_1_*_1> WRT_8333 Rolling back all the targets due to fatal session error.
WRITER_1_*_1> CMN_1761 Timestamp Event: [Wed Aug 17 00:31:32 2011]
WRITER_1_*_1> CMN_1022 Database driver error...
CMN_1022 [
ORA-24795: Illegal ROLLBACK attempt made
Database driver error...
Function Name : Commit
Database driver error...
Function Name : Rollback]
WRITER_1_*_1> CMN_1761 Timestamp Event: [Wed Aug 17 00:31:32 2011]
WRITER_1_*_1> WRT_8183 Rollback failed for the target [W_ACT_PARTY]
WRITER_1_*_1> CMN_1761 Timestamp Event: [Wed Aug 17 00:31:32 2011]
WRITER_1_*_1> WRT_8080 Writer run terminated. [Error loading data and error threshold reached]
WRITER_1_*_1> WRT_8168 End loading table [W_ACT_PARTY] at: Wed Aug 17 00:31:32 2011
WRITER_1_*_1> WRT_8035 Load complete time: Wed Aug 17 00:31:32 2011
LOAD SUMMARY
============
WRT_8036 Target: W_ACT_PARTY (Instance Name: [W_ACT_PARTY])
WRT_8038 Inserted rows - Requested: 488 Applied: 0 Rejected: 1 Affected: 0
WRITER_1__1> WRT_8043 ****END LOAD SESSION*****
TRANSF_1_1_1> CMN_1761 Timestamp Event: [Wed Aug 17 00:31:32 2011]
TRANSF_1_1_1> TM_6085 The transformation [W_ACT_PARTY] encountered fatal error. Terminating session.
TRANSF_1_1_1> CMN_1761 Timestamp Event: [Wed Aug 17 00:31:32 2011]
TRANSF_1_1_1> TM_6085 The transformation [UPDTRANS] encountered fatal error. Terminating session.
TRANSF_1_1_1> CMN_1761 Timestamp Event: [Wed Aug 17 00:31:32 2011]
TRANSF_1_1_1> TM_6085 The transformation [UPDTRANS] encountered fatal error. Terminating session.
TRANSF_1_1_1> CMN_1761 Timestamp Event: [Wed Aug 17 00:31:32 2011]
TRANSF_1_1_1> TM_6085 The transformation [FILTRANS] encountered fatal error. Terminating session.
TRANSF_1_1_1> CMN_1761 Timestamp Event: [Wed Aug 17 00:31:32 2011]
TRANSF_1_1_1> TM_6085 The transformation [FILTRANS] encountered fatal error. Terminating session.
TRANSF_1_1_1> CMN_1761 Timestamp Event: [Wed Aug 17 00:31:32 2011]
TRANSF_1_1_1> TM_6085 The transformation [LKP_ETL_PROC_WID] encountered fatal error. Terminating session.
TRANSF_1_1_1> CMN_1761 Timestamp Event: [Wed Aug 17 00:31:32 2011]
TRANSF_1_1_1> TM_6085 The transformation [LKP_ETL_PROC_WID] encountered fatal error. Terminating session.
TRANSF_1_1_1> CMN_1761 Timestamp Event: [Wed Aug 17 00:31:32 2011]
TRANSF_1_1_1> TM_6085 The transformation [EXPTRANS] encountered fatal error. Terminating session.
TRANSF_1_1_1> CMN_1761 Timestamp Event: [Wed Aug 17 00:31:32 2011]
TRANSF_1_1_1> TM_6085 The transformation [EXPTRANS] encountered fatal error. Terminating session.
TRANSF_1_1_1> CMN_1761 Timestamp Event: [Wed Aug 17 00:31:32 2011]
TRANSF_1_1_1> TM_6085 The transformation [SQ_S_ACT_EMP] encountered fatal error. Terminating session.
TRANSF_1_1_1> CMN_1761 Timestamp Event: [Wed Aug 17 00:31:32 2011]
TRANSF_1_1_1> TM_6085 The transformation [SQ_S_ACT_EMP] encountered fatal error. Terminating session.
TRANSF_1_1_1> CMN_1761 Timestamp Event: [Wed Aug 17 00:31:32 2011]
TRANSF_1_1_1> TM_6085 The transformation [SQ_S_ACT_EMP] encountered fatal error. Terminating session.
TRANSF_1_1_1> DBG_21511 TE: Fatal Transformation Error.
MANAGER> PETL_24031
***** RUN INFO FOR TGT LOAD ORDER GROUP [1], CONCURRENT SET [1] *****
Thread [READER_1_1_1] created for [the read stage] of partition point [SQ_S_ACT_EMP] has completed: Total Run Time = [0.656254] secs, Total Idle Time = [0.390627] secs, Busy Percentage = [40.476246].
Thread [TRANSF_1_1_1] created for [the transformation stage] of partition point [SQ_S_ACT_EMP] has completed: Total Run Time = [0.562504] secs, Total Idle Time = [0.406252] secs, Busy Percentage = [27.777928].
Thread [WRITER_1_*_1] created for [the write stage] of partition point [W_ACT_PARTY] has completed. The total run time was insufficient for any meaningful statistics.
MANAGER> PETL_24005 Starting post-session tasks. : (Wed Aug 17 00:31:33 2011)
MANAGER> PETL_24029 Post-session task completed successfully. : (Wed Aug 17 00:31:33 2011)
MAPPING> TE_7216 Deleting cache files [PMLKUP12190_9W32] for transformation [LKP_W_ACT_PARTY].
MAPPING> TE_7216 Deleting cache files [PMLKUP12190_10W32] for transformation [LKP_ETL_PROC_WID].
MAPPING> TM_6018 Session [VisibilityActivityParty_Full] run completed with [0] row transformation errors.
MANAGER> PETL_24002 Parallel Pipeline Engine finished.
DIRECTOR> PETL_24013 Session run completed with failure.
DIRECTOR> TM_6022
SESSION LOAD SUMMARY
================================================
DIRECTOR> TM_6252 Source Load Summary.
DIRECTOR> CMN_1740 Table: [SQ_S_ACT_EMP] (Instance Name: [SQ_S_ACT_EMP])
     Output Rows [50592], Affected Rows [50592], Applied Rows [50592], Rejected Rows [0]
DIRECTOR> TM_6253 Target Load Summary.
DIRECTOR> CMN_1740 Table: [W_ACT_PARTY] (Instance Name: [W_ACT_PARTY])
     Output Rows [0], Affected Rows [0], Applied Rows [0], Rejected Rows [1]
DIRECTOR> TM_6023
===================================================
DIRECTOR> TM_6020 Session [VisibilityActivityParty_Full] completed at [Wed Aug 17 00:31:33 2011]
Can some one help me to debug the issue please ?
Thanks

Answer 1

Hi Nat,
Thanks a lot for your response.
I am running this ETL task from DAC. This task is a part of Daily Full load ETL. As i mentioned earlier, this task fails sporadically and failing till now. After restarting the ETL, it gets completed.
Please let me know if you have any question.
Thanks,
Neeraj