Sql statement hanging in prod. fast in dev.

Question

Hi,
Sql statement is hanging in production.
In development it is executing in 2 secs.
From explainplan , noticed that taking different indexes.
I have posted the staement and explain plan (prod and dev) below.
Statement:
SELECT
REP_V_SERVICE_REQUEST.SERVICE_REQ_ID,
REP_V_ACTIVITY.EXTERNAL_REF,
REP_V_ACTIVITY.VENUS_PROBLEM_START,
REP_V_ACTIVITY.VENUS_ALERT_ISSUED,
REP_V_ACTIVITY.VENUS_NOTIFIED,
REP_V_ACTIVITY.ACTIVITY_ID,
REP_V_ACTIVITY.CREATED_BY_WORKGROUP,
REP_V_ACTIVITY.ABSTRACT,
REP_V_ACTIVITY.TIME_TO_VENUS_ALERT,
REP_V_ACTIVITY.TIME_TO_VENUS_ISSUE,
REP_V_ACTIVITY.TIME_TO_VENUS_NOTIFIED,
REP_V_SERVICE_REQUEST.TYPE,
REP_V_SERVICE_REQUEST.SUB_TYPE,
REP_V_SERVICE_REQUEST.CLASSIFICATION_TYPE,
REP_V_SERVICE_REQUEST.CLASSIFICATION_SUB_TYPE,
( REP_V_SERVICE_REQUEST.TIME_OPENED ),
REP_V_SERVICE_REQUEST.CREATED_BY_WORKGROUP,
REP_V_ACTIVITY.VENUS_SENT_TO_SDN,
SR_RESOLVER_WG.WORKGROUP
FROM
REP_V_SERVICE_REQUEST,
REP_V_ACTIVITY,
REP_V_WORKGROUP SR_RESOLVER_WG
WHERE
( SR_RESOLVER_WG.WORKGROUP_ID=REP_V_SERVICE_REQUEST.RESOLUTION_GROUP_ID )
AND ( REP_V_ACTIVITY.SERVICE_REQUEST_ROW_ID=REP_V_SERVICE_REQUEST.SERVICE_REQUEST_ROW_ID )
AND (
REP_V_ACTIVITY.PLANNED_START_TIME BETWEEN '01-JAn-2006' AND '19-Jun-2006'
AND REP_V_ACTIVITY.VENUS_PROBLEM_STATUS NOT IN ('Information', 'Planned')
AND REP_V_ACTIVITY.VENUS_ALERT_STATUS != 'Withdrawn'
AND REP_V_ACTIVITY.CREATED_BY_WORKGROUP LIKE '%SSHD%'
AND REP_V_ACTIVITY.STATUS != 'Cancelled'
AND REP_V_ACTIVITY.CREATED_BY_WORKGROUP NOT LIKE 'GLO_SSHD_MTC'
Exp. plan(prod)
24 SELECT STATEMENT
23 NESTED LOOPS (OUTER)
21 NESTED LOOPS (OUTER)
19 NESTED LOOPS
16 NESTED LOOPS
13 NESTED LOOPS
10 NESTED LOOPS (OUTER)
8 NESTED LOOPS
5 NESTED LOOPS
2 TABLE ACCESS (BY INDEX ROWID), S_ORG_EXT (SIEBEL)
1 INDEX (FULL SCAN), S_ORG_EXT_F13 (SIEBEL)
4 TABLE ACCESS (BY INDEX ROWID), S_BU (SIEBEL)
3 INDEX (UNIQUE SCAN), S_BU_P1 (SIEBEL)
7 TABLE ACCESS (BY INDEX ROWID), S_SRV_REQ (SIEBEL)
6 INDEX (RANGE SCAN), S_SRV_REQ_U2 (SIEBEL)
9 INDEX (UNIQUE SCAN), S_SRV_REGN_P1 (SIEBEL)
12 TABLE ACCESS (BY INDEX ROWID), SERVICE_REQUEST (CRMREP_REP)
11 INDEX (UNIQUE SCAN), PK_SR (CRMREP_REP)
15 TABLE ACCESS (BY INDEX ROWID), S_EVT_ACT (SIEBEL)
14 INDEX (RANGE SCAN), S_EVT_ACT_F14 (SIEBEL)
18 TABLE ACCESS (BY INDEX ROWID), ACTIVITY (CRMREP_REP)
17 INDEX (UNIQUE SCAN), PK_A (CRMREP_REP)
20 INDEX (RANGE SCAN), S_EVT_MAIL_U1 (SIEBEL)
22 INDEX (RANGE SCAN), S_EVT_ACT_X_U1 (SIEBEL
Exp plan(Dev):
24 SELECT STATEMENT
23 NESTED LOOPS (OUTER)
21 NESTED LOOPS (OUTER)
19 NESTED LOOPS
16 NESTED LOOPS
13 NESTED LOOPS (OUTER)
11 NESTED LOOPS
8 NESTED LOOPS
5 NESTED LOOPS
2 TABLE ACCESS (BY INDEX ROWID), S_EVT_ACT (SIEBEL)
1 INDEX (RANGE SCAN), S_EVT_ACT_M8 (SIEBEL)
4 TABLE ACCESS (BY INDEX ROWID), S_SRV_REQ (SIEBEL)
3 INDEX (UNIQUE SCAN), S_SRV_REQ_P1 (SIEBEL)
7 TABLE ACCESS (BY INDEX ROWID), S_ORG_EXT (SIEBEL)
6 INDEX (UNIQUE SCAN), S_ORG_EXT_U3 (SIEBEL)
10 TABLE ACCESS (BY INDEX ROWID), S_BU (SIEBEL)
9 INDEX (UNIQUE SCAN), S_BU_P1 (SIEBEL)
12 INDEX (UNIQUE SCAN), S_SRV_REGN_P1 (SIEBEL)
15 TABLE ACCESS (BY INDEX ROWID), SERVICE_REQUEST (REPORT)
14 INDEX (UNIQUE SCAN), PK_SR (REPORT)
18 TABLE ACCESS (BY INDEX ROWID), ACTIVITY (REPORT)
17 INDEX (UNIQUE SCAN), PK_A (REPORT)
20 INDEX (RANGE SCAN), S_EVT_MAIL_U1 (SIEBEL)
22 INDEX (RANGE SCAN), S_EVT_ACT_X_U1 (SIEBEL)
I checked the v$session_wait while it is hanging,
It is waiting for table s_evt_act (prod. 1.6 crores,dev. 1 crore)
In development it is accessing S_EVT_ACT_M8 (SINGLE COLUMN)index,in production
it is accessing S_EVT_ACT_F14(COMBN. TWO COLUMNS BUT DIFFERENT FROM PRODUCTION).
Thanks,
kumar.

Answer 1

This query is not executing for last 5 to 6 months.
I am new to this issue.
pls find the outof v$session_event for this session.
It is waiting for db file sequential read and the wait is keep on increasing.
SID EVENT TOTAL_WAITS TOTAL_TIMEOUTS TIME_WAITED AVERAGE_WAIT MAX_WAIT TIME_WAITED_MICRO
43 db file sequential read 141459 0 130565 1 66 1305647401
43 db file scattered read 77437 0 54259 1 466 542587342
43 direct path write 1222 0 867 1 26 8671937
43 buffer busy waits 570 0 318 1 4 3175286
43 SQL*Net message to client 339 0 0 0 0 866
43 SQL*Net message from client 338 0 84716 251 32623 847156015
43 latch free 14 12 6 0 2 59905
43 direct path read 6 0 1 0 0 12290
43 log file sync 1 0 0 0 0 2268