Slow down Database after upgrading to 10.2.0.5

Question

Hi
I am having performance problems after upgrading to 10.2.0.5
At the beginning I thought the problem was sga too smalle ( ini: 598M , now 1408M ) but even after recreating the database with the new value the problem remains.
I am sending reports so that someone could give me an idea.
Thanks in advance!
DETAILED ADDM REPORT FOR TASK 'TASK_240' WITH ID 240
Analysis Period: 22-JUN-2011 from 08:34:06 to 16:00:13
Database ID/Instance: 2462860799/1
Database/Instance Names: DXT/DXT
Host Name: thoracle
Database Version: 10.2.0.5.0
Snapshot Range: from 71 to 78
Database Time: 6726 seconds
Average Database Load: .3 active sessions
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
FINDING 1: 38% impact (2540 seconds)
SQL statements consuming significant database time were found.
RECOMMENDATION 1: SQL Tuning, 26% benefit (1763 seconds)
ACTION: Investigate the SQL statement with SQL_ID "30rku9qg2y30j" for
possible performance improvements.
RELEVANT OBJECT: SQL statement with SQL_ID 30rku9qg2y30j and
PLAN_HASH 2734400036
select a.owner, a.object_name, INSTR(a.object_type, :"SYS_B_00"),
:"SYS_B_01" from sys.all_objects a where a.object_type IN
(:"SYS_B_02",:"SYS_B_03") and a.status = :"SYS_B_04" and a.owner
like:"SYS_B_05"escape:"SYS_B_06" and a.object_name
like:"SYS_B_07"escape:"SYS_B_08" union all select c.owner,
c.synonym_name, INSTR(a.object_type, :"SYS_B_09"), :"SYS_B_10" from
sys.all_objects a, sys.all_synonyms c where c.table_owner = a.owner
and c.table_name = a.object_name and a.object_type IN
(:"SYS_B_11",:"SYS_B_12") and a.status = :"SYS_B_13" and c.owner
like:"SYS_B_14"escape:"SYS_B_15" and c.synonym_name
like:"SYS_B_16"escape:"SYS_B_17" union all select distinct b.owner,
CONCAT(b.package_name, :"SYS_B_18" || b.object_name),
min(b.position), max(b.overload) from sys.all_arguments b where
b.package_name IS NOT NULL and b.owner
like:"SYS_B_19"escape:"SYS_B_20" and b.package_name
like:"SYS_B_21"escape:"SYS_B_22" group by b.owner,
CONCAT(b.package_name, :"SYS_B_23" || b.object_name) union all select
distinct c.owner, CONCAT(c.synonym_name, :"SYS_B_24" ||
b.object_name), min(b.position), max(b.overload) from
sys.all_arguments b, sys.all_synonyms c where c.table_owner = b.owner
and c.table_name = b.package_name and b.package_name IS NOT NULL and
c.owner like:"SYS_B_25"escape:"SYS_B_26" and c.synonym_name
like:"SYS_B_27"escape:"SYS_B_28" group by c.owner,
CONCAT(c.synonym_name, :"SYS_B_29" || b.object_name) union all select
distinct c.owner, c.synonym_name, min(b.position), max(b.overload)
from sys.all_arguments b, sys.all_synonyms c where c.owner = b.owner
and c.table_owner=b.package_name and c.table_name=b.object_name and
c.owner like:"SYS_B_30"escape:"SYS_B_31" and c.synonym_name
like:"SYS_B_32"escape:"SYS_B_33" group by c.owner, c.synonym_name
RATIONALE: SQL statement with SQL_ID "30rku9qg2y30j" was executed 12270
times and had an average elapsed time of 0.036 seconds.
RATIONALE: Waiting for event "cursor: pin S wait on X" in wait class
"Concurrency" accounted for 7% of the database time spent in
processing the SQL statement with SQL_ID "30rku9qg2y30j".
RECOMMENDATION 2: SQL Tuning, 23% benefit (1550 seconds)
ACTION: Run SQL Tuning Advisor on the SQL statement with SQL_ID
"7yv1ba0c8y86t".
RELEVANT OBJECT: SQL statement with SQL_ID 7yv1ba0c8y86t and
PLAN_HASH 2684283631
Select WSTJ_.ROWID, WSTJ_.*, WMVD_.*
From THPR.STOJOU WSTJ_, THPR.SMVTD WMVD_ Where ((WMVD_.VCRTYP_0(+) =
WSTJ_.VCRTYP_0) AND (WMVD_.VCRNUM_0(+) = WSTJ_.VCRNUM_0) AND
(WMVD_.VCRLIN_0(+) = WSTJ_.VCRLIN_0))
And WMVD_.CCE2_0 = :1 And WSTJ_.IPTDAT_0 <= :2 And
Substr(WMVD_.ITMDES1_0,:"SYS_B_0",:"SYS_B_1") <> :3 And
Substr(WMVD_.ITMDES1_0,:"SYS_B_2",:"SYS_B_3") <> :4 And
Substr(WMVD_.ITMDES1_0,:"SYS_B_4",:"SYS_B_5") <> :5 And
((WSTJ_.TRSFAM_0 = :6) Or (WSTJ_.TRSFAM_0 = :7))
Order by WSTJ_.STOFCY_0,WSTJ_.UPDCOD_0,WSTJ_.ITMREF_0,WSTJ_.IPTDAT_0
Desc,WSTJ_.MVTSEQ_0,WSTJ_.MVTIND_0
RATIONALE: SQL statement with SQL_ID "7yv1ba0c8y86t" was executed 47
times and had an average elapsed time of 32 seconds.
RECOMMENDATION 3: SQL Tuning, 14% benefit (926 seconds)
ACTION: Use bigger fetch arrays while fetching results from the SELECT
statement with SQL_ID "7yv1ba0c8y86t".
RELEVANT OBJECT: SQL statement with SQL_ID 7yv1ba0c8y86t and
PLAN_HASH 2684283631
Select WSTJ_.ROWID, WSTJ_.*, WMVD_.*
From THPR.STOJOU WSTJ_, THPR.SMVTD WMVD_ Where ((WMVD_.VCRTYP_0(+) =
WSTJ_.VCRTYP_0) AND (WMVD_.VCRNUM_0(+) = WSTJ_.VCRNUM_0) AND
(WMVD_.VCRLIN_0(+) = WSTJ_.VCRLIN_0))
And WMVD_.CCE2_0 = :1 And WSTJ_.IPTDAT_0 <= :2 And
Substr(WMVD_.ITMDES1_0,:"SYS_B_0",:"SYS_B_1") <> :3 And
Substr(WMVD_.ITMDES1_0,:"SYS_B_2",:"SYS_B_3") <> :4 And
Substr(WMVD_.ITMDES1_0,:"SYS_B_4",:"SYS_B_5") <> :5 And
((WSTJ_.TRSFAM_0 = :6) Or (WSTJ_.TRSFAM_0 = :7))
Order by WSTJ_.STOFCY_0,WSTJ_.UPDCOD_0,WSTJ_.ITMREF_0,WSTJ_.IPTDAT_0
Desc,WSTJ_.MVTSEQ_0,WSTJ_.MVTIND_0
FINDING 2: 37% impact (2508 seconds)
Time spent on the CPU by the instance was responsible for a substantial part
of database time.
RECOMMENDATION 1: SQL Tuning, 26% benefit (1763 seconds)
ACTION: Investigate the SQL statement with SQL_ID "30rku9qg2y30j" for
possible performance improvements.
RELEVANT OBJECT: SQL statement with SQL_ID 30rku9qg2y30j and
PLAN_HASH 2734400036
select a.owner, a.object_name, INSTR(a.object_type, :"SYS_B_00"),
:"SYS_B_01" from sys.all_objects a where a.object_type IN
(:"SYS_B_02",:"SYS_B_03") and a.status = :"SYS_B_04" and a.owner
like:"SYS_B_05"escape:"SYS_B_06" and a.object_name
like:"SYS_B_07"escape:"SYS_B_08" union all select c.owner,
c.synonym_name, INSTR(a.object_type, :"SYS_B_09"), :"SYS_B_10" from
sys.all_objects a, sys.all_synonyms c where c.table_owner = a.owner
and c.table_name = a.object_name and a.object_type IN
(:"SYS_B_11",:"SYS_B_12") and a.status = :"SYS_B_13" and c.owner
like:"SYS_B_14"escape:"SYS_B_15" and c.synonym_name
like:"SYS_B_16"escape:"SYS_B_17" union all select distinct b.owner,
CONCAT(b.package_name, :"SYS_B_18" || b.object_name),
min(b.position), max(b.overload) from sys.all_arguments b where
b.package_name IS NOT NULL and b.owner
like:"SYS_B_19"escape:"SYS_B_20" and b.package_name
like:"SYS_B_21"escape:"SYS_B_22" group by b.owner,
CONCAT(b.package_name, :"SYS_B_23" || b.object_name) union all select
distinct c.owner, CONCAT(c.synonym_name, :"SYS_B_24" ||
b.object_name), min(b.position), max(b.overload) from
sys.all_arguments b, sys.all_synonyms c where c.table_owner = b.owner
and c.table_name = b.package_name and b.package_name IS NOT NULL and
c.owner like:"SYS_B_25"escape:"SYS_B_26" and c.synonym_name
like:"SYS_B_27"escape:"SYS_B_28" group by c.owner,
CONCAT(c.synonym_name, :"SYS_B_29" || b.object_name) union all select
distinct c.owner, c.synonym_name, min(b.position), max(b.overload)
from sys.all_arguments b, sys.all_synonyms c where c.owner = b.owner
and c.table_owner=b.package_name and c.table_name=b.object_name and
c.owner like:"SYS_B_30"escape:"SYS_B_31" and c.synonym_name
like:"SYS_B_32"escape:"SYS_B_33" group by c.owner, c.synonym_name
RATIONALE: SQL statement with SQL_ID "30rku9qg2y30j" was executed 12270
times and had an average elapsed time of 0.036 seconds.
RATIONALE: Waiting for event "cursor: pin S wait on X" in wait class
"Concurrency" accounted for 7% of the database time spent in
processing the SQL statement with SQL_ID "30rku9qg2y30j".
RATIONALE: Average CPU used per execution was 0.036 seconds.
RECOMMENDATION 2: SQL Tuning, 23% benefit (1550 seconds)
ACTION: Run SQL Tuning Advisor on the SQL statement with SQL_ID
"7yv1ba0c8y86t".
RELEVANT OBJECT: SQL statement with SQL_ID 7yv1ba0c8y86t and
PLAN_HASH 2684283631
Select WSTJ_.ROWID, WSTJ_.*, WMVD_.*
From THPR.STOJOU WSTJ_, THPR.SMVTD WMVD_ Where ((WMVD_.VCRTYP_0(+) =
WSTJ_.VCRTYP_0) AND (WMVD_.VCRNUM_0(+) = WSTJ_.VCRNUM_0) AND
(WMVD_.VCRLIN_0(+) = WSTJ_.VCRLIN_0))
And WMVD_.CCE2_0 = :1 And WSTJ_.IPTDAT_0 <= :2 And
Substr(WMVD_.ITMDES1_0,:"SYS_B_0",:"SYS_B_1") <> :3 And
Substr(WMVD_.ITMDES1_0,:"SYS_B_2",:"SYS_B_3") <> :4 And
Substr(WMVD_.ITMDES1_0,:"SYS_B_4",:"SYS_B_5") <> :5 And
((WSTJ_.TRSFAM_0 = :6) Or (WSTJ_.TRSFAM_0 = :7))
Order by WSTJ_.STOFCY_0,WSTJ_.UPDCOD_0,WSTJ_.ITMREF_0,WSTJ_.IPTDAT_0
Desc,WSTJ_.MVTSEQ_0,WSTJ_.MVTIND_0
RATIONALE: SQL statement with SQL_ID "7yv1ba0c8y86t" was executed 47
times and had an average elapsed time of 32 seconds.
RATIONALE: Average CPU used per execution was 32 seconds.
RECOMMENDATION 3: SQL Tuning, 5.8% benefit (390 seconds)
ACTION: Run SQL Tuning Advisor on the SQL statement with SQL_ID
"cbtd2nt52qn1c".
RELEVANT OBJECT: SQL statement with SQL_ID cbtd2nt52qn1c and
PLAN_HASH 2897530229
Select DAE_.ROWID, DAE_.*, HAE_.*
From THPR.GACCENTRYD DAE_, THPR.GACCENTRY HAE_ Where ((HAE_.TYP_0(+)
= DAE_.TYP_0) AND (HAE_.NUM_0(+) = DAE_.NUM_0))
And HAE_.CPY_0 = :1 And HAE_.ACCDAT_0 >= :2 And HAE_.ACCDAT_0 <= :3
And DAE_.ACC_0 = :4 And HAE_.FCY_0 >= :5 And HAE_.FCY_0 <= :6
Order by DAE_.BPR_0,DAE_.CUR_0,DAE_.ACC_0
RATIONALE: SQL statement with SQL_ID "cbtd2nt52qn1c" was executed 12980
times and had an average elapsed time of 0.03 seconds.
RATIONALE: Average CPU used per execution was 0.029 seconds.
RECOMMENDATION 4: SQL Tuning, 2.1% benefit (138 seconds)
ACTION: Run SQL Tuning Advisor on the SQL statement with SQL_ID
"33t7fszkr29gy".
RELEVANT OBJECT: SQL statement with SQL_ID 33t7fszkr29gy and
PLAN_HASH 2684283631
Select WSTJ_.ROWID, WSTJ_.*, WMVD_.*
From THPR.STOJOU WSTJ_, THPR.SMVTD WMVD_ Where ((WMVD_.VCRTYP_0(+) =
WSTJ_.VCRTYP_0) AND (WMVD_.VCRNUM_0(+) = WSTJ_.VCRNUM_0) AND
(WMVD_.VCRLIN_0(+) = WSTJ_.VCRLIN_0))
And WMVD_.CCE2_0 = :1 And WSTJ_.IPTDAT_0 <= :2 And
Substr(WMVD_.ITMDES1_0,:"SYS_B_0",:"SYS_B_1") <> :3 And
Substr(WMVD_.ITMDES1_0,:"SYS_B_2",:"SYS_B_3") <> :4 And
Substr(WMVD_.ITMDES1_0,:"SYS_B_4",:"SYS_B_5") <> :5 And
(((WSTJ_.TRSFAM_0 = :6) Or (WSTJ_.TRSFAM_0 = :7)))
Order by WSTJ_.STOFCY_0,WSTJ_.UPDCOD_0,WSTJ_.ITMREF_0,WSTJ_.IPTDAT_0
Desc,WSTJ_.MVTSEQ_0,WSTJ_.MVTIND_0
RATIONALE: SQL statement with SQL_ID "33t7fszkr29gy" was executed 1
times and had an average elapsed time of 136 seconds.
RATIONALE: Average CPU used per execution was 138 seconds.
FINDING 3: 15% impact (1008 seconds)
SQL statements with the same text were not shared because of cursor
environment mismatch. This resulted in additional hard parses which were
consuming significant database time.
RECOMMENDATION 1: Application Analysis, 15% benefit (1008 seconds)
ACTION: Look for top reason for cursor environment mismatch in
V$SQL_SHARED_CURSOR.
ADDITIONAL INFORMATION:
Common causes of environment mismatch are session NLS settings, SQL
trace settings and optimizer parameters.
SYMPTOMS THAT LED TO THE FINDING:
SYMPTOM: Hard parsing of SQL statements was consuming significant
database time. (20% impact [1336 seconds])
SYMPTOM: Contention for latches related to the shared pool was
consuming significant database time. (2% impact [135
seconds])
INFO: Waits for "cursor: pin S wait on X" amounted to 1% of
database time.
SYMPTOM: Wait class "Concurrency" was consuming significant
database time. (2.3% impact [154 seconds])
FINDING 4: 8.5% impact (570 seconds)
Wait class "User I/O" was consuming significant database time.
NO RECOMMENDATIONS AVAILABLE
ADDITIONAL INFORMATION:
Waits for I/O to temporary tablespaces were not consuming significant
database time.
The throughput of the I/O subsystem was not significantly lower than
expected.
FINDING 5: 5.3% impact (355 seconds)
The SGA was inadequately sized, causing additional I/O or hard parses.
RECOMMENDATION 1: DB Configuration, 3.2% benefit (215 seconds)
ACTION: Increase the size of the SGA by setting the parameter
"sga_target" to 1740 M.
ADDITIONAL INFORMATION:
The value of parameter "sga_target" was "1392 M" during the analysis
period.
SYMPTOMS THAT LED TO THE FINDING:
SYMPTOM: Hard parsing of SQL statements was consuming significant
database time. (20% impact [1336 seconds])
SYMPTOM: Contention for latches related to the shared pool was
consuming significant database time. (2% impact [135
seconds])
INFO: Waits for "cursor: pin S wait on X" amounted to 1% of
database time.
SYMPTOM: Wait class "Concurrency" was consuming significant
database time. (2.3% impact [154 seconds])
SYMPTOM: Wait class "User I/O" was consuming significant database time.
(8.5% impact [570 seconds])
INFO: Waits for I/O to temporary tablespaces were not consuming
significant database time.
The throughput of the I/O subsystem was not significantly lower
than expected.
FINDING 6: 4.2% impact (281 seconds)
Cursors were getting invalidated due to DDL operations. This resulted in
additional hard parses which were consuming significant database time.
RECOMMENDATION 1: Application Analysis, 4.2% benefit (281 seconds)
ACTION: Investigate appropriateness of DDL operations.
SYMPTOMS THAT LED TO THE FINDING:
SYMPTOM: Hard parsing of SQL statements was consuming significant
database time. (20% impact [1336 seconds])
SYMPTOM: Contention for latches related to the shared pool was
consuming significant database time. (2% impact [135
seconds])
INFO: Waits for "cursor: pin S wait on X" amounted to 1% of
database time.
SYMPTOM: Wait class "Concurrency" was consuming significant
database time. (2.3% impact [154 seconds])
FINDING 7: 4% impact (266 seconds)
Waits on event "log file sync" while performing COMMIT and ROLLBACK operations
were consuming significant database time.
RECOMMENDATION 1: Host Configuration, 4% benefit (266 seconds)
ACTION: Investigate the possibility of improving the performance of I/O
to the online redo log files.
RATIONALE: The average size of writes to the online redo log files was
26 K and the average time per write was 2 milliseconds.
SYMPTOMS THAT LED TO THE FINDING:
SYMPTOM: Wait class "Commit" was consuming significant database time.
(4% impact [266 seconds])
FINDING 8: 2.9% impact (192 seconds)
Soft parsing of SQL statements was consuming significant database time.
RECOMMENDATION 1: Application Analysis, 2.9% benefit (192 seconds)
ACTION: Investigate application logic to keep open the frequently used
cursors. Note that cursors are closed by both cursor close calls and
session disconnects.
RECOMMENDATION 2: DB Configuration, 2.9% benefit (192 seconds)
ACTION: Consider increasing the maximum number of open cursors a session
can have by increasing the value of parameter "open_cursors".
ACTION: Consider increasing the session cursor cache size by increasing
the value of parameter "session_cached_cursors".
RATIONALE: The value of parameter "open_cursors" was "800" during the
analysis period.
RATIONALE: The value of parameter "session_cached_cursors" was "20"
during the analysis period.
SYMPTOMS THAT LED TO THE FINDING:
SYMPTOM: Contention for latches related to the shared pool was consuming
significant database time. (2% impact [135 seconds])
INFO: Waits for "cursor: pin S wait on X" amounted to 1% of database
time.
SYMPTOM: Wait class "Concurrency" was consuming significant database
time. (2.3% impact [154 seconds])
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ADDITIONAL INFORMATION
Wait class "Application" was not consuming significant database time.
Wait class "Configuration" was not consuming significant database time.
Wait class "Network" was not consuming significant database time.
Session connect and disconnect calls were not consuming significant database
time.
The database's maintenance windows were active during 100% of the analysis
period.
The analysis of I/O performance is based on the default assumption that the
average read time for one database block is 10000 micro-seconds.
An explanation of the terminology used in this report is available when you
run the report with the 'ALL' level of detail.

Answer 1

user12023161 wrote:
I have upgraded 10.2.0.3.0 to 10.2.0.5.0 and facing same issue. The database is slow in general after upgrade compared to 10.2.0.3.0.Try setting OPTIMIZER_FEATURE_ENABLE parameter to 10.2.0.3.
Refer following link:
http://download.oracle.com/docs/cd/B19306_01/server.102/b14237/initparams142.htm