Single Query suddenly goes slow until database restart or query change

Question

Hello all,
Configuration I'm using is Oracle 10g EE 10.1.0.4.0 on Windows 2003 Server
I have performance issue with one single query. The query is called using dbExpress from Delphi application.
Since a month, almost every week this query starts going VERY slow. Usually it executes in a 0.5-1 second, but when it's slowed down, it takes approx. 4-5 minutes to execute.
The query runs on a single table (~4 000 000 rows) and runs on primary index. The index itself consists of 5 fields - the query uses the first 4 of them. I know it's not a good design but restructuring the table/index is out of question, AFAIK.
Until now, I figured out two ways to speed things up. The first and obvious is to restart the server or shut/start the database. It works. For a week. Then we're back at first square.
The second way is some kind of a mystery to me: If I change the query a little bit, never mind how - for example, adding 'and (1=1)' in the clause, after deploying the application, it starts running fast. Again for a week or so.
I've attached an jpeg showing the explain plans on slow and fast executing queries. They look almost the same.
I'm running out of ideas for now. It's very annoying (and unsafe) to restart the machine on a week basis. Thus any help is greatly appreciated.
Things I've done so far:
- installed last service packs from Oracle and Microsoft.
- tried different versions of dbexpora.dll.
- Regularly gathered and estimated statistics and analyzings.
- Made sure that the server is not under heavy load (antiviruses, firewalls, archive loggers are stopped or uninstalled for a while).
- made sure that there is enough free space on hard disks.
- the disk array itself is in good health - defragmented, HDD tools doesn't find anything suspicious.
Final thoughts: Maybe the sql cache is going overloaded and someway stops executing the query. That's why when I restart the machine or change the query clause insignificantly, it starts running fast. I took a peek on v$sql and v$sqlarea but there is nothing special, and honestly, I'm not very familiar reading the values in this view - I'm just comparing the values returned by this query with the others - they look similar.
Anyway, if someone could help, it would be great.
Regards,
Ivan
Screenshots showing the explain plans for slow and fast query:
Image: !http://img254.imageshack.us/img254/6674/queryr.jpg!

Answer 1

Sorry for the incomplete information. I tried to gather the mentioned in your second link additional data. Unfortunately, I cannot find the TKPROF utility - it's not in the bin directory, neither in any other folder. Guess it's not installed at all.
About rewriting the query, suggested in the first link you gave: it's so simple and clear that it should not be rewriten. As I mentioned before, every time I change the query even a little bit, it starts running well for a couple of days.
Anyway, here are the stats I've just gathered. Unfortunately, now the query performs well and I'm not able to give you stats, showing the slowed down query.
These are the parameters relevant to the optimizer:
SQL> show parameter optimizer
NAME TYPE VALUE
optimizer_dynamic_sampling integer 2
optimizer_features_enable string 10.1.0.4
optimizer_index_caching integer 0
optimizer_index_cost_adj integer 100
optimizer_mode string ALL_ROWS
SQL> show parameter db_file_multi
NAME TYPE VALUE
db_file_multiblock_read_count integer 32
SQL> show parameter db_block_size
NAME TYPE VALUE
db_block_size integer 8192
SQL> show parameter cursor_sharing
NAME TYPE VALUE
cursor_sharing string EXACT
Screenshots of explain plan is posted in my first post - both for fast and slow executed query.
output of SQL_Plus AUTOTRACE including the TIMING information:
SQL> set autotrace traceonly arraysize 1000
SQL> SELECT * FROM "AOPERITEM"
WHERE (FOPER = 201)
AND (FCLIENT = 1)
AND (FDATEOPER = '2009-10-28')
AND (FNUMOPER = 95532)
Execution Plan
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)|
| 0 | SELECT STATEMENT | | 1 | 132 | 5 (0)|
| 1 | TABLE ACCESS BY INDEX ROWID| AOPERITEM | 1 | 132 | 5 (0)|
| 2 | INDEX RANGE SCAN | AOPERITEM_IDX | 1 | | 4 (0)|
Note
- 'PLAN_TABLE' is old version
Statistics
1 recursive calls
0 db block gets
9 consistent gets
0 physical reads
0 redo size
3098 bytes sent via SQL*Net to client
369 bytes received via SQL*Net from client
2 SQL*Net roundtrips to/from client
0 sorts (memory)
0 sorts (disk)
4 rows processed
The DBMS_XPLAN.DISPLAY_CURSOR output:
SQL> select /*+ gather_plan_statistics */
* FROM "AOPERITEM"
WHERE (FOPER = 201)
AND (FCLIENT = 1)
AND (FDATEOPER = '2009-10-28')
AND (FNUMOPER = 95532)
Execution Plan
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)|
| 0 | SELECT STATEMENT | | 1 | 132 | 5 (0)|
| 1 | TABLE ACCESS BY INDEX ROWID| AOPERITEM | 1 | 132 | 5 (0)|
| 2 | INDEX RANGE SCAN | AOPERITEM_IDX | 1 | | 4 (0)|
Note
- 'PLAN_TABLE' is old version
Statistics
0 recursive calls
0 db block gets
9 consistent gets
4 physical reads
0 redo size
2937 bytes sent via SQL*Net to client
369 bytes received via SQL*Net from client
2 SQL*Net roundtrips to/from client
0 sorts (memory)
0 sorts (disk)
4 rows processed
PS: I'm not a DBA myself, I'm just trying to find a solution to this problem.
Thanks in advance.