Oracle performance, slow for larger and more complex results.

Question

Hello Oracle forum,
At the moment i have a Oracle database running and i'm specifically interested in the efficiency spatial extension for webmaps and GIS.
I've been testing the database with large shape files (400mb - 1gigabyte) loaded them into the database with shp2sdo->sql*loader.
Using Benchmark factory i've test the speed of transactions an these drop relatively quickly. I've started with a simple query:
SELECT id FROM map WHERE id = 3 when I increase the amount of id's to 3-10000 the performance decreases drastically.
so :
SELECT id FROM map WHERE id >=3 and id <= 10000
The explain plan shows the second query , both query's use the index.
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
| 0 | SELECT STATEMENT | | 9828 | 49140 | 22 (0)| 00:00:01 |
|* 1 | INDEX RANGE SCAN| SYS_C009650 | 9828 | 49140 | 22 (0)| 00:00:01 |
Statistics
0 recursive calls
0 db block gets
675 consistent gets
0 physical reads
0 redo size
134248 bytes sent via SQL*Net to client
7599 bytes received via SQL*Net from client
655 SQL*Net roundtrips to/from client
0 sorts (memory)
0 sorts (disk)
9796 rows processed
The statistics does not show very weird stuff, but maybe i'm wrong. Nothing changed in the explain plan except for the range scan instead of a unique scan.
The query returns lots of results and this is I think the reason why my measured time of the query is large. The time it takes returning large amount of rows increases quickly for more rows.
. Can this be solved? The table has been analyzed before starting the query.
The parameters of the database are not really changed from standard, I increased the amount of memory used by Oracle 11g to 1+ gigabyte.
and let the database itself decide how it uses this memory.
The system specs are and db_parameters are:
Oracle 11G
Memory Processor # of CPUs OS OS Version OS B
1.99 gb Intel(R) Core(TM)2 CPU 6600 @ 2.40GHz 2 Microsoft WindowsXP 5.2600
0=Oracle decides which value will be given
cursor_sharing EXACT
cursor_space_for_time FALSE
db_block_size 8192
db_recovery_file_dest_size 2147483648
diagnostic_dest C:\DBBENCHMARK\ORACLE
dispatchers (PROTOCOL=TCP) (SERVICE=gistestXDB)
hash_area_size 131072
log_buffer 5656576
memory_max_target 1115684864
memory_target 1048576000
open_cursors 300
parallel_max_servers 20
pga_aggregate_target 0
processes 150
resumable_timeout 2162688
sort_area_size 65536
Sga=632mb
PGA=368mb
javapool=16mb
largepool=8mb
other=8mb
So I indexed and analyzed the data what did i forget? I can speed it up with soft parsing, but the problem remains . Hopefully this enough information for some analysis, does anyone experienced the same problems ? I tested with SQLdeveloper the speed and is shows the same speed as Benchmark factory. What could be wrong with the parameters?
Thanks,
Jan Martijn
Edited by: user12227964 on 25-jan-2010 4:53
Edited by: user12227964 on 26-jan-2010 2:20

Answer 1

Sand wrote:
select count(id) , resulted in 3669015 counted id's.
The database counted 18,345,075 rows per second without binded variables , which is ten times slower as your result. This can be possible because of hardware but my question is specifically about the number of rows returned thus large amount of results. The idea was not to compare the speed of "+select count(*)+" statements - but to illustrate that even when dealing with a huge number of rows, one can decrease the amount of I/O that needs to be performed to deal with that number of rows.
Select id from map where id <= 1
4000 rows per second are selected, Rows/sec is a meaningless measurement - due to physical I/O (PIO) versus logical I/O (LIO). You can select a 100 rows and these require PIO. Resulting in an elapsed time of 1 sec. You can select a 1000 rows that require only LIO. With an an elapsed time of 0.5 sec.
Is the 2nd method better or faster? No. It simply needed less time to be spend on I/O as the data blocks were in the buffer cache (memory) and did not require very slow and expensive disk access.
Another database i testes returns 6 times 25425 rows back per second for the same query (100 ids). What could be a parameter that limits the output speed of multiple rows in a query?.Every single row that needs to be read/processed by a SQL statement has a cost associated with it. This cost is not consistent! It differs depending on how that row can reached - what I/O paths are available to find that rows? Does the full table need to be scanned? Does an index need to be scanned? Is there a unique index that can be used? Is the table partitioned and can partitioning pruning be applied and local partition indexes used? Are there are user functions that need to be applied to the row's data? Etc. Etc.
All these together determine how fast the client gets a row from the cursor executing that SQL.
The more rows you want to process, the bigger the increase in the cost/expense - specifically more I/O. As I/O is the biggest expense (slowest ito elapsed time).
So you want to do as little I/O as possible and read as little data as possible. For example, instead of a full table scan, a fast full index scan. For example, instead of reading the complete contents of a 10GB table, reading the complete contents of a 12MB index for that table.
I suggest that you read the Oracle Performance Guide to familiarise yourself with basic performance concepts. Use http://tahiti.oracle.com for finding the the guide for your applicable Oracle version.