Spatial Performance

'm building a system which stores Geographical data, and trying to use the spatial functions in the Oracle database. Amongst the tables there are:
blpu - one row per property in the county, each row has a spatial point object indicating its position. Has 480,000 rows.
lcc_boundary - one row per region in the county (parishes, wards, districts, etc.), each row has a spatial polygon indicating its boundary. Has 537 rows.
Now, if I want to determine which parish (say) a given property is in, I do this:
SELECT b.uprn,lb.name area
FROM blpu b,osmm.lcc_boundary lb
WHERE MDSYS.SDO_GEOM.RELATE(lb.geoloc,'CONTAINS',b.geometry, 0.005) = 'CONTAINS'
AND lb.area_code = 'CPC'
AND b.uprn = 100030723225
So far so good - it comes back with the right answer - but it takes 1 second to do so. If I changed the last line to bring back 100 properties, it takes about 100 seconds. Clearly performance is going to be extremely sluggish when doing real queries on thousands of addresses.
Is there anything I can do - e.g. indexes that can be built - that will speed up these spatial functions; or am I just going to have to "denormalise" the spatial data into an ordinary table that cross-references each row in blpu with the rows in lcc_boundaries that enclose it?
I find it hard to believe that Oracle would be selling a product that's so unusably slow.
This is on Oracle9i Enterprise Edition Release 9.2.0.6.0, btw.

Feedback and progress so far...
asturbcn: Yeah, I think that's what I know as a spatial index. I've built them on the relevant columns.
Bryan: I'm building it on 9i 'cos that's what I've got. The "powers that be" here tend to be pretty cautious about database upgrades, so I dunno when the DB in question will go to 10g. Maybe I can use this issue as leverage, but I still need to get something going today.
As noted, I already had spatial indexes, but not optimised for the polygon type. I've recreated them with that refinement added - thanks.
Richard: I wouldn't normally consider area_code as a candidate for an index, as it only has a few distinct values, but I created one anyway.
Dan: Applying all the above advice, and some I got locally, I've ended up with this statement:
SELECT b.uprn,lb.name area
FROM blpu b,osmm.lcc_boundary lb
WHERE sdo_relate(lb.GEOLOC, b.geometry, 'mask=CONTAINS querytype=WINDOW') = 'TRUE'
AND lb.area_code = 'CPC'
AND b.uprn = 100030723225
This runs about three times faster than my original one, the biggest improvement seeming to come from the use in SDO_RELATE instead of MDSYS.SDO_GEOM.RELATE (I tried adding Bryan's /*+ ordered */ hint, but that slowed the query to 1.5mins. A place to trust the optimiser, I think!).
So that's a good result - but it's not yet good enough to be usable in my particular situation. I think I'm still going to use a batch job to create a conventional this-property-is-in-this-region lookup table, and populate it (and maintain it) using spatial functions. Since neither of the tables concerned change much, I think this will be a workable approach - the lookup table will in effect be a kind of spatial index. Maybe when we go to 10g I can investigate a pure spatial solution.
Unless, of course, you know different...

Similar Messages

Oracle Spatial Performance with 10-20.000 users

Does anyone have any experience when Oracle Spatial is used with say 20.000 concurrent users. I am not interested in MapViewer response time, but lets say there is:
- an app using 800 different tables each having an sdo_geometry column
- the app is configured with different tables visible on different view scales
- let's say an average of 40-50 tables is visible at any given time
- some tables will have only a few records, while other can hold millions.
- there is no client side caching
- clients can zoom in/out pan.
Anwers I am interested in:
- What sort of server would be required
- How can Oracle serve all that data (each Refresh renders the map and retrieves the data over the wire as there is no client side caching).
- What sort of network infrastructure would be required.
- Can clients connect to different servers and hence use load balancing or does Oracle have an automatic mechanism for that?
Thanks in advance,
Patrick

Patrick, et al.
There are lots of things one can do to improve performance in mapping environments because of a lot of the visualisation is based on "background" or read-only data. Here are some "tips":
1. Spatially sort read-only data.
This tip makes sure that data that is close to each other in space are next to each other on disk! Dan gave a good suggestion when he referenced Chapter 14, "Reorganize the Table Data to Minimize I/O" pp 580- 582, Pro Oracle Spatial. But just as easily one can create a table as select ... where sdo_filter() where the filtering object is an optimized rectangle across the whole of the dataset. (This is quite quick on 10g and above but much slower on earlier releases.)
When implementing this make sure that the created table is created such that its blocks are next to each other in the tablespace. (Consider tablespace defragmentation beforehand.) Also, if the data is READ ONLY set the PCTFREE to 0 in order to pack the data up into as small a number of blocks as possible.
2. Generalise data
Rendering spatial data can be expensive where the data is geometrically detailed (many vertices) esp where the data is being visualised at smaller scales than it was captured at. So, if your "zoom thresholds" allow 1:10,000 data to be used at 1:100,000 then you are going to have problems. Consider pre-generalising the data (see sdo_util.simplify) before deployment. You can add multiple columns to your base table to hold this data. Be careful with polygon data because generalising polygons that share boundaries will create gaps etc as the data is more generalised. Often it is better to export the data to a GIS which can maintain the boundary relationships when generalising (say via topological relationships).
Oracle's MapViewer has excellent on-the-fly generalisation but here one needs to be careful. Application tier caching (cf Bryan's comments) can help here a lot.
3. Don't draw data that is sub-pixel.
As one zooms out objects become smaller and smaller until they reach a point where the whole object can be drawn within a single pixel. If you have control over your map visualisation application you might want to consider setting the SDO_FILTER parameter "min_resolution" flag dynamically so that its value is the same as the number of meters / pixel (eg min_resolution=10). If this is set Oracle Spatial will only include spatial objects in the returned search set if one side of a geometry's MBR is greater than or equal to this value. Thus any geometries smaller than a pixel will not be returned. Very useful for large scale data being drawn at small scales and for which no selection (eg identify) is required. With Oracle MapViewer this behaviour can be set via the generalized_pixels parameter.
3. SDO_TOLERANCE, Clean Data
If you are querying data other than via MBR (eg find all land parcels that touch each other) then make sure that your sdo_tolerance values are appropriate. I have seen sites where data captured to 1cm had an sdo_tolerance value set to a millionth of a meter!
A corollary to this is make sure that all your data passes validation at the chosen sdo_tolerance value before deploying to visualisation. Run sdo_geom.validate_geometry()/validate_layer()...
4. Rtree Spatial Indexing
At 10g and above lots of great work went in to the RTree indexing. So, make sure you are using RTrees and not QuadTrees. Also, many GIS applications create sub-optimal RTrees by not using the additional parameters available at 10g and above.
4.1 If your table/column sdo_geometry data contains only points, lines or polygons then let the RTree indexer know (via layer_gtype) as it can implement certain optimizations based on this knowledge.
4.2 With 10g you can set the RTree's spatial index data block use via sdo_pct_free. Consider setting this parameter to 0 if the table/column sdo_geometry data is read only.
4.3 If a table/column is in high demand (eg it is the most commonly used table in all visualisations) you can consider loading (a part of) the RTree index into memory. Now, with the RTree indexing, the sdo_non_leaf_tbl=true parameter will split the RTree index into its leaf (contains actual rowid reference) and non-leaf (the tree built on the leaves) components. Most RTrees are built without this so only the MDRT*** secondary tables are built. But if sdo_non_leaf_tbl is set to true you will see the creation of an additional MDNT*** secondary table (for the non_leaf part of the rtree index). Now, if appropriate, the non_leaf table can be loaded into memory via the following:
ALTER TABLE MDNT*** STORAGE(BUFFER_AREA KEEP);
This is NOT a general panacea for all performance problems. One should investigate other options before embarking on this (cf Tom Kyte's books such as Expert Oracle Database Architecture, 9i and 10g Programming Techniques and Solutions.)
4.4 Don't forget to check your spatial index data quality regularly. Because many sites use GIS package GUI tools to create tables, load data and index them, there is a real tendency to not check what they have done or regularly monitor the objects. Check the SDO_RTREE_QUALITY column in USER_SDO_INDEX_METADATA and look for indexes with an SDO_RTREE_QUALITY setting that is > 2. If > 2 consider rebuilding or recreating the index.
5. The rendering engine.
Whatever rendering engine one uses make sure you try and understand fully what it can and cannot do. AutoDesk's MapGuide is an excellent product but I have seen it simply cache table/column data and never dynamically access it. Also, I have been at one site which was running Deegree and MapViewer and MapViewer was so fast in comparison to Deegree that I was called in to find out why. I discovered that Deegree was using SDO_RELATE(... ANYINTERACT ...) for all MBR queries while MapViewer was using SDO_FILTER. Just this difference was causing some queries to perform at < 10% of the speed of MapViewer!!!!
6. Consider "denormalising" data
There is an old adage in databases that is "normalise for edit, denormalise for performance". When we load spatial data we often get it from suppliers in a fairly flat or normalised form. In consort with spatial sorting, consider denormalising the data via aggregations based on a rendering attribute and some sort of spatial unit. For example, if you have 1 million points stored as single points in SDO_GEOMETRY.SDO_POINT which you want to render by a single attribute containing 20 values, consider aggregating the data using this attribute AND some sort of spatial BUCKET or BIN. So, consider using SDO_AGGR_UNION coupled with Spatial Analysis and Mining package functions to GROUP the data BY <<column_name>> and a set of spatial extents.
6. Tablespace use
Finally, talk to your DBA in order to find out how the oracle database's physical and logical storage is organised. Is a SAN being used or SAME arranged disk arrays? Knowing this you can organise your spatial data and indexes using more effective and efficient methods that will ensure greater scalability.
7. Network fetch
If your rendering engine (app server) and database are on separate machines you need to investigate what sort of fetch sizes are being used when returning data from queries to the middle-tier. Fetch sizes for attribute only data rows and rows containing spatial data can be, and normally are, radically different. Accepting the default settings for these sizes could be killing you (as could the sort_area_size of the Oracle session the application server has created on the database). For example I have been informed that MapInfo Pro uses a fixed value of 25 records per fetch when communicating with Oracle. I have done some testing to show that this value can be too small for certain types of spatial data. SQL Developer's GeoRaptor uses 100 which is generally better (but this one can modify this). Most programmers accept defaults for network properties when programming in ADO/ODBC/OLEDB/JDBC: just be careful as to what is being set here. (This is one of the great strengths of ArcSDE: its TCP/IP network transport is well written, tuneable and very efficient.)
8. Physical Format
Finally, while Oracle's excellent MapViewer requires data its spatial data to be in Oracle, other commercial rendering engines do not. So, consider using alternate, physical file formats that are more optimal for your rendering engine. For example, Google Earth Enterprise "compiles" all the source data into an optimal format which the server then serves to Google Earth Enterprise clients. Similarly, a shapefile on local disk to the application server (with spatial indexing) may be faster that storing the data back in Oracle on a database server that is being shared with other business databases (eg Oracle financials). If you don't like this approach and want to use Oracle only consider using a dedicated Oracle XE on the application server for the data that is read only and used in most of your generated maps eg contour or drainage data.
Just some things to think about.
regards
Simon

Spatial Performance Problem

I'm new to GIS development, and hereby got a performance comparison.
Techs adopted time consumed to display the whole map in one time
ESRI SDE + Oracle Spatial 6 Mins
ESRI SDE + Oracle 1 Min 20 Sec
ESRI SDE + SQLSERVER 50 Sec
ESRI SDE + Shapefile 30 Sec
Shape file size 550M, totally 480k rows
Can anyone tell me why SDE + Spatial works so low performance than SDE + SQL SERVER, and what can I do to improve the performance when adopt just SDE + ORACLE(just for storage like normal RDBMS).
I don't think Oracle will work worse than SQL Server in SDE environment. But what should I pay attention to during using Oracle to store map data for SDE.
Is that not a normal scense to use SDE with Oracle Spatial? Why I got so bad performance? I just migrate data from Shapefile to SDO_GEOMETRY type and add indexes as Spatial required.
Any one can leave your comment if you know something about this.
Thanks in advance.

Here is a partitioning example as promised. Note an upcoming talk at Oracle Open World will show how to extend this model to include spatial partitioning, where we will describe some really important optimizations that will bring scalability (already a differentiator) to new levels. After Oracle Open World we'll post that information to OTN as well (please come talk to us if you happen to go).
-- First, remove table partition_sales if it exists
drop table partition_sales;
-- Create table partition_sales. This table is partitioned
-- by date range, breaking each year into 4 quarters.
-- What users see is a single table called partition_sales
-- but in reality there is one smaller table created for
-- each partition
-- For simplicity we are loading all partitions into the same
-- tablespace. Most often they are loaded into separate tablespaces
-- for performance, scalability, and manageability reasons
CREATE TABLE partition_sales
( customer_id number,
sale_date date,
sale_amount number(10,2),
geom mdsys.sdo_geometry
PARTITION BY RANGE (sale_date)
(PARTITION BEFORE_2003
VALUES LESS THAN (TO_DATE('01-JAN-2003','DD-MON-YYYY')),
PARTITION Q1_2003 VALUES LESS THAN (TO_DATE('01-APR-2003','DD-MON-YYYY')),
PARTITION Q2_2003 VALUES LESS THAN (TO_DATE('01-JUL-2003','DD-MON-YYYY')),
PARTITION Q3_2003 VALUES LESS THAN (TO_DATE('01-OCT-2003','DD-MON-YYYY')),
PARTITION Q4_2003 VALUES LESS THAN (TO_DATE('01-JAN-2004','DD-MON-YYYY')),
PARTITION Q1_2004 VALUES LESS THAN (TO_DATE('01-APR-2004','DD-MON-YYYY')),
PARTITION Q2_2004 VALUES LESS THAN (TO_DATE('01-JUL-2004','DD-MON-YYYY')),
PARTITION Q3_2004 VALUES LESS THAN (TO_DATE('01-OCT-2004','DD-MON-YYYY')),
PARTITION Q4_2004 VALUES LESS THAN (TO_DATE('01-JAN-2005','DD-MON-YYYY')),
PARTITION AFTER_2004
VALUES LESS THAN (MAXVALUE));
-- Each of these inserts goes into a separate partition.
-- Although there are only 9 rows inserted, each into a separate
-- partition, if you had 50 million rows they would be divided into
-- smaller, more manageable pieces
insert into partition_sales values (61243, '31-DEC-2002', 18764.23,
mdsys.sdo_geometry(2001, 8307,
mdsys.sdo_point_type(-73.943849, 40.66980, NULL), NULL, NULL) );
insert into partition_sales values (4576, '15-JAN-2003', 27797.05,
mdsys.sdo_geometry(2001, 8307,
mdsys.sdo_point_type(-71.017892, 42.336029, NULL), NULL, NULL) );
insert into partition_sales values (1161, '22-MAY-2003', 16222.50,
mdsys.sdo_geometry(2001, 8307,
mdsys.sdo_point_type(-76.610616, 39.30080, NULL), NULL, NULL) );
insert into partition_sales values (55033, '09-SEP-2003', 1211.33,
mdsys.sdo_geometry(2001, 8307,
mdsys.sdo_point_type(-77.016167, 38.90505, NULL), NULL, NULL) );
insert into partition_sales values (768, '15-DEC-2003', 84397.61,
mdsys.sdo_geometry(2001, 8307,
mdsys.sdo_point_type(-73.943851, 40.66975, NULL), NULL, NULL) );
insert into partition_sales values (61243, '05-JAN-2004', 21764.26,
mdsys.sdo_geometry(2001, 8307,
mdsys.sdo_point_type(-73.943849, 40.66980, NULL), NULL, NULL) );
insert into partition_sales values (6474, '17-JUN-2004', 76411.81,
mdsys.sdo_geometry(2001, 8307,
mdsys.sdo_point_type(-85.256956, 35.066209, NULL), NULL, NULL) );
insert into partition_sales values (505, '27-JUL-2004', 104100.89,
mdsys.sdo_geometry(2001, 8307,
mdsys.sdo_point_type(-74.173245, 40.7241, NULL), NULL, NULL) );
insert into partition_sales values (9151, '11-OCT-2004', 44298.66,
mdsys.sdo_geometry(2001, 8307,
mdsys.sdo_point_type(-79.976702, 40.43920, NULL), NULL, NULL) );
commit;
-- Let's make sure what we thought happened actually did happen
-- The following allows us to see how many rows are stored in each partition
exec dbms_stats.gather_table_stats('SCOTT','PARTITION_SALES');
select partition_name,num_rows
from user_tab_partitions
where table_name='PARTITION_SALES';
-- Now the data is loaded, we can spatially index it
-- First, insert into user_sdo_geom_metadata
delete from user_sdo_geom_metadata where table_name = 'PARTITION_SALES'
and column_name = 'GEOM';
insert into user_sdo_geom_metadata values ('PARTITION_SALES', 'GEOM',
mdsys.sdo_dim_array (
mdsys.sdo_dim_element ('LONG', -180, 180, 1),
mdsys.sdo_dim_element ('LAT', -90, 90, 1)),
8307);
-- Now create the index. Note the keyword local is used
-- which creates a separate index for each partition
-- Because this example is simple, we didn't show storage
-- of each spatial index partition in a separate tablespace,
-- which is the common usage, again for performance, scalability,
-- and manageability.
drop index partition_sales_sidx;
create index partition_sales_sidx on partition_sales (geom)
indextype is mdsys.spatial_index parameters ('layer_gtype=point')
local;
-- Any query that uses the partition key (the date) will only search the
-- partition associated with that date:
select customer_id, sale_amount
from partition_sales
where sale_date = '17-JUN-2004'
and sdo_relate(geom,
mdsys.sdo_geometry(2003, 8307, null, mdsys.sdo_elem_info_array(1,1003,1),
mdsys.sdo_ordinate_array(-86,34, -85,34, -85,36, -86,36, -86,34)),
'mask=anyinteract querytype=window') = 'TRUE';
-- The previous query only touched a single partition.
-- You don't have to specify the partition key, though.
-- All partitions can be searched if required. To speed up
-- these kinds of queries, you can search all partitions in
-- parallel by specifying the keywork parallel in the
-- create index statement, or altering the index
alter index partition_sales_sidx parallel 4;
-- No date specified in this query, so all partitions may be searched
select customer_id, sale_amount, sale_date
from partition_sales
where sdo_relate(geom,
mdsys.sdo_geometry(2003, 8307, null, mdsys.sdo_elem_info_array(1,1003,1),
mdsys.sdo_ordinate_array(-75,38, -70,38, -70,45, -75,45, -75,38)),
'mask=anyinteract querytype=window') = 'TRUE';

Spatial Performance with join

I have a Oracle Spatial table with 3.5 million rows plus another auxillary table with 3.5 million rows. A query over these two tables joined returns a full result (250 rows) based on one to one join - here's an example:
Select count(*)
from F, N
where F.id = n.id and (F.GEOM,mdsys.sdo_geometry (2003,8307, null, mdsys.sdo_elem_info_array (1,1003,1), mdsys.sdo_ordinate_array(-120.0,49.5,-119.0,49.5,-119.0,60.35,-120.0,60.35,-120.0,49.5)),
'mask= ANYINTERACT querytype=WINDOW') = 'TRUE') AND N.PNUM = '4';
It takes an average of 35 seconds to get the full result set back. I've gathered statistics, tweaked memory parameters and this is the best I can get. Does anyone have any suggestions?

This is an interesting problem. It looks like Oracle is doing the right thing for each of the table accesses - use the index and fetch by rowid.
The only thing you have to play with if you don't go to materialized views or temp tables is how the results of the two table queries are joined.
You don't have a lot of options. Hash join seems to be slow, but you don't know if it is faster or slower compared with nested loops or merge join.
I'd compare what you have done with something like the following to test nested loops:
select /*+ no_merge use_nl (f1,n1) */ count(*)
from
(select id
from f
where sdo_anyinteract ( F.GEOM,
sdo_geometry (2003,8307, null, sdo_elem_info_array (1,1003,1),
sdo_ordinate_array (-120.0,49.5,-119.0,49.5,-119.0,60.35,
-120.0,60.35,-120.0,49.5))) = 'TRUE') f1,
(select id
from n
where n.pnum='4') n1
where f1.id=n1.id ;
and presort with a merge join hint to see how it performs:
select /*+ no_merge use_merge (f1,n1) */ count(*)
from
(select id
from f
where sdo_anyinteract ( F.GEOM,
sdo_geometry (2003,8307, null, sdo_elem_info_array (1,1003,1),
sdo_ordinate_array (-120.0,49.5,-119.0,49.5,-119.0,60.35,
-120.0,60.35,-120.0,49.5))) = 'TRUE'
order by id) f1,
(select id
from n
where n.pnum='4'
order by id) n1
where f1.id=n1.id ;
It might be that you already have the best Oracle can do, but I'd be curious to know how you make out.
Dan Abugov
VP Software Support and Services
Acquis Inc.

Oracle Spatial performance question

All,
I am doing a performance test on Oracle 11g Spatial. I am simulating doing searches in 10 degree by 10 degree windows over 6M+ images, six arc minutes per side. Here is my spatial query construction:
String intersectSQL = "SELECT A.name, A.GEOMETRY.Get_WKT() " +
"FROM six_amin_polygons A " +
"WHERE SDO_RELATE(A.GEOMETRY,?, " +
"'mask=inside+coveredby+overlapbdyintersect')='TRUE'";
where the question mark is replaced by the geometry structure of the search window. The results for the first few searches are fast, then the query times balloon very quickly. PostGIS/PostgreSQL performs these searches in an average time of 30 s per window.
Here are the initial (first four rows) of Oracle Spatial results:
area_idx area_name sql_query_time number_results
0 S80.0W90.0 3890 10100
1 S80.0W80.0 3124 10100
2 S80.0W70.0 186484 10100
3 S80.0W60.0 183077 10100
Any ideas? Am I using the best mask for image/area intersection? Please advise.
Thanks,
Jeff

With anyinteract you get
inside+coveredby+overlapbdyintersect+touch
since you are comparing polygons to polygons.
Do you want polygons that touch the window geometry in the result ? Do you want all the geometries
that have some kind of intersection with the window query ? Then you should use ANYINTERACT mask.
siva

11.1.0.6 Spatial Performance Bug ?

Hi All,
Is there a known issue with performance of SDO_ANYINTERACT on 11.1.0.6 ?
I have 2 tables:
MAP_LAKES_US with 16 rows and MAP_PARKFACILITY_SF with 162 rows
The following query:
SELECT count( a1.POLYGON_ID )
FROM MAP_LAKES_US a1, MAP_PARKFACILITY_SF a2
WHERE SDO_ANYINTERACT(A1.GEOMETRY,
SDO_GEOMETRY(2003, 8307, NULL, SDO_ELEM_INFO_ARRAY(1, 1003, 1), SDO_ORDINATE_ARRAY( -122.96677479856513, 37.508862, -122.96677479856513, 37.9221695, -121.89290520143487, 37.9221695, -121.89290520143487, 37.508862, -122.96677479856513, 37.508862 ) )
)='TRUE'
AND SDO_ANYINTERACT(a1.GEOMETRY, a2.GEOMETRY)='TRUE';
Takes 17minutes and 50 seconds on 11.1.0.6 (10minutes 53 seconds on a higher spec machine) ... But only takes 7 seconds on 11.1.0.7 on the same spec machine as the 1st machine.
Without the 2nd where clause the query executes instantly on all machines.
cheers,
Ro

Hi Siva,
Sorry this took so long.... Here are the explain pans:
11.1.0.6 Database
PLAN_TABLE_OUTPUT
Plan hash value: 3455258727
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
| 0 | SELECT STATEMENT | | 1 | 102 | 8 (0)| 00:00:01 |
| 1 | SORT AGGREGATE | | 1 | 102 | | |
| 2 | NESTED LOOPS | | 1 | 102 | 8 (0)| 00:00:01 |
| 3 | TABLE ACCESS BY INDEX ROWID| MAP_LAKES_US | 1 | 42 | 3 (0)| 00:00:01 |
|* 4 | DOMAIN INDEX | SIDX_MAP_LAKES_US | | | | |
|* 5 | TABLE ACCESS FULL | MAP_PARKFACILITY_SF | 2 | 120 | 5 (0)| 00:00:01 |
PLAN_TABLE_OUTPUT
Predicate Information (identified by operation id):
4 - access("MDSYS"."SDO_ANYINTERACT"("A1"."GEOMETRY","MDSYS"."SDO_GEOMETRY"(2003,8307,NULL
,"SDO_ELEM_INFO_ARRAY"(1,1003,1),"SDO_ORDINATE_ARRAY"((-122.96677479856513),37.508862,(-122.9
6677479856513),37.9221695,(-121.89290520143487),37.9221695,(-121.89290520143487),37.508862,(-
122.96677479856513),37.508862)))='TRUE')
5 - filter("MDSYS"."SDO_ANYINTERACT"("A1"."GEOMETRY","A2"."GEOMETRY")='TRUE')
11.1.0.7 Database
PLAN_TABLE_OUTPUT
Plan hash value: 3455258727
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
| 0 | SELECT STATEMENT | | 1 | 7658 | 8 (0)| 00:00:01 |
| 1 | SORT AGGREGATE | | 1 | 7658 | | |
| 2 | NESTED LOOPS | | 1 | 7658 | 8 (0)| 00:00:01 |
| 3 | TABLE ACCESS BY INDEX ROWID| MAP_LAKES_US | 1 | 3835 | 3 (0)| 00:00:01 |
|* 4 | DOMAIN INDEX | SIDX_MAP_LAKES_US | | | | |
|* 5 | TABLE ACCESS FULL | MAP_PARKFACILITY_SF | 2 | 7646 | 5 (0)| 00:00:01 |
PLAN_TABLE_OUTPUT
Predicate Information (identified by operation id):
4 - access("MDSYS"."SDO_ANYINTERACT"("A1"."GEOMETRY","MDSYS"."SDO_GEOMETRY"(2003,8307,NULL
,"SDO_ELEM_INFO_ARRAY"(1,1003,1),"SDO_ORDINATE_ARRAY"((-122.96677479856513),37.508862,(-122.9
6677479856513),37.9221695,(-121.89290520143487),37.9221695,(-121.89290520143487),37.508862,(-
122.96677479856513),37.508862)))='TRUE')
5 - filter("MDSYS"."SDO_ANYINTERACT"("A1"."GEOMETRY","A2"."GEOMETRY")='TRUE')
PLAN_TABLE_OUTPUT
Note
- dynamic sampling used for this statement
The only difference I can see his the values in the Bytes column, I'm not sure what can cause these differences.
As an aside, I've discovered that if I replace the order of the parameters in the final SDO_ANYINTERACT clause (placing the larger dataset a2 first) then the query executes in 0.8 seconds on both databases.
(This also has the effect of changing the full table scan in step 5 to a domain index lookup)
Unfortuantly, since the query is generated from code and can be different every time, this is not a simple code change to fix.
Thanks,
Ronan

Oracle Open World in SanFrancisco and Oracle Spatial

Hi,
This week is Oracle Open World in San Francisco, and while usually we try to keep this message board strictly technical, I wanted to mention that there are some interesting things happening here.
There are a few sessions Wednesday in Room 2016 starting at 3:00 pm:
Data Management on a Budget with Oracle 10g: Exploring Strategies and Tactics with the U.S. Geological Survey
by Nate Booth and Harold House, USGS
Oracle Database 10g Spatial Performance and Manageability Best Practices and German Rail Case Study
by Dan Abugov (me) and Andreas Hoefler, Fichtner Consulting & IT AG
I'm excited to be talking about Partitioning Best Practices, and we'll be posting the white paper here on OTN.
Finally, please come and see us at booth E28 in the Oracle DemoGrounds (we have Locator/Spatial, MapViewer, and Workspace Manager here in the pod). If you are the FIRST person to mention reading this posting on OTN when you visit the booth, we'll be happy to give you a copy of "Pro Oracle Spatial", the first book devoted to developing applications using Oracle Spatial. The book was written by Ravi Kothuri, Albert Godfrind (two Oracle Spatial Developers) and Euro Beinat of Geodan NL. One of the authors (Albert Godfrind) is here, and if you ask I'd bet he'd autograph it for you!
We also have a very limited supply of the book we'll be giving out over the course of the conference.
Of course, everyone who stops by will get the Oracle Spatial mini CD which includes viewlets, white papers, and more.
The book is also available here:
http://www.amazon.com/exec/obidos/tg/detail/-/1590593839/qid=1102358585/sr=1-1/ref=sr_1_1/002-9450919-8639226?v=glance&s=books

Hello,
Take a look at this page, http://www.oracle.com/technology/events/oracle-openworld-2007/index.html
There are links here to the whole OOW (social network,twitter,blog/flicker)osphere I think the new buzzword being "Social Graph" that covers all that.
But anyway there are tons of ways for people to sign up and track the sessions and see who's going to go where, etc.
Regards,
Carl
blog : http://carlback.blogspot.com/
apex examples : http://apex.oracle.com/pls/otn/f?p=11933:5

Query slowness after upgrading to 10.2.0.4

Is anyone else having spatial query problems after applying the 10.2.0.4 patch? We applied it to our DEV and QA environments last weekend and we are now seeing a degragation of spatial queries.
On the database we upgaded a query that has been running fine for years now takes 5.5 hours to complete. The same query on a database that has not been upgraded takes 5 minutes.
Both databases are on servers with the same hardware and operating system (Solaris 10 64bit), the only difference is the database version.
We have an SR open, but we aren't feeling the love yet.

Hi Robenour,
7003151 is not just for Linux, it's for all platforms, and will definitely help 10.2.0.4 and 11.1.0.6 spatial performance.
Below is the current list of Oracle Locator/Spatial/MapViewer patches I recommend.
The only one specific to Solaris is 5888136 for 10.2.0.3.
I recommend all the other patches for all hardware platforms.
Most of the following patches are available on Metalink.
If the patch is not there for your hardware platform/database release, please log a SR with Oracle support and request a backport.
Hope this helps. Thanks.
Dan
For Oracle 10.2.0.3, patches:
· 6980648 - Bug fixes and improved performance
· 6329260 - Bug fixes and improved performance
· 5888136 - Only if you are running Solaris - improved performance
· 6956194 - Bug fix if working with geodetic data
For Oracle 10.2.0.4, patches:
· 7003151 - Bug fixes and improved performance
· 6989483 - Improved performance
· 7046751 - Apply this patch if you applied patch 6989483 (necessary for some GIS software to work)
· 7237687 - Bug fix if working with geodetic data
· 6956194 - Bug fix if working with geodetic data
· 7276032 - Bug fix if working with geodetic data
For Oracle 11.1.0.6, patches:
· 7003151 - Bug fixes and improved performance
· 6989483 - Improved performance
· 7046751 - Apply this patch if you applied patch 6989483 (necessary for some GIS software to work)
· 6956194 - If working with geodetic data
For MapViewer 10.1.3.1
· 7195504 - Bug fixes and improved performance

Mapviewer/Mapbuilder versus other map serving products

Mapviewer/Mapbuilder versus other map serving products
Oracle Mapviewer is an attractive map serving option for organisations that have large data holdings in Oracle Spatial. Similarly, Mapbuilder is very useful for cartographic assembly of data to be delivered via Mapviewer.
Are other map serving options or cartogarphic tools for web mapping worth considering?
I'd appreciate peoples views/links on how Mapviewer and Mapbuilder compare with other map serving products?
Thanks.

I'd also be very interested to know if using MapViewer gives you better Oracle Spatial performance then with say MapServer (from UMN). Our company has a UMN MapServer setup and we're using some Oracle Spatial data to populate this as well and I see a lot of performance decrease when trying to use Oracle Spatial data. One nice thing about UMN MapServer is the fact that it's free and supports many type of GIS data. There are also quite a few application frameworks available for developing a client front-end. We use Chameleon currently.
I would like to know what other people are using with Oracle Spatial.

SGA in memory

I've made an application in VB that works against an Oracle Spatial db. Running the program takes extremely long time, and I'm trying to speed up the whole application. Therefor I've tested locking the SGA in the memory instead of reading from disk. This didn't work at all, even though I have 100 Mb available physical memory and the SGA only takes 55 Mb (according to the 'Show sga').
I've also tried altering the initialization parameters 'db_block_buffers' and 'log_buffer', without any success.
What am I doing wrong? Any tips or hints?
Thanks, Pelle

Hi Pelle,
I would recommend executing the Oracle
Spatial query from SQL*Plus to see
the spatial performance impact in your
application.
Please see the technical white paper on
technet under products/spatial for
performance enhancement tips.
Some critical tips for Oracle 8.1.6 is
use a fixed index and also to analyze
the spatial index table. These items are
discussed in the performance white paper.
You may also want to try the Spatial Index Advisor which is part of OEM to help tune your spatial indexes. The performance white paper discusses how to use the Spatial Index Advisor effectively.
Hope this helps. Thanks.
Dan

Poor performance with Oracle Spatial when spatial query invoked remotely

Is anyone aware of any problems with Oracle Spatial (10.2.0.4 with patches 6989483 and 7003151 on Red Hat Linux 4) which might explain why a spatial query (SDO_WITHIN_DISTANCE) would perform 20 times worse when it was invoked remotely from another computer (using SQLplus) vs. invoking the very same query from the database server itself (also using SQLplus)?
Does Oracle Spatial have any known problems with servers which use SAN disk storage? That is the primary difference between a server in which I see this poor performance and another server where the performance is fine.
Thank you in advance for any thoughts you might share.

OK, that's clearer.
Are you sure it is the SQL inside the procedure that is causing the problem? To check, try extracting the SQL from inside the procedure and run it in SQLPLUS with
set autotrace on
set timing on
SELECT ....If the plans and performance are the same then it may be something inside the procedure itself.
Have you profiled the procedure? Here is an example of how to do it:
Prompt Firstly, create PL/SQL profiler table
@$ORACLE_HOME/rdbms/admin/proftab.sql
Prompt Secondly, use the profiler to gather stats on execution characteristics
DECLARE
l_run_num PLS_INTEGER := 1;
l_max_num PLS_INTEGER := 1;
v_geom    mdsys.sdo_geometry := mdsys.sdo_geometry(2002,null,null,sdo_elem_info_array(1,2,1),sdo_ordinate_array(0,0,45,45,90,0,135,45,180,0,180,-45,45,-45,0,0));
BEGIN
dbms_output.put_line('Start Profiler Result = ' || DBMS_PROFILER.START_PROFILER(run_comment => 'PARALLEL PROFILE')); -- The comment name can be anything: here it is related to the Parallel procedure I am testing.
v_geom := Parallel(v_geom,10,0.05,1); -- Put your procedure call here
dbms_output.put_line('Stop Profiler Result = ' || DBMS_PROFILER.STOP_PROFILER );
END;
SHOW ERRORS
Prompt Finally, report activity
COLUMN runid FORMAT 99999
COLUMN run_comment FORMAT A40
SELECT runid || ',' || run_date || ',' || run_comment || ',' || run_total_time
FROM plsql_profiler_runs
ORDER BY runid;
COLUMN runid       FORMAT 99999
COLUMN unit_number FORMAT 99999
COLUMN unit_type   FORMAT A20
COLUMN unit_owner FORMAT A20
COLUMN text        FORMAT A100
compute sum label 'Total_Time' of total_time on runid
break on runid skip 1
set linesize 200
SELECT u.runid || ',' ||
       u.unit_name,
       d.line#,
       d.total_occur,
       d.total_time,
       text
FROM   plsql_profiler_units u
       JOIN plsql_profiler_data d ON u.runid = d.runid
                                     AND
                                     u.unit_number = d.unit_number
       JOIN all_source als ON ( als.owner = 'CODESYS'
                             AND als.type = u.unit_type
                               AND als.name = u.unit_name
                            AND als.line = d.line# )
WHERE u.runid = (SELECT max(runid) FROM plsql_profiler_runs)
ORDER BY d.total_time desc;Run the profiler in both environments and see if you can see where the slowdown exists.
regards
Simon

Spatial query performance problems

In preparation for making using of spatial data in our oracle database, I wanted to create a view (materialised) that brings together data from a few different tables into one place ready for publishing as a WMS layer.
I'm being stumped at first base by crippling performance of Oracle spatial function. Later joins of ordinary fields are ok, but the spatial joining of two tables using the following sql runs for an absurd length of time (i've given up - I don't know how long it actually takes only that it takes far too long)
SELECT /*+ ordered */
lg.GRIDREF, lg.SYSTEM, lg.PARENT, lg.TYPE,
lrd.REGION_CODE
FROM TABLE (SDO_JOIN('L_GRIDS','BOUNDARY','L_REGION_DEFINITION','BOUNDARY','mask=COVERS')) c,
L_GRIDS lg, L_REGION_DEFINITION lrd
WHERE c.rowid1 = lg.rowid AND c.rowid2 = lrd.rowid
ORDER BY lrd.REGION_CODE
Both tables have spatial indexs. L_REGION_DEFINITION contains 200 rows with complex boundaries stored as spatial objects. L_GRIDS contains 475,000 rows, each with a trivially simple spatial object consisting of a square polygon of 4 points.
The database is 10g patched to latest. The server is dual quad Xeon processors with 16gb of ram. I didn't expect it to be a lightning query, but surely it should be usable?
Any ideas?

Try to upgrade to at least 11.2.0.2 and use the following query
SELECT /*+ leading(lrd lg) */
lg.GRIDREF, lg.SYSTEM, lg.PARENT, lg.TYPE,
lrd.REGION_CODE
FROM L_GRIDS lg, L_REGION_DEFINITION lrd
WHERE sdo_relate(lg.boundary, lrd.boundary, 'mask=COVEREDBY') = 'TRUE'
ORDER BY lrd.REGION_CODE;
And since not sure about your query's intention, maybe it is "mask=INSIDE+COVEREDBY",
please check out oracle spatial developer guide for details about different masks.

Performance problem at bulk insert with spatial index

Hi,
I have a table with SDO_GEOMETRY.
Insert without spatial index is very fast, but with active spatial index it's very slow.
So for the first big import of data, I can drop the index, import the data and again create the index. Thats 10 times faster!
But for an already very big table that is no option.
The 10g1-Users Guide (1) says at 4.1.3 that the spatial index should be set to 'deferred', the data should be inserted and than the index should be synchronized again. That sounds very good, but I can't find this at the 11g1-Users Guide.
I tried it (11g1), but the performance is even worse than with active index!
What could be my mistake? Any hints?
Thank you,
Bjoern Weitzig
create table sggeoptcollection (pt SDO_GEOMETRY);
CREATE INDEX myIdx ON sggeoptcollection (pt) INDEXTYPE IS MDSYS.SPATIAL_INDEX PARAMETERS('sdo_indx_dims=2, layer_gtype=point, sdo_rtr_pctfree=50');
ALTER INDEX myIdx PARAMETERS ('index_status=deferred');
Big import with batch'ed PreparedStatements
ALTER INDEX myIdx PARAMETERS ('index_status=synchronize sdo_batch_size=500');
1) http://download.oracle.com/docs/html/B10826_01/sdo_index_query.htm#g1010227

Hi,
I have a table with SDO_GEOMETRY.
Insert without spatial index is very fast, but with active spatial index it's very slow.
So for the first big import of data, I can drop the index, import the data and again create the index. Thats 10 times faster!
But for an already very big table that is no option.
The 10g1-Users Guide (1) says at 4.1.3 that the spatial index should be set to 'deferred', the data should be inserted and than the index should be synchronized again. That sounds very good, but I can't find this at the 11g1-Users Guide.
I tried it (11g1), but the performance is even worse than with active index!
What could be my mistake? Any hints?
Thank you,
Bjoern Weitzig
create table sggeoptcollection (pt SDO_GEOMETRY);
CREATE INDEX myIdx ON sggeoptcollection (pt) INDEXTYPE IS MDSYS.SPATIAL_INDEX PARAMETERS('sdo_indx_dims=2, layer_gtype=point, sdo_rtr_pctfree=50');
ALTER INDEX myIdx PARAMETERS ('index_status=deferred');
Big import with batch'ed PreparedStatements
ALTER INDEX myIdx PARAMETERS ('index_status=synchronize sdo_batch_size=500');
1) http://download.oracle.com/docs/html/B10826_01/sdo_index_query.htm#g1010227

Performance issue when inserting into spatial indexed table with JDBC

We have a table named 'feature' which has a "sdo_geometry" column, and we created spatial index on that column,
CREATE TABLE feature ( id number, desc varchar, oshape sdo_gemotry)
CREATE INDEX feature_sp_idx ON feature(oshape) INDEXTYPE IS MDSYS.SPATIAL_INDEX;
Then we executed the following SQL to insert about 800 records into that table(We tried this by using DB visualizer and
our Java application, both of them were using JDBC driver to connect to the oracle 11gR2 database) .
insert into feature(id,desc,oshape) values (1001,xxx,xxxxx);
insert into feature (id,desc,oshape) values (1002,xxx,xxxxx);
insert into feature (id,desc,oshape) values (1800,xxx,xxxxx);
We encoutered the same problem as this topic
Performance of insert with spatial index
It takes nearly 1 secs for inserting one record,compare to 50 records inserted per sec without spatial index,
which is 50x drop in peformance when doing insertion with spatial index.
However, when we copy and paste those insertion scripts into Oracle Client(same test and same table with spatial index), we got a totally different performance result:
more than 50 records inserted in 1 secs, just as fast as the insertion without building spatial index.
Is it because Oracle Client is not using JDBC? Perhaps JDBC was got something wrong when updating those spatial indexed tables.
Edited by: 860605 on 19/09/2011 18:57
Edited by: 860605 on 19/09/2011 18:58
Edited by: 860605 on 19/09/2011 19:00

Normally JDBC use auto-commit. So each insert can causes a commit.
I don't know about Oracle Client. In sqlplus, insert is just a insert,
and you execute "commit" to explicitly commit your changes.
So maybe this is the reason.

Performance of Creating Spatial Indexes

I have a spatial database with about 75GB of data in it. When I load this data initially it takes about 6 hours to load using 6 processes. To get this performance level I did not create any of the indexes or contsraints. So after the load I have to run the scripts to create the indexes and constraints.
To improve the performance of this I have written a script on UNIX which parses my SQL files and runs each create or alter statement in a separate process. The script makes sure that there are only a certain number of concurrent processes (10).
Creating the indexes is taking longer than the database load itself. There is one point layer in particular which has serveral hundred million rows which takes around 6 hours to complete. Are the any ways you can improve the performance of index creation. I did try the parallel option once but it did not reliably complete when I did my testing.
Does using Oracle Partitioning improve the performance of index creation? I do have a mapsheet attribute which could be used for the partitioning.
Paul

Hi Paul,
Parallelism should have reliably helped. I've heard a lot of good things about it more recently (i.e. if you tested on an earlier version you may want to upgrade/patch and check it out).
Partitioning should improve index creation speed as well. If you tend to query by an attribute, use that attribute as the partition key. If you tend to query by location (spatial query), try to use a partition key that reflects location (state, county, etc).
When you first create the index, create it local (partitioned) as unusable, then create each partition's index separately in parallel - you can use the script you've already written to do this.
Since it is point data, make sure to use layer_gtype=point
Here is a quick example:
CREATE INDEX partn_table_sidx ON partn_table(geom)
INDEXTYPE IS MDSYS.SPATIAL_INDEX
PARAMETERS ('LAYER_GTYPE=POINT')
LOCAL (
PARTITION P1 PARAMETERS (TABLESPACE=’P1_TBS’),
PARTITION P2 PARAMETERS (TABLESPACE=’P2_TBS’),
PARTITION P3 PARAMETERS (TABLESPACE=’P3_TBS’),
PARTITION P4 PARAMETERS (TABLESPACE=’P4_TBS’))
UNUSABLE;
ALTER INDEX partn_table_sidx REBUILD PARTITION P1;
ALTER INDEX partn_table_sidx REBUILD PARTITION P2;
ALTER INDEX partn_table_sidx REBUILD PARTITION P3;
ALTER INDEX partn_table_sidx REBUILD PARTITION P4;

Spatial Performance

Similar Messages

Maybe you are looking for