Spatial query (point in poly) problem

Question

Hi
One of my users is trying to run the following query in SQL*Plus:
CREATE TABLE erdpwip_join AS
SELECT A.CSNGC, B.PARCELID
FROM RLR.RLR87_SK B, ERDPWIPDATA A
WHERE sdo_relate (a.geom, b.geom, 'mask=ANYINTERACT querytype=Window') = 'TRUE'
He says it works on a small subset of the full dataset, but when he tries to run it on a larger selection of data it stops working. The full dataset has 1.6 million shapes and is held in an Oracle 9.2.0.5.0 database. It isn't clean data, unfortunately: I performed an sdo_geom.validate_layer_with_context on it and removed around 240 shapes with errors (mostly self-intersects), but when trying to access the dataset through ESRI's ArcSDE (version 8.3) a load_buffer error -2035 occurs, which I am informed is because there are other low-level self-intersects which Oracle's validation routine didn't pick up.
There are two questions really: first, is the quality of the data (with these geometric errors) the reason for the user's point-in-polygon query not working on larger recordsets (e.g. about 400000 records), or is because of the volume of data?
Secondly, is there any way that Oracle's validation routines will pick up these low-level self-intersects which were missed in the first pass? ESRI's support forum recommends running a query on the dataset which steps through so many records at a time until it finds an error - the full thread is at http://forums.esri.com/Thread.asp?c=2&f=1720&t=152985&mc=1.
Any help would be appreciated.
Thanks
Sharon Evans

Answer 1

Hi Sharon,
What does "it stops working" mean? Error? Long running query?
Assuming your data is now clean in spatial, I would try creating the table, then loading it with the query using the ordered hint. The number of geometries, while affecting run time, is certainly not more than spatial can handle.
CREATE TABLE erdpwip_join AS
SELECT A.CSNGC, B.PARCELID
FROM RLR.RLR87_SK B, ERDPWIPDATA A
WHERE rownum < 1;
insert into erdpwip_join
SELECT /*+ ordered */ A.CSNGC, B.PARCELID
FROM RLR.RLR87_SK B, ERDPWIPDATA A
WHERE sdo_relate (a.geom, b.geom, 'mask=ANYINTERACT querytype=Window') = 'TRUE';
Regarding drawing in ESRI and tolerance/validation:
In Oracle Spatial the tolerance is specified as the distance two points have to be apart to be considered different.
I think for ESRI, points have to be greater than tolerance apart (not exactly tolerance apart, which is fine in spatial).
So what you might want to try is kicking up the tolerance by some very small increment for the validation, remove or fix those invalid geometries, reset the tolerance back to the smaller value, and then try using ArcSDE again.
I haven't tried this at all, so if you can let us know if this works we'll know this is probably a good general recommendation.