Performance problem at bulk insert with spatial index

Hi,
I have a table with SDO_GEOMETRY.
Insert without spatial index is very fast, but with active spatial index it's very slow.
So for the first big import of data, I can drop the index, import the data and again create the index. Thats 10 times faster!
But for an already very big table that is no option.
The 10g1-Users Guide (1) says at 4.1.3 that the spatial index should be set to 'deferred', the data should be inserted and than the index should be synchronized again. That sounds very good, but I can't find this at the 11g1-Users Guide.
I tried it (11g1), but the performance is even worse than with active index!
What could be my mistake? Any hints?
Thank you,
Bjoern Weitzig
create table sggeoptcollection (pt SDO_GEOMETRY);
CREATE INDEX myIdx ON sggeoptcollection (pt) INDEXTYPE IS MDSYS.SPATIAL_INDEX PARAMETERS('sdo_indx_dims=2, layer_gtype=point, sdo_rtr_pctfree=50');
ALTER INDEX myIdx PARAMETERS ('index_status=deferred');
Big import with batch'ed PreparedStatements
ALTER INDEX myIdx PARAMETERS ('index_status=synchronize sdo_batch_size=500');
1) http://download.oracle.com/docs/html/B10826_01/sdo_index_query.htm#g1010227

Similar Messages

Performance of insert with spatial index

I'm writing a test that inserts (using OCI) 10,000 2D point geometries (gtype=2001) into a table with a single SDO_GEOMETRY column. I wrote the code doing the insert before setting up the index on the spatial column, thus I was aware of the insert speed (almost instantaneous) without a spatial index (with layer_gtype=POINT), and noticed immediately the performance drop with the index (> 10 seconds).
Here's the raw timing data of 3 runs in each 3 configuration (the clock ticks every 14 or 15 or 16 ms, thus the zero when it completes before the next tick):
                                   truncate execute commit
no spatial index                     0.016   0.171   0.016
no spatial index                     0.031   0.172   0.000
no spatial index                     0.031   0.204   0.000
index (1000 default for batch size) 0.141 10.937   1.547
index (1000 default for batch size) 0.094 11.125   1.531
index (1000 default for batch size) 0.094 10.937   1.610
index SDO_DML_BATCH_SIZE=10000       0.203 11.234   0.359
index SDO_DML_BATCH_SIZE=10000       0.094 10.828   0.344
index SDO_DML_BATCH_SIZE=10000       0.078 10.844   0.359As you can see, I played with SDO_DML_BATCH_SIZE to change the default of 1,000 to 10,000, which does improve the commit speed a bit from 1.5s to 0.35s (pretty good when you only look at these numbers...), but the shocking part of the almost 11s the inserts are now taking, compared to 0.2s without an index: that's a 50x drop in peformance!!!
I've looked at my table in SQL Developer, and it has no triggers associated, although there has to be something to mark the index as dirty so that it updates itself on commit.
So where is coming the huge overhead during the insert???
(by insert I mean the time OCIStmtExecute takes to run the array-bind of 10,000 points. It's exactly the same code with or without an index).
Can anyone explain the 50x insert performance drop?
Any suggestion on how to improve the performance of this scenario?
To provide another data point, creating the index itself on a populated table (with the same 10,000 points) takes less than 1 second, which is consistent with the commit speeds I'm seeing, and thus puzzles me all the more regarding this 10s insert overhead...
SQL> set timing on
SQL> select count(*) from within_point_distance_tab;
COUNT(*)
     10000
Elapsed: 00:00:00.01
SQL> CREATE INDEX with6CDF1526$point$idx
2            ON within_point_distance_tab(point)
3    INDEXTYPE IS MDSYS.SPATIAL_INDEX
4    PARAMETERS ('layer_gtype=POINT');
Index created.
Elapsed: 00:00:00.96
SQL> drop index WITH6CDF1526$POINT$IDX force;
Index dropped.
Elapsed: 00:00:00.57
SQL> CREATE INDEX with6CDF1526$point$idx
2            ON within_point_distance_tab(point)
3    INDEXTYPE IS MDSYS.SPATIAL_INDEX
4    PARAMETERS ('layer_gtype=POINT SDO_DML_BATCH_SIZE=10000');
Index created.
Elapsed: 00:00:00.98
SQL>

Thanks for your input. We are likely to use partioning down the line, but what you are describing (partition exchange) is currently beyond my abilities in plain SQL, and how this could be accomplished from an OCI client application without affecting other users and keep the transaction boundaries sounds far from trivial. (i.e. can it made transparent to the client application, and does it require privileges the client does have???). I'll have to investigate this further though, and this technique sounds like one accessible to a DBA only, not from a plain client app with non-privileged credentials.
The thing that I fail to understand though, despite your explanation, is why the slow down is not entirely on the commit. After all, documentation for the SDO_DML_BATCH_SIZE parameter of the Spatial index implies that the index is updated on commit only, where new rows are fed 1,000 or 10,000 at a time to the indexing engine, and I do see time being spent during commit, but it's the geometry insert that slow down the most, and that to me looks quite strange.
It's so much slower that it's as if each geometry was indexed one at a time, when I'm doing a single insert with an array bind (i.e. equivalent to a bulk operation in PL/SQL), and if so much time is spend during the insert, then why is any time spent during the commit. In my opinion it's one or the other, but not both. What am I missing? --DD

Performance issue when inserting into spatial indexed table with JDBC

We have a table named 'feature' which has a "sdo_geometry" column, and we created spatial index on that column,
CREATE TABLE feature ( id number, desc varchar, oshape sdo_gemotry)
CREATE INDEX feature_sp_idx ON feature(oshape) INDEXTYPE IS MDSYS.SPATIAL_INDEX;
Then we executed the following SQL to insert about 800 records into that table(We tried this by using DB visualizer and
our Java application, both of them were using JDBC driver to connect to the oracle 11gR2 database) .
insert into feature(id,desc,oshape) values (1001,xxx,xxxxx);
insert into feature (id,desc,oshape) values (1002,xxx,xxxxx);
insert into feature (id,desc,oshape) values (1800,xxx,xxxxx);
We encoutered the same problem as this topic
Performance of insert with spatial index
It takes nearly 1 secs for inserting one record,compare to 50 records inserted per sec without spatial index,
which is 50x drop in peformance when doing insertion with spatial index.
However, when we copy and paste those insertion scripts into Oracle Client(same test and same table with spatial index), we got a totally different performance result:
more than 50 records inserted in 1 secs, just as fast as the insertion without building spatial index.
Is it because Oracle Client is not using JDBC? Perhaps JDBC was got something wrong when updating those spatial indexed tables.
Edited by: 860605 on 19/09/2011 18:57
Edited by: 860605 on 19/09/2011 18:58
Edited by: 860605 on 19/09/2011 19:00

Normally JDBC use auto-commit. So each insert can causes a commit.
I don't know about Oracle Client. In sqlplus, insert is just a insert,
and you execute "commit" to explicitly commit your changes.
So maybe this is the reason.

Critical performance problem upon bulk load of groups

All (including product development),
I think there are missing indexes in wwsec_flat$ and wwsec_sys_priv$. Anyway, I'd like assistance on fixing the critical performance problems I see, properly. Read on...
During and after bulk load of a few (about 500) users and groups from an external database, it becomes evident that there's a performance problem somewhere. Many of the calls to wwsec_api.addGroupToList took several minutes to finish. Afterwards the machine went 100% CPU just from logging in with the portal30 user (which happens to be group owner for all the groups).
Running SQL trace points in the directions of the following SQL statement:
SELECT ID,PARENT_ID,NAME,TITLE_ID,TITLEIMAGE_ID,ROLLOVERIMAGE_ID,
DESCRIPTION_ID,LAYOUT_ID,STYLE_ID,PAGE_TYPE,CREATED_BY,CREATED_ON,
LAST_MODIFIED_BY,LAST_MODIFIED_ON,PUBLISHED_ON,HAS_BANNER,HAS_FOOTER,
EXPOSURE,SHOW_CHILDREN,IS_PUBLIC,INHERIT_PRIV,IS_READY,EXECUTE_MODE,
CACHE_MODE,CACHE_EXPIRES,TEMPLATE FROM
WWPOB_PAGE$ WHERE ID = :b1
I checked the existing indexes, and see that the following ones are missing (I'm about to test with these, but have not yet done so):
CREATE UNIQUE INDEX "PORTAL30"."WWSEC_FLAT_IX_GROUP_ID"
ON "PORTAL30"."WWSEC_FLAT$"("GROUP_ID")
TABLESPACE "PORTAL" PCTFREE 10 INITRANS 2 MAXTRANS 255
STORAGE ( INITIAL 32K NEXT 160K MINEXTENTS 1 MAXEXTENTS 4096
PCTINCREASE 1 FREELISTS 1)
LOGGING
CREATE UNIQUE INDEX "PORTAL30"."WWSEC_FLAT_IX_PERSON_ID"
ON "PORTAL30"."WWSEC_FLAT$"("PERSON_ID")
TABLESPACE "PORTAL" PCTFREE 10 INITRANS 2 MAXTRANS 255
STORAGE ( INITIAL 32K NEXT 160K MINEXTENTS 1 MAXEXTENTS 4096
PCTINCREASE 1 FREELISTS 1)
LOGGING
CREATE UNIQUE INDEX "PORTAL30"."WWSEC_SYS_PRIV_IX_PATCH1"
ON "PORTAL30"."WWSEC_SYS_PRIV$"("OWNER", "GRANTEE_GROUP_ID",
"GRANTEE_TYPE", "OWNER", "NAME", "OBJECT_TYPE_NAME")
TABLESPACE "PORTAL" PCTFREE 10 INITRANS 2 MAXTRANS 255
STORAGE ( INITIAL 32K NEXT 80K MINEXTENTS 1 MAXEXTENTS 4096
PCTINCREASE 1 FREELISTS 1)
LOGGING
Note that when I deleted the newly inserted groups, the CPU consumption immediately went down from 100% to some 2-3%.
This behaviour has been observed on a Sun Solaris system, but I think it's the same on NT (I have observed it during the bulk load on my NT laptop, but so far have not had the time to test further.).
Also note: In the call to addGroupToList, I set owner to true for all groups.
Also note: During loading of the groups, I logged a few errors, all of the same type ("PORTAL30.WWSEC_API", line 2075), as follows:
Error: Problem calling addGroupToList for child group'Marketing' (8030), list 'NO_OSL_Usenet'(8017). Reason: java.sql.SQLException: ORA-06510: PL/SQL: unhandled user-defined exception ORA-06512: at "PORTAL30.WWSEC_API", line 2075
Please help. If you like, I may supply the tables and the java program that I use. It's fully reproducable.
Thanks,
Erik Hagen (you may call me on +47 90631013)
null

YES!
I have now tested with insertion of the missing indexes. It seems the call to addGroupToList takes just as long time as before, but the result is much better: WITH THE INDEXES DEFINED, THERE IS NO LONGER A PERFORMANCE PROBLEM!! The index definitions that I used are listed below (I added these to the ones that are there in Portal 3.0.8, but I guess some of those could have been deleted).
About the info at http://technet.oracle.com:89/ubb/Forum70/HTML/000894.html: Yes! Thanks! Very interesting, and I guess you found the cause for the error messages and maybe also for the performance problem during bulk load (I'll look into it as soon as possible anbd report what I find.).
Note: I have made a pretty foolproof and automated installation script (or actually, it's part of my Java program), that will let anybody interested recreate the problem. Mail your interest to [email protected].
============================================
CREATE INDEX "PORTAL30"."LDAP_WWSEC_PERS_IX1"
ON "PORTAL30"."WWSEC_PERSON$"("MANAGER")
TABLESPACE "PORTAL" PCTFREE 10 INITRANS 2 MAXTRANS 255
STORAGE ( INITIAL 32K NEXT 32K MINEXTENTS 1 MAXEXTENTS 4096
PCTINCREASE 1 FREELISTS 1)
LOGGING;
CREATE INDEX PORTAL30.LDAP_WWSEC_PERS_IX2
ON PORTAL30.WWSEC_PERSON$('ORGANIZATION')
TABLESPACE PORTAL PCTFREE 10 INITRANS 2 MAXTRANS 255
STORAGE ( INITIAL 32K NEXT 32K MINEXTENTS 1 MAXEXTENTS 4096
PCTINCREASE 1 FREELISTS 1)
LOGGING;
CREATE INDEX PORTAL30.LDAP_WWSEC_PERS_PK
ON PORTAL30.WWSEC_PERSON$('ID')
TABLESPACE PORTAL PCTFREE 10 INITRANS 2 MAXTRANS 255
STORAGE ( INITIAL 32K NEXT 32K MINEXTENTS 1 MAXEXTENTS 4096
PCTINCREASE 1 FREELISTS 1)
LOGGING;
CREATE INDEX PORTAL30.LDAP_WWSEC_PERS_UK
ON PORTAL30.WWSEC_PERSON$('USER_NAME')
TABLESPACE PORTAL PCTFREE 10 INITRANS 2 MAXTRANS 255
STORAGE ( INITIAL 32K NEXT 32K MINEXTENTS 1 MAXEXTENTS 4096
PCTINCREASE 1 FREELISTS 1)
LOGGING;
CREATE INDEX PORTAL30.LDAP_WWWSEC_FLAT_UK
ON PORTAL30.WWSEC_FLAT$("GROUP_ID", "PERSON_ID",
"SPONSORING_MEMBER_ID")
TABLESPACE PORTAL PCTFREE 10 INITRANS 2 MAXTRANS 255
STORAGE ( INITIAL 32K NEXT 256K MINEXTENTS 1 MAXEXTENTS 4096
PCTINCREASE 0 FREELISTS 1);
CREATE INDEX PORTAL30.LDAP_WWWSEC_FLAT_PK
ON PORTAL30.WWSEC_FLAT$("ID")
TABLESPACE PORTAL PCTFREE 10 INITRANS 2 MAXTRANS 255
STORAGE ( INITIAL 32K NEXT 256K MINEXTENTS 1 MAXEXTENTS 4096
PCTINCREASE 0 FREELISTS 1);
CREATE INDEX PORTAL30.LDAP_WWWSEC_FLAT_IX5
ON PORTAL30.WWSEC_FLAT$("GROUP_ID", "PERSON_ID")
TABLESPACE PORTAL PCTFREE 10 INITRANS 2 MAXTRANS 255
STORAGE ( INITIAL 32K NEXT 256K MINEXTENTS 1 MAXEXTENTS 4096
PCTINCREASE 0 FREELISTS 1);
CREATE INDEX PORTAL30.LDAP_WWWSEC_FLAT_IX4
ON PORTAL30.WWSEC_FLAT$("SPONSORING_MEMBER_ID")
TABLESPACE PORTAL PCTFREE 10 INITRANS 2 MAXTRANS 255
STORAGE ( INITIAL 32K NEXT 256K MINEXTENTS 1 MAXEXTENTS 4096
PCTINCREASE 0 FREELISTS 1);
CREATE INDEX PORTAL30.LDAP_WWWSEC_FLAT_IX3
ON PORTAL30.WWSEC_FLAT$("GROUP_ID")
TABLESPACE PORTAL PCTFREE 10 INITRANS 2 MAXTRANS 255
STORAGE ( INITIAL 32K NEXT 256K MINEXTENTS 1 MAXEXTENTS 4096
PCTINCREASE 0 FREELISTS 1);
CREATE INDEX PORTAL30.LDAP_WWWSEC_FLAT_IX2
ON PORTAL30.WWSEC_FLAT$("PERSON_ID")
TABLESPACE PORTAL PCTFREE 10 INITRANS 2 MAXTRANS 255
STORAGE ( INITIAL 32K NEXT 256K MINEXTENTS 1 MAXEXTENTS 4096
PCTINCREASE 0 FREELISTS 1);
CREATE INDEX "PORTAL30"."LDAP_WWSEC_SYSP_IX1"
ON "PORTAL30"."WWSEC_SYS_PRIV$"("GRANTEE_GROUP_ID")
TABLESPACE "PORTAL" PCTFREE 10 INITRANS 2 MAXTRANS 255
STORAGE ( INITIAL 32K NEXT 56K MINEXTENTS 1 MAXEXTENTS 4096
PCTINCREASE 1 FREELISTS 1)
LOGGING;
CREATE INDEX "PORTAL30"."LDAP_WWSEC_SYSP_IX2"
ON "PORTAL30"."WWSEC_SYS_PRIV$"("GRANTEE_USER_ID")
TABLESPACE "PORTAL" PCTFREE 10 INITRANS 2 MAXTRANS 255
STORAGE ( INITIAL 32K NEXT 56K MINEXTENTS 1 MAXEXTENTS 4096
PCTINCREASE 1 FREELISTS 1)
LOGGING;
CREATE INDEX "PORTAL30"."LDAP_WWSEC_SYSP_IX3"
ON "PORTAL30"."WWSEC_SYS_PRIV$"("OBJECT_TYPE_NAME", "NAME")
TABLESPACE "PORTAL" PCTFREE 10 INITRANS 2 MAXTRANS 255
STORAGE ( INITIAL 32K NEXT 56K MINEXTENTS 1 MAXEXTENTS 4096
PCTINCREASE 1 FREELISTS 1)
LOGGING;
CREATE INDEX "PORTAL30"."LDAP_WWSEC_SYSP_PK"
ON "PORTAL30"."WWSEC_SYS_PRIV$"("ID")
TABLESPACE "PORTAL" PCTFREE 10 INITRANS 2 MAXTRANS 255
STORAGE ( INITIAL 32K NEXT 56K MINEXTENTS 1 MAXEXTENTS 4096
PCTINCREASE 1 FREELISTS 1)
LOGGING;
CREATE INDEX "PORTAL30"."LDAP_WWSEC_SYSP_UK"
ON "PORTAL30"."WWSEC_SYS_PRIV$"("OBJECT_TYPE_NAME",
"NAME", "OWNER", "GRANTEE_TYPE", "GRANTEE_GROUP_ID",
"GRANTEE_USER_ID")
TABLESPACE "PORTAL" PCTFREE 10 INITRANS 2 MAXTRANS 255
STORAGE ( INITIAL 32K NEXT 88K MINEXTENTS 1 MAXEXTENTS 4096
PCTINCREASE 1 FREELISTS 1)
LOGGING;
==================================
Thanks,
Erik Hagen
null

How to optimize massive insert on a table with spatial index ?

Hello,
I need to implement a load process for saving up to 20 000 points per minutes in Oracle 10G R2.
These points represents car locations tracked by GPS and I need to store at least all position from the past 12 hours.
My problem is that the spatial index is very costly during insert (For the moment I do only insertion).
My several tries for the insertion by :
- Java and PreparedStatement.executeBatch
- Java and generation a SQLLoader file
- Java and insertion on view with a trigger "instead of"
give me the same results... (not so good)
For the moment, I work on : DROP INDEX, INSERT, CREATE INDEX phases.
But is there a way to only DISABLE INDEX and REBUILD INDEX only for inserted rows ?
I used the APPEND option for insertion :
INSERT /*+ APPEND */ INTO MY_TABLE (ID, LOCATION) VALUES (?, MDSYS.SDO_GEOMETRY(2001,NULL,MDSYS.SDO_POINT_TYPE(?, ?, NULL), NULL, NULL))
My spatial index is created with the following options :
'sdo_indx_dims=2,layer_gtype=point'
Is there a way to optimize these heavy load ???
What about the PARALLEL option and how does it work ? (Not so clear for me regarding the documentation... I am not a DBA)
Thanks in advanced

It is possible to insert + commit 20000 points in 16 seconds.
select * from v$version;
BANNER
Oracle Database 10g Enterprise Edition Release 10.2.0.1.0 - Prod
PL/SQL Release 10.2.0.1.0 - Production
CORE     10.2.0.1.0     Production
TNS for 32-bit Windows: Version 10.2.0.1.0 - Production
NLSRTL Version 10.2.0.1.0 - Production
drop table testpoints;
create table testpoints
( point mdsys.sdo_geometry);
delete user_sdo_geom_metadata
where table_name = 'TESTPOINTS'
and   column_name = 'POINT';
insert into user_sdo_geom_metadata values
('TESTPOINTS'
,'POINT'
,sdo_dim_array(sdo_dim_element('X',0,1000,0.01),sdo_dim_element('Y',0,1000,0.01))
,null)
create index testpoints_i on testpoints (point)
indextype is mdsys.spatial_index parameters ('sdo_indx_dims=2,layer_gtype=point');
insert /*+ append */ into testpoints
select (sdo_geometry(2001,null,sdo_point_type(1+ rownum / 20, 1 + rownum / 50, null),null,null))
from all_objects where rownum < 20001;
Duration: 00:00:10.68 seconds
commit;
Duration: 00:00:04.96 seconds
select count(*) from testpoints;
COUNT(*)
     20000                                                                      The insert of 20 000 rows takes 11 seconds, the commit takes 5 seconds.
In this example there is no data traffic between the Oracle database and a client but you have 60 -16 = 44 seconds to upload your points into a temporary table. After uploading in a temporary table you can do:
insert /*+ append */ into testpoints
select (sdo_geometry(2001,null,sdo_point_type(x,y, null),null,null))
from temp_table;
commit;Your insert ..... values is slow, do some bulk processing.
I think it can be done, my XP computer that runs my database isn't state of the art.

Insert using Spatial Index fails when called from PHP

I have a stored procedure that inserts into a table with a spatial index on one of its fields.
When I run the procedure from SQL Server Management Studio it runs just fine.
When I run it from PHP, it returns the error: "INSERT failed because the following SET options have incorrect settings: 'CONCAT_NULL_YIELDS_NULL, ANSI_WARNINGS, ANSI_PADDING'. Verify that SET options are correct for use with indexed views and/or indexes
on computed columns and/or filtered indexes and/or query notifications and/or XML data type methods and/or spatial index operations"
From within Management Studio, I have tried many different combinations of settings, until I wound up with the list I have in the procedure below.
Within Managment Studio the error changes based on the options I have set, but PHP always replies with: 'CONCAT_NULL_YIELDS_NULL, ANSI_WARNINGS, ANSI_PADDING'
I have configured PHP to use the same SQL user as management studio, and I have tried both mssql_query and mssql_execute with bound parameters.
The spatial index is on the field post.location If I remove the spatial index it works, but I need the spatial index.
CREATE PROCEDURE insert_post ( @subject AS VARCHAR(250), @body AS VARCHAR(2000), @latitude AS FLOAT, @longitude AS FLOAT ) AS BEGIN
SET NOCOUNT ON
SET ANSI_WARNINGS ON
SET ANSI_PADDING ON
SET CONCAT_NULL_YIELDS_NULL ON
SET NUMERIC_ROUNDABORT OFF
DECLARE @location AS geography = geography::Point(@latitude, @longitude, 4326)
INSERT INTO post
    subject,
    body,
    location
) VALUES (
    @subject,
    @body,
    @location
END

Hi Charles
Your issue looks like it have two basic sources: (1) The connection string properties, (2) The table structure and the data which you try to insert.
(1) The connection string properties can be specified in various ways.
You can set most of them during the connection, depending on the provider which you are using. We did not get from you any information about the PHP code, so we have no information regarding this part.
The connection properties can be changed after the connection was made, which is basically what you have try to do. But we don't know what properties
are differents (since you .. the PHP code..) but you can check those properties dynamically, insert them to a table and then compare. To check all the properties you can use this post:
http://ariely.info/Blog/tabid/83/EntryId/153/SQL-Server-Get-Connection-Properties.aspx
(2) We need to understand your
table structure and compare it to yours query. Please post your DDL+DML (queries to create the table and to insert some sample data), and the full SP code
as it is now.
[Personal Site] [Blog] [Facebook]

Error dialog when open index dialog on tables with spatial index

Hi all,
When I want to open in the preferences of a table with a spatial index the index dialog, then there appears the following message:
"Index <myIndex> column GEOMETRY, datatype SDO_GEOMETRY is not a valid column type for use in a text index".
I can only click the ok button, but I am not able to modify any of my set index.
Does anyone else have the same problem?
regards markus
Version:
Java: 1.6.0.16
Oracle IDE: 2.1.1.64.39
OS: Linux, Ubuntu 9.10
Edited by: markusin on Mar 3, 2010 12:32 AM

I have the same problem on SQLDev 2.1.1 for Windows. I hadn't this problem in 1.5.
I must use a normal sql script to create spatial index.
Vittorio

Partial insert with secondary index

I'd like to use partial inserts (DB_DBT_PARTIAL).
However, I also need secondary indexes. Now when I insert partial data, the secondary index function is immediately called, but still I do not have enough data to calculate the secondary key. I tried to supply some app_data do the DBT, but it does not reach the secondary index function.
Is there any way I can use partial inserts together with secondary indexes?

When writing a partial record to the primary database, here's what we do:
1. Look up the existing record via get()
2. Using the existing record and the partial DBT, construct the new record
3. Pass that newly constructed record to the secondary key generation function
You don't need to do anything special, your callback will receive the full record, even though you passed us a partial DBT. If that's not the case, there's a bug in BDB and we'd need to see your code.
app_data is a private field, you should not try to use it or expect it to work predictably. In this situation, we're passing a brand new DBT to the callback function, not the DBT you gave us. That's why app_data is empty.
Thanks,
Bogdan Coman

Insert with unique index slow in 10g

Hi,
We are experiencing very slow response when a dup key is inserted into a table with unique index under 10g. the scenario can be demonstrated in sqlplus with 'timing on':
CREATE TABLE yyy (Col_1 VARCHAR2(5 BYTE) NOT NULL, Col_2 VARCHAR2(10 BYTE) NOT NULL);
CREATE UNIQUE INDEX yyy on yyy(col_1,col_2);
insert into yyy values ('1','1');
insert into yyy values ('1','1');
the 2nd insert results in "unique constraint" error, but under our 10g the response time is consistently in the range of 00:00:00.64. The 1st insert only took 00:00:00.01. BTW, if no index or non-unique index then you can insert many times and all of them return fast. Under our 9.2 DB the response time is always under 00:00:00.01 with no-, unique- and non-unique index.
We are on AIX 5.3 & 10g Enterprise Edition Release 10.2.0.2.0 - 64bit Production.
Has anybody seen this scenario?
Thanks,
David

It seems that in 10g Oracle simply is doing something more.
I used your example and run following script on 9.2 and 10.2. Hardware is the same i.e. these are two instances on the same box.
begin
for i in 1..10000 loop
    begin
      insert into yyy values ('1','1');
    exception when others then null;
    end;
end loop;
end;
/on 10g it took 01:15.08 and on 9i 00:47.06
Running trace showed that in 9i there was difference in plan of following recursive sql:
9i plan:
select c.name, u.name
from
con$ c, cdef$ cd, user$ u where c.con# = cd.con# and cd.enabled = :1 and
c.owner# = u.user#
call     count       cpu    elapsed       disk      query    current        rows
Parse    10000      0.43       0.43          0          0          0           0
Execute 10000      1.09       1.07          0          0          0           0
Fetch    10000      0.23       0.19          0      20000          0           0
total    30000      1.76       1.70          0      20000          0           0
Misses in library cache during parse: 1
Optimizer mode: CHOOSE
Parsing user id: SYS   (recursive depth: 2)
Rows     Row Source Operation
      0 NESTED LOOPS
      0   NESTED LOOPS
      0    TABLE ACCESS BY INDEX ROWID CDEF$
      0     INDEX RANGE SCAN I_CDEF4 (object id 53)
      0    TABLE ACCESS BY INDEX ROWID CON$
      0     INDEX UNIQUE SCAN I_CON2 (object id 49)
      0   TABLE ACCESS CLUSTER USER$
      0    INDEX UNIQUE SCAN I_USER# (object id 11)10g plan
select c.name, u.name
from
con$ c, cdef$ cd, user$ u where c.con# = cd.con# and cd.enabled = :1 and
c.owner# = u.user#
call     count       cpu    elapsed       disk      query    current        rows
Parse    10000      0.21       0.20          0          0          0           0
Execute 10000      1.20       1.31          0          0          0           0
Fetch    10000      2.37       2.59          0      20000          0           0
total    30000      3.79       4.11          0      20000          0           0
Misses in library cache during parse: 1
Misses in library cache during execute: 1
Optimizer mode: CHOOSE
Parsing user id: SYS   (recursive depth: 2)
Rows     Row Source Operation
      0 HASH JOIN (cr=2 pr=0 pw=0 time=301 us)
      0   NESTED LOOPS (cr=2 pr=0 pw=0 time=44 us)
      0    TABLE ACCESS BY INDEX ROWID CDEF$ (cr=2 pr=0 pw=0 time=40 us)
      0     INDEX RANGE SCAN I_CDEF4 (cr=2 pr=0 pw=0 time=27 us)(object id 53)
      0    TABLE ACCESS BY INDEX ROWID CON$ (cr=0 pr=0 pw=0 time=0 us)
      0     INDEX UNIQUE SCAN I_CON2 (cr=0 pr=0 pw=0 time=0 us)(object id 49)
      0   TABLE ACCESS FULL USER$ (cr=0 pr=0 pw=0 time=0 us)So in 10g it had hash join instead of nested loop join at least for this particular select. Probably time to gather stats on sys tables?
The difference in time wasn't so big though 4.11 vs 1.70 so it doesn't explain all the time taken.
But you can probably check whether you haven't more difference.
Also you can download Thomas Kyte runstats_pkg and run it on both environments to compare whether some stats or latches haven't very big difference.
Gints Plivna
http://www.gplivna.eu

Performance Problem ADF-UIX Appl. with Bea 8.1

Hello JDev & J2EE Folks,
I'm running into a performance problem with an ADF-UIX Application (JDev Version 10.1.2.0.0) under Bea 8.1. Application Server. It seems that the the same application is running at least factor 5 slower than on OC4J. I didn't experience such a difference with the older JDev9i Applications, seems to be an issue with 10g. Not done much investigation so far. Are there any ideas or suggestions out there? Anyone who had a similar problem?

Hi ,
Did you get this resolved? Can you share? Thanks!

Create XML format file in bulk insert with a data file with out delimiter

Hello
I have a date file with no delimiter like bellow
0080970393102312072981103378000004329392643958
0080970393102312072981103378000004329392643958
I just know 5 first number in a line is for example "ID of bank"
or 6th and 7th number in a line is for example "ID of employee"
Could you help me how can I create a XML format file?
thanks alot

This is a fixed file format. We need to know the length of each field before creating the format file. Say you have said the first 5 characters are Bank ID and 6th to 7th as Employee ID ... then the XML should look like,
<?xml version="1.0"?>
<BCPFORMAT xmlns="http://schemas.microsoft.com/sqlserver/2004/bulkload/format"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<RECORD>
<FIELD ID="1"xsi:type="CharFixed"LENGTH="5"/>
<FIELD ID="2"xsi:type="CharFixed"LENGTH="2"/>
<FIELD ID="3" xsi:type="CharFixed" LENGTH="8"/>
<FIELD ID="4" xsi:type="CharFixed" LENGTH="14"/>
<FIELD ID="5" xsi:type="CharFixed" LENGTH="14"/>
<FIELD ID="6" xsi:type="CharFixed" LENGTH="1"/>
</RECORD>
<ROW>
<COLUMNSOURCE="1"NAME="c1"xsi:type="SQLNCHAR"/>
<COLUMNSOURCE="2"NAME="c2"xsi:type="SQLNCHAR"/>
<COLUMN SOURCE="3" NAME="c3" xsi:type="SQLCHAR"/>
<COLUMN SOURCE="4" NAME="c4" xsi:type="SQLINT"
/>
<COLUMN SOURCE="5" NAME="c5" xsi:type="SQLINT"
/>
</ROW>
</BCPFORMAT>
Note: Similarly you need to specify the other length as well.
http://stackoverflow.com/questions/10708985/bulk-insert-from-fixed-format-text-file-ignores-rowterminator
Regards, RSingh

Materialized view issue with spatial index and UNION all.

Hi guys,
I'm trying to build the following materialized view:
create materialized VIEW MV_ElectricalStuffs
refresh fast
AS
SELECT jb.ROWID,
    jb.FID,
    JB.NAME_NUMBER
FROM EL_BUS_BAR jb
UNION ALL
SELECT INC.ROWID,
    INC.FID,
    NULL,
    INC.NAME_NUMBER
FROM EL_INTERNAL_CELL INC;
I have this error showing up:
ORA-12015: cannot create a fast refresh materialized view from a complex query
This is because the table EL_INTERNAL_CELL has a SDO_GEOMETRY column that has a spatial index, whose ddl is
CREATE INDEX EL_INTERNAL_CELL_S ON EL_INTERNAL_CELL (GEOM) INDEXTYPE IS MDSYS.SPATIAL_INDEX
PARAMETERS('SDO_INDX_DIMS=2 TABLESPACE=USERS LAYER_GTYPE=COLLECTION');
When I remove the spatial index from EL_INTERNAL_CELL column GEOM, Oracle is very happy and creates the view.
Is there a mean however to keep the spatial index in the materialized view?

I've managed to drop the spatial index prior to create the materialized view and it is ok. After the materialized view creation, I've recreated the spatial index on the table and all ran smooth. Hope nothing will go bad in the future because of this trick on spatial index..

Messaging performance problem on RAID disk with many IMAP users

The performance problem can be solved by tunning the disc RAID system.

The performance problem can be solved by tunning the disc RAID system.

How can we differentiate normal indexes with spatial indexes

how to knwo schema spatial indexes
any queriy pls provide me
thanks

rp0428 wrote:
Thanks for the link. My parents always said I was 'Spatial'. Now I know that is a good thing.(Spatial? Or just Spaced Out?)
There are a few ways to tell, deending on what you are looking at.
1) Spatial Indexes are implementd in regular tables, prefixed by MDRT$ (MultiDimensional R-Tree)
2) They are DOMAIN indexes, and that is reflected by the INDEX_TYPE in ALL_INDEXES

Problems with spatial indexing

Hi,folks,
Thanks, first.
When I try to build a function_based R-tree index on a table, I get the following error message.
Is anyone has idea what I should look into to fix this problem.
Thank you.
ORA-29855: error occurred in the execution of ODCIINDEXCREATE routine
ORA-13203: failed to read USER_SDO_GEOM_METADATA table
ORA-13203: failed to read USER_SDO_GEOM_METADATA table
ORA-06512: at "MDSYS.SDO_INDEX_METHOD_9I", line 7
ORA-06512: at line 1

Hi,
Thanks Dan.
My step is:
I have three relational tables: node, edge, polyline.
They have relationship as following:
each edge is bounded by two nodes,
each polyline is bounded by two nodes and alos composed of a set of edges.
I made functions to get geometry on each table as following:
get_node_geometry(ndid)
get_edge_geometry(edid)
get_polyline_geometry(pdid)
mdsys.sdo_geometry as return type.
Note that coordinates are saved only with node table.
But edge and polyline can construct themselvel by functions above.
After that, I made r-tree index on node, and edge. But it can not work on polyline table for some reasons. I got the error messages from the last postage.
I just note that in my polyline table there are some self-close polylines, which have the same start and terminate node.
Is that possible problem causing the OCIIndexCreate error?
I debugged my code in jdeveloper, by tracing that, there seems no error to return polyline geometry object using get_polyline_geometry function.
Do you have any clue?
many thanks,

Performance problem at bulk insert with spatial index

Similar Messages

Maybe you are looking for