Performance Problem..DBMS_XMLStore or SQL Loader

Hi All,
I have implemented the insertion of data using the following method but it's taking very long.
Once XML and XSL files are there in mapped 'Directory_Path'....
BEGIN
SELECT directory_name INTO v_directory_name
FROM all_directories
WHERE owner = 'SYS'
AND directory_name = 'Directory_Path';
END;
xmldata1 := XMLTYPE(BFILENAME(v_directory_name, 'temp_xml.xml'), nls_charset_id('UTF8'));
xmldata2 := XMLTYPE(BFILENAME(v_directory_name, 'temp_xml.xsl'), nls_charset_id('UTF8'));
--Open a new context, required for these procedures
v_context := DBMS_XMLSTORE.newContext('TEMP_XML');
v_rows := DBMS_XMLStore.insertXML(
v_context,
XMLType.transform(xmldata1, xmldata2));
-- Close the context
DBMS_XMLStore.closeContext(v_context);
END;
May be I am converting data from BFILE to XMLType. But XMLType.transform(xmldata1, xmldata2) accepts XMLType data only.
Or May be this approach where I am not giving the column names explicitly..
Earlier I was loading the transformed XML file(With ROWSET ROW Format) in the the 'Directory_Path' and loading the data as
DECLARE
insCtx DBMS_XMLSTORE.ctxType;
rows NUMBER;
v_directory_name VARCHAR2(300);
src_file BFILE ;
BEGIN
BEGIN
SELECT directory_name INTO v_directory_name
FROM all_directories
WHERE owner = 'SYS'
AND directory_name = 'Directory_Path';
END;
src_file := BFILENAME(v_directory_name, XMLFILE.xml');
insCtx := DBMS_XMLSTORE.newContext( 'TABLE_NAME'); -- Get saved context
DBMS_XMLSTORE.clearUpdateColumnList(insCtx); -- Clear the update settings
-- Set the columns to be updated as a list of values
DBMS_XMLSTORE.setUpdateColumn(insCtx, 'COLUMN1');
DBMS_XMLSTORE.setUpdateColumn(insCtx, 'COLUMN2');
-- Insert the doc.
rows := DBMS_XMLSTORE.insertXML(insCtx, xmltype(src_file,nls_charset_id('AL32UTF8')));
DBMS_OUTPUT.put_line(rows || ' rows inserted.');
-- Close the context
DBMS_XMLSTORE.closeContext(insCtx);
END;
This was happening in a fraction of a second.
But for thousands of XMLs we can't manually generate the Transformed XMLs even if there is a single type of XSL for all those.
AM I MISSING SOMETHING VERY ELEMENTARY OVER HERE???
(I didn't meant to shout by putting in CAPS but highlight it so that it's not missed..Apologies )
The new suggestion being given is PARSE XML FILE into PIPE DELIMITED FORMAT FILE and LOAD that data through SQL LOADER
Personally I don't think it would be better when Oracle is giving us an inbuilt feature.
PLEASE HELP....
How can I improve this. I really think I am taking some wrong step somewhere.....
Regards.....

Thanks A_NON,
But I actually did load within 4 seconds using the below approach without registering as mentioned here.
xmldata1 :=
XMLTYPE (BFILENAME (v_directory_name,
p_xml_file_nme
NLS_CHARSET_ID ('UTF8')
xmldata2 :=
XMLTYPE (BFILENAME (v_directory_name,
p_xsl_file_nme
NLS_CHARSET_ID ('UTF8')
SELECT XMLTRANSFORM (xmldata1, xmldata2) AS temp2
INTO temp2
FROM DUAL;
And then
insCtx := DBMS_XMLSTORE.newContext( 'TABLE_NAME'); -- Get saved context
DBMS_XMLSTORE.clearUpdateColumnList(insCtx); -- Clear the update settings
-- Set the columns to be updated as a list of values
DBMS_XMLSTORE.setUpdateColumn(insCtx, 'COLUMN1');
DBMS_XMLSTORE.setUpdateColumn(insCtx, 'COLUMN2');
I will try to optimise it more now. But 4 seconds is beautiful you see. :-)
Thanks for All the help A_NON and I think I have learnt some Oracle XML DB over here.
Hope to answer some basic questions too in future.. :-)
Regards....

Similar Messages

  • How to avoid performance problems in PL/SQL?

    How to avoid performance problems in PL/SQL?
    As per my knowledge, below some points to avoid performance proble in PL/SQL.
    Is there other point to avoid performance problems?
    1. Use FORALL instead of FOR, and use BULK COLLECT to avoid looping many times.
    2. EXECUTE IMMEDIATE is faster than DBMS_SQL
    3. Use NOCOPY for OUT and IN OUT if the original value need not be retained. Overhead of keeping a copy of OUT is avoided.

    Susil Kumar Nagarajan wrote:
    1. Group no of functions or procedures into a PACKAGEPutting related functions and procedures into packages is useful from a code organization standpoint. It has nothing whatsoever to do with performance.
    2. Good to use collections in place of cursors that do DML operations on large set of recordsBut using SQL is more efficient than using PL/SQL with bulk collects.
    4. Optimize SQL statements if they need to
    -> Avoid using IN, NOT IN conditions or those cause full table scans in queriesThat is not true.
    -> See to queries they use Indexes properly , sometimes Leading index column is missed out that cause performance overheadAssuming "properly" implies that it is entirely possible that a table scan is more efficient than using an index.
    5. use Oracle HINTS if query can't be further tuned and hints can considerably help youHints should be used only as a last resort. It is almost certainly the case that if you can use a hint that forces a particular plan to improve performance that there is some problem in the underlying statistics that should be fixed in order to resolve issues with many queries rather than just the one you're looking at.
    Justin

  • CONNECT BY, Performance problems with hierarchical SQL

    Hello,
    I have a performance problem with the following SQL:
    table name: testtable
    columns: colA, colB, colC, colD, colE, colF
    Following hierarchical SQL:
    SELECT colA||colB||colC AS A1, colD||colE||colF AS B1, colA, colB, colC, level
    FROM testable
    CONNECT BY PRIOR A1 = B1
    START WITH A1 = 'aaa||bbbb||cccc'
    With big tables the performance of this construct is very bad. I can't use functional based indexes to create an Index on "colA||colB||colC" and "colD||colE||colF"
    Has anyone an idea how I can get a real better performance for this hierarchical construct, if I have to combine multiple colums? Or is there any better way (with an PL/SQL or view trigger) solve something like this ??
    Thanks in advance for your investigation :)
    Carmen

    Why not
    CONNECT BY PRIOR colA = colD
    and PRIOR colB = colE
    and ...
    ? It is not the same thing, but I suspect my version is correct:-)

  • SQL Loader need to insert the input filename into output table

    Hi All,
    I've a small problem with my SQL Loader query. My sql loader should read a file and write the data into a file. Also this sql loader should read the filename and write it into the same table. Here's my control file code:
    LOAD DATA
    APPEND
    INTO TABLE XXMW_STG_SOH_HEADER_UK
    WHEN (1:4) = '7010'
    TRAILING NULLCOLS
    (WH POSITION(5:6)
    ,ITEM POSITION(9:26)
    ,PRODUCT_STATUS POSITION(33:34)
    ,BALANCE_ON_HAND POSITION(35:43)
    ,TO_SHIP_QTY POSITION(71:79)
    ,RUSH_TO_SHIP_QTY POSITION(80:88)
    ,RESERVED_QTY POSITION(175:183)
    ,SNAPSHOT_DATE POSITION(134:143) CHAR "TO_DATE(:SNAPSHOT_DATE,'YYYY-MM-DD')"
    ,SNAPSHOT_TIME POSITION(144:151) CHAR "TO_DATE(:SNAPSHOT_TIME,'HH24:MI:SS')"
    ,PROCESSED_IND CONSTANT "N"
    ,PROCESSED_DATETIME SYSDATE
    ,_FILENAME POSITION(184) CHAR TERMINATED BY WHITESPACE_
    My program should read the filename dynamically (means a shell script is calling this .ctl file which is reading multiple input files) and insert into the filename field in xxmw_stg_soh_header_uk table.
    Please let me know for any questions/clarifications.
    Regards,
    Debabrata

    While I think Blu's suggestion to use external tables is better, if you need to use SQL Loader, you could do something like this.
    Create a "generic" control file with a placeholder for the filename, something like:
    LOAD DATA
    APPEND
    INTO TABLE XXMW_STG_SOH_HEADER_UK
    WHEN (1:4) = '7010'
    TRAILING NULLCOLS (
       WH POSITION(5:6),
       ITEM POSITION(9:26),
       PRODUCT_STATUS POSITION(33:34),
       BALANCE_ON_HAND POSITION(35:43),
       TO_SHIP_QTY POSITION(71:79),
       RUSH_TO_SHIP_QTY POSITION(80:88),
       RESERVED_QTY POSITION(175:183),
       SNAPSHOT_DATE POSITION(134:143) CHAR "TO_DATE(:SNAPSHOT_DATE,'YYYY-MM-DD')",
       SNAPSHOT_TIME POSITION(144:151) CHAR "TO_DATE(:SNAPSHOT_TIME,'HH24:MI:SS')",
       PROCESSED_IND CONSTANT "N",
       PROCESSED_DATETIME SYSDATE,
       FILENAME CONSTANT ":FILE"I am assuming that your shell script is looping through a set of file names and loading each one. So make your shell script look something like:
    FILES=`ls *.txt`
    CTL=generic.CTL
    for f in $FILES
    do
       cat $CTL| sed "s/:FILE/$f/g" > $f.ctl
       sqlldr usr/passwd control=$f.ctl data=$f
    doneThe line cat $CTL| sed "s/:FILE/$f/g" > $f.ctl will create a "custom" control file for each file and add the filename as a constant at the end.
    John

  • SQL*Loader error ORA-00600 and ORA-26095

    Hi all,
    We have a problem with the SQL*Loader Utility. We use an 8i-Database (8.1.7) and SQL*Loader 9.2.0.1.0. We have no Trims or other functions active in the control file and do a direct path load.
    It used to work properly until this morning.
    Now we have the following errors (it is in German - but I can translate it):
    Satz 90153: Abgelehnt - Fehler in Tabelle SCHEMA1.T_IMPORT.
    ORA-00600: Interner Fehlercode, Argumente: [klaprs_50], [7], [], [], [], [], [], []
    ... about 50 equal errors
    Specify SKIP=90171 when continuing the load.
    SQL*Loader-926: OCI-Fehler; uldlm:OCIDirPathFinish für Tabelle SCHEMA1.T_IMPORT
    ORA-26095: Nicht verarbeitete Stream-Daten vorhanden
    SQL*Loader-2026: Der Ladevorgang wurde abgebrochen, weil er nicht fortgesetzt werden kann.
    Specify SKIP=90171 when continuing the load.
    Trnaslation:
    00600 - Internal error code, arguments: ...
    OCI-Fehler = OCI error
    für Tabelle = for table
    ORA-26095: Nicht verarbeitete Stream-Daten vorhanden = Non processed stream data existing
    SQL*Loader-2026: Der Ladevorgang wurde abgebrochen, weil er nicht fortgesetzt werden kann. = Load process has been cancelled as it cannot be continued.
    Has anyone got a clue?!?!?!

    Hi,
    It's very clear that import 9i is for an import into 9i database.
    Upgrade 8i to 9i indicates that we can make an export 8i from 8i database and import 9i into 9i database, but never use a tools like export from other version...
    If you want import data from 9i database into 8i use exp tools 8i, and imp 8i.
    I devise you to read Metalink note 132904.1 - compatibility matrix for export & import between different oracl version.
    Nicolas.

  • Sql Loader - Parallel Direct Path Loading

    Hi,
    I want to load a few million records into a table. I read from OTN site that we can make use of Parallel Direct Loading Option. As given in the site, I split my source file into two and tried loading the file from two different sqlldr sessions simultaniously. I used two seperate control files for each session. The sessions started first is getting completed successfully. But the session which started second is giving the following error.
    Will anyone Pls help to sort out the problem.
    Error
    "SQL*Loader-951: Error calling once/load initialization
    ORA-00604: error occurred at recursive SQL level 1
    ORA-00054: resource busy and acquire with NOWAIT specified "
    I'm using oracle 9.0.1.1.1.
    The Options I tried with both the sessions are :
    Direct=true parallel=true.
    loading method is append.
    Thanks in Advance,
    Tom

    I've got a similar problem.
    I'm running Informatica which runs SQL*Loader via OCI.
    With direct & parallel my Indexes give me ORA-26002 - quite understandable.
    As its all wrapped up, I cannot say SKIP_INDEX_MAINTENANCE directly.
    1) Is there a way to give SQL*Loader some Default-Parameters? I read something about a File shrept.lst but I cannot find a reference in the Oracle documentation.
    Something useful down this road?
    2) when I omit parallel, what am I losing? I load one file into one partition of one table. I read the SQL*Loader Documentation but didn't get the message of the 'Intrasegment Parallel Loading' Paragraph. Is this using parallel? Or is it just enabling to use parallel (if there will be a second SQL*Loader)? If its all about 'enabling' I can easily omit this - I know there won't be a second SQL*Loader for this partition.
    regards,
    Kathrin

  • Upload records - SQL loader.

    Dear All,
    We are in the process of getting a big volume data upload project. The data is to be uploaded using SQLLDR in Oracle DB. However the no of records are huge in GB.
    Max volume looks to be for the table: 153 million records x 197 (Max size of all data fields in one record)Unicode characters volume. If I assume 1 byte per Unicode char, it comes to 153,639,503 x 197 = 30,266,982,091 bytes; which is about 26.3 GB.
    Kindly help to answer below queries –
    a. Can you suggest approximately how much time SQLLDR process will take to upload the data of 26.3 GB ?
    b. If it is 4 bytes per Unicode char, it will be 105.2GB. Is there a way to handle file sizes of that volume?
    c. How can we enhance the data upload process for large volume data (performance and tuning)?
    Thanks in advance.
    deb

    882134 wrote:
    We are in the process of getting a big volume data upload project. The data is to be uploaded using SQLLDR in Oracle DB. However the no of records are huge in GB. Relatively small volumes. I am loading over 35 files every 60 seconds. Each CSV file is 21MB in size. 35 files can contain around 4,645,417 rows in total.
    a. Can you suggest approximately how much time SQLLDR process will take to upload the data of 26.3 GB ?How long is a piece of string?
    b. If it is 4 bytes per Unicode char, it will be 105.2GB. Is there a way to handle file sizes of that volume?Not relevant. It is not the size of the data - it is WHAT needs to be done using that data.
    c. How can we enhance the data upload process for large volume data (performance and tuning)?SQL*Loader supports direct and parallel loads. Overheads of the data structure to load into can be reduced by eliminating table constraints, triggers and indexes. Etc.
    Oracle is very capable of dealing with large volumes of data is a performant and scalable fashion. How well it performs and scales depends on how you use Oracle.

  • Insert stmt depends on condiotion using SQL*loader

    Hi everbody
    insert stmt depends on condiotion using SQL*loader
    I have problem regarding to SQL*loader
    i wants to insert data into table depeds on condiotion like
    LOAD DATA
    APPEND
    INTO TABLE TMP
    WHEN (26:33) = '161M1099'
    TMP_FIELD1 CHAR(256),
    TMP_TTYNO CONSTANT'1232:1228'
    INTO TABLE TMP
    WHEN (26:33) = '161P1340'
    TMP_FIELD1 CHAR(256),
    TMP_TTYNO CONSTANT'1232:1228'
    INTO TABLE TMP
    WHEN (26:33) = '161T1001'
    TMP_FIELD1 CHAR(256),
    TMP_TTYNO CONSTANT'1232:1228'
    it is going to 1st condition and insert both tmp_field1 and tmp_ttyno now
    problem occur when its going into 2nd condition
    it is going to insert only constant tmp_ttyno not tmp_field1
    and same in 3rd condition also
    so what to do to insert data from file in tmp_field1
    if any1 knows then please help me.

    You try with external table. A lot of flexibility is there.
    Thanks.

  • Need help to run sql loader from Form6i

    Hi All,
    I have some problem with the sql loader running from form 6i.
    I have done following steps for that.
    1. have created bat file to run sql loader.
    2.Alos created ctl and txt files inside the same folder.
    In the form , in button press i have written below code
    HOST('cmd /c C:\load1.bat > C:\output.txt');
    But it is giving error as -
    "Unable to open Load1.ctl."
    but i have run that bat files from command prompt ; and it is running fine.
    I guess probably it is not getting the correct path.
    Will pls help me, wt exacly I'm doing wrong?
    Rg,
    Subir

    Hi Francois,
    Thanks for reply.
    It is not a problem with the batch files. Coz, I just able to run batch files, but that is not correct solution.
    Once the host command id firing, that time it is going to path -
    "c:\orant\Form60\"
    And in this path it searching for that batch files and all the require files; but the require files are present in some different folder. So i think I need to change the path as well, but that I'm unable to do.
    To resolve the problem, I have copied all the files in "c:\orant\Form60\" folder, and then it is working.
    So I think I need to changed the path, so how to change the path. If I able to change the path I think it will worked. How to change the path?
    Best Rg,
    Subir

  • SQL loader Probelm

    hi, iam using SQL LOADER to insert the but it is giving the following problem ..
    SQL*Loader-510: Physical record in data file (D:\LucyData\18\trialv7.ctl) is longer than the maximum(65536)
    SQL*Loader-2026: the load was aborted because SQL Loader cannot continue.
    thanks in advance
    Edited by: user13340372 on Feb 17, 2011 10:25 PM

    Hi
    Do you get some lines like the below and then getting the error message. if so please check your data your data file . probebly you will have a inconsistant record/s in the file
    Commit point reached - logical record count 59Cheers
    Kanchana.

  • How can I invoke a sql loader from oracle form?

    I'm got a problem on invoking sql loader command from oracle form where the sql loader product has been installed in Window NT.
    Thanks for your help!

    U can use HOST procedure with NO_SCREEN option.
    this will go like
    Host('d:\orant\bin\SQLLDR80 USERID=username/password@connect CONTROL=contril_file.CTL LOG=log_file.LOG discard=discard_file.dis bad=bad_file.bad', NO_SCREEN);
    Hope this will Work
    Good Luck :-)
    Naren

  • Sql loader performance problem with xml

    Hi,
    i have to load a 400 mb big xml file into mz local machine's free oracle db
    i have tested a one record xml and was able to load succesfully, but 400 mb freeying for half an hour and does not even started?
    it is normal? is there any chance i will be able to load it, just need to wait?
    are there any faster solution?
    i ahve created a table below
    CREATE TABLE test_xml
    COL_ID VARCHAR2(1000),
    IN_FILE XMLTYPE
    XMLTYPE IN_FILE STORE AS CLOB
    and control file below
    LOAD DATA
    CHARACTERSET UTF8
    INFILE 'test.xml'
    APPEND
    INTO TABLE product_xml
    col_id filler CHAR (1000),
    in_file LOBFILE(CONSTANT "test.xml") TERMINATED BY EOF
    anything i am doing wrong? thanks for advices

    SQL*Loader: Release 11.2.0.2.0 - Production on H. Febr. 11 18:57:09 2013
    Copyright (c) 1982, 2009, Oracle and/or its affiliates. All rights reserved.
    Control File: prodxml.ctl
    Character Set UTF8 specified for all input.
    Data File: test.xml
    Bad File: test.bad
    Discard File: none specified
    (Allow all discards)
    Number to load: ALL
    Number to skip: 0
    Errors allowed: 5000
    Bind array: 64 rows, maximum of 256000 bytes
    Continuation: none specified
    Path used: Conventional
    Table PRODUCT_XML, loaded from every logical record.
    Insert option in effect for this table: APPEND
    Column Name Position Len Term Encl Datatype
    COL_ID FIRST 1000 CHARACTER
    (FILLER FIELD)
    IN_FILE DERIVED * EOF CHARACTER
    Static LOBFILE. Filename is bv_test.xml
    Character Set UTF8 specified for all input.
    SQL*Loader-605: Non-data dependent ORACLE error occurred -- load discontinued.
    ORA-01652: unable to extend temp segment by 128 in tablespace TEMP
    Table PRODUCT_XML:
    0 Rows successfully loaded.
    0 Rows not loaded due to data errors.
    0 Rows not loaded because all WHEN clauses were failed.
    0 Rows not loaded because all fields were null.
    Space allocated for bind array: 256 bytes(64 rows)
    Read buffer bytes: 1048576
    Total logical records skipped: 0
    Total logical records rejected: 0
    Total logical records discarded: 0
    Run began on H. Febr. 11 18:57:09 2013
    Run ended on H. Febr. 11 19:20:54 2013
    Elapsed time was: 00:23:45.76
    CPU time was: 00:05:05.50
    this is the log
    i have truncated everything i am not able to load 400 mega into 4 giga i cannot understand
    windows is not licensed 32 bit

  • PL/SQL Performance problem

    I am facing a performance problem with my current application (PL/SQL packaged procedure)
    My application takes data from 4 temporary tables, does a lot of validation and
    puts them into permanent tables.(updates if present else inserts)
    One of the temporary tables is parent table and can have 0 or more rows in
    the other tables.
    I have analyzed all my tables and indexes and checked all my SQLs
    They all seem to be using the indexes correctly.
    There are 1.6 million records combined in all 4 tables.
    I am using Oracle 8i.
    How do I determine what is causing the problem and which part is taking time.
    Please help.
    The skeleton of the code which we have written looks like this
    MAIN LOOP ( 255308 records)-- Parent temporary table
    -----lots of validation-----
    update permanent_table1
    if sql%rowcount = 0 then
    insert into permanent_table1
    Loop2 (0-5 records)-- child temporary table1
    -----lots of validation-----
    update permanent_table2
    if sql%rowcount = 0 then
    insert into permanent_table2
    end loop2
    Loop3 (0-5 records)-- child temporary table2
    -----lots of validation-----
    update permanent_table3
    if sql%rowcount = 0 then
    insert into permanent_table3
    end loop3
    Loop4 (0-5 records)-- child temporary table3
    -----lots of validation-----
    update permanent_table4
    if sql%rowcount = 0 then
    insert into permanent_table4
    end loop4
    -- COMMIT after every 3000 records
    END MAIN LOOP
    Thanks
    Ashwin N.

    Do this intead of ditching the PL/SQL.
    DECLARE
    TYPE NumTab IS TABLE OF NUMBER(4) INDEX BY BINARY_INTEGER;
    TYPE NameTab IS TABLE OF CHAR(15) INDEX BY BINARY_INTEGER;
    pnums NumTab;
    pnames NameTab;
    t1 NUMBER(5);
    t2 NUMBER(5);
    t3 NUMBER(5);
    BEGIN
    FOR j IN 1..5000 LOOP -- load index-by tables
    pnums(j) := j;
    pnames(j) := 'Part No. ' || TO_CHAR(j);
    END LOOP;
    t1 := dbms_utility.get_time;
    FOR i IN 1..5000 LOOP -- use FOR loop
    INSERT INTO parts VALUES (pnums(i), pnames(i));
    END LOOP;
    t2 := dbms_utility.get_time;
    FORALL i IN 1..5000 -- use FORALL statement
    INSERT INTO parts VALUES (pnums(i), pnames(i));
    get_time(t3);
    dbms_output.put_line('Execution Time (secs)');
    dbms_output.put_line('---------------------');
    dbms_output.put_line('FOR loop: ' || TO_CHAR(t2 - t1));
    dbms_output.put_line('FORALL: ' || TO_CHAR(t3 - t2));
    END;
    Try this link, http://download-west.oracle.com/docs/cd/B10501_01/appdev.920/a96624/05_colls.htm#23723

  • Performance problems loading an XML file into oracle database

    Hello ODI Guru's,
    I am trying to load and XML file into the database after doing simple business validations. But the interface takes hours to complete.
    1. The XML files are large in size >200 Mb. We have an XSD file for the schema definition instead of a DTD.
    2. We used the external database feature for loading these files in database.
    The following configuration was used in the XML Data Server:
    jdbc:snps:xml?f=D:\CustomerMasterData1\CustomerMasterInitialLoad1.xml&d=D:\CustomerMasterData1\CustomerMasterInitialLoad1.xsd&re=initialLoad&s=CM&db_props=oracle&ro=true
    3. Now we reverse engineer the XML files and created models using ODI Designer
    4. Similar thing was done for the target i.e. an Oracle database table as well.
    5. Next we created a simple interface with one-to-one mapping from the XSD schema to the Oracle database table and executed the interface. This execution takes more than one hour to complete.
    6. We are running ODI client on Windows XP Professional SP2.
    7. The Oracle database server(Oracle 10g 10.2.0.3) for the target schema as well as the ODI master and work repositories are on the same machine.
    8. I tried changing the following properties but it is not making much visible difference:
    use_prepared_statements=Y
    use_batch_update=Y
    batch_update_size=510
    commit_periodically=Y
    num_inserts_before_commit=30000
    I have another problem that when I set batch_update_size to value greater that 510 I get the following error:
    java.sql.SQLException: class org.xml.sax.SAXException
    class java.lang.ArrayIndexOutOfBoundsException said -32413
    at com.sunopsis.jdbc.driver.xml.v.a(v.java)
    The main concern is why should the interface taking so long to execute.
    Please send suggestions to resolve the problem.
    Thanks in advance,
    Best Regards,
    Nikunj

    Approximately how many rows are you trying to insert?
    One of the techniques which I found improved performance for this scenario was to extract from the xml to a flat file, then to use SQL*LOADER or external tables to load the data into Oracle.

  • Critical performance problem upon bulk load of groups

    All (including product development),
    I think there are missing indexes in wwsec_flat$ and wwsec_sys_priv$. Anyway, I'd like assistance on fixing the critical performance problems I see, properly. Read on...
    During and after bulk load of a few (about 500) users and groups from an external database, it becomes evident that there's a performance problem somewhere. Many of the calls to wwsec_api.addGroupToList took several minutes to finish. Afterwards the machine went 100% CPU just from logging in with the portal30 user (which happens to be group owner for all the groups).
    Running SQL trace points in the directions of the following SQL statement:
    SELECT ID,PARENT_ID,NAME,TITLE_ID,TITLEIMAGE_ID,ROLLOVERIMAGE_ID,
    DESCRIPTION_ID,LAYOUT_ID,STYLE_ID,PAGE_TYPE,CREATED_BY,CREATED_ON,
    LAST_MODIFIED_BY,LAST_MODIFIED_ON,PUBLISHED_ON,HAS_BANNER,HAS_FOOTER,
    EXPOSURE,SHOW_CHILDREN,IS_PUBLIC,INHERIT_PRIV,IS_READY,EXECUTE_MODE,
    CACHE_MODE,CACHE_EXPIRES,TEMPLATE FROM
    WWPOB_PAGE$ WHERE ID = :b1
    I checked the existing indexes, and see that the following ones are missing (I'm about to test with these, but have not yet done so):
    CREATE UNIQUE INDEX "PORTAL30"."WWSEC_FLAT_IX_GROUP_ID"
    ON "PORTAL30"."WWSEC_FLAT$"("GROUP_ID")
    TABLESPACE "PORTAL" PCTFREE 10 INITRANS 2 MAXTRANS 255
    STORAGE ( INITIAL 32K NEXT 160K MINEXTENTS 1 MAXEXTENTS 4096
    PCTINCREASE 1 FREELISTS 1)
    LOGGING
    CREATE UNIQUE INDEX "PORTAL30"."WWSEC_FLAT_IX_PERSON_ID"
    ON "PORTAL30"."WWSEC_FLAT$"("PERSON_ID")
    TABLESPACE "PORTAL" PCTFREE 10 INITRANS 2 MAXTRANS 255
    STORAGE ( INITIAL 32K NEXT 160K MINEXTENTS 1 MAXEXTENTS 4096
    PCTINCREASE 1 FREELISTS 1)
    LOGGING
    CREATE UNIQUE INDEX "PORTAL30"."WWSEC_SYS_PRIV_IX_PATCH1"
    ON "PORTAL30"."WWSEC_SYS_PRIV$"("OWNER", "GRANTEE_GROUP_ID",
    "GRANTEE_TYPE", "OWNER", "NAME", "OBJECT_TYPE_NAME")
    TABLESPACE "PORTAL" PCTFREE 10 INITRANS 2 MAXTRANS 255
    STORAGE ( INITIAL 32K NEXT 80K MINEXTENTS 1 MAXEXTENTS 4096
    PCTINCREASE 1 FREELISTS 1)
    LOGGING
    Note that when I deleted the newly inserted groups, the CPU consumption immediately went down from 100% to some 2-3%.
    This behaviour has been observed on a Sun Solaris system, but I think it's the same on NT (I have observed it during the bulk load on my NT laptop, but so far have not had the time to test further.).
    Also note: In the call to addGroupToList, I set owner to true for all groups.
    Also note: During loading of the groups, I logged a few errors, all of the same type ("PORTAL30.WWSEC_API", line 2075), as follows:
    Error: Problem calling addGroupToList for child group'Marketing' (8030), list 'NO_OSL_Usenet'(8017). Reason: java.sql.SQLException: ORA-06510: PL/SQL: unhandled user-defined exception ORA-06512: at "PORTAL30.WWSEC_API", line 2075
    Please help. If you like, I may supply the tables and the java program that I use. It's fully reproducable.
    Thanks,
    Erik Hagen (you may call me on +47 90631013)
    null

    YES!
    I have now tested with insertion of the missing indexes. It seems the call to addGroupToList takes just as long time as before, but the result is much better: WITH THE INDEXES DEFINED, THERE IS NO LONGER A PERFORMANCE PROBLEM!! The index definitions that I used are listed below (I added these to the ones that are there in Portal 3.0.8, but I guess some of those could have been deleted).
    About the info at http://technet.oracle.com:89/ubb/Forum70/HTML/000894.html: Yes! Thanks! Very interesting, and I guess you found the cause for the error messages and maybe also for the performance problem during bulk load (I'll look into it as soon as possible anbd report what I find.).
    Note: I have made a pretty foolproof and automated installation script (or actually, it's part of my Java program), that will let anybody interested recreate the problem. Mail your interest to [email protected].
    ============================================
    CREATE INDEX "PORTAL30"."LDAP_WWSEC_PERS_IX1"
    ON "PORTAL30"."WWSEC_PERSON$"("MANAGER")
    TABLESPACE "PORTAL" PCTFREE 10 INITRANS 2 MAXTRANS 255
    STORAGE ( INITIAL 32K NEXT 32K MINEXTENTS 1 MAXEXTENTS 4096
    PCTINCREASE 1 FREELISTS 1)
    LOGGING;
    CREATE INDEX PORTAL30.LDAP_WWSEC_PERS_IX2
    ON PORTAL30.WWSEC_PERSON$('ORGANIZATION')
    TABLESPACE PORTAL PCTFREE 10 INITRANS 2 MAXTRANS 255
    STORAGE ( INITIAL 32K NEXT 32K MINEXTENTS 1 MAXEXTENTS 4096
    PCTINCREASE 1 FREELISTS 1)
    LOGGING;
    CREATE INDEX PORTAL30.LDAP_WWSEC_PERS_PK
    ON PORTAL30.WWSEC_PERSON$('ID')
    TABLESPACE PORTAL PCTFREE 10 INITRANS 2 MAXTRANS 255
    STORAGE ( INITIAL 32K NEXT 32K MINEXTENTS 1 MAXEXTENTS 4096
    PCTINCREASE 1 FREELISTS 1)
    LOGGING;
    CREATE INDEX PORTAL30.LDAP_WWSEC_PERS_UK
    ON PORTAL30.WWSEC_PERSON$('USER_NAME')
    TABLESPACE PORTAL PCTFREE 10 INITRANS 2 MAXTRANS 255
    STORAGE ( INITIAL 32K NEXT 32K MINEXTENTS 1 MAXEXTENTS 4096
    PCTINCREASE 1 FREELISTS 1)
    LOGGING;
    CREATE INDEX PORTAL30.LDAP_WWWSEC_FLAT_UK
    ON PORTAL30.WWSEC_FLAT$("GROUP_ID", "PERSON_ID",
    "SPONSORING_MEMBER_ID")
    TABLESPACE PORTAL PCTFREE 10 INITRANS 2 MAXTRANS 255
    STORAGE ( INITIAL 32K NEXT 256K MINEXTENTS 1 MAXEXTENTS 4096
    PCTINCREASE 0 FREELISTS 1);
    CREATE INDEX PORTAL30.LDAP_WWWSEC_FLAT_PK
    ON PORTAL30.WWSEC_FLAT$("ID")
    TABLESPACE PORTAL PCTFREE 10 INITRANS 2 MAXTRANS 255
    STORAGE ( INITIAL 32K NEXT 256K MINEXTENTS 1 MAXEXTENTS 4096
    PCTINCREASE 0 FREELISTS 1);
    CREATE INDEX PORTAL30.LDAP_WWWSEC_FLAT_IX5
    ON PORTAL30.WWSEC_FLAT$("GROUP_ID", "PERSON_ID")
    TABLESPACE PORTAL PCTFREE 10 INITRANS 2 MAXTRANS 255
    STORAGE ( INITIAL 32K NEXT 256K MINEXTENTS 1 MAXEXTENTS 4096
    PCTINCREASE 0 FREELISTS 1);
    CREATE INDEX PORTAL30.LDAP_WWWSEC_FLAT_IX4
    ON PORTAL30.WWSEC_FLAT$("SPONSORING_MEMBER_ID")
    TABLESPACE PORTAL PCTFREE 10 INITRANS 2 MAXTRANS 255
    STORAGE ( INITIAL 32K NEXT 256K MINEXTENTS 1 MAXEXTENTS 4096
    PCTINCREASE 0 FREELISTS 1);
    CREATE INDEX PORTAL30.LDAP_WWWSEC_FLAT_IX3
    ON PORTAL30.WWSEC_FLAT$("GROUP_ID")
    TABLESPACE PORTAL PCTFREE 10 INITRANS 2 MAXTRANS 255
    STORAGE ( INITIAL 32K NEXT 256K MINEXTENTS 1 MAXEXTENTS 4096
    PCTINCREASE 0 FREELISTS 1);
    CREATE INDEX PORTAL30.LDAP_WWWSEC_FLAT_IX2
    ON PORTAL30.WWSEC_FLAT$("PERSON_ID")
    TABLESPACE PORTAL PCTFREE 10 INITRANS 2 MAXTRANS 255
    STORAGE ( INITIAL 32K NEXT 256K MINEXTENTS 1 MAXEXTENTS 4096
    PCTINCREASE 0 FREELISTS 1);
    CREATE INDEX "PORTAL30"."LDAP_WWSEC_SYSP_IX1"
    ON "PORTAL30"."WWSEC_SYS_PRIV$"("GRANTEE_GROUP_ID")
    TABLESPACE "PORTAL" PCTFREE 10 INITRANS 2 MAXTRANS 255
    STORAGE ( INITIAL 32K NEXT 56K MINEXTENTS 1 MAXEXTENTS 4096
    PCTINCREASE 1 FREELISTS 1)
    LOGGING;
    CREATE INDEX "PORTAL30"."LDAP_WWSEC_SYSP_IX2"
    ON "PORTAL30"."WWSEC_SYS_PRIV$"("GRANTEE_USER_ID")
    TABLESPACE "PORTAL" PCTFREE 10 INITRANS 2 MAXTRANS 255
    STORAGE ( INITIAL 32K NEXT 56K MINEXTENTS 1 MAXEXTENTS 4096
    PCTINCREASE 1 FREELISTS 1)
    LOGGING;
    CREATE INDEX "PORTAL30"."LDAP_WWSEC_SYSP_IX3"
    ON "PORTAL30"."WWSEC_SYS_PRIV$"("OBJECT_TYPE_NAME", "NAME")
    TABLESPACE "PORTAL" PCTFREE 10 INITRANS 2 MAXTRANS 255
    STORAGE ( INITIAL 32K NEXT 56K MINEXTENTS 1 MAXEXTENTS 4096
    PCTINCREASE 1 FREELISTS 1)
    LOGGING;
    CREATE INDEX "PORTAL30"."LDAP_WWSEC_SYSP_PK"
    ON "PORTAL30"."WWSEC_SYS_PRIV$"("ID")
    TABLESPACE "PORTAL" PCTFREE 10 INITRANS 2 MAXTRANS 255
    STORAGE ( INITIAL 32K NEXT 56K MINEXTENTS 1 MAXEXTENTS 4096
    PCTINCREASE 1 FREELISTS 1)
    LOGGING;
    CREATE INDEX "PORTAL30"."LDAP_WWSEC_SYSP_UK"
    ON "PORTAL30"."WWSEC_SYS_PRIV$"("OBJECT_TYPE_NAME",
    "NAME", "OWNER", "GRANTEE_TYPE", "GRANTEE_GROUP_ID",
    "GRANTEE_USER_ID")
    TABLESPACE "PORTAL" PCTFREE 10 INITRANS 2 MAXTRANS 255
    STORAGE ( INITIAL 32K NEXT 88K MINEXTENTS 1 MAXEXTENTS 4096
    PCTINCREASE 1 FREELISTS 1)
    LOGGING;
    ==================================
    Thanks,
    Erik Hagen
    null

Maybe you are looking for

  • Why does an object collide with itself?

    I'm getting an unusual condition where, after using WakeupOnCollisionEnter(Node node) calls back my program with the triggering path, I am getting objects that collide with themselves. Why is an object colliding with itself?

  • Disjointed Rollovers in FireFox

    I was practicing disjointed rollovers and driving myself crazy. They just wouldn't work in the browser preview (I use FF as my browser) Finally I tried to preview in IE 6 and it worked fine as it did in Opera 8 and Netscape 8. Is there something stan

  • Archiving part of my whole library!

    I want to archive off only part of my iPhoto library and leave part of it on my machine. What is the best procedure for doing this? (I want to eliminate a couple of earlier years worth...) I know how to burn a searchable iPhoto disc. Im just afraid t

  • Why my itunes can't link to the itunes store?  my internet is functional.

    why my itunes can't link to the itunes store?  my internet is functional.

  • How to read Word Document in JAVA

    I use POI for read an write Word Document, but i have a problem of a compatibility (i use Word 2003 & the api poi supportes Word97). Can you help me please !!!