Sql loader performance problem with xml

Hi,
i have to load a 400 mb big xml file into mz local machine's free oracle db
i have tested a one record xml and was able to load succesfully, but 400 mb freeying for half an hour and does not even started?
it is normal? is there any chance i will be able to load it, just need to wait?
are there any faster solution?
i ahve created a table below
CREATE TABLE test_xml
COL_ID VARCHAR2(1000),
IN_FILE XMLTYPE
XMLTYPE IN_FILE STORE AS CLOB
and control file below
LOAD DATA
CHARACTERSET UTF8
INFILE 'test.xml'
APPEND
INTO TABLE product_xml
col_id filler CHAR (1000),
in_file LOBFILE(CONSTANT "test.xml") TERMINATED BY EOF
anything i am doing wrong? thanks for advices

SQL*Loader: Release 11.2.0.2.0 - Production on H. Febr. 11 18:57:09 2013
Copyright (c) 1982, 2009, Oracle and/or its affiliates. All rights reserved.
Control File: prodxml.ctl
Character Set UTF8 specified for all input.
Data File: test.xml
Bad File: test.bad
Discard File: none specified
(Allow all discards)
Number to load: ALL
Number to skip: 0
Errors allowed: 5000
Bind array: 64 rows, maximum of 256000 bytes
Continuation: none specified
Path used: Conventional
Table PRODUCT_XML, loaded from every logical record.
Insert option in effect for this table: APPEND
Column Name Position Len Term Encl Datatype
COL_ID FIRST 1000 CHARACTER
(FILLER FIELD)
IN_FILE DERIVED * EOF CHARACTER
Static LOBFILE. Filename is bv_test.xml
Character Set UTF8 specified for all input.
SQL*Loader-605: Non-data dependent ORACLE error occurred -- load discontinued.
ORA-01652: unable to extend temp segment by 128 in tablespace TEMP
Table PRODUCT_XML:
0 Rows successfully loaded.
0 Rows not loaded due to data errors.
0 Rows not loaded because all WHEN clauses were failed.
0 Rows not loaded because all fields were null.
Space allocated for bind array: 256 bytes(64 rows)
Read buffer bytes: 1048576
Total logical records skipped: 0
Total logical records rejected: 0
Total logical records discarded: 0
Run began on H. Febr. 11 18:57:09 2013
Run ended on H. Febr. 11 19:20:54 2013
Elapsed time was: 00:23:45.76
CPU time was: 00:05:05.50
this is the log
i have truncated everything i am not able to load 400 mega into 4 giga i cannot understand
windows is not licensed 32 bit

Similar Messages

SQL*Loader importing problem, with file with eastern european files

Hello,
on Oracle 11g with UTF-8 encoding, I tried to import a csv file into a table via sqlload, the separator is the semicolon ";" all work fine except for some lines witch are not well integrated (the concerned files come from Eastern European countries like Bulgary, Hungary and Czech Republic).
For example:
For:
text_1; text_2; text_with_char_at_end_like_š; new_text
during the integration instead of have:
| text_1 | text_2 | text_with_char_at_end_like_š| new_text |
I got:
| text_1 | text_2 | text_with_char_at_end_like_š; new_text | null |
does anyone has this problem, I tried to change the delimiter by code X'59', specified in sqlldr ENCODING UTF8 ... but it does not work
do you have an idea about this problem
Thank you in advance

Thanks,
the problem was solved since, the file was not in UTF8 format (for example GREEK FORMAT) and the NLS_LANG was AMERICAN_AMERICA ASCII
then i translate all files to UTF 8 and changed the NLS_LANG to UtF8.
Regards

CONNECT BY, Performance problems with hierarchical SQL

Hello,
I have a performance problem with the following SQL:
table name: testtable
columns: colA, colB, colC, colD, colE, colF
Following hierarchical SQL:
SELECT colA||colB||colC AS A1, colD||colE||colF AS B1, colA, colB, colC, level
FROM testable
CONNECT BY PRIOR A1 = B1
START WITH A1 = 'aaa||bbbb||cccc'
With big tables the performance of this construct is very bad. I can't use functional based indexes to create an Index on "colA||colB||colC" and "colD||colE||colF"
Has anyone an idea how I can get a real better performance for this hierarchical construct, if I have to combine multiple colums? Or is there any better way (with an PL/SQL or view trigger) solve something like this ??
Thanks in advance for your investigation :)
Carmen

Why not
CONNECT BY PRIOR colA = colD
and PRIOR colB = colE
and ...
? It is not the same thing, but I suspect my version is correct:-)

SQL*Loader : ORA-19007 with missing Schema Location Hint

Hi,
These days I'm stuck with a 10.1.0.4 database (OS Linux Red Hat), and I'm trying to find a workaround for the following situation :
XML schema (sfgtest.xsd)
<?xml version="1.0" encoding="UTF-8"?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema"
           xmlns:xdb="http://xmlns.oracle.com/xdb">
<xs:element name="Code" xdb:SQLName="Code" xdb:SQLType="VARCHAR2">
    <xs:simpleType>
      <xs:restriction base="xs:string">
        <xs:maxLength value="16"/>
      </xs:restriction>
    </xs:simpleType>
</xs:element>
<xs:element name="Val" xdb:SQLType="NUMBER" xdb:SQLName="Val">
    <xs:simpleType>
      <xs:restriction base="xs:decimal">
        <xs:totalDigits value="12"/>
      </xs:restriction>
    </xs:simpleType>
</xs:element>
<xs:element name="Chq">
    <xs:complexType xdb:SQLType="SFG_CHQ_TYPE">
      <xs:sequence>
        <xs:element ref="Code"/>
        <xs:element ref="Val"/>
      </xs:sequence>
    </xs:complexType>
</xs:element>
<xs:element name="NbRem" xdb:SQLType="NUMBER" xdb:SQLName="NbRem">
    <xs:simpleType>
      <xs:restriction base="xs:integer">
        <xs:totalDigits value="5"/>
      </xs:restriction>
    </xs:simpleType>
</xs:element>
<xs:element name="Rmbt" xdb:defaultTable="TEST_XML_SFG">
    <xs:complexType xdb:SQLType="SFG_RMBT_TYPE">
      <xs:sequence>
        <xs:element ref="NbRem"/>
        <xs:element ref="Rem" maxOccurs="unbounded" xdb:SQLCollType="SFG_REM_COLL"/>
      </xs:sequence>
    </xs:complexType>
</xs:element>
<xs:element name="CodeAff" xdb:SQLName="CodeAff" xdb:SQLType="VARCHAR2">
    <xs:simpleType>
      <xs:restriction base="xs:string">
        <xs:maxLength value="12"/>
      </xs:restriction>
    </xs:simpleType>
</xs:element>
<xs:element name="Rem" xdb:SQLName="Rem">
    <xs:complexType xdb:SQLType="SFG_REM_TYPE">
      <xs:sequence>
        <xs:element ref="CodeAff"/>
        <xs:element ref="Chq" maxOccurs="unbounded" xdb:SQLCollType="SFG_CHQ_COLL"/>
      </xs:sequence>
    </xs:complexType>
</xs:element>
</xs:schema>
Sample document (sfgtest.xml)
<?xml version="1.0" encoding="iso-8859-1"?>
<Rmbt>
<NbRem>3</NbRem>
<Rem>
<CodeAff>AFF001</CodeAff>
<Chq><Code>X01001</Code><Val>10.00</Val></Chq>
<Chq><Code>X01002</Code><Val>10.00</Val></Chq>
<Chq><Code>X01003</Code><Val>10.00</Val></Chq>
</Rem>
<Rem>
<CodeAff>AFF002</CodeAff>
<Chq><Code>X02001</Code><Val>10.00</Val></Chq>
<Chq><Code>X02002</Code><Val>10.00</Val></Chq>
<Chq><Code>X02003</Code><Val>10.00</Val></Chq>
</Rem>
<Rem>
<CodeAff>AFF003</CodeAff>
<Chq><Code>X03001</Code><Val>10.00</Val></Chq>
<Chq><Code>X03002</Code><Val>10.00</Val></Chq>
<Chq><Code>X03003</Code><Val>10.00</Val></Chq>
<Chq><Code>X03004</Code><Val>10.00</Val></Chq>
<Chq><Code>X03005</Code><Val>10.00</Val></Chq>
</Rem>
</Rmbt>
Schema registration
begin
dbms_xmlschema.registerSchema(
schemaURL => 'sfgtest.xsd',
schemaDoc => xmltype(bfilename('DUMP_DIR','sfgtest.xsd'),nls_charset_id('AL32UTF8')),
local => true,
genTypes => true,
genTables => false
end;
Table creation
create table test_xml_sfg of xmltype
xmltype store as object relational
xmlschema "sfgtest.xsd"
element "Rmbt"
varray xmldata."Rem" store as table test_xml_sfg_rem_tab
( primary key (NESTED_TABLE_ID, ARRAY_INDEX) ) organization index overflow
varray "Chq" store as table test_xml_sfg_chq_tab
( primary key (NESTED_TABLE_ID, ARRAY_INDEX) ) organization index overflow
);I originally wanted to use SQL*Loader to test performance, as I may have to load multiple files in the same time.
It works great on 10gR2 and 11gR2 with XML files up to 100 MB loaded in a matter of minutes.
However, a little issue with 10gR1 :
SQLLDR control file
LOAD DATA
INFILE 'filelist.txt'
APPEND
INTO TABLE test_xml_sfg
XMLTYPE(XMLDATA) (
filename filler char(260),
XMLDATA LOBFILE(filename) TERMINATED BY EOF
)with filelist.txt :
sfgtest.xmlExecution throws "ORA-19007: Schema - does not match expected sfgtest.xsd".
As per the documentation, it's expected behaviour in 10.1 :
http://download.oracle.com/docs/cd/B14117_01/appdev.101/b10790/xdb03usg.htm#BABECCBG
The XML document must include the appropriate attributes from the XMLSchema-instance namespace or the XML document must be explicitly associated with the XML schema using the XMLType constructor or the createSchemaBasedXML() method.Also as expected, the file is loaded OK after adding the following in the root element :
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="sfgtest.xsd"But, as I don't have any latitude on the files received (i.e. I can't add the location hint), is there some workaround, when using SQL*Loader, to treat the XML file as an instance document?
Thanks for any solution/idea.

Not sure if it will work, but out of the top of my head I would make an attempt to create an view (for update) based on the XMLType table and do the insert via the view, matching the incoming XML doc to the schema via
xmltype(xml_messsage).createSchemaBasedXML(xml_schema)

Numbers Import and Load Performance Problems

Some initial results of converting a single 1.9MB Excel spreadsheet to Numbers:
_Results using Numbers v1.0_
Import 1.9MB Excel spreadsheet into Numbers: 7 minutes 3.5 seconds
Load (saved) Numbers spreadsheet (2.4MB): 5 minutes 11.7 seconds
_Results using Numbers v1.0.1_
Import 1.9MB Excel spreadsheet into Numbers: 6 minutes 36.1 seconds
Load (saved) Numbers spreadsheet (2.4MB): 5 minutes 5.8 seconds
_Comparison to Excel_
Excel loads the original 1.9MB spreadsheet in 4.2 seconds.
Summary
Numbers v1.0 and v1.0.1 exhibit severe performance problems with loading (of it's own files) and importing of Excel V.x files.

Hello
It seems that you missed a detail.
When a Numbers document is 1.9MB on disk, it may be a 7 or 8 MB file to load.
A Numbers document s not a file but a package which is a disguised folder.
The document itself is described in an WML extremely verbose file stored in a gzip archive.
Opening such a document starts with an unpack sequence which is a fast one (except maybe if the space available on the support is short).
The unpacked file may easily be 10 times larger than the packed one.
Just an example, the xml.gz file containing the report of my bank operations for 2007 is a 300Kb one but the expanded one, the one which Numers must read, is a 4 MB one, yes 13,3 times the original.
And, loading it is not sufficient, this huge file must be "interpreted" to build the display.
As it is very long, Apple treats it as the TRUE description of the document and so, each time it must display something, it must work as the interpreters that old users like me knew when they used the Basic available in Apple // machines.
Addind a supplemetary stage would have add time to the opening sequence but would have fasten the usage of the document.
Of course, it would also had added a supplementary stage duringthe save it process.
I hope that they will adopt this scheme but of course I don't know if they will do that.
Of course, the problem is quite the same when we import a document from Excel or from AppleWorks.
The app reads the original which is stored in a compact shape then it deciphers it to create the XML code. Optimisation would perhaps reduce a bit these tasks but it will continue to be a time consuming one.
Yvan KOENIG (from FRANCE dimanche 27 janvier 2008 16:46:12)

Problem with XML in APEX ORA-06502

i, I have a problem with XML generation, I developed an application in APEX, and in a html page I have this process:
declare
l_XML varchar2(32767);
begin
select xmlElement
"iva",
xmlElement("numeroRuc",J.RUC),
xmlElement("razonSocial", J.RAZON_SOCIAL),
xmlElement("idRepre", J.ID_REPRE),
xmlElement("rucContador", J.RUC_CONTADOR),
xmlElement("anio", J.ANIO),
xmlElement("mes", J.MES),
xmlElement
"compras",
select xmlAgg
xmlElement
"detalleCompra",
--xmlAttributes(K.ID_COMPRA as "COMPRA"),
xmlForest
K.COD_SUSTENTO as "codSustento",
K.TPLD_PROV as "tpldProv",
K.ID_PROV as "idProv",
K.TIPO_COMPROBANTE as "tipoComprobante",
to_char(K.FECHA_REGISTRO, 'DD/MM/YYYY') as "fechaRegistro",
K.ESTABLECIMIENTO as "establecimiento",
K.PUNTO_EMISION as "puntoEmision",
K.SECUENCIAL as "secuencial",
to_char(K.FECHA_EMISION, 'DD/MM/YYYY') as "fechaEmision",
K.AUTORIZACION as "autorizacion",
to_char(K.BASE_NO_GRA_IVA, 9999999999.99) as "baseNoGraIva",
to_char(K.BASE_IMPONIBLE, 9999999999.99) as "baseImponible",
to_char(K.BASE_IMP_GRAV, 9999999999.99) as "baseImpGrav",
to_char(K.MONTO_ICE, 9999999999.99) as "montoIce",
to_char(K.MONTO_IVA, 9999999999.99) as "montoIva",
to_char(K.VALOR_RET_BIENES, 9999999999.99) as "valorRetBienes",
to_char(K.VALOR_RET_SERVICIOS, 9999999999.99) as "valorRetServicios",
to_char(K.VALOR_RET_SERV_100, 9999999999.99) as "valorRetServ100"
xmlElement
"air",
select xmlAgg
xmlElement
"detalleAir",
xmlForest
P.COD_RET_AIR as "codRetAir",
to_char(P.BASE_IMP_AIR, 9999999999.99) as "baseImpAir",
to_char(P.PORCENTAJE_AIR, 999.99) as "porcentajeAir",
to_char(P.VAL_RET_AIR, 9999999999.99) as "valRetAir"
from ANEXO_COMPRAS P
where P.ID_COMPRA = K.ID_COMPRA
AND P.ID_INFORMANTE_XML = K.ID_INFORMANTE_XML
xmlElement("estabRetencion1", K.ESTAB_RETENCION_1),
xmlElement("ptoEmiRetencion1", K.PTO_EMI_RETENCION_1),
xmlElement("secRetencion1", K.SEC_RETENCION_1),
xmlElement("autRetencion1", K.AUT_RETENCION_1),
xmlElement("fechaEmiRet1", to_char(K.FECHA_EMI_RET_1,'DD/MM/YYYY')),
xmlElement("docModificado", K.DOC_MODIFICADO),
xmlElement("estabModificado", K.ESTAB_MODIFICADO),
xmlElement("ptoEmiModificado", K.PTO_EMI_MODIFICADO),
xmlElement("secModificado", K.SEC_MODIFICADO),
xmlElement("autModificado", K.AUT_MODIFICADO)
from SRI_COMPRAS K
WHERE K.ID IS NOT NULL
AND K.ID_INFORMANTE_XML = J.ID_INFORMANTE
AND K.ID BETWEEN 1 AND 25
).getClobVal()
into l_XML
from ANEXO_INFORMANTE J
where J.ID_INFORMANTE =:P3_MES
and J.RUC =:P3_ID_RUC
and J.ANIO =:P3_ANIO
and J.MES =:P3_MES;
--HTML
sys.owa_util.mime_header('text/xml',FALSE);
sys.htp.p('Content-Length: ' || length(l_XML));
sys.owa_util.http_header_close;
sys.htp.print(l_XML);
end;
Now my table has more than 900 rows and only when I specifically selected 25 rows of the table "ANEXO_COMPRAS" in the where ( AND K.ID BETWEEN 1 AND 25) the XML is generated.+
I think that the problem may be the data type declared "varchar2", but I was trying with the data type "CLOB" and the error is the same.+
declare
l_XML CLOB;
begin
--Oculta XML
sys.htp.init;
wwv_flow.g_page_text_generated := true;
wwv_flow.g_unrecoverable_error := true;
--select XML
select xmlElement
from SRI_COMPRAS K
WHERE K.ID IS NOT NULL
AND K.ID_INFORMANTE_XML = J.ID_INFORMANTE
).getClobVal()
into l_XML
from ANEXO_INFORMANTE J
where J.ID_INFORMANTE =:P3_MES
and J.RUC =:P3_ID_RUC
and J.ANIO =:P3_ANIO
and J.MES =:P3_MES;
--HTML
sys.owa_util.mime_header('text/xml',FALSE);
sys.htp.p('Content-Length: ' || length(l_XML));
sys.owa_util.http_header_close;
sys.htp.print(l_XML);
end;
The error generated is ORA-06502: PL/SQL: numeric or value error+_
Please I need your help. I don`t know how to resolve this problem, how to use the data type "CLOB" for the XML can be generate+

JohannaCevallos07 wrote:
Now my table has more than 900 rows and only when I specifically selected 25 rows of the table "ANEXO_COMPRAS" in the where ( AND K.ID BETWEEN 1 AND 25) the XML is generated.+
I think that the problem may be the data type declared "varchar2", but I was trying with the data type "CLOB" and the error is the same.+
The error generated is ORA-06502: PL/SQL: numeric or value error+_
Please I need your help. I don`t know how to resolve this problem, how to use the data type "CLOB" for the XML can be generate+The likeliest explanation for this is that length of the XML exceeds 32K, which is the maximum size that <tt>htp.p</tt> can output. A CLOB can store much more than this, so it's necessary to buffer the output as shown in +{message:id=4497571}+
Help us to help you. When you have a problem include as much relevant information as possible upfront. This should include:
<li>Full APEX version
<li>Full DB/version/edition/host OS
<li>Web server architecture (EPG, OHS or APEX listener/host OS)
<li>Browser(s) and version(s) used
<li>Theme
<li>Template(s)
<li>Region/item type(s) (making particular distinction as to whether a "report" is a standard report, an interactive report, or in fact an "updateable report" (i.e. a tabular form)
And always post code wrapped in <tt>\...\</tt> tags, as described in the FAQ.
Thanks

Performance problem with ojdbc14.jar

Hi,
We are having performance problem with ojdbc14.jar in selecting and updating (batch updates) entries in a table. The queries are taking minutes to execute. The same java code works fine with classes12.zip ans queries taking sub seconds to execute.
We have Oracle 9.2.0.5 Database Server and I have downloaded the ojdbc14.jar from Oracle site for the same. Tried executing the java code from windows 2000, Sun Solaris and Opteron machines and having the same problem.
Does any one know a solution to this problem? I also tried ojdbc14.jar meant for Oracle 10g, that did not help.
Please help.
Thanks
Yuva

My code is doing some thing which might be working well with classes12.zip and which does not work well with ojdbc14.jar? Any general suggestions to make the code better, especially for batch updates.
But for selecting a row from the table, I am using index columns in the where cluase. In the code using PreparedStatement, setting all the reuired fields. Here is the code. We have a huge index with 14 fields!!. All the parameters are for where clause.
if(longCallPStmt == null) {
longCallPStmt = conn.prepareStatement(longCallQuery);
log(Level.FINE, "CdrAggLoader: Loading tcdragg entry for "
+GeneralUtility.formatDate(cdrAgg.time_hour, "MM/dd/yy HH"));
longCallPStmt.clearParameters();
longCallPStmt.setInt(1, cdrAgg.iintrunkgroupid);
longCallPStmt.setInt(2, cdrAgg.iouttrunkgroupid);
longCallPStmt.setInt(3, cdrAgg.iintrunkgroupnumber);
longCallPStmt.setInt(4, cdrAgg.iouttrunkgroupnumber);
longCallPStmt.setInt(5, cdrAgg.istateregionid);
longCallPStmt.setTimestamp(6, cdrAgg.time_hour);
longCallPStmt.setInt(7, cdrAgg.icalltreatmentcode);
longCallPStmt.setInt(8, cdrAgg.icompletioncode);
longCallPStmt.setInt(9, cdrAgg.bcallcompleted);
longCallPStmt.setInt(10, cdrAgg.itodid);
longCallPStmt.setInt(11, cdrAgg.iasktodid);
longCallPStmt.setInt(12, cdrAgg.ibidtodid);
longCallPStmt.setInt(13, cdrAgg.iaskzoneid);
longCallPStmt.setInt(14, cdrAgg.ibidzoneid);
rs = longCallPStmt.executeQuery();
if(rs.next()) {
cdr_agg = new CdrAgg(
rs.getInt(1),
rs.getInt(2),
rs.getInt(3),
rs.getInt(4),
rs.getInt(5),
rs.getTimestamp(6),
rs.getInt(7),
rs.getInt(8),
rs.getInt(9),
rs.getInt(10),
rs.getInt(11),
rs.getInt(12),
rs.getInt(13),
rs.getInt(14),
rs.getInt(15),
rs.getInt(16)
}//if
end_time = System.currentTimeMillis();
log(Level.INFO, "CdrAggLoader: Loaded "+((cdr_agg==null)?0:1) + " "
+ GeneralUtility.formatDate(cdrAgg.time_hour, "MM/dd/yy HH")
+" tcdragg entry in "+(end_time - start_time)+" msecs");
} finally {
GeneralUtility.closeResultSet(rs);
GeneralUtility.closeStatement(pstmt);
Why that code works well for classes12.zip (comes back in around 10 msec) and not for ojdbc14.jar (comes back in 6-7 minutes)?
Please advise.

Performance problem with sdn_nn - new 10g install

I am having a performance problem with sdn_nn after migrating to a new machine. The old Oracle version was 9.0.1.4. The new is 10g. The new machine is faster in general. Most (non-spatial) batch processes run in half the time. However, the below statement is radically slower. The below statement ran in 45 minutes before. On the new machine it hasn't finished after 10 hours. I am able to get a 5% sample of the customers to finish in 45 minutes.
Does anyone have any ideas on how to approach this problem? Any chance something isn't installed correctly on the new machine (the nth version of the query finishe, albeit 20 times slower)?
Appreciate any help. Thanks.
- Jack
create table nearest_store
as
select /*+ ordered */
a.customer_id,
b.store_id nearest_store,
round(mdsys.sdo_nn_distance(1),4) distance
from customers a,
stores b
where mdsys.sdo_nn(
b.geometry,
a.geometry,
'sdo_num_res=1, unit=mile',
1
) = 'TRUE'
;

Dan,
customers 110,000 (involved in this query)
stores 28,000
Here is the execution plan on the current machine:
CREATE TABLE STATEMENT cost = 81947
LOAD AS SELECT
PX COORDINATOR
PX SEND QC (RANDOM) :TQ10000
ROW NESTED LOOPS
1 1 PX BLOCK ITERATOR
1 1ROW TABLE ACCESS FULL CUSTOMERS
1 3 PARTITION RANGE ALL
1 3 TABLE ACCESS BY LOCAL INDEX ROWID STORES
DOMAIN INDEX STORES_SIDX
I can't capture the execution plan on the old database. It is gone. I don't remember it being any different from the above (full scan customers, probe stores index once for each row in customers).
I am trying the query without the create table (just doing a count). I'll let you know on that one.
I am at 10.0.1.3.
Here is how I created the index:
create index stores_sidx
on stores(geometry)
indextype is mdsys.spatial_index LOCAL
Note that the stores table is partitioned by range on store type. There are three store types (each in its own partition). The query returns the nearest store of each type (three rows per customer). This is by design (based on the documented behavior of sdo_nn).
In addition to running the query without the CTAS, I am also going try running it on a different machine tonight. I let you know how that turns out.
The reason I ask about the install, is that the Database Quick Installation Guide for Solaris says this:
"If you intend to use Oracle JVM or Oracle interMedia, you must install the Oracle Database 10g Products installation type from the Companion CD. This installation optimizes the performance of those products on your system."
And, the Database Installlation Guide says:
"If you plan to use Oracle JVM or Oracle interMedia, Oracle strongly recommends that you install the natively compiled Java libraries (NCOMPs) used by those products from the Oracle Database 10g Companion CD. These libraries are required to improve the performance of the products on your platform."
Based on that, I am suspicious that maybe I have the product installed on the new machine, but not correctly (forgot to set fast=true).
Let me know if you think of anything else you'd like to see. Thanks.
- Jack

Performance problem with Oracle

We are currently getting a system developed in Unix/Weblogic/Tomcat/Oracle environment. We have developed a screen that contains 5 or 6 different parameters to select from. We could select multiple parameters in each of these selections. The idea behind the subsequent screens is to attach information to already existing data/ possible future data that matches the selection criteria.
Based on these selections, existing data located within the system in a table is searched and those that match are selected. Also new rows are created in the table against combinations that do not currently have a match. Frequently multiple parameters are selected, and 2000 different combinations need to be searched in the table. Of these selections, only about 100 or 200 combinations will be available in existing data. So the system is having to insert 1800 rows. The user meanwhile waits for the system to come up with data based on their selections. The user is not willing to wait more than 30 seconds to get to the next screen. In the above mentioned scenario, the system takes more than an hour to insert the new records and bring the information up. We need suggestions to see if the performance can be improved this drastically. If not what are the alternatives? Thanks

The #1 cause for performance problems with Oracle is not using it correctly.
I find it hard to believe that with the small data volumes mentioned, that you can have perfornance problems.
You need to perform a sanity check. Are you using Oracle correctly? Do you know what bind variables are? Are you using indexes correctly? Are you using PL/SQL correctly? Is the instance setup correctly? What about storage, are you using SAME (RAID10) or something else? Etc.
Facts. Oracle peforms exceptionally well. Oracle exceptionally well.
Simple example from a benchmark I did on this exact same subject. App-tier developers not understanding and not using Oracle correctly. Incorrect usage of Oracle doing a 100,000 SQL statements. 24+ minutes elapsed time. Doing those exact same 100,000 SQL statement correctly (using bind variables) - 8 seconds elapsed time. (benchmark using Oracle 10.1.0.3 on a Sunfire V20z server)
But then you need to use Oracle correctly. Are you familiar with the Oracle Concepts Guide? Have you read the Oracle Application Developer Fundamentals Guide?

Performance problem with CR SDK

Hi,
I'am currently on a customer site and I have the following problem :
The client have a performance problem with a J2EE application wich call a Crystal report with th CR SDK. To reproduce the problem on the local machine (the CR server), I have developped a little jsp page wich used the Crystal SDK to open a Crystal report on the server (this report is based on a XML data source), setting the new data source (with a new xml data flow) and refresh the report in PDF format.
The problem is that the 2 first sequences take about 5 seconde each (5 sec for the opening report and 5 seconds for the setting data source). Then the total process take about 15 seconds to open and refresh the document that is very long for a little document.
The document is a 600Ko file, the xml source is a 80Ko file.
My jsp page is directly deployed on the tomcat of the Crystal Report Server (CRXIR2 without Service Pack).
The Filestore and the MySQL database are on the CR server.
The server is a 4 quadripro (16 proc) with 16Go of RAM and is totally dedicated to Crystal Report. For the moment, there is no activity on the server (it is also used for the test).
The mains jsp orders are the followings :
IEnterpriseSession es = CrystalEnterprise.getSessionMgr().logon("administrator", "", "EDITBI:6400", "secEnterprise");
    IInfoStore infoStore = (IInfoStore) es.getService("", "InfoStore");
    IInfoObjects infoObjects = infoStore.query("SELECT * FROM CI_INFOOBJECTS WHERE SI_NAME='CPA_EV' AND SI_INSTANCE=0 ");
    IInfoObject report = (IInfoObject) infoObjects.get(0);
IReportAppFactory reportAppFactory = (IReportAppFactory)es.getService("RASReportFactory");
ReportClientDocument reportClientDoc = reportAppFactory.openDocument(report.getID(), 0, null);
IXMLDataSet xmlDataSet = new XMLDataSet();
xmlDataSet.setXMLData(new ByteArray(ligne_data_xml));
xmlDataSet.setXMLSchema(new ByteArray(ligne_schema_xml));
DatabaseController db = reportClientDoc.getDatabaseController();
db.setDataSource(xmlDataSet, "", "");
ByteArrayInputStream bt = (ByteArrayInputStream)reportClientDoc.getPrintOutputController().export(ReportExportFormat.PDF);
My question is : does this method is the good one to do this ?
Thank's in advance for your help
Best regards
Emmanuel

Hi,
My problem is not resolved and I have'nt news from the support.
If you have any idea/info, don't forget me
Thank's in advance
Emmanuel

Sql Loader Performance

Hi
i have some question about SQL Loader. i have to find answers but did not get from Google or Documentation. i want to know that is there any way to check whether Sql loader inserting records with Direct path or Conventional path. As we know there are restriction in Direct load. Direct-path inserts do not support all objects that conventional inserts do. Their functionality
is restricted. If the database engine is not able to execute a direct-path insert, the operation is silently converted into a conventional insert. i have instruct Sql loader to insert using Direct=true and parallel as well. but it take 15 mint to loader 4 million record in table. i have observed its transfer rate is bit slow. i have oracle 11R2 on windows 2008 with 40GB RAM and SAN. how can i verify during execution either Sql loader load user Direct path or silently converted it conventional path. here is my sample control file. function used in control file will convert direct path to conventional?
/c sqlldr userid='MSNV5Star/Aa123456@(DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(HOST=srv01)(PORT=1521)))(CONNECT_DATA=(SERVICE_NAME=orcl)))' control='C:\ControlFile.txt' log='C:\ Final Data.log' bad= 'C:\ Final Data.bad' direct=true PARALLEL=TRUE skip=1 Errors=5000000
LOAD DATA
INFILE 'C:\adeel loading\in\A06052010.txt'APPEND
INTO TABLE GN_FILE_DATA_TABLE
FIELDS TERMINATED BY " " TRAILING NULLCOLS
Operational_Date "to_date(:Operational_Date, 'YYYYMMDD')" ,
Store_Code "TRIM(:Store_Code)" ,
Txn_Void_Flag "TRIM(:Txn_Void_Flag)" ,
Txn_Staff_Flag "TRIM(:Txn_Staff_Flag)" ,
Txn_Aborted_Flag "TRIM(:Txn_Aborted_Flag)" Edited by: Oracle Studnet on May 30, 2011 7:18 AM

Pl do not post duplicates - sql Loader Performance
Srini

Problem with XML on Linux

hi everybody,
I've a big problem with XML on Linux, in details I see my program stopping on Linux at the instruction
XMLReader xr = XMLReaderFactory.createXMLReader("org.apache.crimson.parser.XMLReaderImpl");
and it's strange because on Windows it runs and there aren't problems about permissions on files, does anyone knows what to do?
thanks in advance!
Stefano

What happens on that line? I'm assuming you get some kind of error or exception.
Make sure the JAR file for Crimson is in your classpath.

Performance problems with jdk 1.5 on Linux plattform

Performance problems with jdk 1.5 on Linux plattform
(not tested on Windows, might be the same)
After refactoring using the new features from java 1.5 I lost
performance significantly:
public Vector<unit> units;
The new code:
for (unit u: units) u.accumulate();
runs more than 30% slower than the old code:
for (int i = 0; i < units.size(); i++) units.elementAt(i).accumulate();
I expected the opposite.
Is there any information available that helps?

Here's the complete benchmark code I used:package test;
import java.text.NumberFormat;
import java.util.ArrayList;
import java.util.Iterator;
import java.util.LinkedList;
import java.util.Vector;
public class IterationPerformanceTest {
     private int m_size;
     public IterationPerformanceTest(int size) {
          m_size = size;
     public long getArrayForLoopDuration() {
          Integer[] testArray = new Integer[m_size];
          for (int item = 0; item < m_size; item++) {
               testArray[item] = new Integer(item);
          StringBuilder builder = new StringBuilder();
          long start = System.nanoTime();
          for (int index = 0; index < m_size; index++) {
               builder.append(testArray[index]);
          long end = System.nanoTime();
          System.out.println(builder.length());
          return end - start;
     public long getArrayForEachDuration() {
          Integer[] testArray = new Integer[m_size];
          for (int item = 0; item < m_size; item++) {
               testArray[item] = new Integer(item);
          StringBuilder builder = new StringBuilder();
          long start = System.nanoTime();
          for (Integer item : testArray) {
               builder.append(item);
          long end = System.nanoTime();
          System.out.println(builder.length());
          return end - start;
     public long getArrayListForLoopDuration() {
          ArrayList<Integer> testList = new ArrayList<Integer>();
          for (int item = 0; item < m_size; item++) {
               testList.add(item);
          StringBuilder builder = new StringBuilder();
          long start = System.nanoTime();
          for (int index = 0; index < m_size; index++) {
               builder.append(testList.get(index));
          long end = System.nanoTime();
          System.out.println(builder.length());
          return end - start;
     public long getArrayListForEachDuration() {
          ArrayList<Integer> testList = new ArrayList<Integer>();
          for (int item = 0; item < m_size; item++) {
               testList.add(item);
          StringBuilder builder = new StringBuilder();
          long start = System.nanoTime();
          for (Integer item : testList) {
               builder.append(item);
          long end = System.nanoTime();
          System.out.println(builder.length());
          return end - start;
     public long getArrayListIteratorDuration() {
          ArrayList<Integer> testList = new ArrayList<Integer>();
          for (int item = 0; item < m_size; item++) {
               testList.add(item);
          StringBuilder builder = new StringBuilder();
          long start = System.nanoTime();
          Iterator<Integer> iterator = testList.iterator();
          while(iterator.hasNext()) {
               builder.append(iterator.next());
          long end = System.nanoTime();
          System.out.println(builder.length());
          return end - start;
     public long getLinkedListForLoopDuration() {
          LinkedList<Integer> testList = new LinkedList<Integer>();
          for (int item = 0; item < m_size; item++) {
               testList.add(item);
          StringBuilder builder = new StringBuilder();
          long start = System.nanoTime();
          for (int index = 0; index < m_size; index++) {
               builder.append(testList.get(index));
          long end = System.nanoTime();
          System.out.println(builder.length());
          return end - start;
     public long getLinkedListForEachDuration() {
          LinkedList<Integer> testList = new LinkedList<Integer>();
          for (int item = 0; item < m_size; item++) {
               testList.add(item);
          StringBuilder builder = new StringBuilder();
          long start = System.nanoTime();
          for (Integer item : testList) {
               builder.append(item);
          long end = System.nanoTime();
          System.out.println(builder.length());
          return end - start;
     public long getLinkedListIteratorDuration() {
          LinkedList<Integer> testList = new LinkedList<Integer>();
          for (int item = 0; item < m_size; item++) {
               testList.add(item);
          StringBuilder builder = new StringBuilder();
          long start = System.nanoTime();
          Iterator<Integer> iterator = testList.iterator();
          while(iterator.hasNext()) {
               builder.append(iterator.next());
          long end = System.nanoTime();
          System.out.println(builder.length());
          return end - start;
     public long getVectorForLoopDuration() {
          Vector<Integer> testVector = new Vector<Integer>();
          for (int item = 0; item < m_size; item++) {
               testVector.add(item);
          StringBuilder builder = new StringBuilder();
          long start = System.nanoTime();
          for (int index = 0; index < m_size; index++) {
               builder.append(testVector.get(index));
          long end = System.nanoTime();
          System.out.println(builder.length());
          return end - start;
     public long getVectorForEachDuration() {
          Vector<Integer> testVector = new Vector<Integer>();
          for (int item = 0; item < m_size; item++) {
               testVector.add(item);
          StringBuilder builder = new StringBuilder();
          long start = System.nanoTime();
          for (Integer item : testVector) {
               builder.append(item);
          long end = System.nanoTime();
          System.out.println(builder.length());
          return end - start;
     public long getVectorIteratorDuration() {
          Vector<Integer> testVector = new Vector<Integer>();
          for (int item = 0; item < m_size; item++) {
               testVector.add(item);
          StringBuilder builder = new StringBuilder();
          long start = System.nanoTime();
          Iterator<Integer> iterator = testVector.iterator();
          while(iterator.hasNext()) {
               builder.append(iterator.next());
          long end = System.nanoTime();
          System.out.println(builder.length());
          return end - start;
      * @param args
     public static void main(String[] args) {
          IterationPerformanceTest test = new IterationPerformanceTest(1000000);
          System.out.println("\n\nRESULTS:");
          long arrayForLoop = test.getArrayForLoopDuration();
          long arrayForEach = test.getArrayForEachDuration();
          long arrayListForLoop = test.getArrayListForLoopDuration();
          long arrayListForEach = test.getArrayListForEachDuration();
          long arrayListIterator = test.getArrayListIteratorDuration();
//          long linkedListForLoop = test.getLinkedListForLoopDuration();
          long linkedListForEach = test.getLinkedListForEachDuration();
          long linkedListIterator = test.getLinkedListIteratorDuration();
          long vectorForLoop = test.getVectorForLoopDuration();
          long vectorForEach = test.getVectorForEachDuration();
          long vectorIterator = test.getVectorIteratorDuration();
          System.out.println("Array      for-loop: " + getPercentage(arrayForLoop, arrayForLoop) + "% ("+getDuration(arrayForLoop)+" sec)");
          System.out.println("Array      for-each: " + getPercentage(arrayForLoop, arrayForEach) + "% ("+getDuration(arrayForEach)+" sec)");
          System.out.println("ArrayList for-loop: " + getPercentage(arrayForLoop, arrayListForLoop) + "% ("+getDuration(arrayListForLoop)+" sec)");
          System.out.println("ArrayList for-each: " + getPercentage(arrayForLoop, arrayListForEach) + "% ("+getDuration(arrayListForEach)+" sec)");
          System.out.println("ArrayList iterator: " + getPercentage(arrayForLoop, arrayListIterator) + "% ("+getDuration(arrayListIterator)+" sec)");
//          System.out.println("LinkedList for-loop: " + getPercentage(arrayForLoop, linkedListForLoop) + "% ("+getDuration(linkedListForLoop)+" sec)");
          System.out.println("LinkedList for-each: " + getPercentage(arrayForLoop, linkedListForEach) + "% ("+getDuration(linkedListForEach)+" sec)");
          System.out.println("LinkedList iterator: " + getPercentage(arrayForLoop, linkedListIterator) + "% ("+getDuration(linkedListIterator)+" sec)");
          System.out.println("Vector     for-loop: " + getPercentage(arrayForLoop, vectorForLoop) + "% ("+getDuration(vectorForLoop)+" sec)");
          System.out.println("Vector     for-each: " + getPercentage(arrayForLoop, vectorForEach) + "% ("+getDuration(vectorForEach)+" sec)");
          System.out.println("Vector     iterator: " + getPercentage(arrayForLoop, vectorIterator) + "% ("+getDuration(vectorIterator)+" sec)");
     private static NumberFormat percentageFormat = NumberFormat.getInstance();
     static {
          percentageFormat.setMinimumIntegerDigits(3);
          percentageFormat.setMaximumIntegerDigits(3);
          percentageFormat.setMinimumFractionDigits(2);
          percentageFormat.setMaximumFractionDigits(2);
     private static String getPercentage(long base, long value) {
          double result = (double) value / (double) base;
          return percentageFormat.format(result * 100.0);
     private static NumberFormat durationFormat = NumberFormat.getInstance();
     static {
          durationFormat.setMinimumIntegerDigits(1);
          durationFormat.setMaximumIntegerDigits(1);
          durationFormat.setMinimumFractionDigits(4);
          durationFormat.setMaximumFractionDigits(4);
     private static String getDuration(long nanos) {
          double result = (double)nanos / (double)1000000000;
          return durationFormat.format(result);
}

Performance problems with new Java Tiger style recommendations

Performance problems with jdk 1.5 on Linux plattform
(not tested on Windows, might be the same)
using the new style recommendations.
I need fast Vector loops for high speed mathematical calculations, some
hints about the fastest way to program that loop would be also great!
After refactoring using the new features from java 1.5 (as recommended from
SUN) I lost performance significantly:
using a vector:
public Vector<unit> units;
The new code (recommended from SUN for Java Tiger for redesign):
for (unit u: units) u.accumulate();
runs more than 30% slower than the old code:
for (int i = 0; i < units.size(); i++) units.elementAt(i).accumulate();
I expected the opposite.
Is there any information available that helps?
The following additional information I got from Mr. Shankar Unni:
I got some fairly anomalous results comparing ArrayList and Vector: for the
1.5-style loops, ArrayList was faster then Vector, but for a loop with get()
calls, Vector was faster. Vector was even faster than that using
elementAt(), which was a surprise:
For a million summing iterations over a 100-element array:
vector elementAt loop took 3446 ms.
vector get loop took 3796 ms.
vector iterator loop took 5469 ms.
arraylist get loop took 4136 ms.
arraylist iterator loop took 4668 ms.

If your topic doesn't change, please stay in your original post.

Performance problems with SAP GUI 7.10 and BEx 3.5 Patch 400?

Hi everybody,
we installed SAP GUI 7.10 and BEx 3.5 Patch 400 and detected hugh performance problems with this version in comparison to the SAP GUI 6.40 and BEx 3.5 or BEx 7.0 Patch 800.
Does anybody detect the same problems?
Best regards,
Ulli

Most important question when you are talking about performance-issues:
which OC are you working on and which excel version?
ciao
Joke

Sql loader performance problem with xml

Similar Messages

Maybe you are looking for