WebDAV Write Unreliability - Large non-XML Files

I am unable to get Oracle's WebDAV implementation on Windows XP to reliably store large (1 Megabyte +) files that are not XML, without unexplained errors occurring, which result in only part files being transferred.
For testing I started using the repository folders created by the XDB Basic Demo, using Windows Explorer WebDAV as the client to insert files of varying sizes. Files in excess of 1 megabyte files fail more often than succeed, but I have occasionally succeeded in storing files of many megabytes. Because they were handy I mainly used ZIP files to insert into the Oracle XML DB repository. When the files were renamed to extension, .xml, they always succeeded.
To eliminate client error, I have also used DavExplorer with the same symptoms observed. No problems were experienced retrieving large files from the Oracle XML DB Repository. My tests were performed using Oracle 9.2.0.5 and 10.1.0.2. The same problem did not occur when using Oracle's FTP interface, and when using the "native" Apache 1.3 and Apache WebDAV implementations.
This failure is a major problem for our application, which would otherwise fit very well with Oracle XML DB features. Our application provides an authoring environment for a large collection of small XML files, but also has to allow storage of some related GIF, PDF and Microsoft Offices files. Our customers are large corporates, who would prefer to use Oracle for this requirement and for related data processing requirements.
In case, it is relevant, we are not restricted to running Oracle on XP, with Unix platforms being a more likely option for our customers.
I could not find details of the error logged by the Oracle software, using either database version. The only information I have gathered about the failure was provided by adding the following line to sqlnet.ora:
TRACE_LEVEL_SERVER=4
The trace output for a failure is as follows:
[19-MAY-2004 13:46:59:735] nsopen: opening transport...
[19-MAY-2004 13:46:59:735] nsopen: transport is open
[19-MAY-2004 13:46:59:735] nsopen: global context check-in (to slot 1) complete
[19-MAY-2004 13:46:59:735] nsanswer: deferring connect attempt; at stage 1064
[19-MAY-2004 13:46:59:735] nsc2addr: (ADDRESS=(PROTOCOL=tcp)(DEV=340)(HOST=132.223.134.163)(PORT=8080))
[19-MAY-2004 13:46:59:735] nttgetport: port resolved to 8080
[19-MAY-2004 13:46:59:735] nttbnd2addr: using host IP address: 132.223.134.163
[19-MAY-2004 13:46:59:735] nstimarmed: no timer allocated
[19-MAY-2004 13:46:59:735] nsclose: global context check-out (from slot 1) complete
[19-MAY-2004 13:46:59:735] nttcnp: Validnode Table IN use; err 0x0
[19-MAY-2004 13:46:59:735] nsopen: opening transport...
[19-MAY-2004 13:46:59:735] nttcnp: Validnode Table IN use; err 0x0
[19-MAY-2004 13:46:59:735] nttcnp: Validnode Table IN use; err 0x0
[19-MAY-2004 13:46:59:735] nttcnp: getting sockname
[19-MAY-2004 13:46:59:735] nttcnp: getting peername
[19-MAY-2004 13:46:59:735] nttcon: set TCP_NODELAY on 296
[19-MAY-2004 13:46:59:745] nsopen: transport is open
[19-MAY-2004 13:46:59:745] nsopen: global context check-in (to slot 1) complete
[19-MAY-2004 13:46:59:745] nstoControlATO: ATO disabled for ctx=0x055D6FC4
[19-MAY-2004 13:46:59:745] nsevdansw: exit
[19-MAY-2004 13:46:59:745] ntt2err: soc 296 error - operation=5, ntresnt[0]=524, ntresnt[1]=997, ntresnt[2]=0
[19-MAY-2004 13:46:59:745] ntt2err: soc 296 error - operation=5, ntresnt[0]=524, ntresnt[1]=997, ntresnt[2]=0
[19-MAY-2004 13:46:59:745] ntt2err: soc 296 error - operation=5, ntresnt[0]=524, ntresnt[1]=997, ntresnt[2]=0
[19-MAY-2004 13:46:59:745] nstoControlATO: ATO disabled for ctx=0x055D3548
[19-MAY-2004 13:46:59:745] ntt2err: soc 296 error - operation=5, ntresnt[0]=524, ntresnt[1]=997, ntresnt[2]=0
[19-MAY-2004 13:46:59:745] nstoControlATO: ATO disabled for ctx=0x055D3548
[19-MAY-2004 13:46:59:745] snttcallback: op = 5, bytes = 232, err = 0
[19-MAY-2004 13:46:59:745] ntt2err: soc 296 error - operation=5, ntresnt[0]=524, ntresnt[1]=997, ntresnt[2]=0
[19-MAY-2004 13:46:59:745] nstoControlATO: ATO disabled for ctx=0x055D3548
[19-MAY-2004 13:46:59:755] snttcallback: op = 5, bytes = 296, err = 0
[19-MAY-2004 13:46:59:755] ntt2err: soc 296 error - operation=5, ntresnt[0]=524, ntresnt[1]=997, ntresnt[2]=0
[19-MAY-2004 13:46:59:755] nstoControlATO: ATO disabled for ctx=0x055D3548
[19-MAY-2004 13:46:59:755] snttcallback: op = 5, bytes = 0, err = 0
[19-MAY-2004 13:46:59:755] ntt2err: Read unexpected EOF ERROR on 296
[19-MAY-2004 13:46:59:755] snttcallback: op = 5, bytes = 0, err = 0
[19-MAY-2004 13:46:59:755] ntt2err: Read unexpected EOF ERROR on 296
[19-MAY-2004 13:46:59:755] snttcallback: op = 5, bytes = 0, err = 0
[19-MAY-2004 13:46:59:755] ntt2err: Read unexpected EOF ERROR on 296
[19-MAY-2004 13:46:59:755] snttcallback: op = 5, bytes = 0, err = 0
[19-MAY-2004 13:46:59:755] ntt2err: Read unexpected EOF ERROR on 296
[19-MAY-2004 13:46:59:755] nsdo: transport read error
[19-MAY-2004 13:46:59:755] nserror: nsres: id=1, op=68, ns=12537, ns2=12560; nt[0]=507, nt[1]=0, nt[2]=0; ora[0]=0, ora[1]=0, ora[2]=0
[19-MAY-2004 13:46:59:755] nstimarmed: no timer allocated
[19-MAY-2004 13:46:59:765] nsclose: closing transport
[19-MAY-2004 13:46:59:765] nsclose: global context check-out (from slot 1) complete
[19-MAY-2004 13:46:59:765] nsopen: opening transport...
[19-MAY-2004 13:46:59:765] nsopen: transport is open
[19-MAY-2004 13:46:59:765] nsopen: global context check-in (to slot 1) complete
[19-MAY-2004 13:46:59:765] nsanswer: deferring connect attempt; at stage 1064
[19-MAY-2004 13:46:59:765] nsc2addr: (ADDRESS=(PROTOCOL=tcp)(DEV=340)(HOST=132.223.134.163)(PORT=8080))
[19-MAY-2004 13:46:59:765] nttgetport: port resolved to 8080
[19-MAY-2004 13:46:59:765] nttbnd2addr: using host IP address: 132.223.134.163
[19-MAY-2004 13:46:59:765] nstimarmed: no timer allocated
[19-MAY-2004 13:46:59:765] nsclose: global context check-out (from slot 1) complete
[19-MAY-2004 13:46:59:765] nttcnp: Validnode Table IN use; err 0x0
[19-MAY-2004 13:46:59:765] nsopen: opening transport...
[19-MAY-2004 13:46:59:765] nttcnp: Validnode Table IN use; err 0x0
[19-MAY-2004 13:46:59:765] nttcnp: Validnode Table IN use; err 0x0
[19-MAY-2004 13:46:59:765] nttcnp: getting sockname
[19-MAY-2004 13:46:59:765] nttcnp: getting peername
[19-MAY-2004 13:46:59:765] nttcon: set TCP_NODELAY on 296
[19-MAY-2004 13:46:59:765] nsopen: transport is open
[19-MAY-2004 13:46:59:765] nsopen: global context check-in (to slot 1) complete
[19-MAY-2004 13:46:59:765] nstoControlATO: ATO disabled for ctx=0x055D6FC4
[19-MAY-2004 13:46:59:765] nsevdansw: exit
[19-MAY-2004 13:46:59:765] ntt2err: soc 296 error - operation=5, ntresnt[0]=524, ntresnt[1]=997, ntresnt[2]=0
[19-MAY-2004 13:46:59:765] ntt2err: soc 296 error - operation=5, ntresnt[0]=524, ntresnt[1]=997, ntresnt[2]=0
[19-MAY-2004 13:46:59:765] ntt2err: soc 296 error - operation=5, ntresnt[0]=524, ntresnt[1]=997, ntresnt[2]=0
[19-MAY-2004 13:46:59:765] ntt2err: soc 296 error - operation=5, ntresnt[0]=524, ntresnt[1]=997, ntresnt[2]=0
[19-MAY-2004 13:46:59:765] nstoControlATO: ATO disabled for ctx=0x055D3548
[19-MAY-2004 13:46:59:765] snttcallback: op = 5, bytes = 302, err = 0
[19-MAY-2004 13:46:59:765] ntt2err: soc 296 error - operation=5, ntresnt[0]=524, ntresnt[1]=997, ntresnt[2]=0
[19-MAY-2004 13:46:59:775] snttcallback: op = 5, bytes = 8192, err = 0
[19-MAY-2004 13:46:59:775] snttcallback: op = 5, bytes = 8192, err = 0
[19-MAY-2004 13:46:59:775] snttcallback: op = 5, bytes = 8192, err = 0
[19-MAY-2004 13:46:59:775] snttcallback: op = 5, bytes = 8192, err = 0
[19-MAY-2004 13:47:00:246] nstoControlATO: ATO disabled for ctx=0x055D3548
[19-MAY-2004 13:47:00:246] nstimarmed: no timer allocated
[19-MAY-2004 13:47:00:246] nsclose: closing transport
[19-MAY-2004 13:47:00:366] nsclose: global context check-out (from slot 1) complete
The trace for a successful transfer is as follows:
[19-MAY-2004 13:57:28:249] nsopen: opening transport...
[19-MAY-2004 13:57:28:249] nsopen: transport is open
[19-MAY-2004 13:57:28:249] nsopen: global context check-in (to slot 1) complete
[19-MAY-2004 13:57:28:249] nsanswer: deferring connect attempt; at stage 1064
[19-MAY-2004 13:57:28:249] nsc2addr: (ADDRESS=(PROTOCOL=tcp)(DEV=352)(HOST=132.223.134.163)(PORT=8080))
[19-MAY-2004 13:57:28:249] nttgetport: port resolved to 8080
[19-MAY-2004 13:57:28:249] nttbnd2addr: using host IP address: 132.223.134.163
[19-MAY-2004 13:57:28:249] nstimarmed: no timer allocated
[19-MAY-2004 13:57:28:249] nsclose: global context check-out (from slot 1) complete
[19-MAY-2004 13:57:28:249] nttcnp: Validnode Table IN use; err 0x0
[19-MAY-2004 13:57:28:249] nsopen: opening transport...
[19-MAY-2004 13:57:28:249] nttcnp: Validnode Table IN use; err 0x0
[19-MAY-2004 13:57:28:249] nttcnp: Validnode Table IN use; err 0x0
[19-MAY-2004 13:57:28:249] nttcnp: getting sockname
[19-MAY-2004 13:57:28:249] nttcnp: getting peername
[19-MAY-2004 13:57:28:249] nttcon: set TCP_NODELAY on 1420
[19-MAY-2004 13:57:28:249] nsopen: transport is open
[19-MAY-2004 13:57:28:249] nsopen: global context check-in (to slot 1) complete
[19-MAY-2004 13:57:28:249] nstoControlATO: ATO disabled for ctx=0x055D6FC4
[19-MAY-2004 13:57:28:249] nsevdansw: exit
[19-MAY-2004 13:57:28:249] nstoControlATO: ATO disabled for ctx=0x055D3548
[19-MAY-2004 13:57:28:249] ntt2err: soc 1420 error - operation=5, ntresnt[0]=524, ntresnt[1]=997, ntresnt[2]=0
[19-MAY-2004 13:57:28:249] ntt2err: soc 1420 error - operation=5, ntresnt[0]=524, ntresnt[1]=997, ntresnt[2]=0
[19-MAY-2004 13:57:28:269] ntt2err: soc 1420 error - operation=5, ntresnt[0]=524, ntresnt[1]=997, ntresnt[2]=0
[19-MAY-2004 13:57:28:269] ntt2err: soc 1420 error - operation=5, ntresnt[0]=524, ntresnt[1]=997, ntresnt[2]=0
[19-MAY-2004 13:57:28:369] nstoControlATO: ATO disabled for ctx=0x055D3548
[19-MAY-2004 13:57:31:374] nstimarmed: no timer allocated
[19-MAY-2004 13:57:31:384] snttcallback: op = 5, bytes = 0, err = 995
[19-MAY-2004 13:57:31:384] ntt2err: soc 1420 error - operation=5, ntresnt[0]=530, ntresnt[1]=995, ntresnt[2]=0
[19-MAY-2004 13:57:31:384] snttcallback: op = 5, bytes = 0, err = 995
[19-MAY-2004 13:57:31:384] ntt2err: soc 1420 error - operation=5, ntresnt[0]=530, ntresnt[1]=995, ntresnt[2]=0
[19-MAY-2004 13:57:31:384] snttcallback: op = 5, bytes = 0, err = 995
[19-MAY-2004 13:57:31:384] ntt2err: soc 1420 error - operation=5, ntresnt[0]=530, ntresnt[1]=995, ntresnt[2]=0
[19-MAY-2004 13:57:31:384] snttcallback: op = 5, bytes = 0, err = 995
[19-MAY-2004 13:57:31:384] ntt2err: soc 1420 error - operation=5, ntresnt[0]=530, ntresnt[1]=995, ntresnt[2]=0
[19-MAY-2004 13:57:31:384] nsclose: closing transport
[19-MAY-2004 13:57:31:484] nsclose: global context check-out (from slot 1) complete

Thanks Mark.
Unfortunately I'm not currently in a position to test on Unix, but understand that I need to do so. It may be a couple of weeks before I can test on Linux, as "IT Support" will need to install an OS version officially supported by Oracle. If we win the work we'll have the required hardware, but we have to demonstrate it working first.
Note, in my test circumstance transfers work fine using FTP, and only fail using WebDAV when the file extension is not .xml.

Similar Messages

How to validate and transform large (180M) xml files

Hi:
I've been looking at various ways to do this with oracle and am getting a bit lost in the sea of documentation and different ways to go about this. I was hoping that something like the XMLParser class and XMLTransform would be smart enough to handle large files by using SAX when it has to but I'm getting "too many nodes" when trying to transform a really large file. I've gotten oraxsl to handle it if I pass in the proper memory parameters on the command line but a) this will still have limits and b) I was trying to do this in a stored procedure which (I think) means I'm looking at XMLParser?
I've also seen documentation on something called "Scalable DOM" but I think that's only in 11g? So I'm thinking I have to write a (Java?) stored procedure to loop through the top elements of this XML file (select extract(...)) and transform each node?
I have the XML, XSD, and XSLT all in clob columns. What's the easiest/quickest path within Oracle to validate the XML against the xsd and translate source XML with XSL?
I'm using Oracle 10gR2 on RH Linux.
Thanks.

So I'm thinking I have to write a (Java?) stored procedure to loop through the top elements of this XML file (select extract(...)) and transform each node? Here's something I've written a while back when I hit the same restrictions ("too many nodes") on 10.2.0.4.
It takes XMLType as input parameters (but it's easy to adapt for CLOB), and streams the transformed XML directly into a file :
create or replace and compile java source named ora_xslt_util as
import oracle.xml.parser.v2.*;
import oracle.xdb.XMLType;
import java.io.*;
import org.w3c.dom.*;
public class oraXSL
private static XMLDocument getXMLDocument(XMLType xml) throws Exception
    XMLDocument doc = null;
    DOMParser parser = new DOMParser();
    parser.setValidationMode(oracle.xml.parser.v2.XMLParser.NONVALIDATING);
    parser.setPreserveWhitespace(true);
    parser.parse(new StringReader(xml.getStringVal()));
    doc = parser.getDocument();
    return doc;
public static void transform(XMLType doc, XMLType xsl, String filename) throws Exception
    OutputStream os = new FileOutputStream(filename);
    XMLDocument xmldoc = getXMLDocument(doc);
    XMLDocument xsldoc = getXMLDocument(xsl);
    XSLProcessor xsp = new XSLProcessor();
    XSLStylesheet xss = xsp.newXSLStylesheet(xsldoc);
    xsp.processXSL(xss, xmldoc, os);
    os.close();
}and the PL/SQL wrapper (originally part of a package) :
PROCEDURE processXSL (
   p_xmldoc IN XMLType
, p_xsldoc IN XMLType
, p_filename IN VARCHAR2
IS
LANGUAGE JAVA NAME 'oraXSL.transform(oracle.xdb.XMLType,oracle.xdb.XMLType,java.lang.String)'
;

Write data to local xml file

Does anyone know if there is a way to write data to a local xml file using the Connection refresh Button??

Hi there Glenn,
That's tricky...
Are you using SP1? You need to have at least SP1.
The XLFs were generated with an internal build newer than SP1, but they should still work in earlier builds.
If you changed the HTML file, that might be the cause. The SWF references in the HTML file must be exact.
Also, the range names must match as defined in the XLF/SWF. The bolded lines following should match. It's easiest if they all use the SWF name.
else if (hasRequestedVersion) {
// if we've detected an acceptable version
// embed the Flash Content SWF when all tests are passed
AC_FL_RunContent(
"src", "consumer",
"width", "380",
"height", "308",
"align", "middle",
"id", "consumer",
"quality", "high",
"bgcolor", "#ffffff",
"name", "consumer",
+"flashvars",'historyUrl=history.htm%3F&lconid=' + lc_id + '',+
"allowScriptAccess","always",
"type", "application/x-shockwave-flash",
"pluginspage", "http://www.adobe.com/go/getflashplayer"
} else { // flash is too old or we can't detect the plugin
var alternateContent = 'Alternate HTML content should be placed here. '
'This content requires the Adobe Flash Player. '
'<a href=http://www.adobe.com/go/getflash/>Get Flash</a>';
document.write(alternateContent); // insert non-flash content
// -->
</script>
<noscript>
<object classid="clsid:D27CDB6E-AE6D-11cf-96B8-444553540000"
id="consumer" width="380" height="308"
codebase="http://fpdownload.macromedia.com/get/flashplayer/current/swflash.cab">
<param name="movie" value="consumer.swf" />
<param name="quality" value="high" />
<param name="bgcolor" value="#ffffff" />
<param name="allowScriptAccess" value="always" />
<embed src="consumer.swf" quality="high" bgcolor="#ffffff"
width="380" height="308" name="consumer" align="middle"
play="true"
loop="false"
quality="high"
allowScriptAccess="always"
type="application/x-shockwave-flash"
pluginspage="http://www.adobe.com/go/getflashplayer">
</embed>

I want to load large raw XML file in firefox and parse by DOM. But, for large XML file the firefox very slow some time crashed . Is there any option to increase DOM handling memory in Firefox

Actually i am using an off-line form to load very large XML file and using firefox to load that form. But, its taking more time to load and some time the browser crashed. through DOM parsing this XML file to my form. Is there any option to increase DOM handler size in firefox

Thank you for your suggestion. I have a question,
though. If I use a relational database and try to
access it for EACH and EVERY click the user makes,
wouldn't that take much time to populate the page with
data?
Isn't XML store more efficient here? Please reply me.You have the choice of reading a small number of records (10 children per element?) from a database, or parsing multiple megabytes. Reading 10 records from a database should take maybe 100 milliseconds (1/10 of a second). I have written a web application that reads several hundred records and returns them with acceptable response time, and I am no expert. To parse an XML file of many megabytes... you have already tried this, so you know how long it takes, right? If you haven't tried it then you should. It's possible to waste a lot of time considering alternatives -- the term is "analysis paralysis". Speculating on how fast something might be doesn't get you very far.

Can I route non-XML files with NW PI 7.0?

Hello,
I have NW 7.0 PI installed. I need to implement very simple scenario:
1. There are two folders Source and Target
2. Any file (non-XML, binary for example) comes to the Source folder
3. File needs to be picked up and routed without any modifications to the Target folder
Questions:
1. Can I implement that with NW? I guess I can
2. How do I do that? Is there any example or some description of implementing such a simple scenario (I have a few samples on how to implement complex scenarios that work with XML files, but I don't need that complexity)
Thanks in advance.

Hi,
Check out the Sravya's blogs for starters, It has lot of links which will help you to learn various pieces of XI.
Aspirant to learn SAP XI...You won the Jackpot if you read this!-Part I
Aspirant to learn SAP XI...You won the Jackpot if you read this!-Part II
Aspirant to learn SAP XI...You won the Jackpot if you read this!-Part III
Thanks,
Pooja

How to write from database to XML file using JAVA and back..

Hi....
I am strugling for a code to write from database to xml output instead of what i have written in flat file....could anyone please help me out...to do this and again read the same back to the database using java...

Hi,
In java world, there is a tutorial on what you are looking for :
http://www.javaworld.com/javaworld/jw-01-2000/jw-01-dbxml_p.html
It uses SAX/DOM parsers to translate XML data into the appropriate fields of the database and visa-versa.
Hope it helps.

Import/Export Large Complex XMl files(700MB) to MySQL tables

Am creating an application that will be mporting large and complex XML file in the range of 500-1000MB size. An xml ile wil have 5-6 levels of xml tags where the first tag will be the client account configuration e.g name, location, address, the second will be orders made, the third will be order items, fourth will be item defination...etc. The following is a sample structure of the file.
<mycompany_sales> <customer name="me"> <location country="Kenya"> <town>nairobi</toen> <order id="20000" date="2008-09-08"> <item id="4000"> <stock>2000</stock> <buying>300.0</buying> </item> ........ <total>5000</total> </order> .............. </location> </customer> ........... </mycompany_sales>
Because of the size, if i process this file using DOM it will result to out of memory error. I have tried parsing with SAX but it takes about 18 hours to process which is not ideal. If i import this data to Microsoft Access it takes about 10 minutes to import including it's relationship. i.e. creates the six tables and the foregin key relations.
I tried using XML-DBMS package but since it relies on DOM, it cant process. Does anyone know of a package that would help me import this data to any JDBC databse and also recreate the same structure of xml from the jdbc db. It would also be ideal if it can create sample map file during import and export.
Thanks in advance,
Simon K.

kagara wrote:
I tried using XML-DBMS package but since it relies on DOM, it cant process. Does anyone know of a package that would help me import this data to any JDBC databse and also recreate the same structure of xml from the jdbc db. It would also be ideal if it can create sample map file during import and export.So you are parsing the XML, extracting the schema, using jdbc to create it, then parsing the data and using jdbc to populate it.
I suggest that you just create SQL directly. Or perhaps a import file if MySQL has it.
Less impact on the database, easier to test, easier to verify, and much faster to actually apply the data.

How to write data to an XML file present under application server

frnds: can ne one tell me, how to write data in to a file, which is present under a application server
Ex: i want to write a string data in to a file test.txt which is present under "http://localhost:8080/<some_webapp>/test.txt"
Note:i have deploted a service<some_webapp> under Tomcat/webapps dir

Very simple. A servlet can writes to that file if it has the good rights.
In the servlet get (or post) method, use a code like this:
import java.io.*;
protected void doGet(HttpServletRequest req, HttpServletResponse resp) {
try {
String filePath = this.getServletContext().getRealPath("/text.txt");//See http://java.sun.com/products/servlet/2.2/javadoc/javax/servlet/ServletContext.html#getRealPath(java.lang.String)
         PrintWriter writer = new PrintWriter (filePath);
        //or BufferedWriter out = new BufferedWriter(new FileWriter(filePath));
        writer .println("aString");
       writer.flush();
        writer .close();
    } catch (Exception e) {
       e.printStackTrace();
}Hope That Helps

How to write multiple ejb-jar.xml file in an ejb module

i am developing an ejb project. I hav a number of entity beans and session beans and their mapping in ejb-jar.xml. As the number of beans increases the entry of beans and mapping is also increasing. So i want to ask u that is there any mechanism so that i can divide the ejb-jar.xml file so that it will be easy to maintain it.

ejb-jar is the smallest composable unit for related EJB components. Your best bet is to create multiple ejb-jars and package them within the same enterprise application (.ear).

When parsing a large size xml file , OutOfMemorry exception(attach code)

Dear all ,
I met a OutOfMemorry exception when I parse a xml file (20M) in my application(I extract data in xml file for building my application data), but I found that it take long time(for 2 more minutes) even just searching the xml file , because my application is viewed by web page , so it's too bad for waiting.
what I used is org.jdom.input.SAXBuilder(jdom1.0beta8-dev) , and my xml file structure is like :
<errors>
<item1>content</item1>
<item2>content</item2>
</errors>
and this is my source code of parsing xml file :
import java.io.*;
import java.text.*;
import java.util.*;
import org.jdom.*;
import org.jdom.input.*;
import org.jdom.output.*;
public class XMLProperties {
    private File file;
    private Document doc;
    private Map propertyCache = new HashMap();
    public XMLProperties(String filename) {
            SAXBuilder builder = new SAXBuilder();
            // Strip formatting
            DataUnformatFilter format = new DataUnformatFilter();
            builder.setXMLFilter(format);
            long time_start = System.currentTimeMillis();
            Runtime run = Runtime.getRuntime();
            doc = builder.build(new File(filename));
            System.out.println("Build doc memory ="+(run.totalMemory()-
                     run.freeMemory())/1024+"K");
            System.out.println("Build doc used time :"+(
                     System.currentTimeMillis()-time_start)/1000+" s");
        catch (Exception e) {
            System.err.println("Error creating XML parser in "
                + "PropertyManager.java");
            e.printStackTrace();
     public String [] getChildrenProperties(String parent) {
        // Search for this property by traversing down the XML heirarchy.
        Element element = doc.getRootElement();
        element = element.getChild(parent);
        if (element == null) {
            // This node doesn't match this part of the property name which
            // indicates this property doesn't exist so return empty array.
            return new String [] { };
        // We found matching property, return names of children.
        List children = element.getChildren();
        int childCount = children.size();
        String [] childrenNames = new String[childCount];
        for (int i=0; i<childCount; i++) {
            childrenNames[i] = ((Element)children.get(i)).getName();
        return childrenNames;
    }the test main class:
import java.util.Map;
import java.util.HashMap;
import org.jdom.*;
import org.jdom.input.*;
public class MyTest {
public static void main(String[] args) {
    long time_start = System.currentTimeMillis();
    Runtime run = Runtime.getRuntime();
    Map childs = new HashMap();
    System.out.println("Used memory before="+(run.totalMemory()-run.freeMemory())/1024 + "K");
    XMLProperties parser = new XMLProperties("D:\\projects\\edr\\jsp\\status-data\\edr-status-2003-09-01.xml");
    String[] child = parser.getChildrenProperties("errors");
    for(int i=0;i<child.length;i++) {
//      childs.put(new Integer(i), child);
if(i%1000 == 0) {
System.out.println("Used memory while="+(run.totalMemory()-run.freeMemory()/1024+"K"));
System.out.println("child.length="+child.length);
System.out.println("Used memory after="+(run.totalMemory()-run.freeMemory())/1024+"K");
System.out.println("Time used: "+(System.currentTimeMillis()-time_start)/1000+"s");
The result is : Used memory before=139K
Used memory while=56963K
Used memory after=51442K
child.length=27343
Time used: 146s
is that some way to solve this problem ?
Thanks for your help

I met a OutOfMemorry exception when I parse a xml
l file (20M) in my application(I extract data in xml
file for building my application data)...Rule of thumb for parsing XML: the memory you need is about 10 times the size of the XML file. So in your case you need 200 MB of free memory.
, but I found
that it take long time(for 2 more minutes) even just
searching the xml file , because my application is
viewed by web page...Then you need to redesign your application. Parsing 20 megabytes of XML for every request is -- as you can see -- impractical. Sorry I can't suggest how, since I have no idea what your application is.

Dbms_xmlgen: write emp content to xml file

Hi.
I have the following procedure to write the content of the emp table in xml to a file:
CREATE OR REPLACE PROCEDURE BSP_DBMSXMLGEN IS
v_ctx DBMS_XMLGen.ctxHandle;
v_file Utl_File.File_Type;
v_xml CLOB;
v_more BOOLEAN := TRUE;
l_exception varchar2(2000);
BEGIN
-- XML context erzeugen
v_ctx := DBMS_XMLGen.newContext('SELECT * from emp_big');
-- Root Element und Zeilentag setzen
DBMS_XMLGen.setRowsetTag(v_ctx, 'Emp');
DBMS_XMLGen.setRowTag(v_ctx, 'Zeile');
-- XML Dokument erzeugen
v_xml := DBMS_XMLGen.GetXML(v_ctx);
DBMS_XMLGen.closeContext(v_ctx);
-- Ausgabedatei erzeugen
v_file := Utl_File.FOpen('XML_DIR', 'BSP_DBMSXMLGEN.xml', 'w');
-- In Datei schreiben
WHILE v_more LOOP
Utl_File.Put(v_file, Substr(v_xml, 1, 32767));
IF Length(v_xml) > 32767 THEN
v_xml := Substr(v_xml, 32768);
ELSE
v_more := FALSE;
END IF;
END LOOP;
Utl_File.FClose(v_file);
EXCEPTION
when OTHERS then
l_exception := sqlerrm;
dbms_output.put_line('Error: ' || l_exception);
END;
The procedure works well with 14 emp rows. But after duplicating the rows a few times the procedure stops working...
insert into emp_big select * from emp_big;
commit;
select count(*) from emp_big;
COUNT(*)
960
exec bsp_dbmsxmlgen;
The file content:
<?xml version="1.0"?>
<Emp>
<Zeile>
<EMPNO>7369</EMPNO>
<ENAME>SMITH</ENAME>
<JOB>CLERK</JOB>
<MGR>7902</MGR>
<HIREDATE>17.12.80</HIREDATE>
<SAL>800</SAL>
<DEPTNO>20</DEPTNO>
</Zeile>
<Zeile>
<EMPNO>7499</EMPNO>
<E
The procedure stops writing to file in the middle of a table row...
Any ideas?
Thanks
Markus

Thanks, Marco, but I think I need to be more clear. The original XML data in the XMLType field looks like:
<?xml version="1.0" encoding="ISO-8859-1"?>
<metadata>
</metadata>where "metadata" is the root node of the entire file. What happens is when I run the script listed above, the name of the field being queried gets added as a root node, which I don't want. I did figure out how to omit the "ROW" and ROWSET" nodes from being placed in the resulting XML flat file. Using the same script, I replaced:
-- Root Element und Zeilentag setzen
DBMS_XMLGen.setRowsetTag( v_ctx, 'Emp' );
DBMS_XMLGen.setRowTag( v_ctx, 'Zeile' );with
-- Root Elements
DBMS_XMLGen.setRowsetTag( v_ctx, '' );
DBMS_XMLGen.setRowTag( v_ctx, '' );I simply removed "Emp" and "Zeile." By doing this, those nodes are omitted. I still can't figure out how to omit the "XML_DATA" node.

Non-XML Files

Can anyone tell me if it is possible for XI to both send and receive files that are not in an XML format. It seems that in all the documentation everything is expected to be in XML, but most of the applications that we need to integrate with do not communicate via XML.

I think what the poster above is referring to is the Adapter Framework, which allows SAP to communicate via many other protocols and message formats. There are many off-the-shelf adapters available and there is the alternative of building your own adapter. It depends on the specifics of the scenario. Please see the service marketplace for details on the Adapter Framework's capabilities.
Cheers,
Steve
If you find a post useful, please help keep the community going by setting a good example and rewarding the poster with points.

Problem with parsing large XML files chunked over HTTP

I'm trying to isolate a bug that was introduced when upgrading the JRE in use from Java 7u51 to 7u71 without changing any code. The problem appears to be very similar to: Bug ID: JDK-8027359 XML parser returns incorrect parsing results.
Further investigation showed that it was also introduced in the same versions (7u71) where that fix was applied. Unlike that bug though, my XML is marked as version 1.0. It also appears to be with only large XML files, on the order of 10MB or so.
The closest I've been able to narrow it down to is the code is using JAXB to unmarshall a stream that the debugger tells me is a org.apache.http.com.EofSensorInputStream / org.apache.http.impl.io.ChunkedInputStream. The exception I get is not consistent, but typically appears to be from chunks being overwritten or shuffled, resulting in letters appearing in attributes that are actually numbers, or like the following where an attribute "testAttribute" gets partially overwritten by the end of a timestamp that was in a different section of the XML.
javax.xml.bind.UnmarshalException
- with linked exception:
[javax.xml.stream.XMLStreamException: ParseError at [row,col]:[1,98748]
Message: Attribute name "testAttribu00Z" associated with an element type "testElement" must be followed by the ' = ' character.]
at com.sun.xml.internal.bind.v2.runtime.unmarshaller.UnmarshallerImpl.handleStreamException(UnmarshallerImpl.java:421)
at com.sun.xml.internal.bind.v2.runtime.unmarshaller.UnmarshallerImpl.unmarshal0(UnmarshallerImpl.java:357)
at com.sun.xml.internal.bind.v2.runtime.unmarshaller.UnmarshallerImpl.unmarshal(UnmarshallerImpl.java:334)
Caused by: javax.xml.stream.XMLStreamException: ParseError at [row,col]:[1,98748]
Message: Attribute name "testAttribu00Z" associated with an element type "testElement" must be followed by the ' = ' character.
at com.sun.org.apache.xerces.internal.impl.XMLStreamReaderImpl.next(XMLStreamReaderImpl.java:598)
at com.sun.xml.internal.bind.v2.runtime.unmarshaller.StAXStreamConnector.bridge(StAXStreamConnector.java:181)
at com.sun.xml.internal.bind.v2.runtime.unmarshaller.UnmarshallerImpl.unmarshal0(UnmarshallerImpl.java:355)
... 6 more
Here's some code that seems to reproduce it if you can connect to an XML server that returns a large chunked XML file:
SchemeRegistry registry = new SchemeRegistry();
registry.register(
new Scheme("http", 80, PlainSocketFactory.getSocketFactory()));
HttpClient client = new DefaultHttpClient(new BasicClientConnectionManager(registry));
String url = "http://someUrlReturningAlargeChunkedXML";
HttpGet method = new HttpGet(url);
HttpResponse response = client.execute(method);
InputStream inputStream = response.getEntity().getContent();
XMLStreamReader responseReader = factory.createXMLStreamReader(inputStream);
JAXBElement<JaxBObjectOfResponse> wot = unmarshaller.unmarshal(responseReader, JaxBObjectOfResponse.class);
If you connect using URL.openStream() to the same service there is no error. If I read bytes directly and write to a file, there is no error. The error only happens when I try to unmarshal it, and it's large, and I'm using Java 7u71 (or later). It can be consistently repeated with the jsp webapp that I'm using, but didn't show the error when I used the same code with a Wikipedia dump XML file.
How can I unmarshal in a different way to avoid this problem? Or, how can I better isolate the bug so it can be posted to the appropriate bug system?

Apparently, adding the Woodstox XML libraries avoids the bug. Is there anyone who can reproduce this on another system? Was there any changes to the Stax implementation between u67 and u71 that may have introduced a bug like this?
Edit: When setting the logging level to DEBUG, I once saw the overwritten buffer being logged as if that was what was received (as in the testAttribu00Z example above). I can't repeat that anymore though, and very rarely it does parses with no exception (though it may have still been corrupted). Now the error seems to be consistently on one of the buffer boundaries, as in:
17:08:09,705 DEBUG wire:63 - << "2000[\r][\n]"
17:08:09,705 DEBUG wire:77 - << "trend>....OTHER XML...<trend hours=""
17:08:09,705 DEBUG wire:77 - << "634.0972777777778" datetime="2013-05-21T00:43:48.350Z" t"
17:08:09,705 DEBUG wire:63 - << "[\r][\n]"
17:08:09,705 DEBUG wire:63 - << "2000[\r][\n]"
17:08:09,705 DEBUG wire:77 - << "rend-mode="0">
Exception in thread "main" java.lang.NumberFormatException: t34.0972777777778
at com.sun.xml.internal.bind.DatatypeConverterImpl._parseDouble(DatatypeConverterImpl.java:213)
at mypackage.Trend_JaxbXducedAccessor_hours.parse(TransducedAccessor_field_Double.java:48)
at com.sun.xml.internal.bind.v2.runtime.unmarshaller.StructureLoader.startElement(StructureLoader.java:194)
at com.sun.xml.internal.bind.v2.runtime.unmarshaller.UnmarshallingContext._startElement(UnmarshallingContext.java:486)
at com.sun.xml.internal.bind.v2.runtime.unmarshaller.UnmarshallingContext.startElement(UnmarshallingContext.java:465)
at com.sun.xml.internal.bind.v2.runtime.unmarshaller.InterningXmlVisitor.startElement(InterningXmlVisitor.java:60)
at com.sun.xml.internal.bind.v2.runtime.unmarshaller.StAXStreamConnector.handleStartElement(StAXStreamConnector.java:231)
at com.sun.xml.internal.bind.v2.runtime.unmarshaller.StAXStreamConnector.bridge(StAXStreamConnector.java:165)
at com.sun.xml.internal.bind.v2.runtime.unmarshaller.UnmarshallerImpl.unmarshal0(UnmarshallerImpl.java:355)
at com.sun.xml.internal.bind.v2.runtime.unmarshaller.UnmarshallerImpl.unmarshal(UnmarshallerImpl.java:334)
Or:
17:19:12,563 DEBUG wire:63 - << "2000[\r][\n]"
17:19:12,563 DEBUG wire:77 - << ...OTHER XML...<trend index="5"
17:19:12,563 DEBUG wire:77 - << "" label="N"
17:19:12,563 DEBUG wire:63 - << "[\r][\n]"
Exception in thread "main" java.lang.NumberFormatException: Not a number: N
at com.sun.xml.internal.bind.DatatypeConverterImpl._parseInt(DatatypeConverterImpl.java:106)
at com.sun.xml.internal.bind.DatatypeConverterImpl._parseShort(DatatypeConverterImpl.java:118)

How to write input data into an xml file

Hi All,
I have some input data and i have to write it
into an xml file.How is it possible send me some related
links regarding this and source code if any.

Hi
Try to go through these links.I hope this will help you to solve your problem.
http://www.xml.com/pub/a/2003/07/09/udell.html
Thanks
Mrutyunjaya Tripathy

How to write POJO for XML file.. i do not want to use JAXB or any other tec

How to generate POJOs for the specific xml files.
Each XML file will have elements name with Type. Which in turn related to some other xml file
For example
student.xml
<student name"mystudent">
<studentname type="firstname"/>
<studentname type="firstname"/>
<studentname type="firstname"/>
</student>
firstname.xml
<studentname name="firstname">
<stufirstname type="myfirstname"/>
</studentname>
myfirstname.xml
<myfirstname name="myfirstname">
<myname1>xxx</myname1>
</myfirstname>
like this there will be so many xml files interlinked.
I would like to write POJOs for these xml files
please let me know what is the best way to write (design the hirarchy of these xml files
Thanks in advance
srinivas

Not at this time. You may submit your polite feedback to Apple using this webpage: http://www.apple.com/feedback/itunesapp.html

WebDAV Write Unreliability - Large non-XML Files

Similar Messages

Maybe you are looking for