Storing XML in memory

Hi,
I want to access a XML file from the application very frequently as in for every request.
I would like to know how can this XML be pushed in memory after being read for the first time and use it till the session closes such that the performance increases
Thanks

user13443455 wrote:
I thought of having the sticky session on for both servers but the customer does not want it and hence I am stuck with the performance issue...No, the customer is stuck with the performance issue. If there is one, that is. You have measured how much the reading of the XML slows down a request, right? How much was it, on average?
Edit: actually what I said about the customer is wrong there. You have a customer who is dictating technical design issues such as whether you should persist session data between servers in a cluster. (Hint: sessions are useless if you don't.) So now you have another technical design issue. The customer should decide that too. Put it on the customer's desk and wait for an answer.

Similar Messages

XML parser memory usage

I try to proove the advantage of SAX (and StAX) parser, i.e. the memory usage over time is very low and quite constant over time while parsing a large XML file.
DOM APIs create a DOM that is stored in the memory and therefore the memory usage is at least the filesize.
To analyse the SAX heap usage over time I used the following source:
import java.io.File;
import java.io.FileInputStream;
import java.io.InputStream;
import org.xml.sax.InputSource;
import org.xml.sax.XMLReader;
import org.xml.sax.helpers.XMLReaderFactory;
public class ParserMemTest {
    public static void main(String[] args) {
        System.out.println("Start");
        try {
            InputStream xmlIS = new FileInputStream(
                    new File("xmlTestFile-0.xml") );
            HeapAnalyser ha = new HeapAnalyser(xmlIS);
            InputSource insource = new InputSource(ha);
            XMLReader SAX2parser = XMLReaderFactory.createXMLReader();
            //SAX2EventHandler handler = new SAX2EventHandler();
            //SAX2parser.setContentHandler(handler);
            SAX2parser.parse(insource);
        } catch (Exception e) {
        System.out.println("Finished.");
}and the HeapAnalyser class:
import java.io.IOException;
import java.io.InputStream;
public class HeapAnalyser extends InputStream {
    private InputStream is = null;
    private int byteCounter = 0;
    private int lastByteCounter = 0;
    private int byteStepLogging = 200000; //bytes between logging times of measurement
    public HeapAnalyser(InputStream is) {
        this.is = is;
    @Override
    public int read() throws IOException {
        int b = is.read();
        if(b!=-1) {
            byteCounter++;
        return b;
    @Override
    public int read(byte b[]) throws IOException {
        int i = is.read(b);
        if(i!=-1) {
            byteCounter += i;
        //LOG
        if ((byteCounter-lastByteCounter)>byteStepLogging) {
            lastByteCounter = byteCounter;
            System.out.println(byteCounter + ": " + getHeapSize() + " bytes.");
        return i;
    @Override
    public int read(byte b[], int off, int len) throws IOException {
        int i = is.read(b, off, len);
        if (i!=-1) {
            byteCounter += i;
        //LOG
        if ((byteCounter-lastByteCounter)>byteStepLogging) {
            lastByteCounter = byteCounter;
            System.out.println(byteCounter + ": " + getHeapSize() + " bytes.");
        return i;
    public static String getHeapSize(){
        Runtime.getRuntime().gc();
        return Long.toString((Runtime.getRuntime().totalMemory() - Runtime.getRuntime().freeMemory())/1000);
}and these are the results:
Start
204728: 1013 bytes.
409415: 1713 bytes.
614073: 2400 bytes.
818763: 3085 bytes.
1023449: 3772 bytes.
1228130: 4458 bytes.
1432802: 5145 bytes.
1637473: 5832 bytes.
1842118: 6519 bytes.
2046789: 7206 bytes.
2251470: 7894 bytes.
2456134: 8580 bytes.
2660814: 9268 bytes.
2865496: 9955 bytes.
3070177: 10625 bytes.
3274775: 11287 bytes.
3479418: 11950 bytes.
3684031: 12612 bytes.
3888695: 13275 bytes.
4093364: 13937 bytes.
4298027: 14600 bytes.
4502694: 15262 bytes.
4707372: 15925 bytes.
4912040: 16586 bytes.
5116662: 17249 bytes.
5321331: 17912 bytes.
5525975: 18574 bytes.
5730640: 19237 bytes.
5935308: 19898 bytes.
Finished.
As you can see while parsing the XML file (200k elements, about 6MB) the heap memory raises. I would expect this result when a DOM API is analysed, but not with SAX .
What could be the reason? The Runtime class measurement, the SAX implementation or what?
thanks!

http://img214.imageshack.us/img214/7277/jprobeparser.jpg
Test with jProbe while parsing the 64MB XML file.
Testsystem: Windows 7 64bit, java version "1.6.0_20" Java(TM) SE Runtime Environment (build 1.6.0_20-b02), Java HotSpot(TM) 64-Bit Server VM (build 16.3-b01, mixed mode), Xerces 2.10.0
Eclipse Console System output:
25818828: 116752 bytes.
26018980: 117948 bytes.
26219154: 99503 bytes.
26419322: 100852 bytes.
26619463: 102275 bytes.
26819642: 103624 bytes.
27019805: 104974 bytes.
27220008: 105649 bytes.
27420115: 106998 bytes.
27620234: 108348 bytes.
27820330: 109697 bytes.
Exception in thread "main" java.lang.OutOfMemoryError: PermGen space
at java.lang.String.intern(Native Method)
at org.apache.xerces.util.SymbolTable$Entry.<init>(Unknown Source)
at org.apache.xerces.util.SymbolTable.addSymbol(Unknown Source)
at org.apache.xerces.impl.XMLEntityScanner.scanQName(Unknown Source)
at org.apache.xerces.impl.XMLNSDocumentScannerImpl.scanStartElement(Unknown Source)
at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl$FragmentContentDispatcher.dispatch(Unknown Source)
at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown Source)
at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
at org.apache.xerces.parsers.AbstractSAXParser.parse(Unknown Source)
at ParserMemTest.main(ParserMemTest.java:47)<img src="file:///C:/Users/*******/AppData/Local/Temp/moz-screenshot.png" alt="" />
Edited by: SUNMrFlipp on Sep 13, 2010 11:48 AM
Edited by: SUNMrFlipp on Sep 13, 2010 11:50 AM

I am trying to download photos stored on a memory stick to my imac, but two of the files are saying unreadable. Why? Are these two sets of photos never going to be copied onto my Mac? If so, how can I get them onto my iphoto folder. Desperate. Glen

I am trying to put photos stored in a memory stick onto my Mac computer. But there are two folders that will not move. I get an error message saying - volumes/FLASH DRIVE/desktop. ini - can someone help me please? The photos are from my windows computer. Please answer back assuming I am a total drip - I am a lot older than you I should imagine !! Thanks Glenys

Can you double click on them on the memory stick, see if preview opens them, then save them to your Mac?
...JER

Using XSU for Storing XML data

Please how can I use the XSU API for storing XML documents in Oracle8i? I know part of the code to store a document but how do I import the XSU API?
Do I have to download it or use it via Oracle?
example:
String xmlDoc = "my_xml_document";
Connection conn = >>DriverManager.getConnection(...);
OracleXMLSave sav = new OracleXMLSave
(conn,"purchaseOrderTab");
sav.insertXML(xmlDoc);But how do I use OracleXMLSave? How do I import the XSU API?
thanks for your help
null

What is XSU? I think you question should be posted at the XML or XDK forum.

Storing XML as XML

Hi, there. Here's my question for XML gurus. What I need to do and I am currently evaluating a number of products to find ones that best fit my needs:
1) store XML documents as they come, in their XML form in some sort of XML repository.
2) Be able to run various queries against that XML repository. For example, if there exist a bunch of documents of DOCTYPE "blah" (among others) I need to query the repository like: for all documents of this doctype select docs that have a value of a tag <PurchaseOrder> between 11-01-2000 and 11-14-2000.
In other words, I'd like to operate just on XML, and nothing else. Store XML and query the repository to get XML back.
Is Oracle's XML products capable of doing something like that ? If so, which ones would I need to utilize to obtain the results. And if not, could anyone point me towards the products that are capable of doing what I need ?
Any help and feedback would be greatly appreciated.
Thank you.

<BLOCKQUOTE><font size="1" face="Verdana, Arial">quote:</font><HR>Originally posted by Dmitriy:
Thank you for your prompt reply. 'Dynamic news' app looks great, but it's not exactly what I need. The emphasis is on storing XML messages as they are, not broken up into relational representation, AND being able to query those messages on the values of the different tags contained in those XML docs.
Can Oracle do that ?<HR></BLOCKQUOTE>
Oracle interMedia could be help you in this problem, go to the interMedia section on this web site and look at in the examples and the documentation.
Best regards, Marcelo.
null

HTTPService + XML Load + Memory Leak

Hi all....
I have noticed a memory leak in my application. This leak was
not apparent when the application was completed some months back
which is what left me a little confused as all I have done since
was upgrade to Flex 3 and possibly updated / changed my Flash
player.
I think I have found the cause to this problem (below) but am
not 100% sure that it is the "actual" problem or any reasons to
back my thoughts up, so have listed what I have checked / tried
along the way (maybe I have missed something)....
My Discovery Process:
I started profiling my application but did not find anything
out of the ordinary. I did a code walk-through double checking I
had cleaned up after myself, removing and even nulling all items
etc but still to now success - the leak is still there.
I have profiled the app in the profiler for reasonably long
periods of time.
All the classes etc being used within the app are consistent
in size and instance amount and there is no sign of any apparent
leak.
I am using a HTTPService that is loading XML data which I am
refreshing every 5 seconds. On this 5 second data refresh some
class instances are incremented but are restored to the expected
amount after a GC has occurred. The GC seems to take longer, the
longer the app is running, therefore more and more instances are
being added to the app, but when the GC eventually runs it "seems"
to clear these instances to the expected amount.
After scratching my head for a while I decided to make a copy
of my application, rip everything out, and focus in my data load,
where I found a problem!
I have now just a HTTPService that loads an XML file every 5
seconds, and this is all I currently have in the app (as I ripped
the rest of the code out), e.g:
Code:
<mx:Application xmlns:mx="
http://www.adobe.com/2006/mxml"
....... creationComplete="initApp()" >
<mx:HTTPService
id="httpServiceResults"
url="
http://myIP:myPort/myRoot/myXML.cfm"
resultFormat="e4x"
result="httpResultHandler(event)" />
<script....... >
private var timerPulse:Timer;
private function initApp():void
httpServiceResults.send();
timerPulse = new Timer(5000, 0);
private function httpResultHandler(event:ResultEvent):void
timerPulse.start();
timerPulse.addEventListener(TimerEvent.TIMER, timerRefresh);
public function timerRefresh(eventObj:TimerEvent):void
timerPulse.stop();
timerPulse.removeEventListener(TimerEvent.TIMER,
timerRefresh);
timerPulse.reset();
httpServiceResults.send();
</script>
</mx:Application>
This is pretty much the code I am currently using and it
leaks.
I ran and monitored this in both the profiler and the
activity / task manager, and after running the app for 1800 seconds
(30 min) in the profiler, the memory size grew from 50mg to 165mg
just sending the HTTPService.
I tried loading the service in multiple ways including in AS
rather than MXML creating new instances of it each time, resetting
it, nulling it etc... but nothing prevented this memory increase.
I then tried to load the XML using different methods such as
using the URLRequest and URLLoader which again caused a memory
leak.
What still confuses me is that this leak did not exist in the
previous version and not a great deal has changed since then apart
from upgrading to Flex 3 and possibly upgrading my Flash payer
(which I believe is a possible cause)
After looking into this issue a bit more deeply, I read a few
blogs / forums and other people are experiencing the same problems
- even with a lot bigger leaks in some cases all when reloading
large sets of XML data into Flex - however, I as of yet found no
solution to this leak - people with a similar problem believe it is
not due to a memory leak more a GC error, and others pointing
towards the Flash Player itself that is leaking - I don't really
know.
Findings so far during investigation of this issue:
* App leaks for both HTTPService and ULRRequest / URLLoader
methods
* App only leaks when calling a data loader
* The size of the leak seems to depend on the size of the
XML being loaded
* The size of the leak also seems to be affected by the
applications heaviness - the greater seems to enhance the leak
An interesting factor I have noticed is that if I copy the
XML from my "myXML.cfm" that I link to in my HTTPService and copy
the contents of the file into my own XML file stored within the
Flex project root itself: ""myXML.xml"" the leak disappears... like
it seems to link when loading the XML over a network, however as my
network knowledge is not great I am not sure what to make of this -
any ideas???
Could the connection to the XML document cause leaks??? is
there anything else that could cause this leak??? have I something
in my code sample that could cause this leak??? or could any of the
other things I have mentioed cause this leak???
Any help / ideas would be greatly appreciated.
Thanks,
Jon.

I also observed heavy memory leak from using httpservice with
XML data. I am using Flex3 builder under Linux. My Flex application
polls httpservice every 10 seconds. The reply is a short XML
message less than 100 bytes. This simple polling will consume 30+
MB of memory every hour. I leave it idling for several hours and it
took 200 MB of memory. No sign of garbage collection at all.

Is DocumentBuilder.parse(..) reading the whole XML into memory?

Hello,
I have a quick question about the following code snippet:
DocumentBuilderFactory factory = DocumentBuilderFactory.newInstance();
factory.setNamespaceAware(true);
DocumentBuilder builder = factory.newDocumentBuilder();
Document doc = builder.parse("books.xml");Please have a look at the last line "builder.parse("books.xml");". I would like to use XPath to retrieve some informations of a xml file without having to deserialize the whole file (since the XML file rather big and I need to go through several of them in a loop).
What exactly does the "builder.parse(...)" method? Is it reading the XML file and writing all the content to memory? Or does it only analyze and read the structure of the XML code? Or in other words, is the text of an element "//books/title" in memory after parsing, or not until I really access the "title" node of that XML?
Thanks a lot for your help
Best Regards,
Ben

Hi ,
I did'nt knew how to start a new thread , so i am asking the question here:
===========================================================
I am trying to read a xml file but i am not able to do so properly. I am new to xml .
am trying to read a xml file of the following format in the staf and stax job xml file:
<?xml version="1.0" encoding="utf-8"?>
<operating_system>
<unix_80sp1>
<tests type="quick_sanity_test">
<prerequisitescript>preparequicksanityscript</prerequisitescript>
<acbuildpath>acbuildpath</acbuildpath>
<testsuitscript>test quick sanity script</testsuitscript>
<testdir>quick sanity dir</testdir>
</tests>
<machine_name>u80sp1_L004</machine_name>
<machine_name>u80sp1_L005</machine_name>
<machine_name>xyz.pxy.dxe.cde</machine_name>
<vmware id="155.35.3.55">144.35.3.90</vmware>
<vmware id="155.35.3.56">144.35.3.91</vmware>
</unix_80sp1>
</operating_system>
Here machine_name tag can be n number. I need to read all the machine_name tags in an array , so that i can use them later.
Also i need to read all the vmware tags in an array (both value and attributes) so that i can use them later. The number of vmware tags can again be n numbers.
I am trying to use the following code :




<function name="parseXML" scope="local">
<function-list-args>
<function-required-arg name="xmlFileName">
Name of file containing XML to be parsed
</function-required-arg>
</function-list-args>
<sequence>

<script>
factory = DocumentBuilderFactory.newInstance();
factory.setValidating(1)
factory.setIgnoringElementContentWhitespace(0)
builder = factory.newDocumentBuilder()
document = builder.parse(xmlFileName)
</script>
<script>
vmware = None
machine_name = None
# Get the text value for the element with tag name "vmware"
nodeList = document.getElementsByTagName("vmware")
for i in range(nodeList.getLength()):
node = nodeList.item(i)
if node.getNodeType() == Node.ELEMENT_NODE:
children = node.getChildNodes()
for j in range(children.getLength()):
thisChild = children.item(j)
if (thisChild.getNodeType() == Node.TEXT_NODE):
machine_name = thisChild.getNodeValue()
# Get the text value for the element with tag name "machine_name"
nodeList = document.getElementsByTagName("vmware")
for i in range(nodeList.getLength()):
node = nodeList.item(i)
if node.getNodeType() == Node.ELEMENT_NODE:
children = node.getChildNodes()
for j in range(children.getLength()):
thisChild = children.item(j)
if (thisChild.getNodeType() == Node.TEXT_NODE):
machine_name = thisChild.getNodeValue()
</script>
<return>[machine_name,vmware]</return>
</sequence>
</function>
I am able to read only one value for machine_name (u80sp1_L005) and also one value for vmware
i.e 144.35.3.91.
What i want is read all the machine_value tags into a list , so that i can use it later
Also i want to read all the vmware tags attributes into a list and all corresponding values into another list.
what i mean is considering the following:_
<machine_name>u80sp1_L004</machine_name>
<machine_name>u80sp1_L005</machine_name>
<machine_name>xyz.pxy.dxe.cde</machine_name>
<vmware id="155.35.3.55">144.35.3.90</vmware>
<vmware id="155.35.3.56">144.35.3.91</vmware>
read all machine_name tag values into one list (u80sp1_L004,u80sp1_L005,xyz.pxy.dxe.cde)
also all vmware attributes into one list *(155.35.3.55,155.35.3.56).....*and all the vmware values into one list *(144.35.3.90,144.35.3.91)*
Also a good link to understand parsing using doumentbilder factory would be good help.
Regards
Sangram
mail: [email protected]

XML SCHEMA registration for XML TYPE (storing XML files in Oracle 10g)

I have created the XML Schema for the XML file stored in Oracle 10g and also added this Schema into the database. I have related that schema with the column in the table which contains the XML file. When i execute the query to fetch the data from the stored file i am getting a blank resultset. Is registering the XML Schema is necessary, if yes then please let me know the process of doing it. I have tried following steps to register Schema, but it is not working
Step1:
DECLARE
v_return BOOLEAN;
BEGIN
v_return := dbms_xdb.createFolder('/home/');
v_return := dbms_xdb.createFolder('/home/DEV/');
v_return := dbms_xdb.createFolder('/home/DEV/xsd/');
v_return := dbms_xdb.createFolder('/home/DEV/messages/');
v_return := dbms_xdb.createFolder('/home/DEV/employees/');
COMMIT;
END;
STEP 2:
Connecting To XML DB
Step3:
Register XML schema
I am failing to execute step number 2 and hence not able to register the schema also.

<BLOCKQUOTE><font size="1" face="Verdana, Arial">quote:</font><HR>Originally posted by sudeepk:
If a java exception is thrown probably during ur install u might have forgotten
grant javauserpriv to scott;
grant javasyspriv to scott;
Thanks
[email protected]
<HR></BLOCKQUOTE>
Thank you!!!

Where is stored xml payload of XI messages

Hi to all,
where can I find the XML payload of each XI Message that I see in the SXI_MONITOR? I want to find the table or the FUNCTION MODULE that contains/retries the XML Payload.
Thanks to all!

Hi,
The messages are stored in the Table.
SXMSPMAST---->Integration Engine: Message Queue (Master)
SXMSSYERR---->XMS: System Error Error Codes
SXMSCLUP -
>XMB: Property Cluster
SXMSCLUR -
>XMB: Resources Cluster
Regards
San

Storing XML schema in database

Hi everybody..I have question here related to the storing of the XML schema in the database. When an XML Schema is created like this.
dbms_xdb.createResource('home/DEV/xml_schema_test.xsd'),
then is the XSD created and stored in one of the database tables or is it stored physcially as an xsd file in the database repository or both?
If it is stored in one of database tables, then which table is it?
If it is stored as an XSD file in the database repository, then how can I access the xsd file if I created it like in the example above.
To access the xsd file over http, do I need to start the xsdhttp server or is it already started if I loaded XDK in my database.
Would the url for accessing the same over http be
http://db_machine:8080/home/DEV/xml_schema_test.xsd.
I think I have got information in bits and pieces but not able to put everything together.
Thanks in advance for the replies.

The questionssss you are asking are dealt with in the FAQ link at the top of the first page on this forum, the rest is described in detail in the XML DB Developers Guide. A little bit of reading wouldn't harm, would it.
Message was edited by:
mgralike

Storing xml in oracle 8i db

Hi,
I'm trying to store a XML string into a long coloumn in Oracle 8i database. I'm doing this thro a Stored Procedure. when the size of the XML statemene goes beyond 4000, i get the following error message on the command.execute statement
"ORA-01460: unimplemented or unreasonable conversion requested"
Has anyone come across this problem OR does anyone have a solution to this. All help is throughly appreciated.
Thank you.
null

OTN has some sample apps that show how to store documents in the database as CLOBs:
[list][*]Customizing Web Content
[*]B2B with XML
[*]Managing Web Content (CMS)
[*]XML Live Demos
[list]
Regards,
-rh

Deletion of messages stored on mass memory on N97

Hi - I was wondering if anybody could help please ?
I've got a N97 which I've had around 180 messages in my Inbox and Sent Items which used the mass memory (E drive) to store the messages on.
I did a backup of the device on the phone via the File Manager application, and did a reinstall of the firmware.
Although the reinstall of the firmware didn't touch the music stored on the mass memory it appears to have deleted the messages I had stored, and unfortunately the backup doesn't appear to have stored the messages ...
Does anyone know if there is anyway of accessing/retrieving the messages which should hopefully still be on my E drive please ?
Thanks in advance.
Regards,
Edward

Ok, I had to try it out.
1. I had 1 message on C: and did a backup in filemanager. It backed up on F: on default. - I deleted the message on C: and restored the backup. My message got back.
2. If I deleted my message on C: and had 1 message on E: and did a backup - it also stored the backup on F: by default. - I set my message memory to E: and restored. - No message to see. - Cause it told me it would restore the backup to phonememory (C: ). I changed the messagememory to C: - but could not find any message.
As I can see, the backupprogram only backs up messages from C....
Message Edited by Fender77 on 29-Jan-2010 09:57 PM

How to view the messages stored in my memory card

hey..
i have just bought the c6-01..and my prev memory card has my messages which are imp ..is there a way frm which i can view those messages
thank u

Symbian^3 currently has no way of looking at stored messages in the memory card. If you would like to access them, load the memory card back to your old phone and sync them to your PC via Ovi suite where these messages can be read on the program. You can move these to the C6-01 via Ovi Suite should you desire to do so.
If you find my post helpful please click the green star on the left under the avatar. Thanks.

XML/C++ Memory problems

I've been experimenting with the C++ version of the XML parser using our own internal XML/XSL formats. I have found the program performs fairly slowly, allocating memory until the system memory (>512Mb) is consumed and then crashes with an out-of-memory error. I'm running about 200 XSL matches across about 200 main XML nodes. I can provide sample XML/XSL if this will help.
Any ideas?

We are aware of the memory usage issues. These are being addressed and you will see improvements across the next several releases.

Storing XML to Oracle

Hii
I am creating a application using Swiing .
The user(client side) stores information locally in a xml file , and whenever he has access to internet , he can connect to server , and save the data from the xml file (stored at client side) to the different tables at the server side
It is similar to SAX Loader Application as given in OTN XML DB Samples.
This document describes the SAX Loader Application that demonstrates an efficient way of breaking a large files containing multiple XML documents outside the database and then inserting them as a set of separate documents
Can anybody tell me how to go abt it.
Bcz i a dont know where to start

i am not familiar with example "SAX Loader Application". For me you do not have to chop huge XML files to insert CLOB fields... Use "getBinaryOutputStream()" method to get output stream of clob field you selected. and than read XML file using "FileInputStream" and write to Output Stream of CLOB field. that's it.... Do not forget to flush ;)

Storing XML in memory

Similar Messages

Maybe you are looking for