Store XML on fly

Question

Hi,
I have an application that parses an XML file, stores data in memory. At the end, it generates a different XML file. Now the original file is too large and the application frequently runs out of memory. I am wondering if instead of storing data in memory during parsing, it can store them as XML somewhere on disk. During parsing, the application will keep on adding nodes to the XML file and occasionally needs to update a few notes already stored.
I've been reading about XML database, not sure if it is over kill here. Any suggestions on which data base, API to use, or a suggestion on different approaches are appreciated

Answer 1

Is your need is to store a big XML file, sometimes get from other data sources some additionnal contents (XML subtrees or flat content...) to complete the big one, and finally merge additional data to XML.
If XML big file is never used during the app processing, except sometimes storing data, you can implement a data cache for storing additional data, and when closing the process, or every n stored data set, you are able to wander (througth SAX parsing) the huge XML container to add data as needed.
When your application decide to merge data, it can parse the big file with a simple SAX contentHandler, you contenthandler implementation will ask the cache to get additional data for current node, and will perform merge as desired.
But if your application need to keep the big file in memory for DOM access, natularilly u encounter memory problems. And database, or a DOM on Disk solution is needed.