Memory issue loading XML schema files

Hi,
I have an application that a startup reads available XML schema files from a folder and creates Validator objects that are put into a HashMap.
I'm running into problems creating validation schema objects when the file size of the schema is just over 30kB in size.
The following error message is given when testing with the "large" schemas:
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
     at com.sun.org.apache.xerces.internal.impl.dtd.models.CMStateSet.<init>(CMStateSet.java:100)
     at com.sun.org.apache.xerces.internal.impl.dtd.models.CMNode.firstPos(CMNode.java:96)
     at com.sun.org.apache.xerces.internal.impl.xs.models.XSCMBinOp.calcFirstPos(XSCMBinOp.java:136)
     at com.sun.org.apache.xerces.internal.impl.dtd.models.CMNode.firstPos(CMNode.java:97)
     at com.sun.org.apache.xerces.internal.impl.xs.XSComplexTypeDecl.getContentModel(XSComplexTypeDecl.java:185)
     at com.sun.org.apache.xerces.internal.impl.xs.XSConstraints.fullSchemaChecking(XSConstraints.java:457)
     at com.sun.org.apache.xerces.internal.impl.xs.XMLSchemaLoader.loadGrammar(XMLSchemaLoader.java:530)
     at com.sun.org.apache.xerces.internal.jaxp.validation.xs.SchemaFactoryImpl.newSchema(SchemaFactoryImpl.java:206)
     at javax.xml.validation.SchemaFactory.newSchema(SchemaFactory.java:489)
The smaller 1-10kB schemas can be stored without any problem regardless of the number I test (+1000).
The problem occurs as soon as it handles these larger schemas in the source directory. Even only one 30kB schema is enough to consume the all memory.
Below is the code uesd for creating the validation objects
for(int i=0;i<files.length;i++){
    SchemaFactory factory = SchemaFactory.newInstance(XMLConstants.W3C_XML_SCHEMA_NS_URI);
    Source schemaFile = new StreamSource(files);
schema = factory.newSchema(schemaFile); // Here is where the process fails
Validator validator = schema.newValidator();
hm.put(documentType, validator); //hm is the HashMap used as placeholder for Validator objects
Regards,
Håkan
Edited by: dezer on Jun 2, 2010 5:04 AM
Edited by: dezer on Jun 2, 2010 5:08 AM

Welcome to the Sun forums.
dezer wrote:
..Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
The smaller 1-10kB schemas can be stored without any problem regardless of the number I test (+1000).
The problem occurs as soon as it handles these larger schemas in the source directory. Even only one 30kB schema is enough to consume the all memory.Are you certain that only one schema is involved? How does the code tell how many schemas there are?
Below is the code uesd for creating the validation objects
for(int i=0;i<files.length;i++){
SchemaFactory factory = SchemaFactory.newInstance(XMLConstants.W3C_XML_SCHEMA_NS_URI);
Source schemaFile = new StreamSource(files);
schema = factory.newSchema(schemaFile); // Here is where the process fails
Validator validator = schema.newValidator();
hm.put(documentType, validator); //hm is the HashMap used as placeholder for Validator objects
It would seem that if files.length is 10000, that would end up in around 30 times as much memory usage as for the 1000x10Kb schemas.

Similar Messages

XML Schema file in data definition

Hi,
I am creating a custom RTF template for a seeded data definition in the eBusiness Suite. The data definition has an XML Schema file loaded however this does not include all the data required for the template. Running the report to just generate the XML (i.e. no template) I can see that the additional data is available at this level just not included in the XSD.
I have built the template including the additional elements from the XML and expected to have to create a custom data definition as well to add the additional elements to the XSD. However the template is working against the seeded definition and successfully picking up the elements that are not included in the XSD.
Has anybody else come across this situation? And are there any potential issues? I could create a custom data definition as well to load an amended XSD but this would be a more invasive change as it would involve end dating the seeded data definition which I would prefer not to do. The report is not being generated from a concurrent request so I cannot just create a custom request and leave the seeded report untouched.
Thanks,
Sarah

The XML Schema is only required if you are using a PDF template (the template file itself is PDF) with field-mapping. For other template types the schema is optional.
The XML Schema is used to define a "class" of XML documents, so to speak. The define the semantics and structure of the XML documents more thoroughly.
If you have closely related reports, you could use a single schema to standardize (in a sense) the XML to be output from the class of related reports. In that case you could have a single schema file uploaded to multiple reports/data-definitions.
The W3C has good documentation on XML Schema.
If you want to see samples, you can use tools such as dtd2xs to create sample schemas from your DTD.

Parse XML Schema File?

Dear Sun Developer Network,
I suppose maybe I should start at the beginning.
Can anyone outline the first few steps involved in parsing an XML schema file into a traversable data structure in Java?
I'd like to get a get a discussion started here about this, because I have some questions about the process as I currently understand it, and the more advanced (i'm supposing they're more advanced) questions that I've been asking in the past have gone completely unanswered in this forum. So perhaps, starting from the beginning, and working through to the difficulties I'm encountering would be a better approach to solving those issues.
Thank you,
Nels

>
Many thanks for your reply! My next question is,
s, if parsing an XML schema file can be done with a
SAX or dom parser, then why has Sun bothered to
develop the JAXB api? What is the difference?
Hi Nels,
SAX and DOM are technologies for parsing and JAXB is a technology for databinding. They are 2 different technologies.
Parsing gives you acess to raw xml
DataBinding does the parsing and goes one step ahed and loads the data from xml to java objects. It is a higher level of abstraction over parsing
schema is a specification which helps to define the rules for a valid xml document. It also happens to be xml. DTD is a kind of limited schema which is non xml.
hope you got the diffrence.
ps: did you try the -J-Xmx switch with javac to resolve yhe OutOfMemory exception I suggested in your previous post
Regards,
Rajagopal

There is not enough free memory to load the entire file

I saw an older reference to this error in a previous post which apparently got resolved, but didn't explain precisely how it got resolved:
Michael Kitzmiller, "There is not enough free memory to load the entire file" #, 27 Jul 2007 5:23 am
I presume that one answer is to rebuild the document from scratch, changing imported images to referenced images. But I want to recover the file as I put hours of work into it and expected that anything I can save that I also should be able to read back in. I will certainly change to referenced images, but I have to be able to open it first.
thank you

Web...
Here is a smattering of ideas.
Do you have ample unused HDD space? If not, defrag it, although in myu experience, this hardly makes a difference. Check your TEMP/TMP folders and clean them out. Consider downloading cCleaner and running it. Do everything you can to make ample room on your HDD and in places like your TEMP folders. If your HDD is congested, temporarily remove big but relatively unimportant applications and move data elsewhere for a while.
Even with 4GB of RAM, Frame can only use somewhere between 2.7 - 3.2 GB of it under the very best of conditions, and that includes the application as well as the files. Unpacking image files can consume considerably more space than it takes to store them. For example, a 1024 x 768 color image requires more than 3.1MB to describe it as a 32-bit color image, yet when stored in JPEG format, it can be as small as 50kB, a compression of 65:1. Of course, it's not reasonable to assume that your document is comprised entirely of such highly-compressed images, but in the unachievable extreme, it would take 980MB to simultaneously open 15MB worth of those kind of files. The point here is to not fall into the trap of thinking a 15MB file comprising a lot of images needs little more than 15MB of RAM.
Make sure ALL other applications are closed when trying to open your file...Outlook, IE or FireFox, Anti-Virus and Spyware apps, Word, etc.
Consider sending a copy of the file to someone else to see if they can open it for you and delete some of the images before resaving it. If you have a backup copy, rename your current file and restore the backup. Maybe your working copy is corrupt. Who knows? Anything is probably worth trying as long as you have a save copy stored somewhere.
Dennis...

Location of XML schema file (XSD)

Hi! Previously a XML schema was imported into the Console Manager and used for the import of data into the repository. Now we have made some changes and I would like to change the XML schema file. However, I would like to get a copy of the previously import schema (XSD) file from the system. Does anyone know where this file can possibly be held? Any way for us to extract this file to update?
Cheers!
SF

Hi SF,
the XSD is actually stored in MDM's database. One option to export is using the transport schema mechanism in MDM console - asuming that you're on MDM 7.1. If so, right-click on your repository in MDM console and choose Transport -> Export Repository Schema in the context menu. Doing this you'll get a huge XML file describing your repository. Open this XML in a suitable editor (e.g. IE, Wordpad, XML Spy, and so on) and check the file contents. Somewhere inside you'll find a section for XML Schemas and your XSD.
If you are on 5.5, the only option is accessing the DB directly...
Best regards
Michael

Loading XML Configuration File into Java

I have a stored procedure wrapper to a Java method (within a jar file) that has been loaded into the database. I need to load an XML configuration file in this method and parse it but get the following error.
<Line 2, Column 192>: XML-24538: (Error) Can not find definition for element 'LODConfigs'
When I run the method standalone through netbeans it parses fine and is able to load the file. I've confirmed that it is able to read the file but I am lost as to what could be the problem. Any suggestions would be welcome.
The header of the file is
<?xml version="1.0" encoding="UTF-8" ?>
<LODConfigs xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.oracle.com/spatial/network/lodLODConfigs.xsd"
xmlns="http://xmlns.oracle.com/spatial/network">
<LODConfig globalNetworkName="$DEFAULT$" networkName="$DEFAULT$">
<networkIO>
</networkIO>
</LODConfig>
</LODConfigs>

Just a guess really, (and this is an old question now - so you may have moved on).. is your XML parser attempting to use the schema location to pull the XML schema from the internet?
If so, that might run fine in netbeans, but hit java permission limits in Oracle. Have you tried running it in netbeans with the network cable unplugged.
If that fails with the same error as the Oracle JVM, that might give a clue as to the problem and you can troubleshoot the permissions from there.

Loading XML(XSD) file into database by validating the format in file

Hi,
I have a file whose format is predefined as listed below
<?xml version="1.0" encoding="utf-8" ?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema">
     <xs:element name="VVV">
          <xs:complexType>
               <xs:sequence>
                    <xs:element name="VVVHeader">
                    <xs:complexType>
                    <xs:sequence>
                    <xs:element name="UniqueIdentifier" type="xs:SEQ0001"/>
                    <xs:element name="VVVFileType" type="xs:123"/>
                    <xs:element name="FileCreatedDatetime" type="xs:23-MAY-2006"/>
                    <xs:element name="ScanDatetime" type="xs:23-MAY-2006"/>
                    <xs:element name="BatchID" type="xs:456"/>
                    <xs:element name="BatchSequence" type="xs:1001"/>
                    <xs:element name="DataEntryDatetime" type="xs:23-MAY-2006"/>
                    </xs:sequence>
                    </xs:complexType>
                    </xs:element>
                    <xs:element name="VVVDetail">
                    <xs:complexType>
                    <xs:sequence>
                    <xs:element name="RRRLocalOffice" type="xs:Mumbai"/>
                    <xs:element name="Duplicate" type="xs:No"/>
                    <xs:element name="ReRegMarker" type="xs:boolean"/>
                    <xs:element name="RegistrationMark" type="xs:Vishnu"/>
                    <xs:element name="RegMarkCheckDigit" type="xs:12S"/>
                    <xs:element name="TaxClassCode" type="xs:67"/>
                    <xs:element name="ExternalMakeCode" type="xs:Maruti"/>
                    <xs:element name="ExternalModelCode" type="xs:800LX"/>
                    <xs:element name="RRRMakeCode" type="xs:Mar"/>
                    <xs:element name="RRRModelCode" type="xs:800LV"/>
                    <xs:element name="RRRVehicleBodyCode" type="xs:98"/>
                    <xs:element name="RRRColourCode" type="xs:red"/>
                    <xs:element name="RegistrationDate" type="xs:23-MAY-2006"/>
                    <xs:element name="ChassisNumber" type="xs:num123"/>
                    <xs:element name="HC" type="xs:1.22"/>
                    <xs:element name="UnWeight" type="xs:90"/>
                    <xs:element name="NoSeats" type="xs:four"/>
                    <xs:element name="NOx" type="xs:2.22"/>
                    <xs:element name="RevenueWeight" type="xs:34"/>
                    <xs:element name="CO2" type="xs:55"/>
                    <xs:element name="Particulates" type="xs:long"/>
                    <xs:element name="CO" type="xs:3.33"/>
                    <xs:element name="HCNOx" type="xs:4.44"/>
                    <xs:element name="TrailerWeight" type="xs:66"/>
                    <xs:element name="StationaryLevel" type="xs:77"/>
                    <xs:element name="EngineSpeed" type="xs:88"/>
                    <xs:element name="DriveBynature" type="xs:99"/>
                    <xs:element name="SMMTFleetCode" type="xs:yel"/>
                    <xs:element name="Purchasercode" type="xs:4004"/>
                    <xs:element name="IndustryofUse" type="xs:office"/>
                    <xs:element name="OriginalDealerCode" type="xs:tr56"/>
                    <xs:element name="SellingDealerCode" type="xs:se23"/>
                    <xs:element name="bill110" type="xs:srira"/>
                    <xs:element name="bill111" type="xs:mula"/>
                    <xs:element name="SalesType" type="xs:krish"/>
                    </xs:sequence>
                    </xs:complexType>
                    </xs:element>
               </xs:sequence>
          </xs:complexType>
     </xs:element>
</xs:schema>
I would like to update this into a a table by validating the format of the XSD file.
I will be having several similar files.
Steps I will be doing on this are
1 > Pick up file from a directory one at a time
2 > Validate the format of XSD file
3 > If valid load into a table
How can I achieve above steps using PL/SQL
Pls mail me ur suggestions on very urgent basis
Thanks in advance
Vishnu

Really this should be in the XML forum, but anyway....
Just a small question... It looks as though you are using an XML schema (XSD) to store your table information along with it's data, am I correct? Seems very bizarre thing to do.
If you are using Oracle 10g you can drop your file into the Oracle WebDAV area (XDB) and from there you can access the XML as an XMLTYPE using something like...
SELECT rv.res.extract('/Resource/Contents/*')
FROM   resource_view rv
WHERE lower(rv.any_path) = lower(lc_filename)Once you have it in the XMLTYPE variable from that query, and because your schema doesn't really define the datatypes etc. it's gonna be up to you to parse the XML using something like the DBMS_XMLDOM package and process that into create table statements or something.
Certainly looks like you've got yourself a nice task to do there. Glad I'm not doing it.
;)

XML Schema validator fails to load XML Schema from URL

I'm trying to validate an XML document against a set of XML Schemas and the parser seems to freak out: even though all the schemas are referred to in the XML and are properly recognized by Schemas field in XML document properties, I get warnings (multiple
instances):
Request for the permission of type 'System.Net.WebPermission, System, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089' failed
Cannot resolve the 'schemaLocation' attribute
The schema referenced from this location in your document contains errors
The warning locations are also seemingly random - errors in schema locations are reported on closing tags inside XML or on whitespaces, and so on.
The first warning (System.Net.WebPermission) suggests it has something to do with access to remote schemas, but all the necessary remote schemas are visible in XML Schema Set editor for the document. I also enabled downloading of XML schemas in XML text
editor options.

Hi Tomasz,
Thank you for posting in MSDN forum.
Since this issue is related to the XML Schemas, so we will move this case to the XML forum:https://social.msdn.microsoft.com/Forums/vstudio/en-US/home?forum=xmlandnetfx
, you will get better support.
Best Regards,
We are trying to better understand customer views on social support experience, so your participation in this interview project would be greatly appreciated if you have time. Thanks for helping make community forums a great place.
Click
HERE to participate the survey.

LoadMovie to load xml button file... undefined

Hi, I am a designer and not familier with programming, my
fiend purchased some flash buttons with xml file, when i run its
swf its work perfect for me. there are 6 six buttons in it and i
can easily change their name from xml file.
ok here is the problem
i want to assemble those button in another file and just
those buttons to run from a different main file.
i made a movie clip container and gave it an instance name
holder_mc
on main file on frame action i wrote this
loadMovie("button.swf","holder_mc");
ok there are loading in my file but problem is that now i see
only one button now and written Undefined on the buttons.
i am not sure why they are not working , i followed many
online tutorials on loadMovie and did exactly same...
one thing is interesting see here.. without xml files are
loading perfect but with xml file i always get error even though
xml file is already in my folder.
I look forward to learn something from your experts.
thank you very much
i am using flash 8 professional. AS 2.0

ok this time i made nav bar my main .. this time i am not
loading nav bar in another movie since i was really undefined error
but this time i have another xml file which is only a simple xml
file but when i click on nav bar to load another swf file which is
txt file, it loads but i don't see anything on it, no text , no
content nothing there... even though when i view it separtly it
works fine..
any help would be appreciated.
thanks

Not enough memory to load the tiff file

I did a search for this problem and there were two threads about it but I can't post to them. I think they are archived. Unfortunately, there was no answer to this problem in there.
The error "not enough memory to load tiff file" has been occuring in our shop on various computers with different resources. I believe that it has happened on our Mac as well. All are running Illustrator CS2 except the Mac which is running Illustrator CS3 and Leopard.
Has there been any progress on what the problem is? Has anyone found a solution that doesn't involve having to resave tif files as other formats to get them into the original Illustrator document? Or a solution that does not involve a huge work around?
I don't know the specs of all the computers in our shop but the one that I'm working on has the following specs:
Pentium 4, 2.8 GHz with 2.5 GB of RAM
Windows XP Pro Version 2002; service pack 3
Any help would be appreciated.
J.

Photo Scaling for Video
http://forums.adobe.com/thread/450798?tstart=0
-Too Large = Crash http://forums.adobe.com/thread/879967?tstart=0

Loading XML Schema into ALSB

I have a rather complex schema that spans 20 .xsd files. I'd like to load the entire schema into ALSB but the only mechanism I can find has me adding individual .xsd files and monkeying around with editing references to resolve includes/imports. For my current shcema
Is there a mechanism that will allow me to load an entire directory (and subdirectories) that contains my .xsd files? (The way I can do in a Weblogic Workshop Schema project)?
Thanks in advance,
Bruce

Thanks for the pointer.. I missed the bulk import option.. looks like I still have issues since when I import my .zip file full of schema I get :
"The import failed with exception: com.bea.wli.sb.common.UserException"
in the ALSB console.. no additional info in the Weblogic server log.. Not sure where else to look.

Problem ragarding load xml(embedded) file

oGlobalMain is the object of global class where every object is created and function is defined
SBO_Application is declared as Public WithEvents SBO_Application As SAPbouiCOM.Application
#Region "Load as XML"
Public Sub LoadAsXml(ByVal FileName As String, Optional ByVal FileType As ResourceType = ResourceType.Content)
Dim oXmlDoc As Xml.XmlDocument
Dim oXmlStream As System.IO.Stream
oXmlDoc = New Xml.XmlDocument
Try
If FileType = ResourceType.Content Then
oXmlDoc.Load(FileName)
Else
oXmlStream = System.Reflection.Assembly.LoadFrom(System.Windows.Forms.Application.ExecutablePath).GetManifestResourceStream(GetType(modStartUp).Namespace & "." & FileName)
oXmlDoc.Load(oXmlStream)
End If
oGlobalMain.SBO_Application.LoadBatchActions(oXmlDoc.InnerXml)
Catch ex As Exception
oGlobalMain.SBO_Application.MessageBox(ex.Message)
End Try
End Sub
#End Region
please send me some suggestion, I would be grateful to you.
Thanks in advance
regards
Nandini

Hi..
use this code
Form load()
use
Me.CreateForm()
Sub CreateForm()
        Try
            SAPXML("AAAAAAA.xml")
            OM_Form = app.Forms.Item("FormID--Yours")
            Catch ex As Exception
            OM_Form = app.Forms.Item("FormID--Yours")
            OM_Form.Select()
            Exit Sub
        End Try
    End Sub
Sub SAPXML(ByVal path As String)
        Try
            Dim xmldoc As New MSXML2.DOMDocument
            Dim Streaming As System.IO.Stream = Assembly.GetExecutingAssembly().GetManifestResourceStream("Your Folder." + path)
            Dim StreamRead As New System.IO.StreamReader(Streaming, True)
            xmldoc.loadXML(StreamRead.ReadToEnd)
            StreamRead.Close()
            app.LoadBatchActions(xmldoc.xml)
        Catch ex As Exception
            app.MessageBox(ex.Message)
        End Try
    End Sub
Regards,
Siva

Convert XML Schema Files (XSD) in HTML Forms

Hi Everybody,
I am new in the topic of the XML, XSD, the situation is the following , I have a few files .XSD and need to generate automatic html forms based on these .XSD to gather information and later convert them into XML files and save them in a repository and later a middleware will process this information.
My first serious question if it is possible to develop the above mentioned task automatic? And the second one where you recommend to me to start investigating?.
Thanks in advance for your answer
Regards from Peru.

There are products which do that already, I think. XMLSpy is one. Check that out to see if I'm right (not sure about the HTML part). Otherwise, welcome to the club of several hundred people who have asked basically that same question (only not with the HTML part) here.

"Source file does not conform to XML Schema!" error at Import Server

Dear SDN,
When I run Import Server the import job stopped with the error "Source file does not conform to XML Schema!" in Log file. I used MDM SP05 Patch01 and HF1.
Both of 2 Import Server have same problems. Of course it works in Import Manager.
What do I have to check?
Regards,
Eric

Hi Eric
Just confirm this...
One needs to load XML schema file in MDM and can be done by specifying this file in Port .
Refer this link.
http://help.sap.com/saphelp_mdm550/helpdata/en/fa/69b20ae0ae4b158a3f6d025b3d3847/content.htm
Just make sure that corresponding XSD file is specified in the port
Regards

How to load a XML-Schema (XSD) from Classpath

Hi
I want to perform XML Schema Validation using a XML-Schema stored in a File "MySchema.xsd" in the classpath.
Unfortunately I have no clue, how I can load this file without using an absolute path. I already tried:
Thread.currentThead().getContextClassLoader().getRessourceAsStream()but this method rejects to load Files with the extension XSD.It does however load the file, if I rename it to MySchema.xml or MySchema.properties, but then I cannot use the Visual Editors of JDeveloper to edit the File.
Has anyone an idea, how I can load a XSD-File stored in the classpath?
Thanks in advance
Frank Brandstetter

My Problem is, that I cannot load the Schema-File during runtime. I try to use code like this:
String Schema = "sam/model/dataaccess/message/NavMessageXSD.xml"; // <== works with .xml, fails with .xsd
ClassLoader loader = Thread.currentThread().getContextClassLoader();
InputStream is = loader.getResourceAsStream(Schema);
if (is==null)
throw new Exception("Ressource not found"); // <== Exception thrown with .xsd
XSDBuilder builder = new XSDBuilder();
schemadoc = builder.build(new InputSource(is));
DOMParser parser = new DOMParser();
parser.setValidationMode(XMLParser.SCHEMA_VALIDATION);
parser.setXMLSchema(schemadoc);This code works if "Schema" points to a file with extension .xml ("NavMessageXSD.xml"). It fails if "Schema" points to a file with extension .xsd ("NavMessageXSD.xsd").
This file is already part of my project. If it is named "NavMessageXSD.xsd" I can edit it using the visual editor. If it is named "NavMessageXSD.xml" it is just an ordenary XML-File.
Thanks for your help
Frank Brandstetter

Memory issue loading XML schema files

Similar Messages

Maybe you are looking for