XML Parsing attributes with encoded ampersand causes wrong order

Question

Hi all,
I am writing in the forum first (because it could be that i am doing something wrong.... but i think it is a bug. Nonetheless, i thought i'd write my problem up here first.
I am using Java 6, and this has been reproduced on both windows and linux.
java version "1.6.0_03"
Problem:
read XML file into org.w3c.dom.Document.
XML File has some attributes which contain ampersand. These are escaped as (i think) is prescribed by the rule of XML. For example:
<?xml version="1.0" encoding="UTF-8"?>
     <lang>
          <text dna="8233" ro="chisturi de plex coroid (&gt;=1.5 mm)" it="Cisti del plesso corioideo(&gt;=1.5mm)" tr="Koro&#305;d pleksus kisti (&gt;=1.5 mm)" pt_br="Cisto do plexo cor&oacute;ide (&gt;=1,5 mm)" de="Choroidplexus Zyste (&gt;=1,5 mm)" el="&Kappa;&#973;&sigma;&tau;&epsilon;&iota;&sigmaf; &chi;&omicron;&rho;&omicron;&epsilon;&iota;&delta;&omicron;&#973;&sigmaf; &pi;&lambda;&#941;&gamma;&mu;&alpha;&tau;&omicron;&sigmaf; (&gt;= 1.5 mm)" zh_cn="&#33033;&#32476;&#33180;&#22218;&#32959;&#65288;&gt;= 1.5 mm&#65289;" pt="Quisto do plexo coroideu (&gt;=1,5 mm)" bg="&#1050;&#1080;&#1089;&#1090;&#1072; &#1085;&#1072; &#1093;&#1086;&#1088;&#1080;&#1086;&#1080;&#1076;&#1085;&#1080;&#1103; &#1087;&#1083;&#1077;&#1082;&#1089;&#1091;&#1089; (&gt;= 1.5 mm)" fr="Kystes du plexus choroide (&gt;= 1,5 mm)" en="Choroid plexus cysts (&gt;=1.5 mm)" ru="&#1082;&#1080;&#1089;&#1090;&#1099; &#1089;&#1086;&#1089;&#1091;&#1076;&#1080;&#1089;&#1090;&#1099;&#1093; &#1089;&#1087;&#1083;&#1077;&#1090;&#1077;&#1085;&#1080;&#1081; (&gt;=1.5 mm)" es="Quiste del plexo coroideo (&gt;=1.5 mm)" ja="&#33032;&#32097;&#33180;&#22178;&#32990;&#65288;&gt;=1.5mm&#65289;" nl="Plexus choroidus cyste (&gt;= 1,5 mm)" />
</lang>As you might understand, we need to have the fixed text '>' for later processing. (not the greater than symbol '>' but the escaped version of it).
Therefore, I escape the ampersand (encode?) and leave the rest of the text as is. And so my > becomes >
All ok?
Symptom:
in fetching attributes, for example by the getAttribute("en") type call, the wrong attribute values are fetched.
Not only that, if i only read to Document instance, and write back to file, the attributes are shown mixed up.
eg:
dna: 8233, ro=chisturi de plex coroid (>=1.5 mm), en=&#1082;&#1080;&#1089;&#1090;&#1099; &#1089;&#1086;&#1089;&#1091;&#1076;&#1080;&#1089;&#1090;&#1099;&#1093; &#1089;&#1087;&#1083;&#1077;&#1090;&#1077;&#1085;&#1080;&#1081; (>=1, de=Choroidplexus Zyste (>=1,5 mm)Here you can see that 'en' is shown holding what looks like greek, ... (what is ru as a country-code anyway?) where it should have obviously had the english text that originally was associated with the attribute 'en'
This seems very strange and unexpected to me. I would have thought that in escaping (encoding) the ampersand, i have fulfilled all requirements of me, and that should be that.
There is also no error that seems to occur.... we simply get the wrong order when fetching attributes.
Am I doing something wrong? Or is this a bug that should be submitted?
Kind Regards, and thanks to all responders/readers.
Sean
p.s. previously I had not been escaping the ampersand. This meant that I lost my ampersand in fetching attributes, AND the attribute order was ALSO WRONG!
In fact, the wrong order was what led me to read about how to correctly encode ampersand at all. I had been hoping that correctly encoding would fix the order problem, but it didn't.
Edited by: svaens on Mar 31, 2008 6:21 AM

Answer 1

Hi kdgregory ,
Firstly, sorry if there has been a misunderstanding on my part. If i did not reply to the question you raised, I appologise.
In this 'reply' I hope not to risk further misunderstanding, and have simply given the most basic example which will cause the problem I am talking about, as well as short instructions on what XML to remove to make the problem disappear.
Secondly, as this page seems to be displayed in ISO 8859-1, this is the reason the xml I have posted looks garbled. The xml is UTF-8. I have provided a link to the example xml file for the sample below
[example xml file UTF-8|http://sean.freeshell.org/java/less2.xml]
As for your most recent questions:
Is it specified as an entity? To my knowledge (so far as my understanding of what an entity is) , yes, I am including entities in my xml. In my below example, the entities are the code for the greater than symbol. I am under the understanding that this is allowed in XML ??
Is it an actual literal character (0xA0)? No, I am specifying 'greater than' entity (code?) in order to include the actual symbol in the end result. I am encoding it in form 'ampersand', 'g character', 't character', 'colon' in order for it to work, according to information I have read on various web pages. A quick google search will show you where I got such information from, example website: https://studio.tellme.com/general/xmlprimer.html
Here is my sample program. It is longer than the one you kindly provided only because it prints out all attributes of the element it looks for. To use it, only change the name of the file it loads.
I have given the xml code seperately so it can be easily copied and saved to file.
Results you can expect from running this small test example?
1. a mixed up list of attributes where attribute node name no longer matches its assigned attribute values (not for all attributes, but some).
2. removing the attribute bg from the 'text' element will reduce most of these symptoms, but not all. Removing another attribute from the element will most likely make the end result look normal again.
3. No exception is thrown by the presence of non xml characters.
IMPORTANT!!! I have only just (unfortunately) noticed what this page does to my unicode characters... all the the international characters get turned into funny codes when previewed and viewed on this page.
Whereas the only codes I am explicitly including in this XML is the greater than symbol. The rest were international characters.
Perhaps that is the problem?
Perhaps there is an international characters problem?
I am quite sure that these characters are all UTF-8 because when I open up this xml file in firefox, It displays correctly, and in checking the character encoding, firefox reports UTF-8.
In order to provide an un-garbled xml file, I will provide it at this link:
link to xml file: [http://sean.freeshell.org/java/less2.xml]
Again, sorry for any hassle and/or delay with my reply, or poor reply. I did not mean to waste anyones time.
It will be appreciated however if an answer can be found for this problem. Chiefly,
1. Is this a bug?
2. Is the XML correct? (if not, then all those websites i've been reading are giving false information? )
Kindest Regards,
Sean
import javax.xml.parsers.DocumentBuilderFactory;
import org.w3c.dom.Document;
import org.w3c.dom.Element;
import org.w3c.dom.NamedNodeMap;
import org.w3c.dom.Node;
import org.w3c.dom.NodeList;
import org.xml.sax.InputSource;
public class Example
    public static void main(String[] argv)
          try
               FileInputStream fis = new FileInputStream("/home/sean/Desktop/chris/less2.xml");
             Document doc = DocumentBuilderFactory.newInstance()
             .newDocumentBuilder()
             .parse(new InputSource(fis));
               Element root = doc.getDocumentElement();
               NodeList textnodes = root.getElementsByTagName("text");
               int len = textnodes.getLength();
               int index = 0;
               int attindex = 0;
               int attrlen = 0;
               NamedNodeMap attrs = null;
               while (index<len)
                    Element te = (Element)textnodes.item(index);
                    attrs = te.getAttributes();
                    attrlen = attrs.getLength();
                    attindex = 0;
                    Node node = null;
                    while (attindex<attrlen)
                         node = attrs.item(attindex);          
                         System.out.println("attr: "+node.getNodeName()+ " is shown holding value: " + node.getNodeValue());
                         attindex++;                         
                    index++;
                    System.out.println("-------------");
             fis.close();
          catch(Exception e)
               System.out.println("we've had an exception, type "+ e);
}  [example xml file|http://sean.freeshell.org/java/less2.xml]
FOR THE XML, Please see link above, as it is UTF-8, and this page is not. Edited by: svaens on Apr 7, 2008 7:03 AM
Edited by: svaens on Apr 7, 2008 7:23 AM
Edited by: svaens on Apr 7, 2008 7:37 AM
Edited by: svaens on Apr 7, 2008 7:41 AM