Different take on XML encoding character emcoding problem

I have the common problem of using non latin characters in my XML code. The XML file is saved in iso-8859-1 format and that is fine when viewing it through an editor. When i try to parse the file, no errors are reported but the special characters are changed into question marks. These characters exist in attributes, they are not content.
This happens when i am looking at the tree through DOM. In other words, I am not even outputting the tree to file. When I do a print statement on the content of the attribute, it comes out as question marks.
Therefore the standard answer of setting the encoding of the outputter does not work for me. The moment that the file is parsed in and is still in memory, the xml parser changes teh characters to question marks.
I am using the latest version of xerces and jaxp with java se 1.4.1
Can anyone help with this urgent problem?

No, I just use a simple System.out.println() statement.
Here it is printing out question marks and that is why even before i output the document the problem arises. Try this
System.out.println(new String(YOUR_STRING.getBytes("UTF8")));

Similar Messages

Character Converting Problem at File Adapter

I am having a character conversion problem bringing in a XML file into XI.
It is converting the data wrong.
I have a file adapter picking up Audit.xml encoding is utf-8, use binary mode to pick it up.
XI takes the file and converts it into a text file.
I compared the hex of Audit.xml source file and the Audit.txt file and the characters are different for a specific few.
See attached image for details
Please let me know if any additional information will help
http://i9.photobucket.com/albums/a68/tkc204/SAP%20XI/Audit.png

Actually for this case Christian is right the source xml that I am receiving wasn't valid UTF-8. The program generating the xml was just putting the encoding - utf-8 tag on top because that's what the code told it to do.
The actual data is Windows-1252 in xml. So when XI reads it, it uses the utf-8 tag on top and is expecting UTF-8, but it was getting something else causing the unsupported characters to get converted.
The HEX was different on the Source XML comparing it to what XI was generating after File Adapter.
After making the output change to Windows 1252. It changes back to the expect HEX.
Source - is at Windows 1252
XI - it will use UTF-8 (no way to change it)
Target - forced to create in Windows 1252
Other helpful tools I used: W3C Validator - upload the file and found that I wasn't receiving UTF-8
I hope this is clear and helps someone else in the future.
Thanks For everyone's input.
Chirag

Issue with XML encoding

Hi,
I am querying an SAP instance for data, and I am getting back the data in an XML document...to which I need apply XSL transformation.
The instance is an German one... so I am getting a few strange characters because of which, the parser is uanble to apply the transformation.
I wrote a small java program to get more info on this particular character causing the problem, it turns out to be an ISOControl character. The properties (got by using the Character class API) are as below.
1. Character 1:
The getDirectionality is:0
The getNumericValue is:21
The getType is:2
The isDefined is:true
The isISOControl is:false
The isMirrored is:false
The isSpaceChar is:false
The isUnicodeIdentifierPart is:true
The isUnicodeIdentifierStart is:true
The isWhitespace is:false
2. character 2:
The getDirectionality is:9
The getNumericValue is:-1
The getType is:15
The isDefined is:true
The isISOControl is:true
The isMirrored is:false
The isSpaceChar is:false
The isUnicodeIdentifierPart is:true
The isUnicodeIdentifierStart is:false
The isWhitespace is:false
These characters appear to be white space in normal text editors...
This is how I am applying the transformation
TransformerFactory tFactory = TransformerFactory.newInstance();
Transformer transformer =
tFactory.newTransformer(
new javax.xml.transform.stream.StreamSource
(XSLURI)
StringReader sr = new StringReader(xmlToApply);
StringWriter sw = new StringWriter();
transformer.transform(
new javax.xml.transform.stream.StreamSource(sr),
new javax.xml.transform.stream.StreamResult(sw)
return sw.toString();
I have tried to use "UTF-8", "ISO-8859-1" and other encodings, but in vain. Appreciate any pointers on this...

Also the postings are from 2 different perspectives... this is to address the issue that I am facing with XML parsing and how to eliminate that using XML apis...
My other posting about a possible work around (encoding) for the problem creating string...

Address book data is XML-encoded in LDAP ?

Hi,
We are trying to find out how address book data is stored in the LDAP, and we've found several different ways apparently. Some format seems to use an XML encoded format (this is when you go in the UWC (JES5) and add a new group and in stead of selecting contacts to add, you just type a name, and an email address).
This is how it looks like in the ldap:
absi@nagios ~/ $ ldapsearch -D "cn=Directory manager" -w Jxxxxxxxxxx -b "o=PiServerDb" -h 192.168.xxx.50 "displayname=test_manueel"
version: 1
dn: piEntryID=e115cc045b373f,piPStoreOwner=adresboekadmin,o=xxxxxxxxxxxxxxt.be
,o=PiServerDb
piEntryID: e115cc045b373f
objectClass: PITYPEGROUP
objectClass: piEntry
objectClass: top
memberOfPIBook: e114db3c213216f6
displayName: test_manueel
pipEntryXMLData:: PG9saT4KIDxHcm91cEV4TWVtYmVyPnRlc3QzJTVFU01UUCU1RXRlc3QzJTQ
wdmxhYW1zcGFybGVtZW50LmJlPC9Hcm91cEV4TWVtYmVyPgogPEdyb3VwRXhNZW1iZXI+a2llcmV
3aWV0JTVFU01UUCU1RWtpZXJld2lldCU0MHZsYWFtc3BhcmxlbWVudC5iZTwvR3JvdXBFeE1lbWJ
lcj4KIDxHcm91cEV4TWVtYmVyPnRlc3Q0NSU1RVNNVFAlNUV0ZXN0NDUlNDBhYnNpLmJlPC9Hcm9
1cEV4TWVtYmVyPgo8L29saT4K
However, I can't find any usable reference to this ldap attribute 'pipEntryXMLData' , except that it's used by JES 2005Q1 onwards
I'm also concerned by the following:
- in case of problems, are there tools to read out / export this XML data in anything that is human readable / parseable ?
- what about scalability, we will be having members with thousands of entries in their addressbook, is this method scalable or will we be experiencing delays when users are looking up contacts in their address books ?
All info on this is more than welcom!
regards,
Tom.

The ldapsearch tools are producing LDIF format (RFC 2849) which requires that non ascii values and values with some specific characters be Base64 encoded.
XML is out of these formats that start with a special character with regards to LDIF ('<') and for which values must be base64 encoded.
There are many decoders for base64 encoding.
Alternately, you can use an LDAP API (C or Java) and the value will not be encoded.
Regards,
Ludovic.

Agent control character set problem

Hi,
here's my problem :
i've got the grid that's running on a RHES4 with an agent. On another RHES4, i've got 10g databases that run and another agent.
The repository database is configured like this :
nsl_language = AMERICAN
nls_territory = AMERICA
character set = AL32UTF8
all the uploads from the agent on the RHES4 where the grid is installed are ok.
On the other server, as soon as there's an UTF8 character in a xml file (like " é " ou " ' "), the upload fails and the agent stops.
in the logs, it s clear it comes from this. I've deleted all the occurences of UTF8 characters in the xml file and restarted the upload and it's ok...
I've tested different configurations but without success.
Any clue ?
Alivetu

Thanks for the reply,
NLS LANG is set on the 2 machines with FRENCHFRANCE.WE8ISO8859P15
I've added the line '<?xml version="1.0" encoding="ISO-8859-1"?>' at the beginning of the xml file that doesn't work and made a 'emctl upload'....it has passed and the agent has stopped arrived to another UTF8 xml file...
So, it really is a character set problem but where to set it ???
Alivetu

How to set the Xml Encoding ISO-8859-1 to Transformer or DOMSource

I have a xml string and it already contains an xml declaration with encoding="ISO-8859-1". (In my real product, since some of the element/attribute value contains a Spanish character, I need to use this encoding instead of UTF-8.) Also, in my program, I need to add more attributes or manipulate the xml string dynamically, so I create a DOM Document object for that. And, then, I use Transformer to convert this Document to a stream.
My problme is: Firstly, once converted through the Transformer, the xml encoding changed to default UTF-8, Secondly, I wanted to check whether the DOM Document created with the xml string maintains the Xml Encoding of ISO-8859-1 or not. So, I called Document.getXmlEncoding(), but it is throwing a runtime error - unknown method.
Is there any way I can maintain the original Xml Encoding of ISO-8859-1 when I use either the DOMSource or Transformer? I am using JDK1.5.0-12.
Following is my sample program you can use.
I would appreciate any help, because so far, I cannot find any answer to this using the JDK documentation at all.
Thanks,
Jin Kim
import java.io.*;
import org.w3c.dom.Document;
import org.w3c.dom.Element;
import org.w3c.dom.Node;
import org.w3c.dom.NodeList;
import org.w3c.dom.Attr;
import org.xml.sax.InputSource;
import javax.xml.parsers.DocumentBuilder;
import javax.xml.parsers.DocumentBuilderFactory;
import javax.xml.transform.Templates;
import javax.xml.transform.Transformer;
import javax.xml.transform.TransformerException;
import javax.xml.transform.TransformerFactory;
import javax.xml.transform.TransformerConfigurationException;
import javax.xml.transform.dom.DOMSource;
import javax.xml.transform.Source;
import javax.xml.transform.stream.StreamSource;
import javax.xml.transform.stream.StreamResult;
public class XmlEncodingTest
    StringBuffer xmlStrBuf = new StringBuffer();
    TransformerFactory tFactory = null;
    Transformer transformer = null;
    Document document = null;
    public void performTest()
        xmlStrBuf.append("<?xml version=\"1.0\" encoding=\"iso-8859-1\"?>\n")
                 .append("<TESTXML>\n")
                 .append("<ELEM ATT1=\"Yes\" />\n")
                 .append("</TESTXML>\n");
        // the encoding is set to iso-8859-1 in the xml declaration.
        System.out.println("initial xml = \n" + xmlStrBuf.toString());
        try
            //Test1: Use the transformer to ouput the xmlStrBuf.
            // This shows the xml encoding result from the transformer which will change to UTF-8
            tFactory = TransformerFactory.newInstance();
            transformer = tFactory.newTransformer();
            StreamSource ss = new StreamSource( new StringBufferInputStream( xmlStrBuf.toString()));
            System.out.println("Test1 result = ");
            transformer.transform( ss, new StreamResult(System.out));
            //Test2: Create a DOM document object for xmlStrBuf and manipulate it by adding an attribute ATT2="No"
            DocumentBuilderFactory dfactory = DocumentBuilderFactory.newInstance();
            DocumentBuilder builder = dfactory.newDocumentBuilder();
            document = builder.parse( new StringBufferInputStream( xmlStrBuf.toString()));
            // skip adding attribute since it does not affect the test result
            // Use a Transformer to output the DOM document. the encoding becomes UTF-8
            DOMSource source = new DOMSource(document);
            StreamResult result = new StreamResult(System.out);
            System.out.println("\n\nTest2 result = ");
            transformer.transform(source, result);
        catch (Exception e)
            System.out.println("<performTest> Exception caught. " + e.toString());
    public static void main( String arg[])
        XmlEncodingTest xmlTest = new XmlEncodingTest();
        xmlTest.performTest();
}

Thanks DrClap for your answer. With your information, I rewrote the sample program as in the following, and it works well now as I intended! About the UTF-8 and Spanish charaters, I think you are right. It looks like there can be many factors involved on this subject though - for example, the real character sets used to create an xml document, and the xml encoding information declared will matter. The special character I had a trouble was u00F3, and somehow, I found out that Sax Parser or even Document Builder parser does not like this character when encoding is set to "UTF-8" in the Xml document. My sample program below may not be a perfect example, but if you replaces ISO-8859-1 with UTF-8, and not setting the encoding property to the transfermer, you may notice that the special character in my example is broken in Test1 and Test2. In my sample, I decided to use ByteArrayInputStream instead of StringBufferInpuptStream because the documentation says StringBufferInputStream may have a problem with converting characters into bytes.
Thanks again for your help!
Jin Kim
import java.io.*;
import java.util.*;
import org.w3c.dom.Document;
import org.w3c.dom.Element;
import org.w3c.dom.Node;
import org.w3c.dom.NodeList;
import org.w3c.dom.Attr;
import org.xml.sax.InputSource;
import javax.xml.parsers.DocumentBuilder;
import javax.xml.parsers.DocumentBuilderFactory;
import javax.xml.transform.Templates;
import javax.xml.transform.Transformer;
import javax.xml.transform.TransformerException;
import javax.xml.transform.TransformerFactory;
import javax.xml.transform.TransformerConfigurationException;
import javax.xml.transform.dom.DOMSource;
import javax.xml.transform.Source;
import javax.xml.transform.stream.StreamSource;
import javax.xml.transform.stream.StreamResult;
* XML encoding test for Transformer
public class XmlEncodingTest2
    StringBuffer xmlStrBuf = new StringBuffer();
    TransformerFactory tFactory = null;
    Document document = null;
    public void performTest()
        xmlStrBuf.append("<?xml version=\"1.0\" encoding=\"iso-8859-1\"?>\n")
                 .append("<TESTXML>\n")
                 .append("<ELEM ATT1=\"Resoluci�n\">\n")
                 .append("Special charatered attribute test")
                 .append("\n</ELEM>")
                 .append("\n</TESTXML>\n");
        // the encoding is set to iso-8859-1 in the xml declaration.
        System.out.println("**** Initial xml = \n" + xmlStrBuf.toString());
        try
            //TransformerFactoryImpl transformerFactory = new TransformerFactoryImpl();
            //Test1: Use the transformer to ouput the xmlStrBuf.
            tFactory = TransformerFactory.newInstance();
            Transformer transformer = tFactory.newTransformer();
            byte xmlbytes[] = xmlStrBuf.toString().getBytes("ISO-8859-1");
            StreamSource streamSource = new StreamSource( new ByteArrayInputStream( xmlbytes ));
            ByteArrayOutputStream xmlBaos = new ByteArrayOutputStream();
            Properties transProperties = transformer.getOutputProperties();
            transProperties.list( System.out); // prints out current transformer properties
            System.out.println("**** setting the transformer's encoding property to ISO-8859-1.");
            transformer.setOutputProperty("encoding", "ISO-8859-1");
            transformer.transform( streamSource, new StreamResult( xmlBaos));
            System.out.println("**** Test1 result = ");
            System.out.println(xmlBaos.toString("ISO-8859-1"));
            //Test2: Create a DOM document object for xmlStrBuf to add a new attribute ATT2="No"
            DocumentBuilderFactory dfactory = DocumentBuilderFactory.newInstance();
            DocumentBuilder builder = dfactory.newDocumentBuilder();
            document = builder.parse( new ByteArrayInputStream( xmlbytes));
            // skip adding attribute since it does not affect the test result
            // Use a Transformer to output the DOM document.
            DOMSource source = new DOMSource(document);
            xmlBaos.reset();
            transformer.transform( source, new StreamResult( xmlBaos));
            System.out.println("\n\n****Test2 result = ");
            System.out.println(xmlBaos.toString("ISO-8859-1"));
            //xmlBaos.flush();
            //xmlBaos.close();
        catch (Exception e)
            System.out.println("<performTest> Exception caught. " + e.toString());
        finally
    public static void main( String arg[])
        XmlEncodingTest2 xmlTest = new XmlEncodingTest2();
        xmlTest.performTest();
}

UTF-8 encoded JSPs compilation problem

Hi,
          I'm using Weblogic 9.0 Beta. I have an XML-format UTF-8 encoded JSP (with the proper encoding declarations). I can see that this is compiled into a UTF-8 Java servlet by WebLogic.
          At the compilation to a class file though, the encoding is corrupted. I guess that the Java compiler is assuming a system-encoded (which would be ISO-8859-1) Java file instead of the actual UTF-8 encoding.
          This problem did not occur with WebLogic 8.1.
          I have tried to explicitly tell the Java compiler to treat the source files as UTF-8 in weblogic.xml, i.e.
          <jsp-param>
          <param-name>compileFlags</param-name>
          <param-value>-encoding UTF8</param-value>
          </jsp-param>
          but that had no effect.
          Anyone else noticed this?
          I assume that correct behaviour is for WebLogic to preserve encoding from JSP to servlet to class file, rather than for me to set encoding in weblogic.xml. Is that correct?
          Is there a workaround?
          Thanks for any help you can offer!

Solved
It is about Tomcat's character encoding not about the codes..
For more info:
[http://wiki.apache.org/tomcat/Tomcat/UTF-8]

IsSchemaValid does chang the xml-encoding header from UTF-8 to WINDOWS-1252

I found the following effect:
isSchemaValid does changing the encoding - entry of the xml-file-header
generating xml-file by using DBMS_XMLGEN :
xmldoc := DBMS_XMLGEN.getXML(ctx);
with the header of the file is
<?xml version="1.0" encoding="UTF-8"?>
change the xmldoc to a xmlType
and validate it against the schema
xmldoc_xmlType:=(xmltype(xmldoc)) ;
xmldoc_xmlType.isSchemaValid ( bSchemalocation)
after this the header of the file is
<?xml version="1.0" encoding="WINDOWS-1252"?>
my DB:
R11_2_0_2 / Windows 64
the same in
R11_2_0_1 / Windows 32
select name, value from v$parameter where upper(name) like '%NLS%'
nls_calendar
nls_comp          BINARY
nls_currency
nls_date_format
nls_date_language
nls_dual_currency
nls_iso_currency
nls_language          AMERICAN
nls_length_semantics     BYTE
nls_nchar_conv_excp     FALSE
nls_numeric_characters
nls_sort
nls_territory          AMERICA
nls_time_format
nls_timestamp_format
nls_timestamp_tz_format
nls_time_tz_format
register my schema by:
dbms_xmlschema.registerSchema(
schemaurl => vschemaurl,
schemadoc => xsd_file,
local      => FALSE,
gentypes      => TRUE,
genbean      => FALSE,
gentables      => TRUE,
force      => FALSE,
owner      => dbuser
,CSID      => nls_charset_id('AL32UTF8')
How can I let or change back the xml-encoding entry to UTF-8 ?
regards

Your solution should not be relied upon...
C:\Temp>sqlplus /nolog @t1 %CD%
SQL*Plus: Release 11.2.0.2.0 Production on Fri Mar 4 09:41:32 2011
Copyright (c) 1982, 2010, Oracle. All rights reserved.
SQL> spool testcase.log
SQL> --
SQL> connect sys/oracle as sysdba
Connected.
SQL> --
SQL> set define on
SQL> set timing on
SQL> --
SQL> def XMLDIR = &1
SQL> --
SQL> def USERNAME = XDBTEST
SQL> --
SQL> def PASSWORD = &USERNAME
SQL> --
SQL> def USER_TABLESPACE = USERS
SQL> --
SQL> def TEMP_TABLESPACE = TEMP
SQL> --
SQL> drop user &USERNAME cascade
2 /
old   1: drop user &USERNAME cascade
new   1: drop user XDBTEST cascade
User dropped.
Elapsed: 00:00:00.24
SQL> grant create any directory, drop any directory, connect, resource, alter session, create view to &USERNAME identified by &PASSWORD
2 /
old   1: grant create any directory, drop any directory, connect, resource, alter session, create view to &USERNAME identified by &PASSWORD
new   1: grant create any directory, drop any directory, connect, resource, alter session, create view to XDBTEST identified by XDBTEST
Grant succeeded.
Elapsed: 00:00:00.07
SQL> alter user &USERNAME default tablespace &USER_TABLESPACE temporary tablespace &TEMP_TABLESPACE
2 /
old   1: alter user &USERNAME default tablespace &USER_TABLESPACE temporary tablespace &TEMP_TABLESPACE
new   1: alter user XDBTEST default tablespace USERS temporary tablespace TEMP
User altered.
Elapsed: 00:00:00.00
SQL> set long 100000 pages 0 lines 256 trimspool on timing on
SQL> --
SQL> connect &USERNAME/&PASSWORD
Connected.
SQL> --
SQL> create or replace directory XMLDIR as '&XMLDIR'
2 /
old   1: create or replace directory XMLDIR as '&XMLDIR'
new   1: create or replace directory XMLDIR as 'C:\Temp'
Directory created.
Elapsed: 00:00:00.00
SQL> create table XML_DEFAULT of XMLTYPE
2 /
Table created.
Elapsed: 00:00:00.11
SQL> create table XML_CLOB of XMLTYPE
2 XMLTYPE store as CLOB
3 /
Table created.
Elapsed: 00:00:00.01
SQL> select *
2    from nls_database_parameters
3   where parameter in ('NLS_LANGUAGE', 'NLS_TERRITORY', 'NLS_CHARACTERSET')
4 /
NLS_LANGUAGE                   AMERICAN
NLS_TERRITORY                  AMERICA
NLS_CHARACTERSET               AL32UTF8
Elapsed: 00:00:00.02
SQL> declare
2    XML_DEFAULT XMLType := xmltype('<?xml version="1.0" encoding="WINDOWS-1252"?><TEST>SELECT</TEST>') ;
3    XML_CLOB    XMLType := xmltype('<?xml version="1.0" encoding="WINDOWS-1252"?><TEST>SELECT</TEST>') ;
4 begin
5    delete XML_DEFAULT;
6    delete XML_CLOB;
7    insert into XML_DEFAULT values (XML_DEFAULT);
8    dbms_xslprocessor.clob2file( XML_DEFAULT.getclobval() , 'XMLDIR','XML_DEFAULT.xml');
9    IF XML_DEFAULT.isSchemaValid ( 'SCHEMALOCATION_DOES_NO_MATTER_FOR_TEST_CASE.XSD', 'SCHEMA_NO_MATTER') = 1 THEN null; ELSE null; END IF;
10    commit;
11    dbms_xslprocessor.clob2file( XML_DEFAULT.getclobval() , 'XMLDIR','XML_DEFAULT_IS_VALID.xml',nls_charset_id('WE8MSWIN1252'));
12    dbms_xslprocessor.clob2file( XML_DEFAULT.getclobval() , 'XMLDIR','XML_DEFAULT_WIN1252.xml');
13    insert into XML_CLOB values (XML_CLOB);
14    dbms_xslprocessor.clob2file( XML_CLOB.getclobval() , 'XMLDIR','XML_CLOB.xml');
15    commit;
16 end ;
17 /
PL/SQL procedure successfully completed.
Elapsed: 00:00:00.27
SQL> --
SQL> host type XML_DEFAULT.xml
<?xml version="1.0" encoding="WINDOWS-1252"?><TEST>SELECT</TEST>
SQL> --
SQL> host type XML_DEFAULT_IS_VALID.xml
<?xml version="1.0" encoding="UTF-8"?>
<TEST>SELECT</TEST>
SQL> --
SQL> host type XML_DEFAULT_WIN1252.xml
<?xml version="1.0" encoding="UTF-8"?>
<TEST>SELECT</TEST>
SQL> --
SQL> host type XML_CLOB.xml
<?xml version="1.0" encoding="WINDOWS-1252"?><TEST>SELECT</TEST>
SQL> --
SQL>First, the character set changes because isSchemaValid() causes the document to be parsed and converted to the internal database character set, as does storing it in a table.
It appear that your solution works in SQL because the semantics of SQL are such that it causes a 'copy' of the XMLType to take place before running the isSchemaValid() processing, were we to optimize away that copy as a result of a patch or performance optimization project then you solution would break...
If you want the output in a particular character set you should force that using XMLSerialize or getBlobVal(charsetid). Unfortunately we don't have a convience method for writing BLOBS on DBMS_XSLPROCESSOR...

Address book data is XML-encoded ?

Hi,
We are trying to find out how address book data is stored in the LDAP, and we've found several different ways apparently. Some format seems to use an XML encoded format (this is when you go in the UWC (JES5) and add a new group and in stead of selecting contacts to add, you just type a name, and an email address).
This is how it looks like in the ldap:
absi@nagios ~/ $ ldapsearch -D "cn=Directory manager" -w Jxxxxxxxxxx -b "o=PiServerDb" -h 192.168.xxx.50 "displayname=test_manueel"
version: 1
dn: piEntryID=e115cc045b373f,piPStoreOwner=adresboekadmin,o=xxxxxxxxxxxxxxt.be
,o=PiServerDb
piEntryID: e115cc045b373f
objectClass: PITYPEGROUP
objectClass: piEntry
objectClass: top
memberOfPIBook: e114db3c213216f6
displayName: test_manueel
pipEntryXMLData:: PG9saT4KIDxHcm91cEV4TWVtYmVyPnRlc3QzJTVFU01UUCU1RXRlc3QzJTQ
wdmxhYW1zcGFybGVtZW50LmJlPC9Hcm91cEV4TWVtYmVyPgogPEdyb3VwRXhNZW1iZXI+a2llcmV
3aWV0JTVFU01UUCU1RWtpZXJld2lldCU0MHZsYWFtc3BhcmxlbWVudC5iZTwvR3JvdXBFeE1lbWJ
lcj4KIDxHcm91cEV4TWVtYmVyPnRlc3Q0NSU1RVNNVFAlNUV0ZXN0NDUlNDBhYnNpLmJlPC9Hcm9
1cEV4TWVtYmVyPgo8L29saT4K
However, I can't find any usable reference to this ldap attribute 'pipEntryXMLData' , except that it's used by JES 2005Q1 onwards
I'm also concerned by the following:
- in case of problems, are there tools to read out / export this XML data in anything that is human readable / parseable ?
- what about scalability, we will be having members with thousands of entries in their addressbook, is this method scalable or will we be experiencing delays when users are looking up contacts in their address books ?
All info on this is more than welcom!
regards,
Tom.

Hi,
tom_absi wrote:
We are trying to find out how address book data is stored in the LDAP, and we've found several different ways apparently. Some format seems to use an XML encoded format (this is when you go in the UWC (JES5) and add a new group and in stead of selecting contacts to add, you just type a name, and an email address).This isn't surprising as there wasn't any pre-existing attribute which could be used to add these values. So it appears they are using an XML 'blob' instead.
This is how it looks like in the ldap:
pipEntryXMLData:: PG9saT4KIDxHcm91cEV4TWVtYmVyPnRlc3QzJTVFU01UUCU1RXRlc3QzJTQ
wdmxhYW1zcGFybGVtZW50LmJlPC9Hcm91cEV4TWVtYmVyPgogPEdyb3VwRXhNZW1iZXI+a2llcmV
3aWV0JTVFU01UUCU1RWtpZXJld2lldCU0MHZsYWFtc3BhcmxlbWVudC5iZTwvR3JvdXBFeE1lbWJ
lcj4KIDxHcm91cEV4TWVtYmVyPnRlc3Q0NSU1RVNNVFAlNUV0ZXN0NDUlNDBhYnNpLmJlPC9Hcm9
1cEV4TWVtYmVyPgo8L29saT4KThis is base64 encoded and decodes as (http://www.opinionatedgeek.com/dotnet/tools/Base64Decode/):
<oli>
<GroupExMember>test3%5ESMTP%5Etest3%40vlaamsparlement.be</GroupExMember>
<GroupExMember>kierewiet%5ESMTP%5Ekierewiet%40vlaamsparlement.be</GroupExMember>
<GroupExMember>test45%5ESMTP%5Etest45%40absi.be</GroupExMember>
</oli> However, I can't find any usable reference to this ldap attribute 'pipEntryXMLData' , except that it's used by JES 2005Q1 onwardsAppears to be a 'private' attribute. Although you can access the data because it is stored in an LDAP server I don't believe it is a public interface so it is bound to change.
I'm also concerned by the following:
- in case of problems, are there tools to read out / export this XML data in anything that is human readable / parseable ?The base64 decoded version seems parseable to me?
- what about scalability, we will be having members with thousands of entries in their addressbook, is this method scalable or will we be experiencing delays when users are looking up contacts in their address books ?Assuming you have a sufficiently scoped/tuned directory server I don't see there is going to be a problem.
Regards,
Shane.

Generate XML - Special Character in data causing error

Hello,
I have a procedure written using DBMS_XMLGEN to create xml output. A problem that I am running into is some of my data has an eacute in it. How can I code the XML to account for this and any other weird characters that may exist?
(ie - i need to be able to specify the encoding part of ?xml version=1.0 encoding=ISO-8859-1?)
Here is an excerpt of what i have so far...the results are fine until I run to include the data with the eacute. Then nothing is generated at all. I tried .convert, but I don't know if that's what I need, or if I am calling it correctly.
Thanks!
Janel
ctx := dbms_xmlgen.newContext(rc_data);
-- set name of root element --
dbms_xmlgen.setRowsetTag(ctx, 'XXFA10022');
-- set name of element separating all rows --
-- default is ROW --
dbms_xmlgen.setRowTag(ctx, 'G_DATA');
-- generate XML document --
xml := dbms_xmlgen.getXml(Ctx);
xml := dbms_xmlgen.convert(xml, dbms_xmlgen.ENTITY_ENCODE);
i := dbms_xmlgen.getNumRowsProcessed(ctx);
dbms_xmlgen.closeContext(Ctx);
apps.fnd_file.put_line(apps.fnd_file.log, 'rows processed -' || i);
apps.fnd_file.put_line(apps.fnd_file.log, 'close');
apps.fnd_file.put_line(apps.fnd_file.log, xml);
apps.fnd_file.put_line(apps.fnd_file.output, xml);
-- free clob content --
dbms_lob.freetemporary(xml);

No. There are no XML components that will clean up malformed XML. Your options are to go to whoever sent you that document and tell them that their XML is malformed and that in future they should not do that, or to write your own component to clean up the malformed XML. Since that would involve more work than writing a full XML parser, I don't recommend the second option.

XML encoding "UTF-8" ignored

HI all,
we try to post an XML message (cXML invoice) to a receiver but it doesn´t work.
We use a communication channel "http-destination" where we configure content-type "text/xml" together with XML encoding "UTF-8".
The configured URL works fine and can be tested in SM59.
The XML payload itself has a declaration <?xml version="1.0" encoding="UTF-8"?>
but the strange thing that happens is that all these declarations are being ignored and instead US-ASCII is used. Which leads to an error due to some special german characters within the contents of the XML (e.g. ä, ö, etc.).
Does anybody of you know how I can achieve that the pre-set UTF-8 will be used as encoding type???
Many thanky in advance!
Willi Wuerstlin

Hi.
I am trying to map the standard cXML invoice to SAP's standard idoc INVOIC01 - This is how the file looks like
<?xml version="1.0" ?>
<!DOCTYPE cXML SYSTEM "http://xml.cXML.org/schemas/cXML/1.2.014/InvoiceDetail.dtd">
<cXML version="" payloadID="2008-07-29T04:51:08-06:00.cXML.TEST4101V002" timestamp="2008-07-29T04:51:08-06:00" xml:lang="en-US">
<Header>...</Header>
...</cxml>
I am having problem with the line <!DOCTYPE cXML SYSTEM "http://xml.cXML.org/schemas/cXML/1.2.014/InvoiceDetail.dtd"> - when the payload is having this line XI is not accepting the message.
Please let me know how to address this.
Thanks.

XML encoding for type XML B/L transaction property

I need to change the XML encoding type from UTF-8 to UTF-16 for XML documents being sent to an external system via the web service action block. Is there any way to change this encoding in XMII?

Musarrat,
Of course it's possible....
Tim,
Instead of using the WebService action us the Post action and set the body of the post to be the WebService SOAP XML. Since the operation of a WebService and POST are the same thing behind the scenes this will work without a problem.
Hope this helps.
Sam

Debian Linux character recognition problem

Maybe this shouldn't be on this forum, but perhaps someone can help me...
I have Jrun 3.1, java 2 1.4 and Debian Linux.
On our test server everything was hunky dory, running perfectly.
But once everything was on the real server, we've had a character recognition problem. All of the higher characters are just replaced by a y or a ?.
I'm pretty sure that we can exclude java from the problem, the i18n.jar, which I believe holds all the international stuff is there and in the classpath (exactly as on the test server) and jrun is exactly as on the test server because we copied every folder as it was.
Therefore I think we can bring it down to Linux (Debian).
I cannot find where to look in Debian for any kind of keyboard or locale problem. Anyone know what to do?
kbdconfig command doesn't exist on our system.
Any help greatly appreciated.
We are in Italy, by the way.
Thanks.

If you check out the default file encoding you'll find that it's 7-bit ASCII; no characters > '\u007F' come out as they should. This applies as well to compiling as to running.
You can compile with another character encoding using the command line options, like: "javac -encoding ISO-8859-1 package/YourClass.java" (available at least in 1.4) and "java -Dfile.encoding=iso-8859-1 package.YourClass".

UNICODE System - XML Encoding Error

Dear Experts,
We've upgraded our SAP system to ECC 6.0 Unicode. One of our applications works with XML files and we've been experimenting some problems with the XML encoding since the machine is Unicode enabled. In the XML structure we obtain strange chinesse characters:
This should be the right xml response:
<?xml version="1.0" encoding="utf-8"?><OpenHR><employee><id>69900004</id><name>Employee 1, Romania</name><orgName>Test ROMANIA</orgName><role>EMPLOYEE</role><flex>NO</flex><company>NE</company></employee><labels><label id="delegation">Delegations</label>
We obtain:
⸰㰿硭氠癥牳楯渽∱⸰u2220敮捯摩湧㴢畴昭㠢㼾㱏灥湈刾㱥浰汯祥放㱩搾㘹㤰〰〴㰯楤㸼湡浥㹅浰汯祥攠ㄬ⁒潭慮楡㰯湡浥㸼潲李慭放呥獴⁒位䅎䥁㰯潲李慭放㱲潬放䕍偌余䕅㰯牯汥㸼晬數㹎似⽦汥砾㱣潭灡湹㹎䕏㰯捯浰慮社㰯敭灬潹敥㸼污扥汳㸼污扥氠楤㴢摥汥条瑩潮∾䑥汥条瑩潮猼⽬慢敬㸼污扥氠楤㴢桥汰∾䡥汰㰯污扥氾㱬慢敬⁩搽≩湣楤敮捥獔楴汥∾健牳潮湥氠䅤浩湩獴牡瑩潮㰯污扥氾㱬慢敬⁩搽≮潄敬敧慴楯渢㹐汥慳攬⁲整畲渠瑯睮慬敮摡爠楦⁹潵⁷慮琠瑯捣敳猠瑯⁤敬敧慴楯湳㰯污扥氾㱬慢敬⁩搽≮潴敃潮晩牭兵敳瑩潮∾偲敳猠慣捥灴⁩映祯甠獨潵汤⁧整⁩琼
Here is the code where we guess the problem is (we have tried with differente encoding types: ISO-8859-1, UTF-8, UTF-16 and DEFAULT).
DATA: S_ENCODING_TYPE TYPE STRING,
             ENCODING TYPE REF TO IF_IXML_ENCODING.
S_ENCODING_TYPE = 'iso-8859-1'.
ENCODING = IXML->CREATE_ENCODING(     CHARACTER_SET = S_ENCODING_TYPE
                                                                       BYTE_ORDER = 0 ).
CALL METHOD ODOCUMENT->SET_ENCODING
    EXPORTING ENCODING = ENCODING.
CALL FUNCTION 'SDIXML_DOM_TO_XML'
       EXPORTING
            DOCUMENT         = ODOCUMENT
       IMPORTING
            SIZE                     = ISIZERAW
       TABLES
            XML_AS_TABLE = IDATA
       EXCEPTIONS
            NO_DOCUMENT = 1
            OTHERS              = 2.
(IDATA is the variable where we see the chinesse text)

It will not give you any effect to set up encoding because this function module sets its own encoding inside:
encoding = pixml->create_encoding( character_set = 'UTF-8' byte_order = 0 ).
Renderer is created by the kernel:
by kernel module abkm_iXML_CreateRenderer.
Something may be wrong with the installation.

TableModel XML Encoding

hi you all
still got a problem with my XML Encoder
I fail encoding my TableModel. I've checked I can serialize it, and serialize it well. But the XML document that is created (encoding a JPanel and everything that's in it) and thus records the table, records no trace of the TableModel.
feel desesperate!
thanx

Can you show me your code block?
Thanks
Tajul islam

Different take on XML encoding character emcoding problem

Similar Messages

Maybe you are looking for