Character conversion and NLS_LANG

Hi,
The Oracle doc says that character encoding conversion for Java programs using the OCI driver is dependent on NLS_LANG. But the description of this was a bit confusing. As per the doc
"The JDBC OCI driver transfers the data from the server to the client in the character set of the database. Depending on the value of the NLS_LANG environment variable, the driver handles character set conversions in one of two ways.
1)If the value of NLS_LANG is not specified, or if it is set to the US7ASCII or WE8ISO8859P1 character set, then the JDBC OCI driver uses Java to convert the character set from US7ASCII or WE8ISO8859P1 directly to UCS-2.
2)If the value of NLS_LANG is set to a non-US7ASCII or non-WE8ISO8859P1 character set, then the driver changes the value of the NLS_LANG parameter on the client to UTF-8. This happens automatically and does not require any user-intervention. OCI uses the value of NLS_LANG to convert the data from the database character set to UTF-8; the JDBC driver then converts the UTF-8 data to UCS-2. "
Now refering to case1, assume the database character set is multibyte.Does this mean that the OCI C libraries first convert this to US7ASCII or WE8ISO8859P1 and then the Java driver does the conversion from US7ASCII or WE8ISO8859P1 directly to UCS-2. If that is the case, wouldnt information get lost during the first conversion.
Thanks,
Tom.

"Now refering to case1, assume the database character set is
multibyte.Does this mean that the OCI C libraries first convert
this to US7ASCII or WE8ISO8859P1 and then the Java driver
does the conversion from US7ASCII or WE8ISO8859P1 directly to
UCS-2. If that is the case, wouldnt information get lost during
the first conversion. "
Yes this is true. For a multibyte database character set caution
must be taken that the client application NLS_LANG is not
US7ASCII or WE8ISO8859P1 or data loss can occur. An effort will
be made to remove the NLS_LANG dependency in a future release
because the current solution is imperfect.

Similar Messages

Character conversion and NLS parameter

Hi forms gurus,
After deploy forms on IAS/Linux, i'm encountring the following situations:
- Conversion of some characters: (Ã©) becomes (e), (Â°) becomes (?).As i'm a french speaker, these subsitution are not interesting for me.
- NLS Parameter: Errors, loggon dialog are in english.I want to put them in french.
What setting may i do to solve these problems?
Thank for ur help!
TYAG

Not familiar with Linux.
But your problem seems to be based on the DB characterset.
If the characters are not proper in the DB tables change the NLS_CHARACTERSET in DB. If the forms labels are showing the error change the NLS_LANG in the forms (default)*.env file

Export/import character data conversion in NLS_LANG FAQ

The [OTN NLS_LANG FAQ|http://www.oracle.com/technology/tech/globalization/htdocs/nls_lang%20faq.htm#_Toc110410566] says:
>
In Oracle9i the Export utility always exports user data, including Unicode data, in the character set of the database. The Import utility automatically converts the data to the character set of the target database.
>
But I don't think this is correct for Oracle 9i. Example:
export NLS_LANG=AMERICAN_AMERICA.US7ASCII
sqlplus /
SQL*Plus: Release 9.2.0.6.0 - Production on Wed Jun 17 10:05:57 2009
Copyright (c) 1982, 2002, Oracle Corporation. All rights reserved.
Connected to:
Oracle9i Enterprise Edition Release 9.2.0.6.0 - 64bit Production
JServer Release 9.2.0.6.0 - Production
SQL> select * from v$version;
BANNER
Oracle9i Enterprise Edition Release 9.2.0.6.0 - 64bit Production
PL/SQL Release 9.2.0.6.0 - Production
CORE    9.2.0.6.0       Production
TNS for HPUX: Version 9.2.0.6.0 - Production
NLSRTL Version 9.2.0.6.0 - Production
SQL> select * from nls_database_parameters where parameter like '%SET%';
PARAMETER                      VALUE
NLS_CHARACTERSET               WE8ISO8859P15
NLS_NCHAR_CHARACTERSET         AL16UTF16
SQL> exit
Disconnected from Oracle9i Enterprise Edition Release 9.2.0.6.0 - 64bit Production
JServer Release 9.2.0.6.0 - Production
exp / owner=test file=test.dmp
Export: Release 9.2.0.6.0 - Production on Wed Jun 17 10:06:42 2009
Copyright (c) 1982, 2002, Oracle Corporation. All rights reserved.
Connected to: Oracle9i Enterprise Edition Release 9.2.0.6.0 - 64bit Production
JServer Release 9.2.0.6.0 - Production
Export done in US7ASCII character set and AL16UTF16 NCHAR character set
server uses WE8ISO8859P15 character set (possible charset conversion)
About to export specified users ...Could you please:
- check and update NLS_LANG FAQ for Oracle 9i
- update NLS_LANG FAQ for Oracle 10g, Oracle 11g and Data Pump.
Thanks.
Edited by: P. Forstmann on Jun 17, 2009 10:14 AM

I was wrong and you are right like the OTN NLS_LANG FAQ.
I made the following test with Oracle 9.2.0.6 and with Oracle XE:
sqlplus test/test
SQL*Plus: Release 10.2.0.1.0 - Production on Ven. Juin 19 10:20:23 2009
Copyright (c) 1982, 2005, Oracle. All rights reserved.
Connecté à :
Oracle Database 10g Express Edition Release 10.2.0.1.0 - Production
SQL> create table t (x varchar2(10));
Table créée.
SQL> insert into t values(chr(233));
1 ligne créée.
SQL> select * from t;
X
é
SQL> select dump(x,1017) from t;
DUMP(X,1017)
Typ=1 Len=1 CharacterSet=WE8MSWIN1252: e9
SQL> exit
Déconnecté de Oracle Database 10g Express Edition Release 10.2.0.1.0 - Production
set NLS_LANG=AMERICAN_AMERICA.US7ASCII
exp test/test tables=t file=t.dmp
Export: Release 10.2.0.1.0 - Production on Fri Jun 19 10:21:54 2009
Copyright (c) 1982, 2005, Oracle. All rights reserved.
Connected to: Oracle Database 10g Express Edition Release 10.2.0.1.0 - Production
Export done in US7ASCII character set and AL16UTF16 NCHAR character set
server uses WE8MSWIN1252 character set (possible charset conversion)
About to export specified tables via Conventional Path ...
. . exporting table                              T          1 rows exported
Export terminated successfully without warnings.
sqlplus test/test
SQL*Plus: Release 10.2.0.1.0 - Production on Fri Jun 19 10:22:07 2009
Copyright (c) 1982, 2005, Oracle. All rights reserved.
Connected to:
Oracle Database 10g Express Edition Release 10.2.0.1.0 - Production
SQL> drop table t;
Table dropped.
SQL> exit
Disconnected from Oracle Database 10g Express Edition Release 10.2.0.1.0 - Production
imp test/test tables=t file=t.dmp
Import: Release 10.2.0.1.0 - Production on Fri Jun 19 10:22:23 2009
Copyright (c) 1982, 2005, Oracle. All rights reserved.
Connected to: Oracle Database 10g Express Edition Release 10.2.0.1.0 - Production
Export file created by EXPORT:V10.02.01 via conventional path
import done in US7ASCII character set and AL16UTF16 NCHAR character set
import server uses WE8MSWIN1252 character set (possible charset conversion)
. importing TEST's objects into TEST
. importing TEST's objects into TEST
. . importing table                            "T"          1 rows imported
Import terminated successfully without warnings.
sqlplus test/test
SQL*Plus: Release 10.2.0.1.0 - Production on Fri Jun 19 10:22:32 2009
Copyright (c) 1982, 2005, Oracle. All rights reserved.
Connected to:
Oracle Database 10g Express Edition Release 10.2.0.1.0 - Production
SQL> select dump(x,1017) from t;
DUMP(X,1017)
Typ=1 Len=1 CharacterSet=WE8MSWIN1252: e9

Character sets and conversions

Hi all,
were facing a quite complex problem, for which I'am not even able to specify were it is going wrong or what needs configuring, partly for lack of experience and partly for combining different tecnical areas from which I'm only responible for some of them.
So I'll sketch breefly the situation, and hopefully you might give me some guidelines or hints as to where to look at.
The setup : web application (so clients access by use of browser) on Weblogic- Linux platform, Tuxedo on Iseries , and as far as I understand some DB internally to Iseries where data is stored.
Data is entered in the DB by use of some data-entry application that comes with the iSeries.
The problem: consulting data by use of the web-aplication , some characters dont show up correctly , e.g. @ in email addresses, e's with accents, ...
For the chain being "browser <-> WL <-> Tuxedo <-> DB" , the problem might be different points. But from trace beeing activated , we could see that the response going out of tuxedo to WL is not correct...
Any hint as to what to look for, what can configuration is important, would be welcome ...
Some sub-questions:
- I understand Tuxedo is always "installed" in English , with no other option. This means that f.e. logs are in English.
But can/need to define some character set?
- Between Tuxedo <-> DB you can use som conversion tables ?
Any help would be apreciated , were quite lost ..

Hi,
Given that you are running Tuxedo on iSeries, I'm guessing you are running Tuxedo 6.5 as the port for the current Tuxedo release on iSeries hasn't been released yet. Tuxedo 6.5 does not directly support multi-byte character strings. The two common buffer formats for string data in Tuxedo are STRING which doesn't support multi-byte characters, or CARRAY which does support multi-byte characters as a CARRAY is essentially a blob. Do you know what buffer type the Tuxedo application is using to send data to WebLogic Server?
In Tuxedo 9.0 and later, direct support for multi-byte strings was added in the form of the MBSTRING buffer type. This buffer type supports multi-byte strings with a variety of character sets and encodings.
Regards,
Todd Little
Oracle Tuxedo Chief Archiitect

XMLELEMENT and suppress character conversion for brackets etc...

Hi,
I want use XMLElement, XMLAgg to create a large XML-file from database.
However I have the problem that I already SELECT valid XML chunks from
the database. I this case I don't want a character conversion,e.g.
open angle bracket (<) becomes <
quotation mark (") becomes "
and so on.
Is it possible with my Oracle 9.2 ?

Hi,
ok finally I think I found now a solution for my problem which is:
printing a large XML file in one transaction within sqlplus.
I let a function create the XML and write it to a CLOB variable.
In sqlplus I do simply
-- 1 GB is acceptable
SET LONG 1073741824;
SPOOL myspoolfile
SELECT MYFUNTION FROM DUAL;
I still have to proof it with mass data.

Datatype conversion and Range

Ok here i am back again..
I am creating a database upgrade tool,
the column types are also subject to change.
I use the CAST(col_name AS newtype) function to cast types,
this often works. But not for all cases.
@see http://download-west.oracle.com/docs/cd/A91202_01/901_doc/server.901/a90125/functions15.htm
For example a change from VARCHAR(255) to CHAR(3) does not work if the row currently
contains a row with more than 3 character.
A simple conversion that solves this problem is:
CAST(CAST(col_name AS VARCHAR(3)) AS CHAR(3))
I tested it using the following statement:
select CAST(CAST('wtfomgbbq' AS VARCHAR(3)) AS CHAR(3)) from dual;
Does this always work?
Some type changes can be done directly on a table like this:
ALTER TABLE table_name MODIFY ( col_name <NEWTYPE>)
But this only works if the range is either the same or increased,
or the table does not contain any rows, or the columns has only null values.
(i checked the reference)
But is there a way to always use this MODIFY statement and force the data to be transformed to the new type?
My current approach is to create a temporary table, drop the current table, and recreate the current (with correct schema) and put the old data in the new table using the CAST function as described above.
Maybe you experts now some way to do this faster?
And how can i handle range decreases without getting oracle errors?
(the update process may take several hours, so the DBA does not want an error message when updating the database)

It would be easier as you still have to convert the output of SUBSTR to CHAR(3), else you are doing an implicit type conversion... Which is not bad, but i really want to be explicit:
CAST(SUBSTR(col1, 1, 3) AS CHAR(3))
Note that the sql code that does the type conversions is automatically generated, so it doesnt matter if it looks more complex.
Anyway the main problem question is:
Can i always use ALTER TABLE MODIFY to modify the data types of columns or do i have to use temporary cols, or temporary tables?
Isnt there an option to force data type conversions and let oracle handle the casting (ans loss of data if the data-range is decreased) ?
Thanks in advance

Printing ZPL (Zebra) data to printer spooler without character conversion

Hi all,
We are printing shipping labels from UPS, with a process where we recive the ZPL label code directly from UPS, and we just need to pass the data to the printer to get the labels. We have already implemented this with Fedex and some custom labels, and it works perfectly. The problem with the UPS label data is that it contains non-printable characters (in the MaxiCode data field). When passed to the SAP printer spooler (see code example below), the data gets corrupted because SAP interprets these non-printable characters as printer control codes.
I have verified this by saving the ZPL data to a local file, before printing it through the SAP spooler. I then print this raw data and compare the output with the labels printed from the spooler. The MaxiCode (the big 2D barcode) is different in these labels. UPS has also tested the labels, and rejected them because of incorrect data in the barcode.
For printing, we are using printers defined as type "PLAIN", but I also tried using the "LZEB2" device type with the same result. The error we see in the spooler entry is this:
Print ctrl S_0D_ is not defined for this printer. Page 1, line 2, col. 2201
Print output may not be as intended
The printer ctrl code differs, depending om the label. I have examined the spooler data in "raw" mode, and there is always an ASCII character 28 (hex 1C) in front of the characters that SAP think are control codes, and this is why I think these non-printable characters are the reason for the problems.
This is the function module I use to print the ZPL data (and as stated above, this works fine for Fedex and custom labels). The ZPL data is converted to binary format before passed to the function module, but I also tried to send the data in text format with another FM, but the result is the same. I have experimented with the "codepage" parameter, and this one gives the least amount of errors, and some labels actually get through without errors. But still at least 50% of the labels gets corrupted, with log entries like above.
CALL FUNCTION 'RSPO_SR_WRITE_BINARY'
      EXPORTING
        handle           = lv_spool_handle
        data             = lv_label_line_bin
        length           = lv_len
        codepage         = '2010'
      EXCEPTIONS
        handle_not_valid = 1
        operation_failed = 2
        OTHERS           = 3.
Does anyone know if there is a way to send data to the spooler without character conversion or interpretation of printer control codes? Or is there any other smart way to get around this problem?
/Leif

I do a more direct output to the spooler, to avoid any issues with the WRITE statement and SAP's report output processing. At the same time, I insert line breaks so that the output is easy to debug in the spooler if needed. Also included is the code to to detect the escape code (ASCII #28) and to insert a control code ZZUPS in its place (you can skip this for Fedex). Here's a simplified example, but please note this is for a Unicode system, some minor changes is required in a non-Unicode system.
CONSTANTS: lc_spcode TYPE c LENGTH 5 VALUE 'ZZUPS',
           lc_xlen TYPE i VALUE 5.
   DATA: lv_print_params TYPE pri_params,
         lv_spool_handle TYPE sy-tabix,
         lv_name TYPE tsp01-rq0name,
         lv_spool_id TYPE rspoid,
         lv_crlf(2) TYPE c,
         lv_lf TYPE c,
         lstr_label_data TYPE zship_label_data_s,
         lv_label_line TYPE char512,
         lv_label_line_bin TYPE x LENGTH 1024,
         lv_len TYPE i,
         ltab_label_data_255 TYPE TABLE OF char512,
         ltab_label_data TYPE TABLE OF x,
         lv_c1 TYPE i,
         lv_c2 TYPE i,
         lv_cnt1 TYPE i,
         lv_cnt2 TYPE i,
         lv_x(2) TYPE x.
   FIELD-SYMBOLS: <n> TYPE x.
   lv_crlf = cl_abap_char_utilities=>cr_lf.
   lv_lf = lv_crlf+1(1).
   lv_name = 'ZPLLBL'.
CALL FUNCTION 'RSPO_SR_OPEN'
     EXPORTING
       dest                   = i_dest
       name                   = lv_name
       prio                   = '5'
       immediate_print        = 'X'
       titleline              = i_title
       receiver               = sy-uname
*      lifetime               = '0'
       doctype                = ''
     IMPORTING
       handle                 = lv_spool_handle
       spoolid                = lv_spool_id
     EXCEPTIONS
       device_missing         = 1
       name_twice             = 2
       no_such_device         = 3
       operation_failed       = 4
       OTHERS                 = 5.
   IF sy-subrc <> 0.
     RAISE spool_open_failed.
   ENDIF.
LOOP AT i_label_data INTO lstr_label_data.
     CLEAR ltab_label_data_255.
     SPLIT lstr_label_data-label_data AT lv_lf INTO TABLE ltab_label_data_255.
     LOOP AT ltab_label_data_255 INTO lv_label_line.
       IF lv_label_line NE ''.
         lv_len = STRLEN( lv_label_line ).
*       Convert character to hex type
         lv_c1 = 0.
         lv_c2 = 0.
         DO lv_len TIMES.
           ASSIGN lv_label_line+lv_c1(1) TO <n> CASTING.
           MOVE <n> TO lv_x.
           IF lv_x = 28.
             lv_cnt1 = 0.
             lv_label_line_bin+lv_c2(1) = lv_x.
             lv_c2 = lv_c2 + 1.
             DO lc_xlen TIMES.
               ASSIGN lc_spcode+lv_cnt1(1) TO <n> CASTING.
               MOVE <n> TO lv_x.
               lv_cnt2 = lv_c2 + lv_cnt1.
               lv_label_line_bin+lv_c2(2) = lv_x.
               lv_c2 = lv_c2 + 2.
               lv_cnt1 = lv_cnt1 + 1.
               lv_len = lv_len + 1.
             ENDDO.
           ELSE.
             lv_label_line_bin+lv_c2(2) = lv_x.
             lv_c2 = lv_c2 + 2.
           ENDIF.
           lv_c1 = lv_c1 + 1.
         ENDDO.
*       Print binary data to spool
         lv_len = lv_len * 2. "Unicode is 2 bytes per character
         CALL FUNCTION 'RSPO_SR_WRITE_BINARY'
           EXPORTING
             handle                 = lv_spool_handle
             data                   = lv_label_line_bin
             LENGTH                 = lv_len
           EXCEPTIONS
             handle_not_valid       = 1
             operation_failed       = 2
             OTHERS                 = 3.
         IF sy-subrc <> 0.
           RAISE spool_write_failed.
         ENDIF.
       ENDIF.
     ENDLOOP.
   ENDLOOP.
   CALL FUNCTION 'RSPO_SR_CLOSE'
     EXPORTING
       handle = lv_spool_handle.
   IF sy-subrc <> 0.
     RAISE spool_close_failed.
   ENDIF.

Character conversion problems when calling FM via RFC from Unicode ECC 6.0?

Hi all,
I faced a Cyrillic character convertion problem while calling an RFC function from R/3 ECC 6.0 (initialized as Unicode system - c.p. 4103). My target system is R/3 4.6C with default c.p. 1500.
The parameter I used in my FM interface in target system is of type CHAR10 (single-byte, obviously).
I have defined rfc-connection (SM59) as an ABAP connection and further client/logon language/user/password are supplied.
The problem I faced is, that Cyrillic symbols are transferred as '#' in the target system ('#' is set as default symbol in RFC-destination definition in case character convertion error is met).
Checking convertions between c.p. 4103 and target c.p. 1500 in my source system using tools of transaction i18n shows no errors - means conversion passed O.K. It seems default character conversion executed by source system whithin the scope of RFC-destination definition is doing something wrong.
Further, I played with MDMP & Unicode settings whithin the RFC-destination definition with no successful result - perhaps due to lack of documentation for how to set and manage these parameters.
The question is: have someone any experience with any conversion between Unicode and non-Unicide systems via RFC-call (non-English target obligatory !!!), or can anyone share valuable information regarding this issue - what should be managed in the RFC-destination in order to get character conversion working? Is it acceptable to use any character parameter in the target function module interface at all?
Many thanks in advance.
Regards,
Ivaylo Mutafchiev
Senior SAP ABAP Consultant

hey,
I had a similar experience. I was interfacing between 4.6 (RFC), PI and ECC 6.0 (ABAP Proxy). When data was passed from ECC to 4.6, RFC received them incorrectly. So i had to send trimmed strings from ECC and receive them as strings in RFC (esp for CURR and QUAN fields). Also the receiver communication channel in PI (between PI and RFC) had to be set as Non unicode. This helped a bit. But still I am getting 2 issues, truncation of values and some additional digits !! But the above changes resolved unwanted characters problem like "<" and "#". You can find a related post in my id. Hope this info helps..

Error occurred during character conversion in SXMB_MONI

Hello Experts,
Good Day!
I would like to seek your help here. When i used tcode SXMB_MONI to search for messages i get this error : Error occurred during character conversion.
So far no problem with the program. Its work for all other dates. Just for one particular day and specific time period, im geting this error.
Does anyone know what is this error means? Please reply..
Thanks for your help.
Looking forward for ur replies..

Hi Presheela,
Basically this problem occurs when ur payload contains any special characters like '&' ,'>'...etc .So you have to take care of how to deal with these characters in XI.
Refer the following documentation:
How to Work with Character Encodings in Process Integration
https://www.sdn.sap.com/irj/sdn/go/portal/prtroot/docs/library/uuid/502991a2-45d9-2910-d99f-8aba5d79fb42
Regards,
Vinod.

Character conversion error

Hi all,
We are getting the following error when trying to parse an xml string resource - Character conversion error: "Illegal ASCII character, 0xc2" (line number may
be too low)-. We have not been able to get around this. We have tried creating the InputSource two different ways:
reader = new StringReader(stringSource);
src = new InputSource( reader );
and
src = new InputSource(new InputStreamReader(new ByteArrayInputStream(stringSource.getBytes())));
The problem does appear to go away if we treat the DTD we are validating against as a file. If we set it has a uri, we get the above problem.
Is anyone else experiencing this problem?
Any help would be greatly appreciated.
Thanks in advance,
Greg

Hi,
2 possible solutions:
1) try using the xerces parser instead of sun's parser
2) look at the posting at the following url and see wether the posted solution solves your problem: http://forums.java.sun.com/thread.jsp?forum=34&thread=67558
Hope this helps,
Kurt.

"character conversion error" while parsing xml files

Hello,
I'm trying to parse MusicXML (Recordare) files, but I'm getting an exception.
I'm using the SAX parser (javax.xml.parsers.SAXParser).
Here is the code I use to instantiate it:
final javax.xml.parsers.SAXParserFactory saxParserFactory = javax.xml.parsers.SAXParserFactory.newInstance();
final javax.xml.parsers.SAXParser saxParser = saxParserFactory.newSAXParser();
final org.xml.sax.XMLReader parser = saxParser.getXMLReader();
I'm using my own handler, but I get the same exception even if I use org.xml.sax.helpers.DefaultHandler.
The error I get is:
Character conversion error: "Illegal ASCII character, 0xc2" (line number may be too low).
The first few lines of my xml files look like this:
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!DOCTYPE score-partwise
PUBLIC "-//Recordare//DTD MusicXML 0.6 Partwise//EN"
"http://www.musicxml.org/dtds/partwise.dtd">
<score-partwise>
[...etc...]
If I delete the <!DOCTYPE ...> line, then I don't get the exception anymore. But the MusicXML files I get (from some other program) always contain this line, and it would be quite some work to delete them from every file manually.
So does anyone know if there is a way to avoid deleting that line in every file, while still being able to parse the xml files without exceptions?
Or maybe does anyone know what the exact cause of the exception is? (because I don't know what exactly causes it)
Thank you in advance.
Greetz,
Jipo

So does anyone know if there is a way to avoid
deleting that line in every file, while still being
able to parse the xml files without exceptions?ok this is side-stepping the real problem but I've used this code to filterout DTD references for other reasons   public static InputStream filterOutDTDRef(InputStream in) throws IOException {
      BufferedReader iniReader = new BufferedReader(new InputStreamReader(in));
      StringBuffer newXML = new StringBuffer();
      for(String line = iniReader.readLine(); line!=null; line = iniReader.readLine())
         newXML.append(line+"\n");
      in.close();
      int s = newXML.indexOf("<!DOCTYPE ");
      if(s!=-1)
         newXML.replace(s,newXML.indexOf(">",s)+1,"");
      return new ByteArrayInputStream(newXML.toString().getBytes());
   }and it actually speeds up the parsing phase too (since the DTD ref.s were on the web and the XML standard mandates that there is a fetch for each xml file parsed..)
you can feed the above into the InputSource constructor that takes an InputStream argument.
Now for the real problem... 0xc2 is "LATIN CAPITAL LETTER A WITH CIRCUMFLEX" according to a unicode chart - which is not an ASCII character (as the error message correctly reports). I'm not sure why the file is being parsed as ASCII though? You could try parsing in a FileReader to the inputsource and hope it picks up the default character encoding of your system, and that that character encoding matches the file. Or you could try passing in a FileReader constructed with a explicit character encoding (eg "UTF8") and see if that does the trick?
asjf

Problem in the character conversion

Hi Guys,
I am facing problem in the character conversion
I am posting data from SAP to third party system using XI , by converting whole input message to a String .I am using SOAP adapter to communicate XI to third party system.
Thirdparty system needs String to be wrapped in CDATA so that it will not choke by looking at the special characters. I did Wrap the output string in CDATA, using ABAP mapping but when I do that XI is converitng arrow brackets < and >. into &lt and u2018&gtu2019 my assumption is it is double encoding.
example -
before map - <AppSystemInfo>
after mapping it is converted as - <![CDATA[ &ltAppSystemInfo&gt]]>
Edited by: Vamsi on Jun 17, 2010 10:00 PM
Edited by: Vamsi on Jun 17, 2010 10:01 PM

Did you try to see the output?
bcz if you are trying this in mapping testing it will show you like this as this conversion if for xml, so xml will not do anything wrong with the special characters, so for that special characters will be converted like that.
Once try to run end to end interface and try to see at receiver side that how data looks like.
Thanks,
Hetal

Character conversion error: Unconvertible UTF-8 character beginning..

Hello,
I'm using TrAX for XSLT transformations, and having a following
problem
Character conversion error: "Unconvertible UTF-8 character beginning with 0xa9" (line number may be too low).
org.xml.sax.SAXParseException: Character conversion error: "Unconvertible UTF-8 character beginning with 0xa9" (line number may be too low).
        at org.apache.crimson.parser.InputEntity.fatal(InputEntity.java:1100)
        at org.apache.crimson.parser.InputEntity.fillbuf(InputEntity.java:1072)
        at org.apache.crimson.parser.InputEntity.isXmlDeclOrTextDeclPrefix(InputEntity.java:914)
        at org.apache.crimson.parser.Parser2.maybeTextDecl(Parser2.java:2795)
        at org.apache.crimson.parser.Parser2.externalParameterEntity(Parser2.java:2880)
        at org.apache.crimson.parser.Parser2.maybeDoctypeDecl(Parser2.java:1167)
        at org.apache.crimson.parser.Parser2.parseInternal(Parser2.java:489)
        at org.apache.crimson.parser.Parser2.parse(Parser2.java:305)
        at org.apache.crimson.parser.XMLReaderImpl.parse(XMLReaderImpl.java:442)
        at mlts.converter.XMLImport.outputESGML(XMLImport.java:311)
        at mlts.converter.Converter.processFile(Converter.java:312)
        at mlts.converter.Converter.Convert(Converter.java:229)
        at test.main(test.java:7)Following the source code, I've found that the exception is thrown
when it reads DTD. I tried to read DTD using InputSource
in ascii, in latin-1 and my program reads it without any problem.
I really appreciate any help,
Thanks

http://forum.java.sun.com/thread.jsp?forum=34&thread=254927

Character conversion error: "Unconvertible UTF-8 character beginning with 0

Hi All,
I developed an Adapter Module and added to Adapter Framework.
package sample;
import java.io.InputStream;
import java.text.SimpleDateFormat;
import java.util.Date;
import javax.ejb.CreateException;
import javax.ejb.SessionBean;
import javax.ejb.SessionContext;
import javax.xml.parsers.DocumentBuilder;
import javax.xml.parsers.DocumentBuilderFactory;
import org.w3c.dom.Document;
import org.w3c.dom.Element;
import org.w3c.dom.NodeList;
import com.sap.aii.af.mp.module.Module;
import com.sap.aii.af.mp.module.ModuleContext;
import com.sap.aii.af.mp.module.ModuleData;
import com.sap.aii.af.mp.module.ModuleException;
import com.sap.aii.af.ra.ms.api.Message;
import com.sap.aii.af.ra.ms.api.XMLPayload;
@ejbHome <{com.sap.aii.af.mp.module.ModuleHome}>
@ejbLocal <{com.sap.aii.af.mp.module.ModuleLocal}>
@ejbLocalHome <{com.sap.aii.af.mp.module.ModuleLocalHome}>
@ejbRemote <{com.sap.aii.af.mp.module.ModuleRemote}>
@stateless
public class SetAttachmentName implements SessionBean, Module {
     private SessionContext myContext;
     private String mailFileName = "UStN";
     public void ejbRemove() {
     public void ejbActivate() {
     public void ejbPassivate() {
     public void setSessionContext(SessionContext context) {
          myContext = context;
     public void ejbCreate() throws CreateException {
     public ModuleData process(
          ModuleContext moduleContext,
          ModuleData inputModuleData)
          throws ModuleException {
          // create a second attachment for the receiver mail adapter
          try {
               //                  get the XI message from the environment
               Message msg = (Message) inputModuleData.getPrincipalData();
               //               creating parsable XML document
               InputStream XIStreamData = null;
               XMLPayload xmlpayload = msg.getDocument();
               XIStreamData = xmlpayload.getInputStream();
               DocumentBuilderFactory docBuilderFactory =
                    DocumentBuilderFactory.newInstance();
               DocumentBuilder docBuilder = docBuilderFactory.newDocumentBuilder();
               Document doc = docBuilder.parse(XIStreamData);
               //            finding the tag's name from the Modules tab in the Directory that will hold the attachment's name
               String absenderIDTag = null;
               absenderIDTag = moduleContext.getContextData("<RCVPRN>");
               //            finding the content of the tag that will be used as the attachment's name (assuming it's the only tag with this name)
               Element element = doc.getDocumentElement();
               NodeList list = doc.getElementsByTagName(absenderIDTag);
               mailFileName += "_" + list.item(0).getFirstChild().toString();
               String anIDTag = null;
               anIDTag = moduleContext.getContextData("<CREDAT>");
               element = doc.getDocumentElement();
               list = doc.getElementsByTagName(anIDTag);
               mailFileName += "_" + list.item(0).getFirstChild().toString();
               Date date = new Date(System.currentTimeMillis());
               //            Add date to the Message
               SimpleDateFormat dateFormat = new SimpleDateFormat("yyyyMMdd");
               mailFileName += "_" + dateFormat.format(date);
               String belegNummerTag = null;
               belegNummerTag = moduleContext.getContextData("<BULK_REF>");
               element = doc.getDocumentElement();
               list = doc.getElementsByTagName(belegNummerTag);
               mailFileName += "_" + list.item(0).getFirstChild().toString();
               //               creating the attachment
               byte by[] = xmlpayload.getText().getBytes();
               XMLPayload attachmentPDF = msg.createXMLPayload();
               attachmentPDF.setName(mailFileName);
               attachmentPDF.setContentType("application/pdf");
               attachmentPDF.setContent(by);
               //adding the message to the attachment
               msg.addAttachment(attachmentPDF);
               //                  provide the XI message for returning
               inputModuleData.setPrincipalData(msg);
          } catch (Exception e) {
               //                  raise exception, when an error occurred
               ModuleException me = new ModuleException(e);
               throw me;
          //                  return XI message
          return inputModuleData;
I get the following error
Character conversion error: "Unconvertible UTF-8 character beginning with 0xaa" (line number may be too low).
Any tips, pointers ?
Thanks in Advance
Mukhtar

Hi Henrique,
I am using .getNodeValue()
import java.io.InputStream;
import java.text.SimpleDateFormat;
import java.util.Date;
import javax.ejb.CreateException;
import javax.ejb.SessionBean;
import javax.ejb.SessionContext;
import javax.xml.parsers.DocumentBuilder;
import javax.xml.parsers.DocumentBuilderFactory;
import org.w3c.dom.Document;
import org.w3c.dom.Element;
import org.w3c.dom.NodeList;
import com.sap.aii.af.mp.module.*;
import com.sap.aii.af.ra.ms.api.*;
@ejbHome <{com.sap.aii.af.mp.module.ModuleHome}>
@ejbLocal <{com.sap.aii.af.mp.module.ModuleLocal}>
@ejbLocalHome <{com.sap.aii.af.mp.module.ModuleLocalHome}>
@ejbRemote <{com.sap.aii.af.mp.module.ModuleRemote}>
@stateless
public class UStNAttachmentName3 implements SessionBean, Module {
     private SessionContext myContext;
     private String mailFileName = "UStN";
     public void ejbRemove() {
     public void ejbActivate() {
     public void ejbPassivate() {
     public void setSessionContext(SessionContext context) {
          myContext = context;
     public void ejbCreate() throws CreateException {
     public ModuleData process(
          ModuleContext moduleContext,
          ModuleData inputModuleData)
          throws ModuleException {
          // create a second attachment for the receiver mail adapter
          try {
               // get the XI message from the environment
               Message msg = (Message) inputModuleData.getPrincipalData();
               // creating parsable XML document
               InputStream XIStreamData = null;
               XMLPayload xmlpayload = msg.getDocument();
               XIStreamData = xmlpayload.getInputStream();
               DocumentBuilderFactory docBuilderFactory =
                    DocumentBuilderFactory.newInstance();
               DocumentBuilder docBuilder = docBuilderFactory.newDocumentBuilder();
               Document doc = docBuilder.parse(XIStreamData);
               // finding the tag's name from the Modules tab in the Directory that will hold the attachment's name
               String absenderIDTag = null;
               absenderIDTag = moduleContext.getContextData("<RCVPRN>");
               // finding the content of the tag that will be used as the attachment's name (assuming it's the only tag with this name)
               Element element = doc.getDocumentElement();
               NodeList list = doc.getElementsByTagName(absenderIDTag);
               mailFileName += "_" + list.item(0).getFirstChild().getNodeValue();
               String anIDTag = null;
               anIDTag = moduleContext.getContextData("<CREDAT>");
               element = doc.getDocumentElement();
               list = doc.getElementsByTagName(anIDTag);
               mailFileName += "_" + list.item(0).getFirstChild().getNodeValue();
               Date date = new Date(System.currentTimeMillis());
               // Add date to the Message
               SimpleDateFormat dateFormat = new SimpleDateFormat("yyyyMMdd");
               mailFileName += "_" + dateFormat.format(date);
               String belegNummerTag = null;
               belegNummerTag = moduleContext.getContextData("<BULK_REF>");
               element = doc.getDocumentElement();
               list = doc.getElementsByTagName(belegNummerTag);
               mailFileName += "_" + list.item(0).getFirstChild().getNodeValue();
               // creating the attachment
               byte by[] = xmlpayload.getText().getBytes();
               XMLPayload attachmentPDF = msg.createXMLPayload();
               attachmentPDF.setName(mailFileName);
               attachmentPDF.setContentType("application/pdf");
               attachmentPDF.setContent(by);
               //adding the message to the attachment
               msg.addAttachment(attachmentPDF);
               // provide the XI message for returning
               inputModuleData.setPrincipalData(msg);
          } catch (Exception e) {
               // raise exception, when an error occurred
               ModuleException me = new ModuleException(e);
               throw me;
          // return XI message
          return inputModuleData;
Still I get the same error.
org.xml.sax.SAXParseException: Character conversion error: "Unconvertible UTF-8 character beginning with 0xaa" (line number may be too low).
Adapter-Framework: Character conversion error: "Unconvertible UTF-8 character beginning with 0xaa" (line number may be too low).
Regards,
Mukhtar

SMQ2 (Inbound Queue) : Error occurred during character conversion

Hi,
In SMQ2 an Inbound Queue has failed with status text "Error occurred during character conversion". The XML message inside this queue is in waiting status with status text " Scheduled for Outbound Processing" .
Even after cancelling this message from SXMB_MONI, and reposting the PO, still again the queue and the XML message inside the queue are in same status.
As this is in XI production system, requesting to provide some suggestion ASAP.
Thanks,
Mateen.

The queue status is still sysfail. There is only 1 message waiting in the queue.
I think that the queue might be locked for some reason and after unlocking the queue and restarting the message, the message will get processed successfully.
But before working on this I want to make sure that the queue is locked.
Do you or anyone else know how and from where can I check whether the queue is locked or unlocked?
Regards,
Mateen.

Character conversion and NLS_LANG

Similar Messages

Maybe you are looking for