Encoding from UTF-16 to UTF-8

Hi,
I need to convert from UTF-16 to UTF-8 encoding.
I receive an CSV file in encoding UTF-16 for our backend system. but our external partner needs the encoding to be UTF-8
How can I change the encoding ?

Hello Frank,
We have used TextCodePageConversionBean to meet such a requirement in one of our scenarios using CSV files.
http://help.sap.com/saphelp_nw04/helpdata/en/45/da2deb47812e98e10000000a155369/content.htm
Can you please try this and let us know if this helps?
Thanks.
Best Regards,
Shweta

Similar Messages

Change character encoding from UTF-8 to EUC-KR

We are receiving data in UTF-8 in the querystring from a partner formatted as:
%EA%B3%A0%EB%AF%BC%ED%95%98%EC%9E%90%21
Our site uses EUC-KR so using this text for search/display/etc is not possible. Does anyone know how we can convert this to the proper Korean EUC encoding so it can be displayed properly using JSP? Basically it should be:
%B0%ED%B9%CE%C7%CF%C0%DA%21
Thanks in advance.

I'm not sure where you are getting %xx encoded UTF-8.... Is it cuz you have it in a GET method form and that's what you are seeing in the browser's location bar? ...
Let's assume you have a form on a page, and the page's charset is set to UTF-8, and you want to generate a URL encoded string (%xx format, although URLEncoder will not encode ASCII chars that way...).
In the page processing the form, you need to do this:
request.setCharacterEncoding("UTF-8"); // makes bytes read as UTF-8 strings(assumes that the form page was properly set to the UTF-8 charset)
String fieldValue = request.getParameter("fieldName"); // get value
// the value is now a Unicode String in Java, generated from reading the bytes submitted from the form as UTF-8 encoded text...
String utf8EncString = URLEncoder.encode(fieldValue, "UTF-8");
// now utf8EncString is a URL encoded (%xx) string of UTF-8 values
String euckrEncString = URLEncoder.encode(fieldValue, "EUC-KR");
// now euckrEncString is a URL encoded (%xx) string of EUC-KR valuesWhat is probably screwing things up for you mostly is this:
euckrValue = new String(utf8Value.getBytes(), "EUC-KR");
What this does is takes the bytes of the string utf8Value (which is not really UTF-8... see below) in the local encoding (possibly Cp1252 (Windows) or ISO8895-1 (Linux), or EUC-KR if it's Korean Windows), and then reads them as if they were EUC-KR... which they aren't.
The key here is that Strings in Java are not of any encoding. They are pure Unicode values. Encodings only matter when converting to or from bytes. The strings stored in a file or sent over the net have to convert to bytes since that's what is stored/sent, just bytes. The encoding defines how the characters can be encoded into 1 or more bytes, and thus reconstructed.

Encoding non english characters with utf 8 on jsp (Critical!!)

I am inserting hebrew characters from JSP into oracle db and everything is fine until this point. But when I try to retrieve the information from the database, the characters are not displayed properly (I get some garbage characters). I am sure that the data stored in the database is correct, but not sure why there is a problem in displaying the data in the JSP.
I came across a thread on TSS
http://www.theserverside.com/discussions/thread.tss?thread_id=28944
and followed the suggestions given there like having
<%@ page contentType="text/html; charset=UTF-8" pageEncoding="UTF-8" %>
<META http-equiv="Content-Type" content="text/html; charset=UTF-8">and also this
<%
//Some JDBC and sql statement query UTF-8 data and then ...
String str = rs.getString("utf8_data");
str = new String(str.getBytes("ISO-8859-1"),"UTF-8");
%>
<%= str %>Now, the data getting displayed is partly correct, I mean to say, some characters are still coming as squares.
Any ideas will be of great help.

even i doubt the database charset for this issue. But what I dont understand is how only certain hebrew characters are getting stored properly and why others are corrupted?
Also, can anyone let me know how i can view the Non-English characters present in the database directly, as TOAD is not able to display them

Can any version of Excel save to a CSV file that is either UTF-8 or UTF-16 encoded (unicode)?

Are there any versions of Excel (chinese, japanese, russian... 2003, 2007, 2010...) that can save CSV files in Unicode (either UTF-8 or UTF-16)?
If not, is the only solution to go with tab-delimited files (save as Unicode-text option)?

Hi Mark,
I have the same problem. Trying to save my CSV file in UTF8 encoding. After several hours in searching and trying this also in my VSTO Add-In I got nothing. Saving file as Unicode option in Excel creates file as TAB separated. Because I'd like to save the
file in my Add-In application, the best to do is (for my problem) saving file as unicode tab delimited and then replacing all tabs with commas in the file automatically.
I don't think there is a direct way to save CSV as unicode in Excel. And I don't understand why.

Convertion from UTF-16 to UTF-8 in XI

Hi,
From Source system (MDM), sometimes data are coming in UTF-16 format in to XI. My target system is R/3 which is UTF-8. Here's the scenario:-
MDM->MQ Queue-> Local JMS Queue-> XI->R/3
Here I am using sender JMS Queue adapter to receive the data from Local JMS Queue and using receiver IDOC adapter to send the IDOC into R/3. I am using ABAP mapping for this scenario.
Since the target system in UTF-8 and the data are coming sometimes in UTF-16, how can I change the format UTF-16 to UTF-8 in sender JMS adapter.
Please advice.
Reply with details would be appreciated.
BR
Soumya

Hi Soumya ,
You can do this in Adapter module in JMS sender adapter .
obj = inputModuleData.getPrincipalData();
msg = (Message) obj;
XMLPayload xmlpayload = msg.getDocument();
xmlpayload.getContent()
convert from UTF 16 to UTF 8 then
xmlpayload.setContent();
Hope this works.
Cheers,
Reddy

How is the largest cde point differs from UTF-8 to UTF-16

how is the largest cde point differs from UTF-8 to UTF-16
the largest code point is 10FFFF for both of them then how is differ from the fromat
thank you,
Regards,
Jagrut BharatKumar Shukla

In this specific case there are no differences for code points storing character data because used character set is the same.
But what is your Oracle 4 digits version ?
Are you sure that database character set and national character set are the same ?
In recent Oracle versions, database character set and national character set are different. For example:
SQL> select * from nls_database_parameters where parameter like '%SET%';
PARAMETER                      VALUE
NLS_CHARACTERSET               AL32UTF8
NLS_NCHAR_CHARACTERSET         AL16UTF16Edited by: P. Forstmann on 28 sept. 2011 18:51

How to block encoded subject header with =?utf-8?B?

What is the best way to block email with encoded subject header with
=?utf-8?B?WzM0MDkzODld5ZyG5Lit5Zu95bel5Lia5qKm77ya44CK5Lqn5ZOB5aSn5YWo5ZKM5bqU55So5qGI5L6L5omL5YaM44CLKDIwMTPlubQp5YWN6LS55LiL6L29KDMy6aG1LOmrmOa4hSw3TSxQREbmoLzlvI8pWzYwZzJqMDk5aHVzZnMwYWhja2FjZm00c3dyazlwd3o5M2dmbm1nbHo0Ym50ZV0=?=

Customize the adapter to write the files out in UTF-8.
OR
Write code to re-create the .DAT output file in UTF-8 format.
I'm not sure the exact coding on either option, but anything is possible.

How to convert UTF-16 to UTF-8

data source is 'ъѓъѓ№ѓфчр Фюыр№ 80Ъ ', it is Ukraine.
I want to remove the blank, but no matter which key word in SAP I use, it doesn't work. i checked hexadecimal of the space from the text above , it is 00A0, but actually system only regard 0020 as space. i checked on internet, the space of the text should be encoded with UTF-16 and system is UTF-8, 00A0 is extended ASCII, so 00A0 can't be seen in SAP system.
my question is in this situation, how can a remove the space?

Hi Eric,
This Document might help u,
Link: [how to convert UTF-16 to UTF-8|How to convert xml utf 16 to utf 8;
-Dileep .C

XML Encoding from Business One - Unable to Read in a 3rd Party APP

Hi:
I have created an Add-On app that produces an XML document file when a new item is created or an existing item is updated. However, the 3rd party app that I am trying to integrate into is unable to read this XML. After doing some research, the following appears to be the issue.
SBO produces the data with UTF-16 which is a double byte encoding for XML's. The 3rd party app that I have appears to be able to handle only single byte encoding formats such as UTF-8 or ISO-8859-1.
I would like to know if there is a way in SDK to convert the output XML format to a single byte encoding such as ISO-8859-1. Can anyone please tell me how to achieve that in my Add-On code?
Thank you.
Kushal Dutta

Hi Kushal,
You can use a XLST transformation to convert from UTF-16 to UTF-8.
The coding goes something like this:
XmlReader xmlRdr = XmlReader.Create(new StringReader(sXML));
XmlWriterSettings xmlWtrSettings = new XmlWriterSettings();
xmlWtrSettings.Encoding = Encoding.Unicode;
XmlWriter xmlWtr = XmlWriter.Create(sOutFile, xmlWtrSettings);
XslCompiledTransform xslt = new XslCompiledTransform();
xslt.Load(sTransformFile);
xslt.Transform(xmlRdr, xmlWtr);
xmlWtr.Flush();
xmlWtr.Close();
xmlRdr.Close();
Where sXML is the XML string generated by the SBO GetAsXML method, sOutFile is the path and filename where the transformed XML will be created and sTransformFile is the path of the XLST file.
Assuming that you wish to retain the same XML structure as before, the XLST file should look something like the following:
<?xml version="1.0" encoding="utf-8"?>
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
<xsl:template match="/">
     <BOM>
          <xsl:for-each select="BOM/BO">
          <BO>
               <AdmInfo>
                    <Object>
                         <xsl:value-of select="AdmInfo/Object"/>
                    </Object>
               </AdmInfo>
               <ORDR>
                    <xsl:for-each select="ORDR/row">
                    <row>
                         <DocEntry>
                              <xsl:value-of select="DocEntry"/>
                         </DocEntry>
                         <DocNum>
                              <xsl:value-of select="DocNum"/>
                         </DocNum>
                         <DocDate>
                              <xsl:value-of select="DocDate"/>
                         </DocDate>
                         <DocDueDate>
                              <xsl:value-of select="DocDueDate"/>
                         </DocDueDate>
                         <DocStatus>
                              <xsl:value-of select="DocStatus"/>
                         </DocStatus>
                         <Cancelled>
                              <xsl:value-of select="CANCELED"/>
                         </Cancelled>
                         <CardCode>
                              <xsl:value-of select="CardCode"/>
                         </CardCode>
                         <CardName>
                              <xsl:value-of select="CardName"/>
                         </CardName>
                         <NumAtCard>
                              <xsl:value-of select="NumAtCard"/>
                         </NumAtCard>
                    </row>
                    </xsl:for-each>
               </ORDR>
               <RDR1>
                    <xsl:for-each select="RDR1/row">
                    <xsl:sort select="LineNum" />
                    <row>
                         <LineNum>
                              <xsl:value-of select="LineNum"/>
                         </LineNum>
                         <ItemCode>
                              <xsl:value-of select="ItemCode"/>
                         </ItemCode>
                         <Dscription>
                              <xsl:value-of select="Dscription"/>
                         </Dscription>
                         <Quantity>
                              <xsl:value-of select="Quantity"/>
                         </Quantity>
                         <WhsCode>
                              <xsl:value-of select="WhsCode"/>
                         </WhsCode>
                         <BaseType>
                              <xsl:value-of select="BaseType"/>
                         </BaseType>
                         <BaseEntry>
                              <xsl:value-of select="BaseEntry"/>
                         </BaseEntry>
                         <BaseLine>
                              <xsl:value-of select="BaseLine"/>
                         </BaseLine>
                         <FreeTxt>
                              <xsl:value-of select="FreeTxt"/>
                         </FreeTxt>
                    </row>
                    </xsl:for-each>
               </RDR1>
          </BO>
          </xsl:for-each>
     </BOM>
</xsl:template>
</xsl:stylesheet>
The above example shows a XLST transform for the sales order document type. This example is for use with the xet_ValidNodesOnly setting for the XmlExportType company property. If you are using a different XmlExportType setting then the XLST will be slightly different but the principal is the same. For simplicity, I have shown only a cut-down list of the fields from the sales order but other fields and tables can be included (assuming that they are in the original XML from SBO).
Kind Regards,
Owen
P.S. Sorry for the lack of indentation on the XML file. It was there when I created this post but the forum removes the indentation.

Detecting character encoding from BLOB stream... (PLSQL)

I'am looking for a procedure/function which can return me the character encoding of a "text/xml/csv/slk" file stored in BLOB..
For example...
I have 4 files in different encodings (UTF8, Utf8BOM, ISO8859_2, Windows1252)...
With java I'can simply detect the character encoding with JuniversalCharDet (http://code.google.com/p/juniversalchardet/)...
thank you

Solved...
On my local PC I have installed Java 1.5.0_00 (because on DB is 1.5.0_10)...
With Jdeveloper I have recompiled source code from:
http://juniversalchardet.googlecode.com/svn/trunk/src/org/mozilla/universalchardet
http://code.google.com/p/juniversalchardet/
After that I have made a JAR file and uploaded it with loadjava to my database...
C:\>loadjava -grant r_inis_prod -force -schema insurance2 -verbose -thin -user username/password@ip:port:sid chardet.jarAfter that I have done a java procedure and PLSQL wrapper example below:
   public static String verifyEncoding(BLOB p_blob) {
       if (p_blob == null) return "-1";
       try
        InputStream is = new BufferedInputStream(p_blob.getBinaryStream());
        UniversalDetector detector = new UniversalDetector(null);
        byte[] buf = new byte[p_blob.getChunkSize()];
        int nread;
        while ((nread = is.read(buf)) > 0 && !detector.isDone()) {
            detector.handleData(buf, 0, nread);
        detector.dataEnd();
        is.close();
       return detector.getDetectedCharset();
       catch(Exception ex) {
           return "-2";
   }as you can see I used -2 for exception and -1 if input blob is null.
then i have made a PLSQL procedure:
function f_preveri_encoding(p_blob in blob) return varchar2 is
language Java name 'Zip.Zip.verifyEncoding(oracle.sql.BLOB) return java.lang.String';After that I have uploaded 2 different txt files in my blob field.. (first one is encoded with UTF-8, second one with WINDOWS-1252)..
example how to call:
declare
   l_blob blob;
   l_encoding varchar2(100);
begin
select vsebina into l_blob from dok_vsebina_dokumenta_blob where id = 401587359 ;
l_encoding := zip_util.f_preveri_encoding(l_blob);
if l_encoding = 'UTF-8' then
   dbms_output.put_line('file is encoded with UTF-8');
elsif l_encoding = 'WINDOWS-1252' then
   dbms_output.put_line('file is encoded with WINDOWS-1252');
else
    dbms_output.put_line('other enc...');
end if;
end;Now I can get encoding from blob and convert it to database encoding and store datas in CLOB field..
Here you have a chardet.jar file if you need this functionality..
https://docs.google.com/open?id=0B6Z9wNTXyUEeVEk3VGh2cDRYTzg
Edited by: peterv6i.blogspot.com on Nov 29, 2012 1:34 PM
Edited by: peterv6i.blogspot.com on Nov 29, 2012 1:34 PM
Edited by: peterv6i.blogspot.com on Nov 29, 2012 1:38 PM

How we represent largest code point in UTF-8 and UTF-16 whats the differenc

how we represent largest code point in UTF-8 and UTF-16 whats the differenc
points will be awarded

There are standards from for CHARACTER encoding.
See below for a brief description:
UTF-16 (16-bit Unicode Transformation Format) is a variable-length character encoding for Unicode, capable of encoding the entire Unicode repertoire. The encoding form maps code points (characters) into a sequence of 16-bit words, called code units. For characters in the Basic Multilingual Plane (BMP) the resulting encoding is a single 16-bit word. For characters in the other planes, the encoding will result in a pair of 16-bit words, together called a surrogate pair. All possible code points from U0000 through U10FFFF, except for the surrogate code points UD800UDFFF, are uniquely mapped by UTF-16 regardless of the code point's current or future character assignment or use.
UTF-8 (8-bit UCS/Unicode Transformation Format) is a variable-length character encoding for Unicode. It is able to represent any universal character in the Unicode standard, yet the initial encoding of byte codes and character assignments for UTF-8 is consistent with ASCII (requiring little or no change for software that handles ASCII but preserves other values). For these reasons, it is steadily becoming the preferred encoding for e-mail, web pages, and other places where characters are stored or streamed.
Check this site for details.
http://unicode.org/.

Convert UTF-16 to UTF-8

Hi
My source file is UTF-16 and Target file is UTF-8. I am using XSLT mapping . If i m testing in Altova XML its working fine. But when i am testing the same thing using my scenario its not wroking.
I have tested this using Test option in ID. If i change the UTF-16 to UTF-8 while testing in ID but if i m trying to change it directly in XML file its not accepting.
How to change UTF-16 to UTF-8 while XSLT mapping. How to reslove this problem
Regards
Sowmya

Which Adapter you are using?
If you are using the file adapter then you can use the File adapter property as file.encoding=<codepage>
you can refer to below link
http://help.sap.com/saphelp_nw04/helpdata/en/0d/00453c91f37151e10000000a11402f/frameset.htm
Gaurav Jain

UTF-16 to UTF-8

Does anyone know of a easy way to go from UTF-16 National character set to UTF-8 ?? I don't have the time to rebuild RAC databases sitting on raw devices.
Any help would be appreciated.
Thanks

NO, there's no relationship between a RAC db and character sets, nor did I say there was. I stated that I don't have the time to recreate it because it is a RAC db on raw devices. It's Oracle 9i, the only version you would have a RAC database sitting on raw devices. I need to go from a national character set of UTF-16 to UTF-8 as I have already stated.

Convert UTF-8 to UTF-16

How do I convert UTF-8 to UTF-16. I want code to be like this..
public String convert (String string)
     //do something with the String
     return string;
}Thanks in advance..

From Unicode standard:
<quote>
Below code only supports three byte CJKs hex dumps
character string:
public class UTF8toUC16{
public static void main(String[] args){
String utf8 = "e799be";
String bin, binrep, uchex;
String[] bins, uc;
if (args.length > 0){
utf8 = args[0];
if (utf8.charAt(0) != 'e' || utf8.length() !=
6){
System.err.println("This program accepts utf8
hex-string for CJK");
System.exit(1);
bin =
Integer.toBinaryString(Integer.parseInt(utf8, 16));
binrep = "";
for (int i = 0; i < bin.length(); ++i){
binrep += (bin.charAt(i));
if ((i + 1) % 4 == 0 && (i != bin.length() - 1)){
binrep += ' ';
System.out.println(binrep);
bins = binrep.split("\\s");
uc = new String[4];
uc[0] = bins[1];
uc[1] = bins[2].substring(2) + bins[3].substring(0,
2);
uc[2] = bins[3].substring(2) +
bins[4].substring(2);
uc[3] = bins[5];
uchex = "";
for (int i = 0; i < 4; ++i){
System.out.print(uc[i] +" ");
uchex += Integer.toHexString(Integer.parseInt(uc,
2));
System.out.println();
System.out.println(uchex);
System.out.println((char)(Integer.parseInt(uchex,
16)));
thanks! its solved my problem.. thanks :-)

Media Encoder From AP-Pro not working

Hi,
I'm trying to export a premiere pro file, I go through the "export settings" like normal and then "ok", it goes to "exporting data" but then doesn't go anywhere.
Normally it should access the "Adobe Media Encoder" but it does nothing. When I try to access "Adobe media encoder from the programs it comes up with the error message:
"Adobe Media Encoder.exe - Entry Point Not Found" - "The procedure entry point ??0NewHandlerInitializer@ErrorManager@config@@QAE@XZ could not be located in the dynamic link library dvacore.dll".
So I'm guessing it needs reinstalling but where do I get it from? It is not in the Premiere setup files and it's not on the adobe website, only the updates are.
I'm using Windows XP and AP-Pro CS4
Regards,

Just had the same problem. I have had cs2 cs3 and cs4 on the same PC Windows XP Pro.
The exact error message as the o.p. was appearing during export from Premier and when opening Encore.
I was able to resolve this issue after weeks by simply ...
Close all Adobe programs and restart computer.
Go to Windows>Control Panel>Adobe Version Cue CS4
Adobe Version Cue Server window will appear.
Make sure your firewall is not blocking Version Cue or Adobe Updater.
Click Updates tab.
Click "Check for Updates" button.
Adobe Updater will connect with the update server. Click Details.
Checkmark Media Encoder and Photoshop Media Encoder.
Download the latest Adobe Media Encoder updates.
Im finally encoding right now so Im not stopping to look at the update version, lol. but my current version of Media Encoder is now 4.2.0.006
[email protected] wrote:
Hi,
I'm trying to export a premiere pro file, I go through the "export settings" like normal and then "ok", it goes to "exporting data" but then doesn't go anywhere.
Normally it should access the "Adobe Media Encoder" but it does nothing. When I try to access "Adobe media encoder from the programs it comes up with the error message:
"Adobe Media Encoder.exe - Entry Point Not Found" - "The procedure entry point ??0NewHandlerInitializer@ErrorManager@config@@QAE@XZ could not be located in the dynamic link library dvacore.dll".
So I'm guessing it needs reinstalling but where do I get it from? It is not in the Premiere setup files and it's not on the adobe website, only the updates are.
I'm using Windows XP and AP-Pro CS4
Regards,

Encoding from UTF-16 to UTF-8

Similar Messages

Maybe you are looking for