UTF-8 & UTF-16

My document was encoded through UTF-16. And whenever I tried to
upload that file into my database I am getting the error of
java.io.UTFDataFormatException: Invalid UTF8 encoding
at java.lang.Throwable.<init>(Compiled Code)
at java.lang.Exception.<init>(Compiled Code)
at java.io.IOException.<init>(Compiled Code)
at java.io.UTFDataFormatException.<init>(Compiled Code)
at
oracle.xml.parser.v2.XMLUTF8Reader.checkUTF8Byte(Compiled Code)
at
oracle.xml.parser.v2.XMLUTF8Reader.readUTF8Char(Compiled Code)
at
oracle.xml.parser.v2.XMLUTF8Reader.fillLastBuffer(Compiled Code)
at
oracle.xml.parser.v2.XMLByteReader.fillByteBuffer(Compiled Code)
at oracle.xml.parser.v2.XMLUTF8Reader.fillBuffer(Compiled
Code)
at oracle.xml.parser.v2.XMLReader.pushXMLReader(Compiled
Code)
at oracle.xml.parser.v2.XMLReader.pushXMLReader(Compiled
Code)
at oracle.xml.parser.v2.XMLParser.parse(Compiled Code)
at oracle.xml.sql.dml.OracleXMLSave.insertXML(Compiled
Code)
at OracleXML.Put_XML(Compiled Code)
at OracleXML.ExecutePutXML(Compiled Code)
at OracleXML.Execute(Compiled Code)
at OracleXML.main(Compiled Code)
oracle.xml.sql.OracleXMLSQLException: Invalid UTF8 encoding
at java.lang.Throwable.<init>(Compiled Code)
at java.lang.Exception.<init>(Compiled Code)
at java.lang.RuntimeException.<init>(Compiled Code)
at oracle.xml.sql.OracleXMLSQLException.<init>(Compiled
Code)
at oracle.xml.sql.dml.OracleXMLSave.insertXML(Compiled
Code)
at OracleXML.Put_XML(Compiled Code)
at OracleXML.ExecutePutXML(Compiled Code)
at OracleXML.Execute(Compiled Code)
at OracleXML.main(Compiled Code)
Invalid UTF8 encoding
===============
And other problem: Though I mentioned my column size as
varchar(4000), if the data contains more than 1000 char the
putXMl is saying:
oracle.xml.sql.OracleXMLSQLException: ORA-01401: inserted value
too large for column
at java.lang.Throwable.<init>(Compiled Code)
at java.lang.Exception.<init>(Compiled Code)
at java.lang.RuntimeException.<init>(Compiled Code)
at oracle.xml.sql.OracleXMLSQLException.<init>(Compiled
Code)
at oracle.xml.sql.dml.OracleXMLSave.insertXML(Compiled
Code)
at oracle.xml.sql.dml.OracleXMLSave.insertXML(Compiled
Code)
at OracleXML.Put_XML(Compiled Code)
at OracleXML.ExecutePutXML(Compiled Code)
at OracleXML.Execute(Compiled Code)
at OracleXML.main(Compiled Code)
ORA-01401: inserted value too large for column
Means I cannot upload data which is in UTF-16? And Can't I upload
columns which is more than 1000 chars?
V Prakash
null

Try to save the file you want to upload inthe format of Unicode.
You can choose to save your file in the format of Unicode in the
Save dialogue.

Similar Messages

  • WLS 8.1.2 : unsupported encoding: 'UTF-8, UTF-16'

    Hello,
    We are porting a web service from WLS 7.0.4 to WLS 8.1.2.0.
    It is a stateless session bean, we use "servicegen" to generate the WS deployment
    descriptor and the client is PocketSoap 1.5
    This web service worked fine with WLS 7.0.4, but with WLS 8.1.2.0, the server
    can't build the HTTP response. It seems it doesn't understand the "Accept-Charset:
    UTF-8, UTF-16" included in the HTTP request generated by PocketSOAP.
    Is there some extra configuration parameter on the server since v8 to handle this,
    or is it a regression ?
    Here is the HTTP request :
    ===================
    POST /ws-powertest/powertest HTTP/1.1
    Host: diffool:82
    Accept-Charset: UTF-8, UTF-16;q=0.8, iso-8859-1;q=0.8
    Accept-Encoding: deflate, gzip
    Content-Type: text/xml; charset=UTF-8
    SOAPAction: ""
    User-Agent: PocketSOAP/1.5.b1/PocketHTTP/1.1
    Content-Length: 412
    Authorization: Basic ZXJlbmF1ZDoxMjM0NTY3OA==
    <S:Envelope
         xmlns:S="http://schemas.xmlsoap.org/soap/envelope/"
         xmlns:a="http://www.bnpparibas.com/eqd/powertest"
         xmlns:XS="http://www.w3.org/2001/XMLSchema"
         xmlns:XI="http://www.w3.org/2001/XMLSchema-instance"><S:Body><a:getPositionAuthenticated
    S:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/"><string XI:type="XS:string">IXOEURWLD</string></a:getPositionAuthenticated></S:Body></S:Envelope>
    And the server-side stack trace :
    =======================
    java.lang.IllegalArgumentException: unsupported encoding: 'UTF-8, UTF-16': java.io.UnsupportedEncodingException:
    UTF-8, UTF-16
         at weblogic.servlet.internal.ServletResponseImpl.setEncoding(ServletResponseImpl.java:865)
         at weblogic.servlet.internal.ServletResponseImpl.setHeader(ServletResponseImpl.java:674)
         at weblogic.servlet.internal.ServletResponseImpl.setContentType(ServletResponseImpl.java:269)
         at weblogic.webservice.binding.soap.HttpServerBinding.send(HttpServerBinding.java:131)
         at weblogic.webservice.server.Dispatcher.dispatch(Dispatcher.java:105)
         at weblogic.webservice.server.WebServiceManager.dispatch(WebServiceManager.java:98)
         at weblogic.webservice.server.servlet.ServletSecurityHelper$3.run(ServletSecurityHelper.java:173)
         at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:353)
         at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:144)
         at weblogic.webservice.server.servlet.ServletSecurityHelper.authenticatedPortInvoke(ServletSecurityHelper.java:170)
         at weblogic.webservice.server.servlet.WebServiceServlet.serverSideInvoke(WebServiceServlet.java:302)
         at weblogic.webservice.server.servlet.ServletBase.doPost(ServletBase.java:485)
         at weblogic.webservice.server.servlet.WebServiceServlet.doPost(WebServiceServlet.java:268)
         at javax.servlet.http.HttpServlet.service(HttpServlet.java:760)
         at javax.servlet.http.HttpServlet.service(HttpServlet.java:853)
         at weblogic.servlet.internal.ServletStubImpl$ServletInvocationAction.run(ServletStubImpl.java:971)
         at weblogic.servlet.internal.ServletStubImpl.invokeServlet(ServletStubImpl.java:402)
         at weblogic.servlet.internal.ServletStubImpl.invokeServlet(ServletStubImpl.java:305)
         at weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.run(WebAppServletContext.java:6350)
         at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:317)
         at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:118)
         at weblogic.servlet.internal.WebAppServletContext.invokeServlet(WebAppServletContext.java:3635)
         at weblogic.servlet.internal.ServletRequestImpl.execute(ServletRequestImpl.java:2585)
         at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:197)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:170)
    thanx for your help,
    Emmanuel.

    Hi Emmanuel,
    Did you already try setting the -Dweblogic.webservice.i18n.charset Java system
    property, on the command-line that starts the WLS instance, hosting the WebLogic
    Web Service?
    -Dweblogic.webservice.i18n.charset=utf-16
    This sets the Accept-Charset for all WebLogic Web Services, so you probably want
    to just set it in the web-services.xml, for powertest. I don't know what this
    currently looks like, but you want to edit it (there is not servicegen option
    to set it) to look something like:
    <web-services>
    <web-service
    name="powertest"
    targetNamespace="http://www.bnpparibas.com/eqd/powertest"
    uri="/ws-powertest/powertest"
    charset="UTF-16"
    >
    </web-service>
    </web-services>
    Refer to http://e-docs.bea.com/wls/docs81/webserv/i18n.html
    HTH,
    Mike Wooten
    "Emmanuel Renaud" <[email protected]> wrote:
    >
    Hello,
    We are porting a web service from WLS 7.0.4 to WLS 8.1.2.0.
    It is a stateless session bean, we use "servicegen" to generate the WS
    deployment
    descriptor and the client is PocketSoap 1.5
    This web service worked fine with WLS 7.0.4, but with WLS 8.1.2.0, the
    server
    can't build the HTTP response. It seems it doesn't understand the "Accept-Charset:
    UTF-8, UTF-16" included in the HTTP request generated by PocketSOAP.
    Is there some extra configuration parameter on the server since v8 to
    handle this,
    or is it a regression ?
    Here is the HTTP request :
    ===================
    POST /ws-powertest/powertest HTTP/1.1
    Host: diffool:82
    Accept-Charset: UTF-8, UTF-16;q=0.8, iso-8859-1;q=0.8
    Accept-Encoding: deflate, gzip
    SOAPAction: ""
    User-Agent: PocketSOAP/1.5.b1/PocketHTTP/1.1
    Content-Length: 412
    Authorization: Basic ZXJlbmF1ZDoxMjM0NTY3OA==
    <S:Envelope
         xmlns:S="http://schemas.xmlsoap.org/soap/envelope/"
         xmlns:a="http://www.bnpparibas.com/eqd/powertest"
         xmlns:XS="http://www.w3.org/2001/XMLSchema"
         xmlns:XI="http://www.w3.org/2001/XMLSchema-instance"><S:Body><a:getPositionAuthenticated
    S:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/"><string XI:type="XS:string">IXOEURWLD</string></a:getPositionAuthenticated></S:Body></S:Envelope>
    And the server-side stack trace :
    =======================
    java.lang.IllegalArgumentException: unsupported encoding: 'UTF-8, UTF-16':
    java.io.UnsupportedEncodingException:
    UTF-8, UTF-16
         at weblogic.servlet.internal.ServletResponseImpl.setEncoding(ServletResponseImpl.java:865)
         at weblogic.servlet.internal.ServletResponseImpl.setHeader(ServletResponseImpl.java:674)
         at weblogic.servlet.internal.ServletResponseImpl.setContentType(ServletResponseImpl.java:269)
         at weblogic.webservice.binding.soap.HttpServerBinding.send(HttpServerBinding.java:131)
         at weblogic.webservice.server.Dispatcher.dispatch(Dispatcher.java:105)
         at weblogic.webservice.server.WebServiceManager.dispatch(WebServiceManager.java:98)
         at weblogic.webservice.server.servlet.ServletSecurityHelper$3.run(ServletSecurityHelper.java:173)
         at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:353)
         at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:144)
         at weblogic.webservice.server.servlet.ServletSecurityHelper.authenticatedPortInvoke(ServletSecurityHelper.java:170)
         at weblogic.webservice.server.servlet.WebServiceServlet.serverSideInvoke(WebServiceServlet.java:302)
         at weblogic.webservice.server.servlet.ServletBase.doPost(ServletBase.java:485)
         at weblogic.webservice.server.servlet.WebServiceServlet.doPost(WebServiceServlet.java:268)
         at javax.servlet.http.HttpServlet.service(HttpServlet.java:760)
         at javax.servlet.http.HttpServlet.service(HttpServlet.java:853)
         at weblogic.servlet.internal.ServletStubImpl$ServletInvocationAction.run(ServletStubImpl.java:971)
         at weblogic.servlet.internal.ServletStubImpl.invokeServlet(ServletStubImpl.java:402)
         at weblogic.servlet.internal.ServletStubImpl.invokeServlet(ServletStubImpl.java:305)
         at weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.run(WebAppServletContext.java:6350)
         at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:317)
         at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:118)
         at weblogic.servlet.internal.WebAppServletContext.invokeServlet(WebAppServletContext.java:3635)
         at weblogic.servlet.internal.ServletRequestImpl.execute(ServletRequestImpl.java:2585)
         at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:197)
         at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:170)
    thanx for your help,
    Emmanuel.

  • Utf-7/utf-8 encryption

    Hi,
    We have a requirement in which the payment list file generated after payment run(f110) is to be encrypted using utf-7 /  utf-8  encoding.We are now using sap 4.6c version.
    i tried using MD5 generator and client does't want that.
    Are there any Function modules to encrypt the file into utf-7/utf-8 ?
    Can you please share your views and help me to proceed further in the development.
    Thankyou
    Gowri

    If you want to upload it to Application Server use
    OPEN DATASET dset FOR OUTPUT IN TEXT MODE ENCODING UTF-8.
    "then transfer
    If for download on desktop use
    DATA itab_str TYPE TABLE OF string.
    APPEND 'Some text in UTf-8' TO itab_str.
    CALL FUNCTION 'GUI_DOWNLOAD'
      EXPORTING
        filename                        = 'C:\utf.txt'
        FILETYPE                        = 'ASC'
        CODEPAGE                        = '4110' "UTF-8 codepage
      tables
        data_tab                        = itab_str.
    Regards
    Marcin

  • DMEE: XML Encoding UTF-8/UTF-16

    Hi,
    I'm generating automatic payment files via program run SAPFPAYM_SCHEDULE.
    The file is generated under UTF-8 but when generated on SAP server become UTF-16.
    => When double clicking on the Z_XML20022 and clicking on glasses you can see "<?xml version="1.0"
    encoding="utf-8"?>"
    => When you have a look under AL11
    /interfaces_sec/DECCLNT300/FI_payment_SAP032/out/PSAPCIT.PAYMENT_CN10_1. it's <?xml version="1.0"
    encoding="utf-16"?>
    I need to have utf-8.
    Thanks in advance for your help.

    Hi,
    If you using XSLT, you may use this transformation (based on your XML example, but simplified):
    <xsl:transform xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
          xmlns:sap="http://www.sap.com/sapxsl"
          xmlns:am="http://www.xxx.com/ActionMessage"
          version="1.0">
      <xsl:strip-space elements="*"/>
      <xsl:output encoding="utf-8"/>
         "your rest XML data"
    </xsl:transform>
    PS: I advise you to read the Note 1017101 - Encoding Problems with ABAP XML Processing, which explains better what I said above.

  • Message Mapping Problem with UTF-16LE Encoded XML

    Hello,
    we have the following scenario:
    IDoc > BPM > HTTP Sync Call > BPM > IDoc
    Resonse message of the HTTP call is a XML file with UTF-16LE processing instruction. This response should then be mapped to a SYSTAT IDoc. However the message mapping fails "...XML Parser: No data allowed here ...".
    So obviously the XML is not considered as well-formed.
    When taking a look at SXMB_MONI the following message appears: "Switch from current encoding to specific encoding not supported.....".
    Strange thing however is if I save the response file as XML and use the same XML file in the test tab message mapping is executed successfully.
    I also tried to use a Java Mapping to switch encodings before executing message mapping, but the error remains.
    Could the problem be, that the codepage UTF-16LE is not installed on the PI system ? Any idea on that ?
    Thank you!
    Edited by: Florian Guppenberger on Feb 2, 2010 2:29 PM
    Edited by: Florian Guppenberger on Feb 2, 2010 2:29 PM

    Hi,
    thank your for your answer.
    This is what I have tried to achieve. I apply the java conversion mapping when receiving the response message - i tried to convert the response to UTF-16, UTF-8 but none of them has helped to solve the problem.
    I guess that using adapter modules is not an option either as it would modify the request message, but not the response, right?

  • UTF-16: better than UTF-8? How to use?

    I read about UTF-8/UTF-16 and found out, that UTF-16 characters all have the same encopded length while UTF-8 uses a more compact representation. But why does Eclipse just shows one line of strange characters (most times little squares) in my .java file when i switch Eclispe's encoding from UTF-8 to UTF-16? I thought Java is using UTF-16 internally all the time?

    String aString = "jschell";
    byte[] whateverEncodingBytes = myString.getBytes("whateverEncoding");aString is in UTF-16, no matter what. The encoding specified in the getBytes() method is the target encoding. So whateverEncodingBytes is an array of bytes stored in a whatever encoding, rather than UTF-16. If you just use the method getBytes() the array of bytes would be stored in the default encoding which is iso-8859-1 unless otherwise specified.
    String aNewString = new String(whateverEncodingBytes, "whateverEncoding");
    aNewString is UTF-16 again, no matter what. The encoding specified in the constructor is the source encoding. So whateverEncodingBytes are still in an encoding. But building a new String out of them requires an encoding conversion to occur, as the String must be in UTF-16. So we need to let the constructor know which encoding to convert from. It would be like an interpretter to know which language to interpret from.
    A Java byte can be in any encoding it likes - it's simply eight raw bits of data stored in an arbitrary order. Anything extending a Java object is stored in UTF-8, except for String, which keeps its internal characters in UTF-16 encoding.
    In short, if you pull data into Java from an external source, the encoding it sits in within Java sticks to the rules above. If it first hits Java as a byte stream (say from an InputStream), it's going to be in whatever encoding it was already in. If it first hits Java as a String (say from request.getParameter() in a servlet or JSP), it will be in UTF-16, as Java will convert its encoding immediately.
    The same holds when writing back out. If you write out a series of bytes (such as with OutputStream), the resulting output is in the encoding the bytes were in to begin with. If you're outputting a String (although this is a lot rarer), it will be in UTF-16. Or to be more specific as somebody pointed out Java uses Unicode characters (16-bits, unsigned) in memory to hold characters.

  • Reading text file in ASCII or UTF-8 or UTF-16 or UTF-32?

    The following code will include the UTF-8 byte-order-mark (EF BB BF) in the first line from the source file:
    BufferedReader reader = new BufferedReader(new FileReader(sourceFile));
    String firstLine = reader.readLine();
    This isn't desirable. I don't want to get the UTF-8 BOM in the text contents that I get from the IO API.
    Now, I can do something like this:
    InputStreamReader reader = new InputStreamReader(new FileInputStream(sourceFile), "UTF8");
    However, that assumes that the program knows the encoding of the input file at design time. Unfortunately, my app takes files from the user who may supply files in UTF-8, UTF-16, ASCII, or some other text encoding. Doesn't Java have some sort of simple file reading API to auto-detect the specific text encoding, strip out any internal BOM type markings and return my a simple Java string of just the actual file contents?

    MassimoH wrote:
    However, that assumes that the program knows the encoding of the input file at design time.Well, the bottom line is that the program DOES have to know the encoding of the input file. Not necessarily at design time, but it does have to know it at run time. The BOMs are only the first problem you have encountered.
    I believe there are algorithms used in browsers to guess at the encoding of HTML documents provided by bozos who don't specify an encoding, but I don't have any good information about them.

  • Write file in UTF-16BE Format

    I am programming web page that runs SWF file (Flash movie) and servlet.
    The SWF file sends with URL command data to the server.
         send_lv.sendAndLoad("http://�", result_lv, "POST");The data at � UTF-8 (UTF-8 is the standard encoding for exchanging text, such as online mail systems. UTF is an 8-bit system.).
    At the servlet in doPost command I get the data with
    String temp = request.getParameter("smstext");I need to write the parameter �temp� into file in UTF-16BE Format, how I do it?
    I try almost everything

    Once you've got a String, you can get bytes corresponding to a specific encoding with the getBytes(String charset) method.
    In your case:byte[] bytes = temp.getBytes("UTF-16BE");Then you may write those bytes to a file using a FileOutputStream.
    http://java.sun.com/j2se/1.4.2/docs/api/java/lang/String.html#getBytes(java.lang.String)
    http://java.sun.com/j2se/1.4.2/docs/api/java/io/FileOutputStream.html
    You should obtain the same result with an OutputStreamWriter, for which you can specify the encoding in the contructor, and then write the String directly:OutputStreamWriter writer = new OutputStreamWriter(new FileOutputStream(fileName), "UTF-16BE");http://java.sun.com/j2se/1.4.2/docs/api/java/io/OutputStreamWriter.html

  • Howto set up proper utf-8 locales and german umlaute?

    Hi there,
    have some issues here and i think it has something to do with my locales... I have them since one of the last updates i think...
    First problem:
    In Kopete, i can send the german "umlaute" (ä, ö, ü) to others and they are sent and displayed correctly at their side... But when they send me these characters, i get only a square and a deleted next-letter-in-the-word by them. This happens with every theme and font combination in kopete. Here is an example:
    Second problem:
    In Konsole, the "umlaute" are correctly displayed on my folders, but in the messages i get the chars are simply deleted and not visible. This happens in the VC and in Xorg too. Here is another example:
    (It should mean "Keine Handbuch Seite für pacman.conf")
    My config:
    I think i have configured my System properly, well at least i hope that Here are the relevant parts of my config files:
    /etc/rc.conf:
    LOCALE="de_DE.utf8"
    KEYMAP="de"
    CONSOLEFONT=
    CONSOLEMAP=
    /etc/profile:
    export LANG="de_DE.utf8"
    export LANGUAGE="de_DE.utf8"
    /etc/locale.gen:
    de_DE.UTF-8     UTF-8
    en_US.UTF-8     UTF-8
    And "locale -a" gives me:
    C
    POSIX
    de_DE.utf8
    en_US.utf8
    The fonts I use:
    KDE GUI: Bitstream Vera Sans (everywhere)
    Y-Terminal (Konsole) Font: Bitstream Vera Sans mono
    So, how can i configure a proper german utf-8? I have already searched the forums (both here and the DE one) and the wiki, but found no solution to this....
    THX
    Funkyou

    Thx for the suggestions, but none of them seems to solve it...
    baze:
    This line was already uncommented and i have generated my locales for several times now... I have updated my post with the uncommented lines in /etc/locale.gen, just to collect all information...
    Romashka:
    Ok, but what if the two variables contain the same content? I mean, when i define LANG="de_DE.utf8" in /etc/profile and it is then overwritten by /etc/profile.d/lang.sh with LANG="de_DE.utf8", then this is simply the same variable defined twice with the same content... I have tried to unset the one in /etc/profile, but it did not solve it...
    As for CONSOLEFONT, i had never one defined, can anyone suggest me a proper one? And the other question is: Its not working on the terminals, but it is also not working in Xorg, so is this really a CONSOLEFONT issue?
    I have tried another thing and switched my locales to de_DE@euro     ISO-8859-15, and with this setting the chars appear correctly... But i cannot be the only one where it does not work, i feel so excluded without UTF-8
    Have also tried another terminal in Xorg... In XTerm the chars are not deleted like in Kopete or on the VC, but i see an inverted question mark instead of them...

  • Codepage Conversionerror UTF-8 from System-Codepage to Codepage iso-8859-1

    Hello,
    we have on SAP PI 7.1 the problem that we can't process a IDOC to Plain HTTP.
    The channel throws "Codepage Conversionerror UTF-8 from System-Codepage to Codepage iso-8859-1".
    The IDOC is 25 MB. Does anybody have a idea how we can find out what is wrong with the IDOC?
    Thanks in advance.

    In java strings are always unicode i.e. utf16. Its the byte arrays that are encoded. So use the following codeString iso,utf,temp = "����� � �����";
    byte b8859[] = temp.getBytes("ISO-8859-1");
    byte butf8= temp.getBytes("utf8");
    try{
      iso = new String(b8859,"ISO-8859-1");
      utf = new String(butf8,"UTF-8");
      System.out.println("ISO-8859-1:"+iso);
      System.out.println("UTF-8:"+utf);
      System.out.println("UTF to ISO-8859-1:"+new String(utf.getBytes("iso8859_1"),"ISO-8859-1"));
    System.out.println(utf);
    System.out.println(iso);
    }catch(Exception e){ }Also keep in mind that DOS window doesnot support international characters so write it to a file

  • Unicode or UTF-8?

    Hi, all,
    I'm developing a JSP application that will work with international characters, both displaying them on webpages and storing them in a MySQL database. I'm a bit confused about whether I should use Unicode or UTF-8 for those character strings. (I've read up on both of these encodings and they appear to be very similar in many respects.)
    Can anyone give me any suggestions as to which I should use and why?
    Thanks,
    Dmitri.

    UTF-16 uses 2 bytes for all characters.
    UTF-8 generally uses anywhere from 1 to 6 bytes toActually, UTF-16 uses 16-bit tokens, and represents characters with one or more tokens, like all UTF encodings.
    Generally, the encodings 'UTF-N' use N-bit tokens, and encode the 32-bit UNICODE scalar values (character set) with one or more tokens. Typically, the lower values in the encoding represent the UNICODE scalar values directly.
    UNICODE defines 'UTF-8', 'UTF-16', and 'UTF-32', the latter two in big- and little-endian forms as well as self-specifying forms (using initial bytes of a file or stream). 'UTF-32' encoding just uses the UNICODE scale values directly.
    There is also 'UTF-7', which is used in mime encoding to get through 7bit character sets. There are also unofficial 'UTF-6' etc for specialist use.
    UTF-8 has the advantage that it does not contain null (zero valued) bytes, which means that it works transparently with code expecting to see one byte characters (assuming the legacy code doesn't try to manipulate multi-byte characters!).

  • Converting uft-16 to utf-8

    Hi All,
    I have file to IDoc Scenario.
    Message Mapping is working fine for xml encoding UFT-8.
    However Source file I am getting from client has xml encoding UTF-16.
    because which my end-to-end mapping is failing.
    Can you please suggest me something by which i can change UFT-16 to UFT-8 before i execute my mapping.
    Regards,
    Manisha

    Hi Manisha,
    In your sender file channel Specify the File Type Text.
    Once  you have selected Text, specify a code page under File Encoding. The default setting is to use the system code page that is specific to the configuration of the installed operating system. The file content is converted to the UTF-8 code page before it is sent.
    Following are the values you can use. I think in ur case use UTF-16.
    ○       US-ASCII
    Seven-bit ASCII, also known as ISO646-US, or Basic Latin block of the Unicode character set
    ○       ISO-8859-1
    ISO character set for Western European languages (Latin Alphabet No. 1), also known as ISO-LATIN-1
    ○       UTF-8
    UTF-8 (BC-ABA)
    ○       UTF-16BE
    16-bit Unicode character format, big-endian byte order
    ○       UTF-16LE
    16-bit Unicode character format, little-endian byte order
    ○       UTF-16
    UTF-16 (BC-ABA), byte order
    Regards,
    Deepak.

  • UTF???

    I read a many articles from web related to the UTF-8, UTF-16 encoding.They used very difficult language... :(
    But, I did not get the UTF encoding in terms of why we it need? Advantages and disadvanates of its.
    Please let me know same.
    Thanks,
    Rahul

    EDanaII wrote:
    You need UTF if you want to use an alphabet other than Roman. UTF goes beyond ASCII by providing 65k symbols to choose from instead of 256. This is useful if you want to display Chinese, Arabic or some other non English alphabet.
    Ideograph languages were in use on computers before Unicode existed.
    Excluding those most other languages, including arabic, are fairly easy to use if you work in that language.
    Excluding explicit exclusive native support (such as java strings) with an exclusive language it wouldn't necessarily be the best code set choice either. If for no other reason than, except for english, it can take more storage space than using a different code set. And even for english UTF8 is the only one that takes the same space.
    Unicode's primary strength lies in creating applications that must support multiple languages at the same time.
    Keep in mind that selecting a code set for an application is probably the easiest part of creating an application that targets multiple cultures.
    Currently there is also 32 bit and perhaps even a 64 bit code set.
    Java actually uses a multibyte format of UTF16 where a sequence of 16 bit values can represent a single character.
    [http://java.sun.com/docs/books/jls/third_edition/html/lexical.html#3.1]
    The unicode site
    [http://www.unicode.org/]

  • [SOLVED] Anki won't start. "Exception: Anki requires a UTF-8 locale."

    When starting Anki, I get this output:
    Traceback (most recent call last):
    File "/usr/bin/anki", line 5, in <module>
    import aqt
    File "/usr/share/anki/aqt/__init__.py", line 7, in <module>
    import anki.lang
    File "/usr/share/anki/anki/__init__.py", line 12, in <module>
    raise Exception("Anki requires a UTF-8 locale.")
    Exception: Anki requires a UTF-8 locale.
    I have en_US.UTF-8 UTF-8 and en_US ISO-8859-1 uncommented in my locale.gen file, and have run "#locale-gen", but the out put of "$ locale" is still
    LANG=C
    LC_CTYPE="C"
    LC_NUMERIC="C"
    LC_TIME="C"
    LC_COLLATE="C"
    LC_MONETARY="C"
    LC_MESSAGES="C"
    LC_PAPER="C"
    LC_NAME="C"
    LC_ADDRESS="C"
    LC_TELEPHONE="C"
    LC_MEASUREMENT="C"
    LC_IDENTIFICATION="C"
    LC_ALL=
    Last edited by adeligen (2013-01-09 15:53:24)

    Found the answer on the systemd wiki page. I guess I need to run "# localectl set-locale LANG="en_US.utf8"" to get it to configure /etc/locale.conf.
    All is working now.

  • Zsh will not use UTF-8 but uses C [solved]

    I have a new install and am using zsh-shell-config and zsh from the official repos. I have my locale set and it has been generated, but I see strange characters and cannot figure out why. en_US.UTF-8 UTF-8 is the only thing uncommented in /etc/locale.gen.
    If I type
    % echo $LANG
    C
    the shell says C and not UTF-8 which causes the funny characters to be displayed.
    % prompt -p
    How can I fix this?
    Last edited by maggie (2014-01-04 17:18:08)

    maggie wrote:
    % echo $LANG
    C
    Why is it not UTF8 like I set it to be?
    That's your problem.  Are you use that you defined the locale and generated the locales?  Post the output of: `sed -e 's/#.*$//' -e '/^$/d' /etc/locale.gen` which will show all non-commented.
    Last edited by graysky (2014-01-04 13:21:40)

Maybe you are looking for

  • SharePoint Designer Workflow 2013 Issue

    Hi All, We have an issue with the SharePoint designer workflow with the below error message. "Something went wrong. To Try again, reload the page and then start the workflow" suggest... Arun Prakash

  • Thread Pool executor

    I run a set of java programs as a single process in solaris os. Those java programs make use of thread pool executor to process threads and it runs in an infinite loop for ever. Thread processing is done whenever data is populated in database from an

  • Setting up .mac addresses in iWeb sites

    My .mac site (www.chaibabyusa.com) has been setup and published on iWeb '08 in .Mac. I would like to have email addresses setup ([email protected]) so that an email could be forwarded to an external site (@gail.com, for example). Thank you in advance

  • How do I open the Database Tables in the Object Explorer window.

    I tried clicking on them and right clicking but they dont open.

  • Uploading multiple files in Contribute CS3

    I have a website where I am constantly creating new pages almost everyday. I use Adobe Contribute CS3. I began creating drafts (all from one template) and saving them for later. Now, I'm have almost 200 drafts to publish to the server and the process