Diacritic: unicode to ascii

I'm looking for an algorithm to convert Unicode ro ASCII characters. The algorithm does not have to cover all unicode characters, but Latin1 and Latin 2 should be covered. If the character cannot be converted (no representation in ascii), then i can do it manually (or use a "?")
thanx.
trev

From your title I assume you want to just drop any diacritics attached to the characters?
Some time ago I downloaded a file from the Unicode site which contains lines like010C;LATIN CAPITAL LETTER C WITH CARON;Lu;0;L;0043 030C;;;;N;LATIN CAPITAL LETTER C HACEK;;;010D;This contains the Unicode description of the character Č in particular that it can be decomposed to 0043 (C) and 030C (combining hook above). I don't see that property exposed in the java.lang.Character class, but you could use it to create a hard-coded mapping table.

Similar Messages

Convert unicode to ascii

I have a unicode like "100", which represents ASCII code: "d"
I can simply convert unicode to ascii code if using Visual Basic function: chr(); but how can I do the same job under cocoa ?
thank you.
G5 Mac OS X (10.4)

Well, if you wanted to create a single-character NSString from an integer value, you could do something like:
unichar oneChar = 100;
NSString *oneCharStr = [NSString stringWithCharacters:&oneChar length:1];
What are you trying to accomplish?

Unicode to ascii

hello,everyone,
i have a problem in java programming.
how can i convert the unicode to ascii?

If you just need to convert a file then java provide a tool: native2ascii.exe
if you need to do it in java read the file in to a string then save it using
OutputStreamWriter out = new OutputStreamWriter(fout, "US-ASCII");

Unicode and ascii conversion help needed

I am trying to read passwords from a foxpro .dbf. The encrpytion of the password is crude, it takes the ascii value of each char entered and adds an integer value to it, then stores the complete password to the table. So to decode, just subtract same integer value from each chars retieved from .dbf. pretty simple.
The problem is that java chars and strings are unicode, so when my java applet retrieves these ascii values from the .dbf they are treated as unicode chars, if the ascii value is over 127 I have problems.
The question. how can i retrieve these ascii values as ascii values in java?
Should I use an InputStream like:
InputStream is=rs.getAsciiStream("password");
Is there a way to convert from unicode to extended ascii?
Some examples would be helpful, Thanks in advance.

version 1
import java.nio.charset.Charset;
import java.nio.ByteBuffer;
import java.nio.CharBuffer;
class Test {
    static char[] asciiToChar(byte[] b) {
        Charset cs = Charset.forName("ASCII");
        ByteBuffer bbuf = ByteBuffer.wrap(b);
        CharBuffer cbuf = cs.decode(bbuf);
        return cbuf.array();
    static byte[] charToAscii(char[] c) {
        Charset cs = Charset.forName("ASCII");
        CharBuffer cbuf = CharBuffer.wrap(c);
        ByteBuffer bbuf = cs.encode(cbuf);
        return bbuf.array();
}version 2
import java.io.*;
import java.nio.charset.Charset;
class Test {
    static char[] asciiToChar(byte[] b) throws IOException {
        Charset cs = Charset.forName("ASCII");
        ByteArrayInputStream bis = new ByteArrayInputStream(b);
        InputStreamReader isr = new InputStreamReader(bis, cs);
        char[] c = new char[b.length];
        isr.read(c, 0, c.length);
        return c;
    static byte[] charToAscii(char[] c) throws IOException {
        Charset cs = Charset.forName("ASCII");
        ByteArrayOutputStream bos = new ByteArrayOutputStream();
        OutputStreamWriter osw = new OutputStreamWriter(bos, cs);
        osw.write(c, 0, c.length);
        osw.flush();
        byte[] b = bos.toByteArray();
        return b;
}

SetString() and Unicode to ASCII/ISO conversion

Hi,
I encountered next problem. My database works in ISO-Latin-1 character set. Still, if I call PreparedStatement.setString(String arg) I would expect this call would convert arg parameter to ISO-Latin-1 using standard conversion mechanism. But after some testing I found out that instead of converting Java characters Oracle's driver just skips hi-byte. So if I have character with hex code 0x2019, which is absolutely normal unicode character. Oracle's driver converts it into 0x19 code, which is not even printable character and not valid ASCII code. Java's standard CharToByteConverter gives 0x63 ('?') code, which, at least, is valid ASCII/ISO-1 character.
And some soft reports "Invalid Character" errors encountering 0x19 and other such codes.
Here's a fragement from Oracle's JDBC log:
DBCV DBG2 UCS-2 bytes (10 bytes):
20 19 20 1c
DBCV FUNC DBConversion.stringToAsciiBytes(str)
DBCV DBG2 DBAccess bytes (5 bytes):
19 1c
Has anyone encountered this problem? Has anyone found work-around?
Checking and converting all strings before calls to setString() would be too overwhelming.
Thanks,
Dmitry

You may not store a string as ASCII values.
Any string you create is stored in Unicode.
The first 128 elements of Unicode are the same as ASCII.
If you want the ASCII representation of the letter "a", then:
System.out.println( java.lang.Character().getNumericValue( 'a' ) );
As DrClap said, there is no reason to save a string in ASCII format since it will be saved in unicode format and thus Java already knows what the ASCII values will be.
If you want to output a string in ASCII format, do it like this:
public Vector convert( String input ) {
StringBuffer tmpIn = new StringBuffer( input );
Vector tmpOut = new Vector
for ( int index = 0; index < tmpIn.length; index++ ) {
char tmpCharacter = tmpIn[ index ];
int tmpValue = java.lang.Character().getNumbericValue( tmpCharacter ) );
tmpOut.addElement( tmpValue );
return tmpOut;
And then display each element as you see fit.
Fixing the bugs in this code is an exercise for the reader. If you want to do something like this, you should be thinking about why you want to do this as well as how to do this.

How to translate native Unicode to ascii Unicode

I want to read in the user input and then store this input as ascii unicode into the database. Can anyone tell me how?!

Wow that looks like one big hack after another. :)
well.. you see..... I designed my database system with
the user interface being the web browser. Because the
Chinese words are 2-byte fonts, hence, there will be
characters like ' and \ which are actually the
keywards of SQL! so I solve that problem by encode the
user input string (via POST) to big5 and then pass to
my database. However!!!! There's a magic word of
unicode \u92b4 not been displayed properly!! It just
basically showed up as question mark.... Hence I
thought about the native2ascii utility which convert
native Chinese words into \u notation. Through this
forum, I actually managed to find the answer, but yet
another question arise again -- I discovered that it
does not even show up properly after the \u notation
process!! Within Java, all characters are 2 bytes (UTF-16), so there's nothing special about Chinese characters. The characters ' and \ are not subcomponents of Chinese characters or any other character. You just have to be careful when you load them into Java, that you specify the correct encoding.
I think the problem is due to the inproper big5
encoding... this is usually how it's supposed to be
done:
String big5Str = new
String(myInputString.getBytes("8859_1"),"Big5");No this is not how it is supposed to be done. I see this idiom very frequently in this forum, and it is wrong. If you don't believe me, read the javadoc for the String constructor, and the javadoc for String.getBytes(). They say: "The behavior of this (constructor | method) when the given bytes are not valid in the given charset is unspecified."
You seem to think that myInputString contains data in a specific encoding. It does not, it just contains a string of characters. If it was constructed with mismatched data/encoding, then that's where your problem lies.

Unicode to ASCII transform...

I write a little class the transforma Unicode string into an ASCII one.
it works good if i don't use char as ?, ?, ?, ?, ? and i dont'know how i can solve the problem. the code i write is this:
[tt]public class Unicode2ASCII{
public static void main(String argv[]){
    String s = argv[0];
    byte b[] = null;
    b = s.getBytes();
    for(int i=0; i<b.length; i++){
      System.out.print(b[i] + " ");
    System.out.println();
    System.out.println("====");
    try{
      s = new String(b, "ASCII");
      System.out.println(s);
      System.out.println("====");
      for(int h=0; h<s.length(); h++){
        char c = s.charAt(h);
        System.out.print(c + " ");
        System.out.print((int)c % 127);
      System.out.println();
      System.out.println("====");
    }catch (Exception e){
      System.out.println(e);
}[tt]

You may not store a string as ASCII values.
Any string you create is stored in Unicode.
The first 128 elements of Unicode are the same as ASCII.
If you want the ASCII representation of the letter "a", then:
System.out.println( java.lang.Character().getNumericValue( 'a' ) );
As DrClap said, there is no reason to save a string in ASCII format since it will be saved in unicode format and thus Java already knows what the ASCII values will be.
If you want to output a string in ASCII format, do it like this:
public Vector convert( String input ) {
StringBuffer tmpIn = new StringBuffer( input );
Vector tmpOut = new Vector
for ( int index = 0; index < tmpIn.length; index++ ) {
char tmpCharacter = tmpIn[ index ];
int tmpValue = java.lang.Character().getNumbericValue( tmpCharacter ) );
tmpOut.addElement( tmpValue );
return tmpOut;
And then display each element as you see fit.
Fixing the bugs in this code is an exercise for the reader. If you want to do something like this, you should be thinking about why you want to do this as well as how to do this.

Why Java is following Unicode & not ASCII standard?

Hi all,
I am new to Java as well as to this forum.
Could you please tell me why Java is not following ASCII format?
And why is it following Unicode?
Thanks,
Reni

Unicode is chosen because it's an abbreviation for Unique Ode, which means that it provides a special sort of poetry to give Java an aura of mystique, intelligence and worldliness that sets it apart from other languages.
To support languages and characters other than English.Who doesn't read/speak English, really?! And why isn't there multiple language keywords for Java. Why can't I write code en Espa?ol?
p?blico vac?o hagaAlgo(Secuencia s) {
para(nent i = 0; i < s.longitud(); i++) {
Sistema.fuera.impresi?nL?nea(i + " = " + s.car?cterEn(i));
}(int abreviated for "n?mero entero")
Message was edited by:
bsampieri

Java Unicodes and ASCII conversion

Hi,
Does anyone know where I could find the unicodes for the ascii codes from ascii 0 to ascii 32 as when I try printing characters using S.o.p("\n"); it does not seem to work for some reason (to give a new line. So I want to use the unicode values but I cant find them anywhere.
Can anyone help? Thanks in advance.
John Loughran

The Unicode values should be 0-32 also.
If you are having problems writing line breaks, try writing a whole carriage return + line feed instead of just the line feed, like:
write("\r\n");
Some text editors and viewers don't like just plain line feeds. I believe that using a PrintWriter to write the data should translate CRLF's for you.

NSString with Unicode + ASCII characters

Hi All,
I am developing an mac application in 10.5.
I need to deal with normal ASCII characters and Unicode characters.
I want to calculate number of bytes taken to store the string.
I am using [[unicodeStr dataUsingEncoding:NSUTF8StringEncoding]bytes].
This is working fine if "unicodeStr" string has only unicode characters.
If "unicodeStr" it is combination of (unicode characters + ASCII Characters) "NSUTF8StringEncoding" is not working.
Can anybody tell me how to proceed??
Thanks in advance.

There is no such thing as a combination of Unicode and ASCII. ASCII is part of Unicode. Any ASCII string is a valid UTF8 string.

'Open Dataset' command - add ASCII code in a unicode system

Hi,
I use following comand in order to save a file on an application server.
open dataset <path> for output in text mode encoding default.
Is it possible to convert file to ASCII code instead of the system setting unicode and to ignore conversion errors? If yes, please tell me how.
Thanks!

problem exporting text in hebrew
Similar kind of problem has been solved here.. check that link first
Use the class CL_ABAP_CONV_OUT_CE and use it to convert file from unicode to ascii.

Sqlldr does not understand unicode characters in file names

Hello,
I am trying to call sqlldr from a .net application on Windows to bulk load some data. The parameter, control, data, log files used by sqlldr, are all located in the C:\Configuración directory (note the unicode character in the directory name).
Here is my parfile:
control='C:\Configuración\SystemResource.ctl'
direct=true
errors=0
log='C:\Configuración\SystemResource.log'
userid=scott/tiger@orasrv
When I make a call as
sqlldr -parfile='C:\Configuración\SystemResource.par'I am getting
SQL*Loader-100: Syntax error on command-line
If I run it as
sqlldr -parfile='C:\Config~1\SystemResource.par'I am getting
SQL*Loader-522: lfiopn failed for file (C:\Configuraci├│n\SystemResource.log)
If I remove the log= parameter from the parameter file, I am getting
SQL*Loader-500: Unable to open file (C:\Configuraci├│n\SystemResource.ctl)
SQL*Loader-553: file not found
SQL*Loader-509: System error: The system cannot find the file specified.
Can anyone suggest a way to handle unicode/extended ASCII characters in file names?
Thanks,
Alex.

Werner, thank you for replying to my post.
In my real application, I actually store the files in %TEMP%, which on Spanish and Portuguese Windows has "special" characters (e.g. '...\Administrador\Configuración local\Temp\'). In addition, you can have a user with the "special" characters in the name which will become part of %TEMP%.
Another problem is that 8.3 name creation may be disabled on NTFS partitions.
Problem #3 is that the short file names that have "special" characters are not converted correctly by GetShortPathName windows API, e.g. "Configuración" will be converted to "Config~1", but for "C:\ración.txt" the api will return the same "C:\ración.txt", even though dir /x displays "RACIN~1.TXT". Since I am creating the parameter and control files programmatically from a .net application, I have to PInvoke GetShortPathName.
Any other ideas?
Thanks,
Alex.

Unicode in UNICHAR and UNICODE Excel Functions Is Decimal

CASE # 12 64 08 79 83
Dear Microsoft Engineers,
There is a lack of information in Microsoft Support pages*, where UNICODE and UNICHAR in Excel 2013 are out of real context, given that unicode is always hexadecimal.
Please add a notice on these pages about the fact that these Excel functions process decimal unicodes. You might also add a hint to use HEXDEC and DECHEX conversion when working with UNICHAR and UNICODE.
Please note that low decimal unicodes equal ASCII. To highlight the new performance, you'd better take some other examples.
I need UNICHAR in my Excel developement tool related to a Unicode-using application (MSKLC).
I contacted Microsoft Support by phone today and got the above case ID.
Best regards,
Marcel Schneider
P.S.: My first post noted these functions to be ASCII. So I am sorry not to have considered the system (whether it is hexadecimal, or decimal!).
* Note: I was not allowed to post this with hyperlinks to the Support Pages. Links are the following:
http://office.microsoft.com/en-us/excel-help/unicode-function-HA102753274.aspx
http://office.microsoft.com/en-gb/excel-help/unichar-function-HA102753273.aspx

Hi Marcel,
Thanks for your feedback, I'll collect the information and submit it with internal ways.
Have a good time.
Regards,
George Zhao
TechNet Community Support
It's recommended to download and install
Configuration Analyzer Tool (OffCAT), which is developed by Microsoft Support teams. Once the tool is installed, you can run it at any time to scan for hundreds of known issues in Office
programs.

Retrieving Unicode characters with MS Query

Hi.
I am using MS Query to retrieve data into Excel from an Oracle database. The data contains several different characters (such as degree and diameter symbols) which are stored as Unicode, but the query returns the same character for all of them.
In the MS Query window the special characters appear as upside down question marks; in Excel they show as white question marks in a black diamond. The ASCII code of the character displayed in Excel is '63', and the UNICODE() value is 65533. This
isn't the character that is stored in the Oracle database.
Is there a way to correctly retrieve these characters?
The ODBC driver is Oracle in OraClient11g_home1, version 11.02.00.01. I've tried it with the 'Force SQL_WCHAR Support' setting on, but it didn't make a difference. So, I've pretty much exhausted my knowledge now...
Thanks in advance, Steve

Hi
Steve,
As far as I know the function of MS Query didn’t update anymore after the version of excel 2003. I suppose the issue might be caused by the different rules between Unicode
and ASCII code, so you get different result in excel from Oracle database. I suggest you can try to use PowerQuery and PowerPivot. Their function are stronger than MS Query, and you can get latest function. Probably they can help you to solve your issue.
Hope it’s helpful.
Regards,

Unicode strings support

Anyone else have problems with unicode strings with ADFm?
I found very strange behavior (bug?) with unicode strings passed through ADFm bindings to EJB method parameters.
The method is invoked only once when non-unicode (plain ASCII) characters involved.
The same PageDef invokes method twice(?!) when parameter receives unicode characters from page! Not only that, but the second time method is invoked with completely messed-up string values (two-byte unicode chars replaced with ?).
Obviously this behavior is making any globalized application (with unicode inputs) development impossible.
Am I doing something wrong here?
In several other threads I found complains on JDev (even 11g) not handling Unicode/UTF-8 properly. But I could not find any Oracle statements on this issue and commitment to true and easy Unicode application development support.

Hi Steve,
I have just sent you an email with test-case. You are the 5th person from Oracle I'm sending this same email. :)) But never received any answer, comment, confirmation or even rejection of the case. Please, understand that for use from non-ASCII regions this issue is BIG issue. If I cannot have my local letters in ADF then everything else is not relevant. No one will buy my SW even if I give it for free (and especially if they have to pay for Oracle licenses which are somehow far from free).
Kind regards,
Pavle

Diacritic: unicode to ascii

Similar Messages

Maybe you are looking for