(Internationalization) - Unicode and Other ... Encoding Schemes

Question

Hello,
I am developing a application that requires multiple languages
(Chinese/Japanese/English, French/German) support.
I plan to use utf-8 encoding, and not individual encoding for each language
like SHIFT_JIS for Japanese, BIG5 for Chinese etc.
This is more so because i would need to display multiple languages on the
same page, and allow the user to enter data in any language he/she chooses.
1. So, is the assumption that nothing but utf-8 can be used here, correct ?
2. If this is the case, why do people go for SHIFT_JS for Japanese or BIG5
for Chinese at all ? After the advent of Unicode, why cant they just use
utf-8.
3. I am using Weblogic 6. And my app is composed of JSPs alone at the
moment. It is working fine with utf-8 encoding, without me setting anything
at all in properties files etc. anywhere. I am getting data entered by user
in forms (in chinese/japanese etc) fine, and able to insert it into the
database and get it back too, without any problems.
So, why is it that people are talking of parameters to be set in properties
files to tell the app abt encoding being used etc.
4. My resource bundles are ASCII text files (.properties) which have name
value pairs. Hex Unicode numbers of the form /uXXXX represent the value. And
this works fine.
For example :
UserNameLabel = \u00e3\ufffd\u2039\u00e3
instead of -
UserNameLabel = ãf¦ãf¼ã
If the properties files have the original characters where values shud be
present, my java code is not able to read the name value pairs in Resource
Bundle.
Am i following the right approach ?
The problem with the current approach is after i create the Resource
Bundles, i must use native2ascii compiler to convert the characters into
their equivalent Hex Code values.
Thanks
JSB

Answer 1

charllescuba1008 wrote:
Unicode states that each character is assigned a number which is unique, this number is called code point. Right.
The relationship between characters and code points is 1:1.Uhm .... let's assume "yes" for the moment. (Note that the relationship between the Java type char and code point is not 1:1 and there are other exceptions ...)
Eg: the String *"hello"* (which is sequence of character literals) can be represent by the following Code Points
*\u0065 \u0048 \u006c \u006c \u006f*Those are the Java String unicode escapes. If you want to talk about Unicode Codepoints, then the correct notation for "Hello" would be
U+0048 U+0065 U+006C U+006C U+006F
Note that you swapped the H and e.
I also read that a certain character code point must be recognized by a specific encoding or else a question mark (?) is output in place of the character.This one is Java specific. If Java tries to translate some unicode character to bytes using some encoding that doesn't support that character then it will output the byte(s) for "?" instead.
Not all code points can be recognized by an encoding.Some encodings (such as UTF-8) can encode all codepoints, others (such as ISO-8859-*, EBCDIC or UCS-2) can not.
So, the letter *ל* would not be recognized by all encodings and should be replaced by a question mark (?) right?Only in a very specific case in Java. This is not a genral Unicode-level rule.
(disclaimer: the HTML code presented was using decimal XML entities to represent the unicode characters).
What you are seing is possibly the replacement character that your text rendering system uses to represent characters that it knows, but can't display (possibly because the current font has no character for them).