UTF-16: better than UTF-8? How to use?

Question

I read about UTF-8/UTF-16 and found out, that UTF-16 characters all have the same encopded length while UTF-8 uses a more compact representation. But why does Eclipse just shows one line of strange characters (most times little squares) in my .java file when i switch Eclispe's encoding from UTF-8 to UTF-16? I thought Java is using UTF-16 internally all the time?

Answer 1

String aString = "jschell";
byte[] whateverEncodingBytes = myString.getBytes("whateverEncoding");aString is in UTF-16, no matter what. The encoding specified in the getBytes() method is the target encoding. So whateverEncodingBytes is an array of bytes stored in a whatever encoding, rather than UTF-16. If you just use the method getBytes() the array of bytes would be stored in the default encoding which is iso-8859-1 unless otherwise specified.
String aNewString = new String(whateverEncodingBytes, "whateverEncoding");
aNewString is UTF-16 again, no matter what. The encoding specified in the constructor is the source encoding. So whateverEncodingBytes are still in an encoding. But building a new String out of them requires an encoding conversion to occur, as the String must be in UTF-16. So we need to let the constructor know which encoding to convert from. It would be like an interpretter to know which language to interpret from.
A Java byte can be in any encoding it likes - it's simply eight raw bits of data stored in an arbitrary order. Anything extending a Java object is stored in UTF-8, except for String, which keeps its internal characters in UTF-16 encoding.
In short, if you pull data into Java from an external source, the encoding it sits in within Java sticks to the rules above. If it first hits Java as a byte stream (say from an InputStream), it's going to be in whatever encoding it was already in. If it first hits Java as a String (say from request.getParameter() in a servlet or JSP), it will be in UTF-16, as Java will convert its encoding immediately.
The same holds when writing back out. If you write out a series of bytes (such as with OutputStream), the resulting output is in the encoding the bytes were in to begin with. If you're outputting a String (although this is a lot rarer), it will be in UTF-16. Or to be more specific as somebody pointed out Java uses Unicode characters (16-bits, unsigned) in memory to hold characters.