Weird character filter

Question

I used to code in RedHat. Something happened (something nasty) and I changed to Mandrake to give it a try. Among my source file I had a Class with a method that filtered characters, some usual characters (dollar, punctuation) and some not so usual (yen, euro, dot in the middle of the line, long hyphen, long underline,...). When i reopened my source file in Mandrake, all of the characters in the second group had turned into "?\200" , "?\202\2004" and things of the sort, sometimes even worse. I guess this has to do with encoding and Mandrake, but it's not only a question of display, my classes (which have compiled for a long time without any problems) are now unable to go through this source file, due to the character stuff. Is there a way of going around this? I've tried to find out what character set those escape sequences come from, but I've been unable to find out.
Any ideas?
Here's (a version of) the method:
    private String filterDoc(String in){
     char c;
     char prec;
     StringBuffer out = new StringBuffer();
     for (int i = 0; i < in.length(); i++){
         c = in.charAt(i);
         if ( ( c >= 'a' ) && ( c <= 'z' ) ) out.append(c);
         else switch ( c ) {
         case '.' : {
          prec = in.charAt( i > 1 ? i-2 : i+2);
          if ( (prec == '.') || (prec == ' ') ) out.append(".");
          else out.append("\n");
         } break;
         case '��' : out.append('.');break;  // this line : mid-height dot
         case ',' : out.append('\n'); break;
         case ';' : out.append('\n'); break;
         case ':' : out.append('\n'); break;
         case '?' : out.append('\n'); break;
         case '$' : out.append('\n'); break;
         case '��' : out.append('\n'); break;  // this line : pound symbol
         case '�,B,(B' : out.append('\n'); break; // this line : euro symbol
         case '��' : out.append('\n'); break;  // this line : yen symbol
         case '��' : out.append('\n'); break;  // this line: copyright symbol
         case '*' : out.append('\n'); break;
         case '-' : out.append('-'); break;
         case '��$(D' (B: out.append('-'); break; // this line : hyphen
         case '_' : out.append('-'); break;
         default : out.append(c); break;
     return out.toString();
    }I say it's 'a version' of the method, because in windows it looks just like in mandrake, and when I copy paste it here I can see 'some' things have changed, for instance, I see squares (or black squares) up here, but I don't see any squares on emacs, I get the \20something.

Answer 1

The Java compiler uses the platform's default character encoding to read source files, so for portability you shouldn't write non-ASCII characters directly, instead you should use the \uXXXX unicode escape sequences, for instance \u20AC for euro.
It appears that in RedHat your default encoding was UTF-8 in which the characters you mention are encoded with multiple bytes, and when you transferred to Mandrake the default encoding changed and now it's some encoding where each character is encoded with a single byte.
There are a number of things you can do:
o Convert the source files to use unicode escape sequences. The easiest way is to use the native2ascii tool that comes with the SDK: run this command to all your source files:    native2ascii -encoding UTF-8 inputfile outputfileAlso, if you are looking for the escape sequence of a particular character you can find it from http://www.unicode.org/charts/
o Continue using UTF-8 and pass the option -encoding UTF-8 to the compiler:    javac -encoding UTF-8 MyClass.javao Change the platform's default character encoding by modifying locale settings, for instance     export LANG=en_US.UTF-8You can use the command "locale -a" to list all available locales.