Re: How to change the NLS_NCHAR_CHARACTERSET from WE8ISO8859P1 to AL16UTF16 ?

Question

Hi,
Now I have a new problem.
I have a Oracle9i instance, with this configurations.
PARAMETER VALUE
NLS_LANGUAGE AMERICAN
NLS_TERRITORY AMERICA
NLS_CURRENCY $
NLS_ISO_CURRENCY AMERICA
NLS_NUMERIC_CHARACTERS .,
NLS_CHARACTERSET WE8MSWIN1252
NLS_CALENDAR GREGORIAN
NLS_DATE_FORMAT DD-MON-RR
NLS_DATE_LANGUAGE AMERICAN
NLS_SORT BINARY
NLS_TIME_FORMAT HH.MI.SSXFF AM
NLS_TIMESTAMP_FORMAT DD-MON-RR HH.MI.SSXFF AM
NLS_TIME_TZ_FORMAT HH.MI.SSXFF AM TZR
NLS_TIMESTAMP_TZ_FORMAT DD-MON-RR HH.MI.SSXFF AM TZR
NLS_DUAL_CURRENCY $
NLS_COMP BINARY
NLS_LENGTH_SEMANTICS BYTE
NLS_NCHAR_CONV_EXCP FALSE
NLS_NCHAR_CHARACTERSET AL16UTF16
NLS_RDBMS_VERSION 9.2.0.1.0
I have a xHarbour DOS Client program that reads, shows in the screen, and write "perfectly fine" a text with Brazilian accentuation stored in a CLOB field.
Following you can see the sample of the text with Brazilian accentuation in my DOS application http://www.screencast.com/t/U5PXwCEo8
In order to my xHarbour DOS Client program works fine with this Brazilian accented characters, I must to set:
SET NLS_LANG=PORTUGUESE_BRAZIL.WE8MSWIN1252
And all is OK from my xHarbour DOS Client program.
My problem is because when I query this data manually using a SELECT from any client DOS/WINDOWS program like SQLPLUS, SQLDEVELOPER, TOAD, etc, I get bad characters instead the correct Brazilian accented characters.
Following you can see the same text into the Toad http://www.screencast.com/t/A1tal2Rtg
(you will see bad characters instead the correct Brazilian accented characters).
Following you can see the result of querying this field using a SELECT from SQLPLUS.
Certifico que por decis„o proferida no processo n§ @@@@@@@@@@@@@@, 
foi reconhecida a n„o incidˆncia do ITBI na transa‡„o do(s) im¢vel(is) 
abaixo caracterizado(s), com base no art.156, @ 2§, I, da Constitui‡„o 
Federal de 1988 e no art.6§, II, da Lei Municipal n§ 1.364 de 19/12/1988. 
(you will see bad characters instead the correct Brazilian accented characters).
How to make "SQLPLUS", "TOAD" (and others Windows or DOS Clients programs) show me the Brazilian accented characters correctly?
Thanks in advance,
Luigggye

Answer 1

OK, if you look at the word "decisäo" in your DOS program (shouldn't it be "decisão", by the way?), it is encoded by the following bytes: 64,65,63,69,73,84,6f.
d = 0x64
e = 0x65
c = 0x63
i = 0x69
s = 0x73
ä = 0x84
o = 0x6fSo, the code for "ä" in your database is 0x84. If you look at the definition of the WE8MSWIN1252 character set, for example here: http://msdn.microsoft.com/en-us/goglobal/cc305145 , you will see that the expected code is 0xe4. On the other hand, if you look at the definition of WE8PC850 here: http://msdn.microsoft.com/en-us/goglobal/cc305160 , you will see that the code for "ä" is 0x84 -- the one you have in your database.
This proves my initial diagnosis that you have incorrectly labeled data in your database. By setting the NLS_LANG to the same character set as the database, you created a so-called pass-through configuration. This configuration does not require any character set conversion, hence what you put into the database is what you can retrieve unchanged. As CHCP shows, your DOS program (the correct name is actually "console mode program") works in the Microsoft OEM Code Page 850, which corresponds to the Oracle character set WE8PC850. So this is the character set that your application sends to Oracle and this is what gets stored in the database. If you set the NLS_LANG to .WE8PC850 before inserting data, the database would know that the received data is in WE8PC850 and would convert it properly to the database character set WE8MSWIN1252. This data could then be properly converted as required for other clients such as Toad or SQL Developer. As the NLS_LANG was WE8MSWIN1252, Oracle wrongly assumed that the application data is already in WE8MSWIN1252 and did not do any conversion. You ended up with mislabeled data.
To fix the issue, you should repair the database character set declaration to correspond to the actual database content, i.e. WE8PC850. You should then change the NLS_LANG for the DOS program (and only for the DOS program, not for Toad or other Windows GUI apps) to WE8PC850 as well. I am assuming here, and this is important!, that all non-ASCII (e.g. Portuguese) characters in your database come from this one DOS program.
To repair the database character set declaration:
1) Upgrade to 10g or 11g
2) Run the csscan utility with parameters FROMCHAR=WE8PC850 TOCHAR=WE8PC850. The scan report from the utility (scan.txt) should report only changeless data.
3) After running the scan, execute the ?/rdbms/admin/csalter.plb script in SQL*Plus.
-- Sergiusz