Oracle 10g R2 - Unicode is converted to unknown ?����

Question

Ok, I've read a thousand posts. Tried everything still stuck.
I am using Oracle 10g : 10.2.0.5.0
I'm on a Mac, using SQL Developer 3.1.07
When I enter "ė€£¥©", the insertion into the database converts it to: ?����
This is straight from a SQL INSERT command, in SQLDeveloper, no java code or anything.
I tried adding a file called SQLDeveloper.app/Resources/sqldeveloper/sqldeveloper/sqldeveloper.conf and added these lines:
-Doracle.jdbc.defaultNChar=true
-Doracle.jdbc.convertNcharLiterals=true
Didn't help.
I tried created a test column and made the datatype NVARCHAR2. same results. VARCHAR column does the same corruption but that's expected.
I tried N'©' - doesn't work
I tried INSERT INTO testchar (column1) VALUES ( unistr('©') );
results is still : �
I check NSL_NVAR_CHARACTERSET and it's UTF8
NLS_LANGUAGE     AMERICAN
NLS_TERRITORY     AMERICA
NLS_CURRENCY     $
NLS_ISO_CURRENCY     AMERICA
NLS_NUMERIC_CHARACTERS     .,
NLS_CHARACTERSET     US7ASCII
NLS_CALENDAR     GREGORIAN
NLS_DATE_FORMAT     DD-MON-RR
NLS_DATE_LANGUAGE     AMERICAN
NLS_SORT     BINARY
NLS_TIME_FORMAT     HH.MI.SSXFF AM
NLS_TIMESTAMP_FORMAT     DD-MON-RR HH.MI.SSXFF AM
NLS_TIME_TZ_FORMAT     HH.MI.SSXFF AM TZR
NLS_TIMESTAMP_TZ_FORMAT     DD-MON-RR HH.MI.SSXFF AM TZR
NLS_DUAL_CURRENCY     $
NLS_COMP     BINARY
NLS_LENGTH_SEMANTICS     BYTE
NLS_NCHAR_CONV_EXCP     FALSE
NLS_NCHAR_CHARACTERSET     UTF8
NLS_RDBMS_VERSION     10.2.0.5.0I don't know what else to do. I am not a DBA. I'm just trying to fix a bug or at least tell a customer what's wrong. Entering special characters is required. Can anyone help? thank you!
I have read that the insertion into the database does its own conversion but I don't know if that applies to me since I'm in 10g R2
Edited by: 947012 on Aug 14, 2012 11:43 AM
My mac's locales are also :
$ locale
LANG="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_CTYPE="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_ALL=Edited by: 947012 on Aug 14, 2012 11:45 AM
Edited by: 947012 on Aug 14, 2012 11:49 AM

Answer 1

NLS_NCHAR_CHARACTERSET specifies the encoding of the NVARCHAR2 variables but not the SQL statements. SQL statements, when they arrive to the database server, are always encoded in the database character set (NLS_CHARACTERSET). This is true for the whole statement, including any character literals. Therefore, even if a literal is valid in SQL Developer (which works in Unicode), when the literal arrives to the database, it is already stripped of all characters that the database character set does not support. The characters are not actually removed but they are converted to a replacement character. In case of the US7ASCII database character set, anything non-ASCII is lost.
The trick to avoid this problem is to:
- mark the literals that need to be preserved for NVARCHAR2 columns as NVARCHAR2 literals by prefixing them with "n".
- set the mentioned property to activate the client side encoding mechanism with basically does a rough parsing of the statement and replaces all N-literals with U-literals. Undocumented U literals are similar to UNISTR calls. They encode non-ASCII characters with Unicode escape sequences. For example, n'€' (U+20AC) is encoded as u'\20AC'. As u'\20AC' contains only ASCII characters, it arrives to the database unchanged. The SQL parser recognizes the U-literals and converts them (unescapes them) to NVARCHAR2 constants before further SQL processing, such as INSERT.
-- Sergiusz