SQL*Loader characterset

Question

Database: 10.2.0.1
sql> select * from nls_database_parameters t where t.parameter like '%CHARACTERSET%';
PARAMETER VALUE
==============================================
NLS_CHARACTERSET AL32UTF8
NLS_NCHAR_CHARACTERSET AL16UTF16
OS: RHEL5
$ export | grep LANG
declare -x LANG="en_US.UTF-8"
declare -x NLS_LANG="GERMAN_GERMANY.WE8ISO8859P1"
I have file in latin1 codepage.
When I create external table (organization external type oracle_loader) I have problems with loading the Ü symbols in this file to the database:
a;b;c;Ü;d <- row inserted
Ü;a;b;c;d <- row inserted
a;b;c;d;Ü <- row failed to insert
KUP-04021: field formatting error for 1 field
KUP-04101: record 1 rejected in file latin1.csv
When I explicitly create external table with "characterset WE8ISO8859P1" clause the latin1 file loads successfully.
When I load the UTF8 file using external table which was created without clause "characterset WE8ISO8859P1" the file is loads OK too.
I want to know the process - why the characterset conversion from latin1 file (WE8ISO8859P1) to UTF8 database for some reason fails?
Is there possible to import latin1 files and do not specify the codepage in the external table DDL (seems that NLS_LANG env variable does not affect SQL*Loader external table)?
From documentation:
Specifying the CHARACTERSET parameter tells SQL*Loader the character set of the input datafile. The default character set for all datafiles, if the CHARACTERSET parameter is not specified, is the session character set defined by the NLS_LANG parameter. Only character data (fields in the SQL*Loader datatypes CHAR,  VARCHAR, VARCHARC, numeric EXTERNAL, and the datetime and interval datatypes) is affected by the character set of the datafile.
Edited by: lynx™ on 15.07.2010 7:46

Answer 1

Hi Andre,
this is how I teach my classes normally!
You need to be aware of some changes you make with NLS_TERRITORY, which is part of NLS_LANG:
--> if you set NLS_TERRITORY to anohter value then you implicitly change the settings for NLS_NUMERIC_CHARACTERS => , although it is the same with america and australia obviousely the first digit here is the decimal separator and the second one is the grand seperator, thisn can destroy all numeric values if it is set impropper !!!
NLS_DATE_FORMAT,
NLS_TIMESTAMP_FORMAT,
AND NLS_CURRENCY.
SYS @10gR2 SQL select * from v$nls_parameters
PARAMETER VALUE
NLS_LANGUAGE AMERICAN
NLS_TERRITORY AMERICA
NLS_CURRENCY $
NLS_ISO_CURRENCY AMERICA
NLS_NUMERIC_CHARACTERS .,NLS_CALENDAR GREGORIAN
NLS_DATE_FORMAT DD-MON-RRNLS_DATE_LANGUAGE AMERICAN
NLS_CHARACTERSET WE8ISO8859P1
NLS_SORT BINARY
NLS_TIME_FORMAT HH.MI.SSXFF AM
NLS_TIMESTAMP_FORMAT DD-MON-RR HH.MI.SSXFF AM
NLS_TIME_TZ_FORMAT HH.MI.SSXFF AM TZR
NLS_TIMESTAMP_TZ_FORMAT DD-MON-RR HH.MI.SSXFF AM TZR
NLS_DUAL_CURRENCY $
NLS_NCHAR_CHARACTERSET AL16UTF16
NLS_COMP BINARY
NLS_LENGTH_SEMANTICS BYTE
NLS_NCHAR_CONV_EXCP FALSE
19 rows selected.
SYS @10gR2 SQL > alter session set NLS_TERRITORY=australia;
SYS @10gR2 SQL > select * from v$nls_parameters
2 ;
PARAMETER VALUE
NLS_LANGUAGE AMERICAN
NLS_TERRITORY AUSTRALIA
NLS_CURRENCY $
NLS_ISO_CURRENCY AUSTRALIA
NLS_NUMERIC_CHARACTERS .,
NLS_CALENDAR GREGORIAN
NLS_DATE_FORMAT DD/MON/RR
NLS_DATE_LANGUAGE AMERICAN
NLS_CHARACTERSET WE8ISO8859P1
NLS_SORT BINARY
NLS_TIME_FORMAT HH12:MI:SSXFF AM
NLS_TIMESTAMP_FORMAT DD/MON/RR HH12:MI:SSXFF AM
NLS_TIME_TZ_FORMAT HH12:MI:SSXFF AM TZR
NLS_TIMESTAMP_TZ_FORMAT DD/MON/RR HH12:MI:SSXFF AM TZR
NLS_DUAL_CURRENCY $
NLS_NCHAR_CHARACTERSET AL16UTF16
NLS_COMP BINARY
NLS_LENGTH_SEMANTICS BYTE
NLS_NCHAR_CONV_EXCP FALSE