Multibyte character error in SqlLoader when utf8 file with chars like €Ää

Question

hello,
posting from Germany, special charactes like german umlaute and euro sign in UTF8 Textfile, SqlLoader rejecting row with Multibyte character error
Oracle Database 11g Release 11.2.0.2.0 - 64bit Production
Database Characterset: WE8MSWIN1252
OS: SLES 11 x86_64
Testcase SqlDeveloper:
CREATE TABLE utf8file_to_we8mswin1252 (
ID NUMBER,
text VARCHAR2(40 CHAR)
can't enter euro symbol in this posting, end's in '€' (?)
SELECT ascii(euro symbol) FROM dual;
128
SELECT chr(128) from dual;
INSERT INTO utf8file_to_we8mswin1252 (ID, text) VALUES (1, '0987654321098765432109876543210987654321');
INSERT INTO utf8file_to_we8mswin1252 (ID, text) VALUES (2, 'äüöäüöäüöäÄÖÜÄÖÜÄÖÜÄßßßßßßßßß߀€€€€€€€€€');
INSERT INTO utf8file_to_we8mswin1252 (ID, text) VALUES (3, 'äüöäüöäüöäÄÖÜÄÖÜÄÖÜÄäüöäüöäüöäÄÖÜÄÖÜÄÖÜÄ');
INSERT INTO utf8file_to_we8mswin1252 (ID, text) VALUES (4, 'ۧۧۧۧۧۧۧۧۧۧ1');
INSERT INTO utf8file_to_we8mswin1252 (ID, text) VALUES (5, 'äüöäüöäüöäÄÖÜÄÖÜÄÖÜÄäüöäüöäüöä');
INSERT INTO utf8file_to_we8mswin1252 (ID, text) VALUES (6, 'ßßßßßßßßß߀€€€€€€€€€1');
INSERT INTO utf8file_to_we8mswin1252 (ID, text) VALUES (7, 'ßßßßßßßßß߀€€€€€€€€€äüöäüöäüöäÄÖÜÄÖÜÄÖÜÄ');
commit;
Select shows correct result, no character is wrong or missing!!!!
put this in a UTF8 file without delimiter and enclosure like
10987654321098765432109876543210987654321
the SqlLoader controlfile:
LOAD DATA characterset UTF8
TRUNCATE                              
INTO TABLE utf8file_to_we8mswin1252
ID CHAR(1)     
, TEXT CHAR(40)
on a linux client machine, NOT the Oracle-Server
export NLS_LANG=AMERICAN_AMERICA.WE8MSWIN1252
sqlldr user/pwd@connectstring CONTROL=TEST01.ctl DATA=TEST01.dat LOG=TEST01.log
Record 6: Rejected - Error on table UTF8FILE_TO_WE8MSWIN1252, column TEXT.
Multibyte character error.
Record 7: Rejected - Error on table UTF8FILE_TO_WE8MSWIN1252, column TEXT.
Multibyte character error.
Select shows missing characters in row 4 and 5, SqlLoader loads only the first 20 characters (maybe at random)
and as shown above, row 6 and 7 never loaded
Problem:
can't load UTF8 Flatfiles with SqlLoader when german umlaute and special characters like euro symbol included.
Any hint or help would be appreciated
Regards
Michael

Answer 1

## put this in a UTF8 file without delimiter and enclosure like
The basic question is how you put the characters into the file. Most probably, you produced a WE8MSWIN1252 file and not an UTF8 file. To confirm, a look at the binary codes in the file would be necessary. Use a hex-mode-capable editor. If the file is WE8MSWIN1252, and not UTF8, then the SQL*Loader control file should be:
LOAD DATA characterset WE8MSWIN1252
TRUNCATE
INTO TABLE utf8file_to_we8mswin1252
ID CHAR(1)
, TEXT CHAR(40)
)-- Sergiusz