Unicode Issues

Question

Hi All,
I have a requirement where i need to get content from a file and then loading it into a table.
The file is containing Some Unicode Characters.
The content is inserted into a CLOB Column in a table.
I placed the file in a directory located in DB Server and loaded into table using below proc.
-- (1) Insert a new row into html_documents with an empty CLOB, and
-- (2) Retrieve the empty CLOB into a variable with RETURNING..INTO
INSERT INTO clob_docs(id,code,
title,
html_document,
category)
VALUES( id, Code,
Title,
empty_clob(),
Category)
RETURNING html_document INTO theCLob;
-- (3) Get a BFile handle to the external file
theBFile := BFileName(dir,file);
-- (4) Open the file
dbms_lob.fileOpen(theBFile);
-- (5) Copy the contents of the BFile into the empty CLOB
dbms_lob.loadCLOBFromFile(dest_lob => theCLob,
src_bfile => theBFile,
amount => dbms_lob.getLength(theBFile),
DEST_OFFSET => dst_offset,
SRC_OFFSET => src_offset,
BFILE_CSID => DBMS_LOB.DEFAULT_CSID,
LANG_CONTEXT => lang_ctx,
WARNING => warning);
-- (6) Close the file and commit
dbms_lob.fileClose(theBFile);
COMMIT;
The file is loaded fine, but inverted commas are loaded in the place of unicode characters.
Example: *¿acc¿¿¿*
Characterset in database is NLS_CHARACTERSET -->AL32UTF8
NLS_NCHAR_CHARACTERSET --> AL16UTF16
Please help me out to avoid this situation and all the characters to be properly loaded.
Thanks.

Answer 1

All text files are, by definition, encoded using some character set. You have to know what character set you are using to know how to translate the series of 1's and 0's in the file into a particular character. The same character will have very different binary representations in different character sets.
We need to know what character set your file is encoded using to know how to load it into the database properly. If you don't specify, Oracle assumes that the file is encoded using the database character set (AL32UTF8). If that is not the actual character set used to encode the file, you'll get character set translation errors which would result in replacement characters getting stored (those are the question marks that you're seeing).
Justin