Oracle XE 10.2.0.1.0 - Problem indexing PDF

Question

I am using Oracle XE 10.2.0.1.0 with Czech national settings set.
I need to make PDF with czech national characters working. Indexing TXT, HTML and DOC2003 documents with the same content works fine.
Below is my configuration.
h1. National Language Support
NLS_CALENDAR GREGORIAN
NLS_CHARACTERSET AL32UTF8
NLS_COMP BINARY
NLS_CURRENCY Kč
NLS_DATE_FORMAT DD.MM.RR
NLS_DATE_LANGUAGE CZECH
NLS_DUAL_CURRENCY Kč
NLS_ISO_CURRENCY CZECH REPUBLIC
NLS_LANGUAGE CZECH
NLS_LENGTH_SEMANTICS BYTE
NLS_NCHAR_CHARACTERSET AL16UTF16
NLS_NCHAR_CONV_EXCP FALSE
NLS_NUMERIC_CHARACTERS ,.
NLS_SORT CZECH
NLS_TERRITORY CZECH REPUBLIC
NLS_TIME_FORMAT HH24:MI:SSXFF
NLS_TIMESTAMP_FORMAT DD.MM.RR HH24:MI:SSXFF
NLS_TIMESTAMP_TZ_FORMAT DD.MM.RR HH24:MI:SSXFF TZR
NLS_TIME_TZ_FORMAT HH24:MI:SSXFF TZR
h1. Datastore
PDF url: http://www.mpsv.cz/files/clanky/6981/tiskove_avizo_CJ.pdf
I renamed the pdf to SummitKZamestnanosti_PDF-40.
CREATE TABLE file_datastore
id NUMBER PRIMARY KEY,
fmt varchar2(10),
docs VARCHAR2(2000)
-- INSERT data INTO File Datastore TABLE
INSERT INTO file_datastore VALUES
(111560,'binary','C:\Docs\SummitKZamestnanosti_PDF-40'
-- Configure DATASTORE
EXEC ctx_ddl.drop_preference('NO_PATH');
EXEC ctx_ddl.create_preference('NO_PATH','FILE_DATASTORE');
-- Configure LEXER
EXEC ctx_ddl.drop_preference('LEXER');
EXEC ctx_ddl.create_preference('LEXER','BASIC_LEXER');
-- CREATE CONTEXT INDEX
DROP INDEX idx_file_datastore_text FORCE;
CREATE INDEX idx_file_datastore_text ON file_datastore
( docs )
indextype IS ctxsys.context parameters
( 'format column fmt Datastore NO_PATH filter ctxsys.AUTO_FILTER lexer LEXER' );
-- QUERIES that should work.
-- SELECT id, docs, score(1) FROM file_datastore WHERE contains ( docs,'Bližší',1 ) > 0;
SELECT id, docs, score(1) FROM file_datastore WHERE contains ( docs,'dopadů ',1 ) > 0;
-- SELECT id, docs, score(1) FROM file_datastore WHERE contains ( docs,'sociálních',1 ) > 0;
SELECT id, docs, score(1) FROM file_datastore WHERE contains ( docs,'Španělsko',1 ) > 0;
-- SELECT id, docs, score(1) FROM file_datastore WHERE contains ( docs,'Švédska',1 ) > 0;
SELECT id, docs, score(1) FROM file_datastore WHERE contains ( docs,'Summit',1 ) > 0;
h1. My Output
ID DOCS
0 rows selected
ID DOCS
0 rows selected
ID DOCS
111560 C:\Docs\SummitKZamestnanosti_PDF-40
1 rows selected
Regards

Answer 1

I have used the 100% same configuration as above, but now for the Oracle Database 11g R1 11.1.0.7.0 – Production instead of Oracle 10g XE.
The result is that AUTO_FILTER for Oracle 11g is able to parse Czech language characters from the sample PDF file without any problems.
The problem with Oracle Text 10g R2 may be I guess:
1. In embedded fonts as mentioned in the Link: [documentation | http://download-west.oracle.com/docs/cd/B12037_01/text.101/b10730/afilsupt.htm] (I tried to embbed all fonts and the whole character set, but it did not helped)
2. in the character encoding of the text within the PDF documents.
I would like to add that also other third party PDF2Text converters have similar issues with the Czech characters in the PDF documents – after text extraction Czech national characters were displayed incorrectly.
If you have any other remarks, ideas or conclusions please reply :-)