File_datastore not indexing correctly (oracle version 10.2.0.1.0)

Question

Problem:
(1) Created an index on a VARCHAR2 column that contained text from a given file (single row).
Used direct_datastore to create index.
Selected count(*) from the index and received a count of 109.
(2) Created an index on VARCHAR2 column that contained a file location for a file on the file system which contains the same data as in (1).
Used file_datastore to create index.
Selected count(*) from the index and received a count of 17.
The data that is getting indexed is Arabic.
Index for direct_datastore works as expected when performing a query but the file_datastore does not.
Any ideas why the files are not getting parsed correctly and what I could do to resolve?
Database properties
NLS_LANGUAGE = AMERICAN
NLS_CHARACTERSET = AL32UTF8
NLS_NCHAR_CHARACTERSET = AL16UTF16
Commands used to create indexes:
(1) create index globaldoc_index on globaldoc(text_data)
indextype is ctxsys.CONTENT
parameters ('datastore CTXSYS.DIRECT_DATASTORE,
filter ctxsys.null_filter, lexer world_lexer, languange column language,
charset column characterset')
(2) create index globaldoc_index on globaldoc(filepath)
indextype is ctxsys.CONTENT
parameters ('datastore CTXSYS.FILE_DATASTORE,
filter ctxsys.null_filter, lexer world_lexer, languange column language,
charset column characterset')
Data in table globaldoc:
language = ar
characterset = AL32UTF8

Answer 1

Can you try the the second case with Auto Lexer?
create index globaldoc_index on globaldoc(filepath)
indextype is ctxsys.CONTENT
parameters ('datastore CTXSYS.FILE_DATASTORE,
filter ctxsys.null_filter, lexer auto_lexer, languange column language,
charset column characterset')
Auto lexer should determine the language of the document automatically.
Also, you can create your own lexer specifying the language of the lexer and use the same while creating the index.