Bulk text data insertion in Partitioned Table

Question

Hi All,
I want to insert more than 200 crore records from 5 text data files in a single table say TAB1 (PARTITIONED table with RANGE type).
Currently I am using EXTERNAL TABLE concept and providing 5 filenames (of total 45 crore records) at a time to external table definition access criteria LOCATION() with COMMA ",".
After that using following sql statement:
INSERT /*+ APPEND PARALLEL(6) */ INTO TAB1 SELECT * FROM TAB1_EXT;
Presently The execution time statistics for 45 crore records is as follows:
1)     With /*+ PARALLEL(6) */ hint, execution time : 81 Mins     
2)     with /*+ APPEND PARALLEL */ hint, execution time : 73 Mins     
3)     With cursor loop; bulk collect and FORALL with interval of 1000000 records, execution time : 98 Mins
Earlier the execution time was too much but after changing and trying with variation in SQL logic, DB parameter, and H/w, now it is somewhat good.
Now INDEXES are also removed from table. NOLOGGING is purposely not using as it will not be considerable at PROD environment.
Also Database is not in ARCHIVE mode now.
I also tried with change in DB parameters like SGA_MAX_SIZE which is now 16GB. I am thinking even to change DB_BLOCK_SIZE value from 8K to 32K.
O/S: LINUX
ORACLE DB: ORACLE 11g
RAM: 256 GB
48 CPU CORES, AMD Opteron (HP DL 585 Model)
If you have any suggestion, please pass it to me.

Answer 1

Instead of disturbing parallel coordinators you could just submit n jobs each doing a simple <b>insert into big_table select * from external_table_n</b> or something similar - all external_tables_i having the same structure just different location parameters.
Regards
Etbin
Edited by: Etbin on 26.2.2011 12:25
possibly grouping text data files according to partitions the data would end within