How to import a large file of 5 gb

Question

Hi all,
I have a file of 5gb to be loaded into Oracle 10g in to a single column and add a primary key to it in another column, actually I need to correct some records in the file after loading for which I have script
one records in file is as below
could any one of you help in writing a loader or any other way of loading this file as below
11232| kklauire| ajskldfjakl kljdklsf| kkkk|  |  |  |  |   |iuueoiui
I want it to be loaded as below
Id   col
1    11232| kklauire| ajskldfjakl kljdklsf| kkkk|  |  |  |  |   |iuueoiui
2    44444| kklauire| ajskldfjakl kljdklsf| kkkk|  |  |a|c |   |iuueoiui
.I have only sql*plus as the tool for this, is any more information needed plz let me know
Regards,
A
Edited by: user605246 on Apr 1, 2010 2:45 AM

Answer 1

Hi Blushadow,
I have to clean it before loading it into the final table, as the records got splitted in to multiple line, along with delimiter and with out delimiter. remember for this you have given me a script to clean a week back ( Re: help needed for correcting records in a table )
with res as (select dtt_id,
                    txt,
                    ceil(sum(length(txt) - length(replace(txt, '|'))) over (order by dtt_id)/4) grp
             from   t), -- assuming that your table is called t.
   final as (select replace(sys_connect_by_path(txt, '$'), '$') full_txt,
                    level
             from   res
             WHERE CONNECT_BY_ISLEAF = 1
             connect by grp = prior grp
             and        dtt_id = prior dtt_id + 1)
select case when length(full_txt) - length(replace(full_txt, '|')) = 11 then full_txt end full_txt
from   final
where  case when length(full_txt) - length(replace(full_txt, '|')) = 11 then full_txt end is not null;
IN TSQL  I have used the below script for the above task, as finally I want to load this file to sql
DECLARE @max bigint,@c  bigint
SELECT @max = MAX(rn),@c = 1 from @X
DECLARE @targ varchar(max)
WHILE @max <> 0
BEGIN
      SELECT @targ = txt FROM @X WHERE rn = @max
      IF (@targ not like '%|%|%|%|%')  -- | = number of delimiter in the file
      BEGIN
            UPDATE @X SET txt = txt+@targ WHERE rn = @max - 1
            DELETE FROM @X WHERE rn = @max
      END
      SET @max = @max - 1
ENDEdited by: user605246 on Apr 1, 2010 4:13 AM
Edited by: user605246 on Apr 1, 2010 4:15 AM