Issue with flat file loading timing out

Question

Hello
I have a scenario, where I am loading a flat file with 65k records into a cube. The problem is, in the loading process, I have to look up the 0Material table which has a million records.
I do have an internal table in the program, where I select a subset of the Material table ie around 20k to 30k records. But my extraction process takes more than 1 1/2 hrs and is failing (timed out).
How can i address this issue? I tried building indexes on the Material table and its not helping.
Thanks,
Srinivas.

Answer 1

Unfortunately, this is BW 3.5, so there is no END routine option here. And I tried both .csv and notepad file methods, and both are creating problems for us.
This is the total code, do you guyz see any potential issues:
Start Routine (main code)
refresh i_oldmats.
refresh ZI_MATL.
data: wa_datapak type transfer_structure.
loop at datapak into wa_datapak. (** I collect all the old material numbers from my flat file into an internal table i_oldmats**)
   i_oldmats-/BIC/ZZOLDMATL = wa_datapak-/BIC/ZZOLDMATL.
   collect i_oldmats.
endloop.
sort i_oldmats.
  SELECT /BIC/ZZOLDMATL MATERIAL (** ZI_MATL only has recs. where old materials exist, this gets about 300k records out of 1M**)
         FROM /BI0/PMATERIAL
         INTO ZI_MATL FOR ALL
         ENTRIES in i_oldmats WHERE
          /BIC/ZZOLDMATL = i_oldmats-/BIC/ZZOLDMATL .
            collect    ZI_MATL.
  Endselect.
  Sort ZI_MATL.
Transfer rule routine (main code)
IF TRAN_STRUCTURE-MATERIAL = 'NA'.
    READ TABLE ZI_MATL INTO ZW_MATL
    WITH KEY /BIC/ZZOLDMATL = TRAN_STRUCTURE-/BIC/ZZOLDMATL
    BINARY SEARCH.
    IF SY-SUBRC = 0.
      RESULT = ZW_MATL-MATERIAL.
    ENDIF.
  ELSE.
    RESULT = TRAN_STRUCTURE-MATERIAL.
  ENDIF.
Regards,
Srinivas.