Data packets/size - Generic extractor

Question

Hi all,
We built a custom function module based datasource and it is extracting data to BW in one big packet of 900,000+ records and the load is taking about 18 hours. We are trying to spilt the BW extraction into smaller data packets to improve performance but unable to do so. Following is our extraction program...
Please let me know where we are doing it wrong...
This Program fetches/build e_t_data. The issue is, program does not splitting into packets as the SAP standard program does.
""Local interface:
*"  IMPORTING
*"     VALUE(I_REQUNR) TYPE  SRSC_S_IF_SIMPLE-REQUNR
*"     VALUE(I_DSOURCE) TYPE  SRSC_S_IF_SIMPLE-DSOURCE OPTIONAL
*"     VALUE(I_MAXSIZE) TYPE  SRSC_S_IF_SIMPLE-MAXSIZE OPTIONAL
*"     VALUE(I_INITFLAG) TYPE  SRSC_S_IF_SIMPLE-INITFLAG OPTIONAL
*"     VALUE(I_READ_ONLY) TYPE  SRSC_S_IF_SIMPLE-READONLY OPTIONAL
*"     VALUE(I_DATAPAKID) TYPE  SBIWA_S_INTERFACE-DATAPAKID OPTIONAL
*"  TABLES
*"      E_T_DATA STRUCTURE  Z0333W OPTIONAL
*"      I_T_SELECT TYPE  SRSC_S_IF_SIMPLE-T_SELECT
*"      I_T_FIELDS TYPE  SRSC_S_IF_SIMPLE-T_FIELDS
*"  EXCEPTIONS
*"      NO_MORE_DATA
  DATA: lr_range_name TYPE rsselect-fieldnm.
  DATA: st_e_t_data TYPE z0333w.
  STATICS: l_cursor TYPE cursor.
  STATICS: called(1) TYPE c VALUE 'N'.
Maximum number of lines for DB table
  STATICS: l_maxsize TYPE sbiwa_s_interface-maxsize.
  FIELD-SYMBOLS: <l_range> TYPE ANY,
                 <l_t_range> TYPE STANDARD TABLE.
  IF i_initflag = 'X'.
Initialization: check input parameters
                buffer input parameters
                prepare data selection
Fill parameter buffer for data extraction calls
    g_s_interface-requnr    = i_requnr.
    g_s_interface-isource   = i_dsource.
    g_s_interface-maxsize   = i_maxsize.
    g_s_interface-initflag  = i_initflag.
    g_s_interface-datapakid = i_datapakid.
    g_flag_interface_initialized = sbiwa_c_flag_on.
    REFRESH g_t_select.
    REFRESH g_t_fields.
    APPEND LINES OF i_t_select TO g_t_select.
    APPEND LINES OF i_t_fields TO g_t_fields.
  ELSE.
first data package of first table -> open cursor
    IF g_counter_datapakid = 0.
*--Get selection ranges
      LOOP AT i_t_select.
        MOVE-CORRESPONDING i_t_select TO <l_range>.
        APPEND <l_range> TO <l_t_range>.
      ENDLOOP.
      l_maxsize = g_s_interface-maxsize.
fetch plants for the company code
      PERFORM get_plants.
fetch mast data into internal table as we will be using MAST for validation
of whether BOM exist or not.
      SELECT * FROM mast INTO TABLE it_mast
                         WHERE stlan = '1' OR stlan = '6'.
      SORT it_mast BY matnr werks stlan.
Material BOM information
First data package -> OPEN CURSOR
     OPEN CURSOR WITH HOLD l_cursor FOR
    SELECT mast~matnr
           mast~werks
           mast~stlnr
           mast~stlan
           mast~stlal
           stko~stlty
   FROM mast INNER JOIN stko
     ON stkostlnr = maststlnr AND
        stkostlal = maststlal
   FOR ALL entries IN gt_werks
  WHERE mast~matnr IN gr_matnr AND
        mast~werks IN gr_werks AND
        mast~stlan IN gr_stlan AND
        mast~werks = gt_werks-werks AND
        mast~stlal = '01' AND
        stko~stlty = 'M'  AND                    "Material BOM only
        ( maststlan = '1' OR maststlan = '6' ).
    ENDIF.
Fetch records into interface table.
  named E_T_'Name of extract structure'.
    REFRESH: gt_mat_bom,gt_mat_bom1.
    FETCH NEXT CURSOR l_cursor
               APPENDING CORRESPONDING FIELDS
               OF TABLE  gt_mat_bom
               PACKAGE SIZE i_maxsize.
    IF sy-subrc <> 0.
      CLOSE CURSOR l_cursor.
      RAISE no_more_data.
    ELSE.
get BOM data and fill E_T_DATA
      PERFORM get_bom_data TABLES e_t_data.
    ENDIF.
Increment Package
    g_counter_datapakid = g_counter_datapakid + 1.
  ENDIF.
ENDFUNCTION
Thanks,
Anirudh.

Answer 1

I'm not sure, but this might help:
* Fetch records into interface table.
* named E_T_'Name of extract structure'.
  DO.
    REFRESH: gt_mat_bom,gt_mat_bom1.
    FETCH NEXT CURSOR l_cursor
      APPENDING CORRESPONDING FIELDS
      OF TABLE gt_mat_bom
      PACKAGE SIZE i_maxsize.
    IF sy-subrc <> 0.
      EXIT.
    ELSE.
* get BOM data and fill E_T_DATA
      PERFORM get_bom_data TABLES e_t_data.
    ENDIF.
* Increment Package
    g_counter_datapakid = g_counter_datapakid + 1.
  ENDDO.
  CLOSE CURSOR l_cursor.
  RAISE no_more_data.
Rob