Ssis flat file to ole db skipping some rows...

for some reason, I can't seem to figure out why I'm missing some lines when I import a csv file that I exported using sqlplus. The ssis package gives no warnings or errors as well.
The reason I know this is because when I loaded the csv file into a separate analysis application it gets the correct totals for numeric columns and row counts.
But for some reason, SSIS doesn't seem to capture all of the lines...
there were 6313052 rows in the csv file and ssis imports 6308607...
any thoughts?
I've tried different code pages too (1250, 1252, and UTF8) but they didn't seem to have an affect
I checked out this link: Why
all the records are not being copied from CSV to SQL table in a SSIS package
and numbers I've checked on numbers 2, 3, and 4.
Although, for number 1, I'm using a for each loop container descirbed in this site:http://help.pragmaticworks.com/dtsxchange/scr/FAQ%20-%20How%20to%20loop%20through%20files%20in%20a%20specified%20folder,%20load%20one%20by%20one%20and%20move%20to%20archive%20folder%20using%20SSIS.htm to
loop through files in a folder and import them.
I've also thought about missing delimiters, but I exported the files myself using sqlplus like this:
select
trim(to_char(t1.D_DTM, 'yyyy-mm-dd hh24:mm:ss'))||','||
trim(t1.TECHNOLOGY)||','||
trim(t1.VOICEDATA)||','||
trim(t2.MRKT_NM)||','||
trim(t2.REGION_NM)||','||
trim(t2.CLUSTER_NM)||','||
trim(t3.BSC_NM)||','||
trim(t1.BTS_ID)||','||
trim(t1.CSCD_NM)||','||
trim(t1.SECT_SEQ_ID)||','||
trim(t1.BND_ID)||','||
trim(t1.FA_ID)||','||
trim(t1.TCE_DTCT_CALL_SETUP_F_CNT)||','||
trim(t1.MS_ACQ_CALL_SETUP_F_CNT)||','||
trim(t1.SIGN_CALL_SETUP_F_CNT)||','||
trim(t1.BAD_FRM_CALL_SETUP_F_CNT)||','||
trim(t1.REORG_ATT_CNT)||','||
trim(t1.TCE_CALL_SETUP_F_CNT)||','||
trim(t1.WCD_CALL_SETUP_F_CNT)||','||
trim(t1.L_CALL_SETUP_F_CNT)||','||
trim(t1.TRAF_FRM_MISS_CNT)||','||
trim(t1.BCP_TMO_CALL_SETUP_F_CNT)
from table
I'm not sure how I could miss delimiters if I export a file like that...
Also, I tested importing these files into MySql using LOAD DATA INFILE and that seems to work fine and import all of the data...

Hi rpatel18,
The issue might occur if there are missing either row or column delimiters in the CSV file. Please have a look at the following blog to see if it is the issue:
http://blogs.msdn.com/b/dataaccesstechnologies/archive/2013/03/13/flat-file-source-cannot-handle-file-with-uneven-number-of-columns-in-each-row.aspx.
In addition, the issue might also happen due to the design structure of the SSIS package. You could post one screenshots of your SSIS package Control Flow Tasks and Data Flow Tasks for further analysis.
Regards,
Mike Yin
TechNet Community Support

Similar Messages

SSIS Flat File Source not populating varchar column

I have SSIS package that imports flat files with column separator | from "Flat File Source" into the database trough "OLE DB Destination".
For one of the columns (file contains UIDs in curly brackets) destination column is varchar(250) null-able, when "Retain nulls" is set to true complete data have been imported, but when "Retain nulls" is set to false beside fact that in
the file data is not null and it is not empty string it is not imported and in the destination column is empty string.
I know that this "Retain nulls" applies to the columns that contains NULLs but this is not the case.
If someone have experience with such issue please help.
Thank you in advance.

From your statement it looks like when you have curly brackets it skips the value and inserts nulls instead. And that may be the reason why its working fine when you change retain null value.
I would suggest
Make your setting "retain null" to true and load the flat file, then check what happens to the values having curly braces (is there any value or null) , if there is value we need to check for that(please share a sample file). And if it having null
instead of "{value}", I would suggest to put a script task to remove { from your flat file and then try load the data.
Hope this helps.
Regards, -Amit

How to remove the date extensions from a filename in SSIS Flat File Connection Manager dynamically at run time

Hello,
I have to load data from a csv file to SQL Database. The file is placed into a directory by another program but the file name being same, has different extensions based on time of the day that it is placed in the directory. Since I know the file name
ahead of time, so, I want to strip off the date/time extension from the file name so that I can load the file using Flat File Connection Manager. I am trying to use 'variable' and 'expression editor' so that I can specify the file name dynamically. But I
don't know how to write it correctly. I have a variable 'FileLocation' that holds the folder location where the file will be placed. The file for example: MyFileName201410231230 (MyFileName always the same, but the date/time will be different)
Thanks,
jkrish

I don't want to use ForEach Loop because the files are placed by a FTP process 3 times a day at a specific time, for ex. at 10 AM, 12 PM and 3 PM. These times are pretty much fixed. The file name is same but the extension will have this day time stamp. I
have to load each file only once for a particular reason, meaning I don't want to load the ones I already loaded. I am planning on setting up the SSIS process to load at 10:05, 12:05 and 3:05 daily. The files will be piling up in the folder. As it comes,
I load them. At some point in time, I can remove the old ones so that they won't take up space in the server. In fact, I don't have to keep the old ones at all since they are saved in a different folder anyways. I can ask the FTP process to
remove the previous one when the new one arrives. So, at any point in time, there will be one file, but that file will have different extensions every time.
I am thinking of removing the extensions before I load every time. If the file name is 'MyFileNamexxxxxxx.csv', then I want to change it to 'MyFileName.csv' and load it.
Thanks,
jkrish
You WILL need to use it eventually because you need to iterate over each file.
Renaming is unnecessary as one way or another you will need to put a processed file away.
And having the file with the original extension intact will also help you troubleshoot.
Arthur
MyBlog
Twitter

How can I assign " " in ssis flat file ?

Hi,
I want to each record should had start " and end "
i.e., "SJDD", "DKLJJ"
So flat file conncetion manager text qualifier entered _x0022_
but finall output generated like::It is an error
_x0022_4139801_x0022_,_x0022_NJ270_x0022
So plz help to resolve it
-Madhu

Thank you,
Previously It has used by using text qualifier entered _x0022_
So I dont know difference between using " and _x0022_
Please explain about text qualifier = _x0022_
and If I use " it Is any changes in Text file (as your experience)
Mostly I should use _x0022_
So How can I apply in production server?
-Madhu

"Skip" in Import Format not skipping some rows

We are currently using FDM 11.1.1.1.00 and we have a problem with skipping lines based on currency (which is our AccountDescription).
We first extract 28392 lines from Oracle EBS (27921 CAD and 471 USD). No mather what we put in the skip condition, the generated .dat contains exactly the same data (file compared with UltraEdit).
If we set our import format to skip USD lines, 11 of them are still imported. It's the same behavior if we set the import format to skip CAD lines (1136 of them are still imported).
For example this line still load
2021;1115562;50308;E;;006;806050;0000;55365;001;0000;;;;;;;;;;;;;;;;;;;;;;;;;USD;P02-13;A;;;2013;3.06;22.08;-19.02
while this one don't
2021;4368;50308;L;;101;000000;0000;21350;001;0000;;;;;;;;;;;;;;;;;;;;;;;;;USD;P02-13;A;;;2013;2400000;17100000;-14700000
***(Skip is on field 36 (USD))
I looked at some of the lines still loaded in hexadecimal to find any invisble character or differences in codes used to display characters but they are the same as lines that are skipped.
Three persons has been searching for pattern in data that could lead us to the problem without success.
Any ideas on why thoses lines are still imported ? Where should I look at ?
This may be easy to solve, but I'm new at this stuff (I actually had to learn FDM by myself as all technical administrators knowing it left the company).
Thanks.

Thanks for your help. It works fine with the script. Now it return only the rows I need.
I only inverted the condition so if it's CAD, the function return the value, else it skip the line (to exclude USD, EUR or any other currency it may encounter in the future).
Thanks again.
My script looks like that :
Function SelectCADLines(strField, strRecord)
'UpStream WebLink DM DataPump Import Script:
'Created By: Francois Robineau
'Date Created: 2/19/2013 10:35 AM
'Purpose: Select only CAD (skipping USD, EUR or any other possible currency)
Dim strAccount
'Store first value of account
strAccount = strField
'Check to see if value is CAD
If strAccount = "CAD" Then
'Return line
SelectCADLines=strAccount
Else
'Skip line
Res.PblnSkip=True
End if
End Function

How to skip first record while inserting data from a flat file to BW system

Hi Experts,
In my project we have to upload flat file into a BW system. I have written a program and it is working fine.
Now we have got another requirement. The flat file will have a header record (first row). While uploading the flat file we have to skip this record. How I can do so?
The code is as below:
FORM upload1.
DATA : wf_title    TYPE string,
          lt_filetab TYPE filetable,
          l_separator TYPE char01,
          l_action    TYPE i,
          l_count     TYPE i,
          ls_filetab TYPE file_table,
          wf_delemt TYPE rollname,
          wa_fieldcat TYPE lvc_s_fcat,
          tb_fieldcat TYPE lvc_t_fcat,
          rows_read TYPE i,
          p_error   TYPE char01,
          l_file      TYPE string.
DATA: wf_object(30) TYPE c,
          wf_tablnm TYPE rsdchkview.
wf_object = 'myprogram'.
DATA i TYPE i.
DATA:
       lr_mdmt                TYPE REF TO cl_rsdmd_mdmt,
       lr_mdmtr               TYPE REF TO cl_rsdmd_mdmtr,
       lt_idocstate           TYPE rsarr_t_idocstate,
       lv_subrc               TYPE sysubrc.
TYPES : BEGIN OF test_struc,
           /bic/myprogram TYPE /bic/oimyprogram,
           txtmd   TYPE rstxtmd,
           END OF test_struc.
DATA :    tb_assum TYPE TABLE OF /bic/pmyprogram.
DATA: wa_ztext TYPE /bic/tmyprogram,
        myprogram_temp TYPE ziott_assum,
        wa_myprogram TYPE /bic/pmyprogram.
DATA : test_upload TYPE STANDARD TABLE OF test_struc,
         wa2 TYPE test_struc.
DATA : wa_test_upload TYPE test_struc,
         ztable_data TYPE TABLE OF /bic/pmyprogram,
         ztable_text TYPE TABLE OF /bic/tmyprogram,
         wa_upld_text TYPE /bic/tmyprogram,
         wa_upld_data TYPE /bic/pmyprogram,
          t_assum TYPE ziott_assum.
DATA : wa1 LIKE test_upload.
wf_title = text-026.
CALL METHOD cl_gui_frontend_services=>file_open_dialog
    EXPORTING
      window_title            = wf_title
      default_extension       = 'txt'
      file_filter             = 'Tab delimited Text Files (*.txt)'
    CHANGING
      file_table              = lt_filetab
      rc                      = l_count
      user_action             = l_action
    EXCEPTIONS
      file_open_dialog_failed = 1
      cntl_error              = 2
      OTHERS                  = 3.                          "#EC NOTEXT
IF sy-subrc <> 0.
    EXIT.
ENDIF.
LOOP AT lt_filetab INTO ls_filetab.
    l_file = ls_filetab.
ENDLOOP.
CHECK l_action = 0.
IF l_file IS INITIAL.
    EXIT.
ENDIF.
l_separator = 'X'.
wa_fieldcat-fieldname = 'test'.
wa_fieldcat-dd_roll = wf_delemt.
APPEND wa_fieldcat TO tb_fieldcat.
CALL FUNCTION 'MESSAGES_INITIALIZE'.
CLEAR wa_test_upload.
Upload file from front-end (PC)
File format is tab-delimited ASCII
CALL FUNCTION 'GUI_UPLOAD'
    EXPORTING
      filename                = l_file
      has_field_separator     = l_separator
    TABLES
     data_tab                = i_mara
    data_tab                   = test_upload
    EXCEPTIONS
      file_open_error         = 1
      file_read_error         = 2
      no_batch                = 3
      gui_refuse_filetransfer = 4
      invalid_type            = 5
      no_authority            = 6
      unknown_error           = 7
      bad_data_format         = 8
      header_not_allowed      = 9
      separator_not_allowed   = 10
      header_too_long         = 11
      unknown_dp_error        = 12
      access_denied           = 13
      dp_out_of_memory        = 14
      disk_full               = 15
      dp_timeout              = 16
      OTHERS                  = 17.
IF sy-subrc <> 0.
    EXIT.
ELSE.
   CALL FUNCTION 'MESSAGES_INITIALIZE'.
    IF test_upload IS NOT INITIAL.
      DESCRIBE TABLE test_upload LINES rows_read.
      CLEAR : wa_test_upload,wa_upld_data.
      LOOP AT test_upload INTO wa_test_upload.
        CLEAR : p_error.
        rows_read = sy-tabix.
        IF wa_test_upload-/bic/myprogram IS INITIAL.
          p_error = 'X'.
          MESSAGE s153 WITH wa_test_upload-/bic/myprogram sy-tabix.
          CONTINUE.
        ELSE.
          TRANSLATE wa_test_upload-/bic/myprogram TO UPPER CASE.
          wa_upld_text-txtmd = wa_test_upload-txtmd.
          wa_upld_text-txtsh = wa_test_upload-txtmd.
          wa_upld_text-langu = sy-langu.
          wa_upld_data-chrt_accts = 'xyz1'.
          wa_upld_data-co_area = '12'.
          wa_upld_data-/bic/zxyzbcsg = 'Iy'.
          wa_upld_data-objvers = 'A'.
          wa_upld_data-changed = 'I'.
          wa_upld_data-/bic/zass_mdl = 'rrr'.
          wa_upld_data-/bic/zass_typ = 'I'.
          wa_upld_data-/bic/zdriver = 'yyy'.
          wa_upld_text-langu = sy-langu.
          MOVE-CORRESPONDING wa_test_upload TO wa_upld_data.
          MOVE-CORRESPONDING wa_test_upload TO wa_upld_text.
          APPEND wa_upld_data TO ztable_data.
          APPEND wa_upld_text TO ztable_text.
        ENDIF.
      ENDLOOP.
      DELETE ADJACENT DUPLICATES FROM ztable_data.
      DELETE ADJACENT DUPLICATES FROM ztable_text.
      IF ztable_data IS NOT INITIAL.
        CALL METHOD cl_rsdmd_mdmt=>factory
          EXPORTING
            i_chabasnm     = 'myprogram'
          IMPORTING
            e_r_mdmt       = lr_mdmt
          EXCEPTIONS
            invalid_iobjnm = 1
            OTHERS         = 2.
   CALL FUNCTION 'MESSAGES_INITIALIZE'.
   **Lock the Infoobject to update
        CALL FUNCTION 'RSDG_IOBJ_ENQUEUE'
          EXPORTING
            i_objnm      = wf_object
            i_scope      = '1'
            i_msgty      = rs_c_error
          EXCEPTIONS
            foreign_lock = 1
            sys_failure = 2.
        IF sy-subrc = 1.
          MESSAGE i107(zddd_rr) WITH wf_object sy-msgv2.
          EXIT.
        ELSEIF sy-subrc = 2.
          MESSAGE i108(zddd_rr) WITH wf_object.
          EXIT.
        ENDIF.
*****Update Master Table
        IF ztable_data IS NOT INITIAL.
          CALL FUNCTION 'RSDMD_WRITE_ATTRIBUTES_TEXTS'
                EXPORTING
                  i_iobjnm                     = 'myprogram'
                  i_tabclass                   = 'M'
       I_T_ATTR                     = lt_attr
                TABLES
                  i_t_table                    = ztable_data
                  EXCEPTIONS
                 attribute_name_error         = 1
                 iobj_not_found               = 2
                 generate_program_error       = 3
                 OTHERS                       = 4.
          IF sy-subrc <> 0.
            CALL FUNCTION 'MESSAGE_STORE'
              EXPORTING
                arbgb = 'zddd_rr'
                msgty = 'E'
                txtnr = '054'
                msgv1 = text-033
              EXCEPTIONS
                OTHERS = 3.
            MESSAGE e054(zddd_rr) WITH 'myprogram'.
          ELSE.
            CALL FUNCTION 'MESSAGE_STORE'
              EXPORTING
                arbgb = 'zddd_rr'
                msgty = 'S'
                txtnr = '053'
                msgv1 = text-033
              EXCEPTIONS
                OTHERS = 3.
          ENDIF.
*endif.
*****update Text Table
          IF ztable_text IS NOT INITIAL.
            CALL FUNCTION 'RSDMD_WRITE_ATTRIBUTES_TEXTS'
              EXPORTING
                i_iobjnm               = 'myprogram'
                i_tabclass             = 'T'
              TABLES
                i_t_table              = ztable_text
              EXCEPTIONS
                attribute_name_error   = 1
                iobj_not_found         = 2
                generate_program_error = 3
                OTHERS                 = 4.
            IF sy-subrc <> 0.
              CALL FUNCTION 'MESSAGE_STORE'
                EXPORTING
                  arbgb = 'zddd_rr'
                  msgty = 'E'
                  txtnr = '055'
                  msgv1 = text-033
                EXCEPTIONS
                  OTHERS = 3.
            ENDIF.
          ENDIF.
        ELSE.
          MESSAGE s178(zddd_rr).
        ENDIF.
      ENDIF.
      COMMIT WORK.
      CALL FUNCTION 'RSD_CHKTAB_GET_FOR_CHA_BAS'
        EXPORTING
          i_chabasnm = 'myprogram'
        IMPORTING
          e_chktab   = wf_tablnm
        EXCEPTIONS
          name_error = 1.
      IF sy-subrc <> 0.
        MESSAGE ID sy-msgid TYPE sy-msgty NUMBER sy-msgno
                WITH sy-msgv1 sy-msgv2 sy-msgv3 sy-msgv4.
      ENDIF.
****Release locks on Infoobject
      CALL FUNCTION 'RSDG_IOBJ_DEQUEUE'
        EXPORTING
          i_objnm = 'myprogram'
          i_scope = '1'.
    ENDIF.
ENDIF.
PERFORM data_selection .
PERFORM update_alv_grid_display.
CALL FUNCTION 'MESSAGES_SHOW'.
ENDFORM.
Please let me know how I can skip first record of the flat file?
Regards,
S

go through this hope u can get some idea
REPORT ztest no standard page heading line-size 255.
                      Declaration                            *
TYPES t_itab1 TYPE alsmex_tabline.
types: begin of t_csks,
       kostl like csks-kostl,
      end of t_csks.
types: begin of t_cska,
       kstar like cska-kstar,
      end of t_cska.
data: begin of t_flatfile,
      docdate like COHEADER-BLDAT,
      postdate like COHEADER-BUDAT,
      doctext like COHEADER-BLTXT,
       costele like RK23F-KSTAR,
       amount like RK23F-WTGBTR,
       scostctr like RK23F-SKOSTL,
       rcostctr like RK23F-EKOSTL,
       rintorder like RK23F-EAUFNR,
     end of t_flatfile.
data: begin of t_flatfile1,
      docdate like COHEADER-BLDAT,
      postdate like COHEADER-BUDAT,
      doctext like COHEADER-BLTXT,
       costele like RK23F-KSTAR,
       amount like RK23F-WTGBTR,
       scostctr like RK23F-SKOSTL,
       rcostctr like RK23F-EKOSTL,
       rintorder like RK23F-EAUFNR,
       NUM LIKE SY-INDEX,
     end of t_flatfile1.
data: itab like table of t_flatfile with header line.
data: itab2 like table of t_flatfile1 with header line.
DATA: it_itab1 TYPE STANDARD TABLE OF t_itab1 WITH HEADER LINE,
      MESSTAB1 LIKE BDCMSGCOLL OCCURS 0 WITH HEADER LINE,
      MESSTAB LIKE BDCMSGCOLL OCCURS 0 WITH HEADER LINE.
data: begin of bdcdata occurs 0.
        include structure bdcdata.
data: end of bdcdata.
data:t_lin type i VALUE '0',
     u_rec type i VALUE '0',
     s_rec type i VALUE '0'.
data: it_csks type standard table of t_csks,
      wa_csks type t_csks.
data: it_cska type standard table of t_cska,
      wa_cska type t_cska.
*Selection Screen
SELECTION-SCREEN BEGIN OF BLOCK b1 WITH FRAME TITLE text-010.
parameters: p_docdat LIKE COHEADER-BLDAT obligatory,
            p_postda LIKE COHEADER-BUDAT obligatory,
            p_doctxt LIKE COHEADER-BLTXT.
SELECTION-SCREEN END OF BLOCK b1.
SELECTION-SCREEN BEGIN OF BLOCK b2 WITH FRAME TITLE text-011.
parameters: p_file LIKE RLGRAP-FILENAME obligatory,
            DIS_MODE LIKE CTU_PARAMS-DISMODE DEFAULT 'N'.
SELECTION-SCREEN END OF BLOCK b2.
              A T S E L E C T I O N   S C R E E N                  *
AT SELECTION-SCREEN ON VALUE-REQUEST FOR p_file.
PERFORM get_local_file_name USING p_file.
*Start of Selection
START-OF-SELECTION.
Perform get_Excel_data.
perform validate_data.
Perform Process_Data.
                end-of-selection
end-of-selection.
perform display_data.
*&      Form get_local_file_name
      text
     -->P_P_FILE text
FORM get_local_file_name USING    P_P_FILE.
CALL FUNCTION 'KD_GET_FILENAME_ON_F4'
    CHANGING
      file_name = p_file.
ENDFORM.                    " get_local_file_name
*&      Form get_Excel_data
      text
--> p1        text
<-- p2        text
FORM get_Excel_data .
FIELD-SYMBOLS : <FS>.
DATA : V_INDEX TYPE I.
CALL FUNCTION 'ALSM_EXCEL_TO_INTERNAL_TABLE'
    EXPORTING
      filename                = p_file
      i_begin_col             = 1
    i_begin_row             = 2
      i_begin_row             = 1
      i_end_col               = 256
      i_end_row               = 9999                        "65536
    TABLES
      intern                  = it_itab1
    EXCEPTIONS
      inconsistent_parameters = 1
      upload_ole              = 2
      OTHERS                  = 3.
IF sy-subrc <> 0.
MESSAGE ID SY-MSGID TYPE SY-MSGTY NUMBER SY-MSGNO
        WITH SY-MSGV1 SY-MSGV2 SY-MSGV3 SY-MSGV4.
    Message text-013 type 'E'.
ENDIF.
IF IT_ITAB1[] IS INITIAL.
    Message text-001 type 'E'.
else.                                    "IF IT_ITAB1[] IS INITIAL.
    data: itab2 like itab occurs 0 with header line.
    SORT IT_ITAB1 BY ROW COL.
    LOOP AT IT_ITAB1.
      MOVE :IT_ITAB1-COL TO V_INDEX.
      ASSIGN COMPONENT V_INDEX OF STRUCTURE itab2 TO <FS>.
      MOVE : IT_ITAB1-VALUE TO <FS>.
      AT END OF ROW.
        MOVE-CORRESPONDING itab2 TO itab.
        APPEND itab.
        CLEAR:itab,itab2.
      ENDAT.
    endloop.
    describe table itab lines t_lin.
endif.               "IF IT_ITAB1[] IS INITIAL.
ENDFORM.                    " get_Excel_data
*&      Form Process_Data
      text
--> p1        text
<-- p2        text
FORM Process_Data .
data:l_tabix type sy-tabix.
data:l_periv like t001-periv,
       l_monat like bkpf-monat,
       l_gjahr like bkpf-gjahr,
       l_amt(21) type c.
data: l_ddate(10),
        l_pdate(10).
WRITE p_docdat TO l_ddate.
WRITE p_postda TO l_pdate.
clear: l_periv,l_monat,l_gjahr.
select single periv from t001 into l_periv where bukrs = '5000'. "P_bukrs
if sy-subrc eq 0.
    l_gjahr = p_postda+0(4).
    call function 'FI_PERIOD_DETERMINE'
      EXPORTING
        i_budat = p_postda
        i_bukrs = '5000'     "p_bukrs
        i_periv = l_periv
        i_gjahr = l_gjahr
      IMPORTING
        e_monat = l_monat.
    clear:l_periv.
endif.
loop at itab2.
    refresh:bdcdata.
    clear:bdcdata.
    l_tabix = sy-tabix.
    perform bdc_dynpro      using 'SAPLK23F1' '1200'.
    perform bdc_field       using 'BDC_OKCODE'
                                  '/00'.
    perform bdc_field       using 'COHEADER-SEND_REC_REL'
                                  '10SAP'.
    perform bdc_field       using 'RK23F-STATUS'
                                  'S'.
    perform bdc_field       using 'COHEADER-BLDAT'
                             itab-docdate.
                                  l_ddate.
    perform bdc_field       using 'COHEADER-BUDAT'
                             itab-postdate.
                                  l_pdate.
    perform bdc_field       using 'COHEADER-PERIO'
                                   l_monat.                 "'9'.
    perform bdc_field       using 'COHEADER-BLTXT'
                             itab-doctext.
                                  p_doctxt.
    perform bdc_field       using 'RK23F-KSTAR'
                                  itab2-costele.
    WRITE itab2-amount TO l_amt.
l_amt = itab-amount.
    condense l_amt no-gaps.
    perform bdc_field       using 'RK23F-WTGBTR'
                                   l_amt.
                             itab-amount.
    perform bdc_field       using 'RK23F-WAERS'
                                  'USD'.
*perform bdc_field       using 'RK23F-SGTXT'
                             itab-doctext.
    perform bdc_field       using 'RK23F-SKOSTL'
                                  itab2-scostctr.
    perform bdc_field       using 'BDC_CURSOR'
                                  'RK23F-EAUFNR'.
    perform bdc_field       using 'RK23F-EKOSTL'
                                  itab2-rcostctr.
    perform bdc_field       using 'RK23F-EAUFNR'
                                  itab2-rintorder.
    perform bdc_dynpro      using 'SAPLK23F1' '1200'.
    perform bdc_field       using 'BDC_OKCODE'
                                  '=POST'.
    perform bdc_field       using 'COHEADER-SEND_REC_REL'
                                  '10SAP'.
    perform bdc_field       using 'RK23F-STATUS'
                                  'S'.
    perform bdc_field       using 'COHEADER-BLDAT'
                             itab-docdate.
                                  l_ddate.
    perform bdc_field       using 'COHEADER-BUDAT'
                              itab-postdate.
                                  l_pdate.
    perform bdc_field       using 'COHEADER-PERIO'
                             '9'.
                                    l_monat.
    perform bdc_field       using 'COHEADER-BLTXT'
                              itab-doctext.
                                  p_doctxt.
    perform bdc_field       using 'BDC_CURSOR'
                                  'RK23F-KSTAR'.
    perform bdc_field       using 'RK23F-WAERS'
                                  'USD'.
    CALL TRANSACTION 'KB15N' USING BDCDATA MODE DIS_MODE MESSAGES INTO MESSTAB.
    If sy-subrc = 0.
      s_rec = s_rec + 1.
    ELSE.
      u_rec = u_rec + 1.
      move ITAB2-NUM to messtab1-msgv1.
      concatenate itab2-costele ' | ' itab2-scostctr ' | ' itab2-rcostctr ' | ' itab2-rintorder into messtab1-msgv2.
      condense messtab1-msgv2.
      condense messtab1-msgv1.
      append messtab1.
    endif.
    clear:itab2.
endloop.
ENDFORM.                    " Process_Data
      BDC_DYNPRO                                                     *
FORM BDC_DYNPRO USING PROGRAM DYNPRO.
CLEAR BDCDATA.
BDCDATA-PROGRAM = PROGRAM.
BDCDATA-DYNPRO   = DYNPRO.
BDCDATA-DYNBEGIN = 'X'.
APPEND BDCDATA.
ENDFORM.                    "BDC_DYNPRO
       BDC_FIELD                                                     *
FORM BDC_FIELD USING FNAM FVAL.
IF FVAL <> ''. "NODATA.
    CLEAR BDCDATA.
    BDCDATA-FNAM = FNAM.
    BDCDATA-FVAL = FVAL.
    APPEND BDCDATA.
ENDIF.
ENDFORM.                    "BDC_FIELD
*&      Form display_data
      text
--> p1        text
<-- p2        text
FORM display_data .
skip 2.
write:/15 text-002.
skip 2.
write:/8 text-003.
SKIP.
write:/12 text-008,
         25 P_DOCDAT.
SKIP.
write:/12 text-009,
         25 P_POSTDA.
SKIP.
write:/12 text-012,
          25 P_DOCTXT.
SKIP.
write:/12 text-004,
         25 p_file.
skip 2.
write:/8 text-005,
        60 t_lin.
skip.
write:/8 text-006,
        60 s_rec.
skip.
write:/8 text-007,
        60 u_rec.
skip.
write:/10 'row no',
         20 'Information'.
skip.
loop at messtab1.
    write:/10 messtab1-msgv1,
           20 messtab1-msgv2.
    clear:messtab1.
endloop.
ENDFORM.                    " display_data
*&      Form validate_data
      text
--> p1        text
<-- p2        text
FORM validate_data .
data: l_tabix1 type sy-tabix.
data: l_tabix2 type sy-tabix.
if not itab[] is initial.
    select kostl from CSKS into table it_csks.
    if sy-subrc eq 0.
      sort it_csks by kostl.
    endif.
    select kstar from CSKA into table it_cska.
    if sy-subrc eq 0.
      sort it_cska by kstar.
    endif.
    loop at itab.
      l_tabix1 = sy-tabix.
CALL FUNCTION 'CONVERSION_EXIT_ALPHA_INPUT'
EXPORTING
    INPUT         = itab-scostctr
IMPORTING
   OUTPUT        = itab-scostctr .
      read table it_csks into wa_csks with key kostl = itab-scostctr.
      if sy-subrc ne 0.
        u_rec = u_rec + 1.
       L_TABIX2 = l_tabix1 + 1.
       move l_tabix2 to messtab1-msgv1.
        move l_tabix1 to messtab1-msgv1.
        move itab-rintorder to messtab1-msgv2.
        concatenate itab-costele ' | ' itab-scostctr ' | ' itab-rcostctr ' | ' itab-rintorder into messtab1-msgv2.
        condense messtab1-msgv2.
        condense messtab1-msgv1.
        append messtab1.
        clear:wa_csks.
       CLEAR:L_TABIX2.
        continue.
      endif.
CALL FUNCTION 'CONVERSION_EXIT_ALPHA_INPUT'
EXPORTING
    INPUT         = itab-rcostctr
IMPORTING
   OUTPUT        = itab-rcostctr .
      read table it_csks into wa_csks with key kostl = itab-rcostctr.
      if sy-subrc ne 0.
        u_rec = u_rec + 1.
       L_TABIX2 = l_tabix1 + 1.
       move l_tabix2 to messtab1-msgv1.
        move l_tabix1 to messtab1-msgv1.
          concatenate itab-costele ' | ' itab-scostctr ' | ' itab-rcostctr ' | ' itab-rintorder into messtab1-msgv2.
        condense messtab1-msgv2.
        condense messtab1-msgv1.
        append messtab1.
        clear:wa_csks.
       CLEAR:L_TABIX2.
        continue.
      endif.
CALL FUNCTION 'CONVERSION_EXIT_ALPHA_INPUT'
EXPORTING
    INPUT         = itab-costele
IMPORTING
   OUTPUT        = itab-costele .
      read table it_cska into wa_cska with key kstar = itab-costele.
      if sy-subrc ne 0.
        u_rec = u_rec + 1.
       L_TABIX2 = l_tabix1 + 1.
       move l_tabix2 to messtab1-msgv1.
        move l_tabix1 to messtab1-msgv1.
        concatenate itab-costele ' | ' itab-scostctr ' | ' itab-rcostctr ' | ' itab-rintorder into messtab1-msgv2.
        condense messtab1-msgv2.
        condense messtab1-msgv1.
        append messtab1.
        clear:wa_csks.
       CLEAR:L_TABIX2.
        continue.
      endif.
move-corresponding itab to itab2.
MOVE l_tabix1 TO ITAB2-NUM.
append itab2.
clear: itab2.
      clear:itab.
    endloop.
else.
    message 'No records in File' type 'S'.
endif.
ENDFORM.                    " validate_data

Reject columns which exceed a particular length during OWB flat file import

Hi,
I have a file in csv format. This file has a lot of columns out of which i need to use only a few. I created a file in OWB using the given csv file using the file import wizard. I am using this file to create an external table.
The import is working fine, But the problem is that certain columns (which are not required to be processed) have very huge data. This is leading to rejection of certain rows (when i deploy the external table) which have valid data for the required columns, but too huge data was columns which i do not need.
But these rows must not be rejected.
I wanted to know if there any way by which we can truncate/ insert null in these unwanted columns (either during flat file import / or during creation of external table), so that the row is taken in partially(with the wanted columns)?
Thanks a lot!
NS

Hi Julie,
As Jim posted, you could try a RAW file instead of a CSV file. In SSIS, the Raw File destination writes raw data to a file. Because the format of the data is native to the destination, the data requires no translation and little parsing. So does the Raw
File source. Besides, the Raw File destination/source can improve the package performance because they write data more quickly than other destinations such as the Flat File, the OLE DB, and Recordset destinations.
References:
http://www.katieandemil.com/ssis-raw-file-source-example-ssis-2012
http://www.jasonstrate.com/2011/01/31-days-of-ssis-raw-files-are-awesome-131/
Regards,
Mike Yin
TechNet Community Support

Transfer structure sequence for Flat File

I am wondering is it realy important to maintain particular sequence in DS/TR str. for InfoObjects which are set up for getting constant value in TR rules. I am having trouble loading flat file. TR str. has some compounded characteristics like 0Fisvar for 0Fiscper and 0Co_area for 0profit_ctr both getting fix value in transfer rules. how to maintain flat file and TR str. sequence?

Hi Vishan,
This is what I used to do.
If there is a compounded InfoObject or Key Figure that requires a unit (even though in your case, the compounded InfoCObject & the unita are always the same), You "need" add them in TS.
After adding them, this is what I will do:
Move them around to match the Flat File structure.
Now, the fields that are always constants & not coming from Flat Files, move them to the end.
They will be ignored. Your TR will populate them.
If you get an error now, that means your Flat File is not formatted correctly, apart from that, you are fine.
Ram Chamarthy
Message was edited by: Ram Chamarthy

What is the best way to load and convert data from a flat file?

Hi,
I want to load data from a flat file, convert dates, numbers and some fields with custom logic (e.g. 0,1 into N,Y) to the correct format.
The rows where all to_number, to_date and custom conversions succeed should go into table STG_OK. If some conversion fails (due to an illegal format in the flat file), those rows (where the conversion raises some exception) should go into table STG_ERR.
What is the best and easiest way to archive this?
Thanks,
Carsten.

Hi,
thanks for your answers so far!
I gave them a thought and came up with two different alternatives:
Alternative 1
I load the data from the flat file into a staging table using sqlldr. I convert the data to the target format using sqlldr expressions.
The columns of the staging table have the target format (date, number).
The rows that cannot be loaded go into a bad file. I manually load the data from the bad file (without any conversion) into the error table.
Alternative 2
The columns of the staging table are all of type varchar2 regardless of the target format.
I define data rules for all columns that require a later conversion.
I load the data from the flat file into the staging table using external table or sqlldr without any data conversion.
The rows that cannot be loaded go automatically into the error table.
When I read the data from the staging table, I can safely convert it since it is already checked by the rules.
What I dislike in alternative 1 is that I manually have to create a second file and a second mapping (ok, I can automate this using OMB*Plus).
Further, I would prefer using expressions in the mapping for converting the data.
What I dislike in alternative 2 is that I have to create a data rule and a conversion expression and then keep the data rule and the conversion expression in sync (in case of changes of the file format).
I also would prefer to have the data in the staging table in the target format. Well, I might load it into a second staging table with columns having the target format. But that's another mapping and a lot of i/o.
As far as I know I need the data quality option for using data rules, is that true?
Is there another alternative without any of these drawbacks?
Otherwise I think I will go for alternative 1.
Thanks,
Carsten.

Regarding flat file load to DSO...

Hi Experts,
I am trying to load transaction data from flat file to DSO in BI7.0 that comprises two characteristics and one keyfigure. The characteristics type is char and keyfigure's type is int. Till PSA, data is fine. In transformation, I have just done one-to-one mapping with source field and target field, but while executing DTP, the error popped up as follows,
Data package 1:Errors during processing
Extraction completed, processing termination
Please help. I tried various combinations, but not able to schedule. Valuable answers would be awarded remarkably.
Thanks,

Hi there...
Here are some ideas to Ur problem
Assuming that u defined info objects correctly, length and types... and u mapped transformations correctly
And ur data in flat file is absolutely correct...
Of course very common but suggestible is plz check the data lengths in flat file and defined length of the info object.. If yes
Come to Data source and check the settings for the flat file ,separater and escape sign and rows to ignored..and check the data preview...
Plz compare the data in PSA and Flat file... plz do this very carefully... If yes...
Go to data source and at the FIELDS tab and check whether u assigned info objects correctly...if yes...
Check the conversion routines in the same screen of the tab FIELDS.. if for key figure u see PER16.. Remove it and try to load to PSA and do load the cube..
Hope this can give u an idea...
Regrds...KP

Flat file in http folder

Hi,
Is there anyway of downloading a flat file into a folder in some http address rather than the application server or local directory.
Regards,
Sukumar.

You can run the VB script on any PC where the SAP Gateway and SDK have been installed.
I am working on a project where we use vbscript and ASP to extract data from SAP and use it on the intranet.
You can use the VB-Script in an excel or word document and also on a webpage. The example below I have used in a excel spreadsheet (use a macro to call the subroutine).
I hope this helps you.
Example below retrieves a list of vendors.
VBScript to list vendors from SAP table LFA1
Sub GetTable()
'Logon
Dim sapConn As Object 'Declare variant
Set sapConn = CreateObject("SAP.Functions") 'Create ActiveX object
If sapConn.Connection.Logon(0, False) <> True Then 'Try Logon
   MsgBox "Cannot Log on to SAP"
End If
'Define function
Dim objRfcFunc As Object
Set objRfcFunc = sapConn.Add("RFC_READ_TABLE")
'Set import parameters
Dim objQueryTab, objRowCount As Object
'Table name
Set objQueryTab = objRfcFunc.Exports("QUERY_TABLE")
objQueryTab.Value = "LFA1"
'Set max nr of rows
Set objRowCount = objRfcFunc.Exports("ROWCOUNT")
objRowCount.Value = "10"
'Set table parameters
Dim objOptTab, objFldTab, objDatTab As Object
Set objOptTab = objRfcFunc.Tables("OPTIONS")
Set objFldTab = objRfcFunc.Tables("FIELDS")
Set objDatTab = objRfcFunc.Tables("DATA")
'First we set the condition
objOptTab.FreeTable    'Refresh table
'Then set values where statement (TEXT field in table parameter OPTIONS)
objOptTab.Rows.Add
objOptTab(objOptTab.RowCount, "TEXT") = "KTOKK = 'HSUB' and "
objOptTab.Rows.Add
objOptTab(objOptTab.RowCount, "TEXT") = "ORT01 = 'London'"
'Next we set fields to obtain (fields we want to retrieve from the table)
objFldTab.FreeTable    'Refresh table
'Then set values (FIELDNAME field in table parameter FIELDS)
objFldTab.Rows.Add
objFldTab(objFldTab.RowCount, "FIELDNAME") = "LIFNR" 'vendor nr
objFldTab.Rows.Add
objFldTab(objFldTab.RowCount, "FIELDNAME") = "NAME1" 'vendor name
objFldTab.Rows.Add
objFldTab(objFldTab.RowCount, "FIELDNAME") = "ORT01" 'city
objFldTab.Rows.Add
objFldTab(objFldTab.RowCount, "FIELDNAME") = "ADRNR" 'address nr
objFldTab.Rows.Add
objFldTab(objFldTab.RowCount, "FIELDNAME") = "ERNAM" 'user who created the vendor
If objRfcFunc.Call = False Then
   MsgBox objRfcFunc.Exception
End If
Dim objDatRec As Object
Dim objFldRec As Object
For Each objDatRec In objDatTab.Rows
   For Each objFldRec In objFldTab.Rows
      Cells(objDatRec.Index, objFldRec.Index) = Mid(objDatRec("WA"), objFldRec("OFFSET") + 1, objFldRec("LENGTH"))
   Next
Next
End Sub

Unsorted Flat File into IDoc with multiple use of nodes

Hi Experts!
I am facing a little problem. I have a source flat file for a classification where some fields appear several times.
My source flat file looks like this:
item1; field1a
item2; fieldA
item3; fieldxa
item1; field1b
as you can see the item1 exists twice (further appearances are also possible).
Now i have to map the flat file into an IDoc structure
My target IDoc looks like this
Header
-- node1
attribute1
-- node2
the "field1a" and "field1b" has to be mapped into the "attribute1" in "node1". "node1" has to be duplicated for each time an "item1" appears (.. and if item2, item3 etc. appears twice, three ... four times...).
So how can i reach it that the node1 will be duplicated automatically when an item appears twice or more times? I know that it could be possible to work with "SplitByValue"... but for this i need all item1 in an straight order.... but i dont have them in a correct order.
I am looking forward to your suggestions.
Thank you in advance.
Udo

Complex sorting is not or not easy possible with the grafical mapping tool.
Use a sequence mapping. The first mapping is a simple XSLT which does the sort. The second mapping works as usual.
I have an example XSLT which I used for a different purpose:
<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet version="1.0"
xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
<xsl:template match="/">
    <ORDERLIST>
      <xsl:for-each select="ORDERLIST/ITEM">
        <xsl:sort select="ID"/>
        <xsl:copy-of select="."/>
      </xsl:for-each>
    </ORDERLIST>
</xsl:template>
</xsl:stylesheet>
Regards
Stefan

How to built a hierarchy with flat file?

Hi Experts,
for a better representation of the report, i need to built a hierarchy with flat file.
The Data load is via Flat file.
Do you have some documentation about that?
Thanks
Cheers
Gilo

Hi Gilo,
Check these docs...
<a href="https://www.sdn.sap.comhttp://www.sdn.sap.comhttp://www.sdn.sap.com/irj/servlet/prt/portal/prtroot/docs/library/uuid/0403a990-0201-0010-38b3-e1fc442848cb">How to Download a Hierarchy to a Flat File</a>
<a href="http://help.sap.com/saphelp_nw04/helpdata/en/fa/e92637c2cbf357e10000009b38f936/content.htm">Uploading Hierarchies from Flat Files</a>
<a href="http://sapbwneelam.blogspot.com">More Docs...</a>
Hope it Helps
Srini

Call Pipeline in orchestration for converting XML to flat file

Hi ,
Can we call Pipeline in orchestration for converting XML to flat file ?
Or we have some another option for this task ?
Prakash

Hi,
You can refer this article from Abdul Rafay on how to work with Flat files and how to call receive and send pipelines within your orchestration. He
has provided detailed step by step explanation of the process.
Calling Send and Receive Pipelines
from the orchestration expression shapes
The code shared by Abdul at http://abdulrafaysbiztalk.wordpress.com/files/2009/06/processingflatfiles.doc (Change the extention to rar) and try it.
One common mistake I have seen people doing will working with Flat File Assembler is that they miss to set the Document schema property in the Properties
window to the flat file schema.
If the Document schema property is not specified, runtime schema discovery will be attempted. During schema discovery, BizTalk Server attempts
to determine the correct flat file schema to assemble the message with, based on the namespace and root node of the message, its added performance hit.
Rachit

Format of flat file!!!

hello friends!!!
We are uploading opportunities in our system from a flat file. I am using LSMW with CRMXIF_ORDER_save for that. All has been the test data is working fine.
But the problem i am facing is in which format the file should be for?? in regards to item and partner function?? one opportunity can have several products and many partner functions.... does it should have a seperate field name for each item and each partner function..???
Please help its a bit urgent and also if you have a sample file please send to me on [email protected]
i hope iwas able to define my requirement clearly;
Many many thanks!!
Pankaj

This is now going into a general discussion please input your views.
In my opinion the flat file will have all the fields sequentially including all the items. but the problem is if the company has 50 or more products to offer then it will be a mess in the flat file having 50 items in a row and then only have value for the one product we are offering to a particular opportunity.
My question can´t we somehow break flat files?? or have some solution for this??
many thanks
Pankaj

Ssis flat file to ole db skipping some rows...

Similar Messages

Maybe you are looking for