Flat File Split

Hi,
I will be loading more than 3 million records from a flat file every week.
I tuned RSCUSTV6, the infocube dimensions (creating line-itemsdims), use number range buffering, have a fixed length ASCII file, load Target and PSA in parallell. The load is now taking 2 hours.
In order to speed this up I want to split the Flat file in 4 pieces and load these in parallell. However, these loads are locking each other (SM12, ENQUEUE) when I start them manually.
What to do?
SOlved the problem by changing the setting in the infopackage; Only load Data Target. The load now takes 30 min. for 3million records
Message was edited by: F. Padt

Hi,
If that's happening then I would suggest to try different upload methods (try to put your files on the application server and load them in background).
If that's not working then I would try to create 4 different flatfile infosources (and datasources) to be able to load one piece from each infosource.
Hope it helps,
David.

Similar Messages

Flat file splitting

Can i upload a flat file which contains entries around 12 lakhs.where in when i am trying to upload with ws_upload it is reading the entries but thrwoing error when trying to open a file.
Is there any way so that the flat file can be splitted in to parts and do the validation

Hi Teja
here check this program i am spliting the record depend upon ',' and dont use ws_upload etc fm's are obsolete....
data: begin of itab_string occurs 0,
      record type char255,
      end of itab_string.
data: L_FILETABLE TYPE FILETABLE,
L_FILETAB_H TYPE FILETABLE WITH HEADER LINE.
data: p_file1 type string.
selection screen .
PARAMETERS: P_FILE TYPE LOCALFILE.
initialization.
at selection-screen on value-request for P_FILE.
IF THE USER SELECT EXTENTION BUTTON IT WILL OPEN THE LOCAL DIRECTORY FOR SELECTING THE FILE LOCATION.
CALL METHOD CL_GUI_FRONTEND_SERVICES=>FILE_OPEN_DIALOG
EXPORTING
   WINDOW_TITLE            =
   DEFAULT_EXTENSION       = 'CSV'
   DEFAULT_FILENAME        = 'C:\Documents and Settings\196093\Desktop\STATUS.csv'
   FILE_FILTER             =
   INITIAL_DIRECTORY        = 'C:\Documents and Settings\196093\Desktop\'
   MULTISELECTION          =
   WITH_ENCODING           =
CHANGING
    FILE_TABLE              = L_FILETABLE
    RC                      = RC
   USER_ACTION             =
   FILE_ENCODING           =
EXCEPTIONS
    FILE_OPEN_DIALOG_FAILED = 1
    CNTL_ERROR              = 2
    ERROR_NO_GUI            = 3
    NOT_SUPPORTED_BY_GUI    = 4
    others                  = 5
IF SY-SUBRC <> 0.
ELSE.
LOOP AT l_filetable INTO L_FILETAB_H.
P_FILE = L_FILETAB_H-FILENAME.
move p_file to p_file1.
EXIT.
ENDLOOP.
ENDIF.
passing the selected file name to gui_upload for loading the data
into internal table
CALL FUNCTION 'GUI_UPLOAD'
EXPORTING
    FILENAME                      = p_file1
FILETYPE                      = 'ASC'
HAS_FIELD_SEPARATOR           = ' '
HEADER_LENGTH                 = 0
READ_BY_LINE                  = 'X'
DAT_MODE                      = ' '
CODEPAGE                      = ' '
IGNORE_CERR                   = ABAP_TRUE
REPLACEMENT                   = '#'
CHECK_BOM                     = ' '
NO_AUTH_CHECK                 = ' '
IMPORTING
FILELENGTH                    =
HEADER                        =
TABLES
    DATA_TAB                      = itab_string
EXCEPTIONS
   FILE_OPEN_ERROR               = 1
   FILE_READ_ERROR               = 2
   NO_BATCH                      = 3
   GUI_REFUSE_FILETRANSFER       = 4
   INVALID_TYPE                  = 5
   NO_AUTHORITY                  = 6
   UNKNOWN_ERROR                 = 7
   BAD_DATA_FORMAT               = 8
   HEADER_NOT_ALLOWED            = 9
   SEPARATOR_NOT_ALLOWED         = 10
   HEADER_TOO_LONG               = 11
   UNKNOWN_DP_ERROR              = 12
   ACCESS_DENIED                 = 13
   DP_OUT_OF_MEMORY              = 14
   DISK_FULL                     = 15
   DP_TIMEOUT                    = 16
   OTHERS                        = 17
IF SY-SUBRC <> 0.
MESSAGE I000(Z00) WITH 'PLEASE PROVIDE CORRECT FILE NAME'.
ENDIF.
loop at itab_string.
now split the statuses
split itab_string at ',' into itab_status-aufnr itab_status-asttx itab_status-asttx1.
and move one internal table
append itab_status.
clear itab_status.
endloop.
reward points to all helpful answers
kiran.M

Flat File Splitting into two Messages in Orchestration

In BizTalk, Orchestration receives the following formatted file. After receiving the e file AS-IS to go to one subscriber and for another subscriber where Mode=”N” contains in a row those rows should be go as a another file.
Please let me know how to implement this scenario
INPUTFILE
InvoiceNo InvoiceDate Mode Amount
1000            20042015    Y          1244
1001            20042015 N         2345
1003            20042015    N          1244
1004            20042015   Y       2345
1005            20042015    Y          1244
10046           20042015   N     2345
From Above File,expecting two output files
One File OutPut
1000            20042015    Y          1244
1001            20042015 N         2345
1003            20042015    N          1244
1004            20042015   Y       2345
1005            20042015    Y          1244
10046           20042015   N       2345
Second File Output
1001            20042015 N         2345
1003            20042015    N          1244
10046           20042015   N       2345

Hi BizQ.
If you are receiving the flat file directly in the orchestration, it will be very dificult to apply rules based on its content. Probably you will need to pass the entire message content to a C# helper that will read it and return the rows that will
be sent to each subscriber.
A better option could be creating a custom pipeline component that will be executed in the receive port. this component will process the entire document and will create a new document that will contain the rows with the Mode value set to N. As a result it
will return the two documents required by the subscribers.
Other option could be disassembling the request document in a hierarchical XML structure before it's processed inside the orchestration. The orchestration will generate the different documents that will be sent to each subscriber based on the
Mode property of each row. Next use a assembler pipeline to build the flat files again before sending to the target subscribers.
Regards.

SSIS Script Component Conditional Split to Flat File Destination

I have a flat file which needs to be split into multiple flat files based on value in RecordType column.
For example, if (RecordType == 20), then direct all rows to a new text file,
I have around 15 different record types. I have managed to write some C# code for Conditional Split, but
still trying to figure out what is the next step to save these rows to a text file.
I will be grateful if someone please point me to the right direction.
Many Thanks
#region Namespaces
using System;
using System.Data;
using Microsoft.SqlServer.Dts.Pipeline.Wrapper;
using Microsoft.SqlServer.Dts.Runtime.Wrapper;
using System.IO;
#endregion
[Microsoft.SqlServer.Dts.Pipeline.SSISScriptComponentEntryPointAttribute]
public class ScriptMain : UserComponent
string copiedAddressFile;
private StreamWriter textWriter;
private string columnDelimiter = ",";
private string filepath = @"C:\DestFiles";
private string[] columns;
public override void PreExecute()
IDTSInput100 input = ComponentMetaData.InputCollection[0];
columns = new string[input.InputColumnCollection.Count];
for (int i = 0; i < input.InputColumnCollection.Count; i++)
columns[i] = input.InputColumnCollection[i].Name;
public override void Input0_ProcessInputRow(Input0Buffer Row)
if (Row.intRecordType ==20)
Row.DirectRowToRecordType20();
else if (Row.intRecordType ==10)
Row.DirectRowToRecordType10();

see similar example
http://www.sqlis.com/sqlis/post/Using-the-Script-Component-as-a-Conditional-Split.aspx
Please Mark This As Answer if it solved your issue
Please Mark This As Helpful if it helps to solve your issue
Visakh
My MSDN Page
My Personal Blog
My Facebook Page

How to split column wise into separate flat files in ssis

IN SSIS...
1.---->I have a sales table country wise regions like (india, usa, srilanka) ....
india usa
srilanka
a b
c
d e
f
so i want output like in
flat file1.txt has india flat file2.txt has usa flat file3.txt has srilanka
a b
c
d e
f
2.----->i dont know how many regions in my table....dynamically split into separate flat files ....
pls help me.....thank u

I think what you can do is this
1. Do a query based on UNPIVOT to get the data as rows instead of columns
For that you can use a query like this
IF OBJECT_ID('temp') IS NOT NULL DROP TABLE temp
CREATE TABLE temp
Country varchar(100),
Val decimal(25,5)
DECLARE @CountryList varchar(3000),@SQL varchar(max)
SELECT @CountryList = STUFF((SELECT ',[' + Column_Name + ']' FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_NAME = '<SalesTableNameHere>' FOR XML PATH('')),1,1,'')
SET @SQL= 'SELECT * FROM <SalesTableNameHere> t UNPIVOT (Val FOR Country IN (' + @CountryList + '))p'
INSERT temp
EXEC (@SQL)
Once this is done you'll get data unpivoted to table
Then you can use a execute sql task with query like above
SELECT DISTINCT Country FROM Temp
Use resultset option as full resultset and store the result to an object variable
Then add a ForEach loop container with ADO enumerator and map to the object variable created above. Have variables inside loop to get inidvidual country values out.
Inside loop place a data flow task. Use a variable to store source query , make EvaluateAsExpression true for it and set Expression as below
"SELECT Val FROM Temp WHERE Country = " + @[User::LoopVariable]
Where LoopVariable is variable created inside loop for getting iterated values
Inside data flow task place a oledb source, choose option as SQL command from variable and map to the above query variable.
Link this to flat file destination create a flat file connection manager. Set a dynamic flat file connection using expression builder. Make it based on a variable and set variable to increment based on loop iteration
The core logic looks similar to this
http://visakhm.blogspot.ae/2013/09/exporting-sqlserver-data-to-multiple.html
dynamic file naming can be seen here
http://jahaines.blogspot.ae/2009/07/ssis-dynamically-naming-destination.html
Please Mark This As Answer if it solved your issue
Please Vote This As Helpful if it helps to solve your issue
Visakh
My Wiki User Page
My MSDN Page
My Personal Blog
My Facebook Page

Split flat file column data into multiple columns using ssis

Hi All, I need one help in SSIS.
I have a source file with column1, I want to split the column1 data into
multiple columns when there is a semicolon(';') and there is no specific
length between each semicolon,let say..
Column1:
John;Sam;Greg;David
And at destination we have 4 columns let say D1,D2,D3,D4
I want to map
John -> D1
Sam->D2
Greg->D3
David->D4
Please I need it ASAP
Thanks in Advance,
RH
sql

Imports System
Imports System.Data
Imports System.Math
Imports Microsoft.SqlServer.Dts.Pipeline.Wrapper
Imports Microsoft.SqlServer.Dts.Runtime.Wrapper
Imports System.IO
Public Class ScriptMain
Inherits UserComponent
Private textReader As StreamReader
Private exportedAddressFile As String
Public Overrides Sub AcquireConnections(ByVal Transaction As Object)
Dim connMgr As IDTSConnectionManager90 = _
Me.Connections.Connection
exportedAddressFile = _
CType(connMgr.AcquireConnection(Nothing), String)
End Sub
Public Overrides Sub PreExecute()
MyBase.PreExecute()
textReader = New StreamReader(exportedAddressFile)
End Sub
Public Overrides Sub CreateNewOutputRows()
Dim nextLine As String
Dim columns As String()
Dim cols As String()
Dim delimiters As Char()
delimiters = ",".ToCharArray
nextLine = textReader.ReadLine
Do While nextLine IsNot Nothing
columns = nextLine.Split(delimiters)
With Output0Buffer
cols = columns(1).Split(";".ToCharArray)
.AddRow()
.ID = Convert.ToInt32(columns(0))
If cols.GetUpperBound(0) >= 0 Then
.Col1 = cols(0)
End If
If cols.GetUpperBound(0) >= 1 Then
.Col2 = cols(1)
End If
If cols.GetUpperBound(0) >= 2 Then
.Col3 = cols(2)
End If
If cols.GetUpperBound(0) >= 3 Then
.Col4 = cols(3)
End If
End With
nextLine = textReader.ReadLine
Loop
End Sub
Public Overrides Sub PostExecute()
MyBase.PostExecute()
textReader.Close()
End Sub
End Class
Put this code in ur script component. Before that add 5 columns to the script component output and name them as ID, col1, co2..,col4. ID is of data type int. Create a flat file destination and name it as connection and point it to the flat file as the source.
Im not sure whats the delimiter in ur flat file between the 2 columns. I have use a comma change it accordingly.
This is the output I get:
ID Col1
Col2 Col3
Col4
1 john
Greg David
Sam
2 tom
tony NULL
NULL
3 harry
NULL NULL
NULL

Splitting flat file data

I want to store the flat file data for temporary purpose and as a backup also, but its size is so huge that processing them at one instance is a difficult task. Is there any 'Z' program that will split the records puts it in a new file. Anyone please give an idea and your answer ll b rewarded with maximum points if it helps me on this issue.

Hi Manjula,
Check out the below program this will help you solve your requirement, you split the records as per the limit specified and puts them in a new file. Think so it help you.
REPORT zc1_split_file MESSAGE-ID ztestmsg.
TABLES: mara.
DATA: BEGIN OF input,
mandt LIKE mara-mandt,
matnr LIKE mara-matnr,
ersda LIKE mara-ersda,
ernam LIKE mara-ernam,
matkl LIKE mara-matkl,
END OF input.
DATA: i_mara_tab LIKE TABLE OF input WITH HEADER LINE,
i_mara_temp LIKE TABLE OF input,
w_mara_tab LIKE LINE OF i_mara_tab,
v_newfile(120) TYPE c,
v_no_lines TYPE i,
v_split(4) TYPE n VALUE 1,
v_count(4) TYPE n VALUE 1.
SELECTION-SCREEN BEGIN OF BLOCK b1 WITH FRAME TITLE text-001.
PARAMETERS: p_file TYPE rlgrap-filename MEMORY ID file,
p_count(4) TYPE n.
SELECTION-SCREEN END OF BLOCK b1.
AT SELECTION-SCREEN ON VALUE-REQUEST FOR p_file.
Diplaying the dialog box for opening the file ************
CALL FUNCTION 'WS_FILENAME_GET'
EXPORTING
mask = '.,..'
IMPORTING
filename = p_file.
START-OF-SELECTION.
IF p_file IS INITIAL. " check to ensure file.
MESSAGE i001.
EXIT.
ENDIF.
IF p_count IS INITIAL. " check to ensure file.
MESSAGE i002.
EXIT.
ENDIF.
CALL FUNCTION 'WS_UPLOAD'
EXPORTING
filename = p_file
filetype = 'DAT'
TABLES
data_tab = i_mara_tab
EXCEPTIONS
conversion_error = 1
file_open_error = 2
file_read_error = 3
invalid_type = 4
no_batch = 5
unknown_error = 6
invalid_table_width = 7
gui_refuse_filetransfer = 8
customer_error = 9
OTHERS = 10.
IF sy-subrc <> 0.
MESSAGE ID sy-msgid TYPE sy-msgty NUMBER sy-msgno
WITH sy-msgv1 sy-msgv2 sy-msgv3 sy-msgv4.
ENDIF.
LOOP AT i_mara_tab INTO w_mara_tab.
IF v_count < p_count.
APPEND w_mara_tab TO i_mara_temp.
v_count = v_count + 1.
ELSE.
APPEND w_mara_tab TO i_mara_temp.
v_count = v_count + 1.
PERFORM split_records.
REFRESH i_mara_temp.
v_count = 1.
v_split = v_split + 1.
ENDIF.
ENDLOOP.
IF v_count NE 1.
PERFORM split_records.
ENDIF.
--> p1 text
<-- p2 text
FORM split_records.
CONCATENATE p_file v_split INTO v_newfile.
CALL FUNCTION 'WS_DOWNLOAD'
EXPORTING
filename = v_newfile
filetype = 'DAT'
mode = 'A'
TABLES
data_tab = i_mara_temp
EXCEPTIONS
file_open_error = 1
file_write_error = 2
invalid_filesize = 3
invalid_type = 4
no_batch = 5
unknown_error = 6
invalid_table_width = 7
gui_refuse_filetransfer = 8
customer_error = 9
no_authority = 10
OTHERS = 11.
IF sy-subrc <> 0.
MESSAGE ID sy-msgid TYPE sy-msgty NUMBER sy-msgno
WITH sy-msgv1 sy-msgv2 sy-msgv3 sy-msgv4.
ENDIF.
WRITE:/'Data has been written into:' , v_newfile.
ENDFORM.
**Reward points if found useful....
Regards,
Manikandan.A

Issue with uploading flat file from App Server

Hi,
Im loading a flat file to an ODS.
When I try to load in from my Local Workstation, it works fine.
All data is loaded green.
However, when I try to load the file after i copied it to my App Server, it cannot load.
The error is get is:
Error 2 while splitting CSV data: see long text
Message no. RSDS_ACCESS011
Diagnosis
Error 2 occurred while splitting the CSV data string.
1 = Could not find a closing escape character
2 = Invalid escape character
3 = Conversion error
4 = OTHERS
The thing is that it worked fine (exact same file no changes) when I loaded from my Desktop.
Appreciate any help or input.
Rgds,
Gary.

Gary,
You can load it from the App server. Just check the file name whther you are giving it correctly and also check the readio button from App Server.
Go to AL11 (Tcode) and cross check whther your file is there. See in which directory your file is there. Mention the same path in the extraction tab in IP.
Thanks...
Shambhu

How to find out duplicate record contained in a flat file

Hi Experts,
For my project I have written a program for flat file upload.
Requirement 1
In the flat file there may be some duplicate record like:
Field1   Field2
11        test1
11        test2
12        test3
13        test4
Field1 is primary key.
Can you please let me know how I can find out the duplicate record.
Requirement 2
The flat file contains the header row as shown above
Field1   Field2
How our program can skip this record and start reading / inserting records from row no 2 ie
11        test1
onwards.
Thanks
S
FORM upload1.
DATA : wf_title TYPE string,
lt_filetab TYPE filetable,
l_separator TYPE char01,
l_action TYPE i,
l_count TYPE i,
ls_filetab TYPE file_table,
wf_delemt TYPE rollname,
wa_fieldcat TYPE lvc_s_fcat,
tb_fieldcat TYPE lvc_t_fcat,
rows_read TYPE i,
p_error TYPE char01,
l_file TYPE string.
DATA: wf_object(30) TYPE c,
wf_tablnm TYPE rsdchkview.
wf_object = 'myprogram'.
DATA i TYPE i.
DATA:
lr_mdmt TYPE REF TO cl_rsdmd_mdmt,
lr_mdmtr TYPE REF TO cl_rsdmd_mdmtr,
lt_idocstate TYPE rsarr_t_idocstate,
lv_subrc TYPE sysubrc.
TYPES : BEGIN OF test_struc,
/bic/myprogram TYPE /bic/oimyprogram,
txtmd TYPE rstxtmd,
END OF test_struc.
DATA : tb_assum TYPE TABLE OF /bic/pmyprogram.
DATA: wa_ztext TYPE /bic/tmyprogram,
myprogram_temp TYPE ziott_assum,
wa_myprogram TYPE /bic/pmyprogram.
DATA : test_upload TYPE STANDARD TABLE OF test_struc,
wa2 TYPE test_struc.
DATA : wa_test_upload TYPE test_struc,
ztable_data TYPE TABLE OF /bic/pmyprogram,
ztable_text TYPE TABLE OF /bic/tmyprogram,
wa_upld_text TYPE /bic/tmyprogram,
wa_upld_data TYPE /bic/pmyprogram,
t_assum TYPE ziott_assum.
DATA : wa1 LIKE test_upload.
wf_title = text-026.
CALL METHOD cl_gui_frontend_services=>file_open_dialog
EXPORTING
window_title = wf_title
default_extension = 'txt'
file_filter = 'Tab delimited Text Files (*.txt)'
CHANGING
file_table = lt_filetab
rc = l_count
user_action = l_action
EXCEPTIONS
file_open_dialog_failed = 1
cntl_error = 2
OTHERS = 3. "#EC NOTEXT
IF sy-subrc 0.
EXIT.
ENDIF.
LOOP AT lt_filetab INTO ls_filetab.
l_file = ls_filetab.
ENDLOOP.
CHECK l_action = 0.
IF l_file IS INITIAL.
EXIT.
ENDIF.
l_separator = 'X'.
wa_fieldcat-fieldname = 'test'.
wa_fieldcat-dd_roll = wf_delemt.
APPEND wa_fieldcat TO tb_fieldcat.
CALL FUNCTION 'MESSAGES_INITIALIZE'.
CLEAR wa_test_upload.
Upload file from front-end (PC)
File format is tab-delimited ASCII
CALL FUNCTION 'GUI_UPLOAD'
EXPORTING
filename = l_file
has_field_separator = l_separator
TABLES
data_tab = i_mara
data_tab = test_upload
EXCEPTIONS
file_open_error = 1
file_read_error = 2
no_batch = 3
gui_refuse_filetransfer = 4
invalid_type = 5
no_authority = 6
unknown_error = 7
bad_data_format = 8
header_not_allowed = 9
separator_not_allowed = 10
header_too_long = 11
unknown_dp_error = 12
access_denied = 13
dp_out_of_memory = 14
disk_full = 15
dp_timeout = 16
OTHERS = 17.
IF sy-subrc 0.
EXIT.
ELSE.
CALL FUNCTION 'MESSAGES_INITIALIZE'.
IF test_upload IS NOT INITIAL.
DESCRIBE TABLE test_upload LINES rows_read.
CLEAR : wa_test_upload,wa_upld_data.
LOOP AT test_upload INTO wa_test_upload.
CLEAR : p_error.
rows_read = sy-tabix.
IF wa_test_upload-/bic/myprogram IS INITIAL.
p_error = 'X'.
MESSAGE s153 WITH wa_test_upload-/bic/myprogram sy-tabix.
CONTINUE.
ELSE.
TRANSLATE wa_test_upload-/bic/myprogram TO UPPER CASE.
wa_upld_text-txtmd = wa_test_upload-txtmd.
wa_upld_text-txtsh = wa_test_upload-txtmd.
wa_upld_text-langu = sy-langu.
wa_upld_data-chrt_accts = 'xyz1'.
wa_upld_data-co_area = '12'.
wa_upld_data-/bic/zxyzbcsg = 'Iy'.
wa_upld_data-objvers = 'A'.
wa_upld_data-changed = 'I'.
wa_upld_data-/bic/zass_mdl = 'rrr'.
wa_upld_data-/bic/zass_typ = 'I'.
wa_upld_data-/bic/zdriver = 'yyy'.
wa_upld_text-langu = sy-langu.
MOVE-CORRESPONDING wa_test_upload TO wa_upld_data.
MOVE-CORRESPONDING wa_test_upload TO wa_upld_text.
APPEND wa_upld_data TO ztable_data.
APPEND wa_upld_text TO ztable_text.
ENDIF.
ENDLOOP.
DELETE ADJACENT DUPLICATES FROM ztable_data.
DELETE ADJACENT DUPLICATES FROM ztable_text.
IF ztable_data IS NOT INITIAL.
CALL METHOD cl_rsdmd_mdmt=>factory
EXPORTING
i_chabasnm = 'myprogram'
IMPORTING
e_r_mdmt = lr_mdmt
EXCEPTIONS
invalid_iobjnm = 1
OTHERS = 2.
CALL FUNCTION 'MESSAGES_INITIALIZE'.
**Lock the Infoobject to update
CALL FUNCTION 'RSDG_IOBJ_ENQUEUE'
EXPORTING
i_objnm = wf_object
i_scope = '1'
i_msgty = rs_c_error
EXCEPTIONS
foreign_lock = 1
sys_failure = 2.
IF sy-subrc = 1.
MESSAGE i107(zddd_rr) WITH wf_object sy-msgv2.
EXIT.
ELSEIF sy-subrc = 2.
MESSAGE i108(zddd_rr) WITH wf_object.
EXIT.
ENDIF.
*****Update Master Table
IF ztable_data IS NOT INITIAL.
CALL FUNCTION 'RSDMD_WRITE_ATTRIBUTES_TEXTS'
EXPORTING
i_iobjnm = 'myprogram'
i_tabclass = 'M'
I_T_ATTR = lt_attr
TABLES
i_t_table = ztable_data
EXCEPTIONS
attribute_name_error = 1
iobj_not_found = 2
generate_program_error = 3
OTHERS = 4.
IF sy-subrc 0.
CALL FUNCTION 'MESSAGE_STORE'
EXPORTING
arbgb = 'zddd_rr'
msgty = 'E'
txtnr = '054'
msgv1 = text-033
EXCEPTIONS
OTHERS = 3.
MESSAGE e054(zddd_rr) WITH 'myprogram'.
ELSE.
CALL FUNCTION 'MESSAGE_STORE'
EXPORTING
arbgb = 'zddd_rr'
msgty = 'S'
txtnr = '053'
msgv1 = text-033
EXCEPTIONS
OTHERS = 3.
ENDIF.
*endif.
*****update Text Table
IF ztable_text IS NOT INITIAL.
CALL FUNCTION 'RSDMD_WRITE_ATTRIBUTES_TEXTS'
EXPORTING
i_iobjnm = 'myprogram'
i_tabclass = 'T'
TABLES
i_t_table = ztable_text
EXCEPTIONS
attribute_name_error = 1
iobj_not_found = 2
generate_program_error = 3
OTHERS = 4.
IF sy-subrc 0.
CALL FUNCTION 'MESSAGE_STORE'
EXPORTING
arbgb = 'zddd_rr'
msgty = 'E'
txtnr = '055'
msgv1 = text-033
EXCEPTIONS
OTHERS = 3.
ENDIF.
ENDIF.
ELSE.
MESSAGE s178(zddd_rr).
ENDIF.
ENDIF.
COMMIT WORK.
CALL FUNCTION 'RSD_CHKTAB_GET_FOR_CHA_BAS'
EXPORTING
i_chabasnm = 'myprogram'
IMPORTING
e_chktab = wf_tablnm
EXCEPTIONS
name_error = 1.
IF sy-subrc 0.
MESSAGE ID sy-msgid TYPE sy-msgty NUMBER sy-msgno
WITH sy-msgv1 sy-msgv2 sy-msgv3 sy-msgv4.
ENDIF.
****Release locks on Infoobject
CALL FUNCTION 'RSDG_IOBJ_DEQUEUE'
EXPORTING
i_objnm = 'myprogram'
i_scope = '1'.
ENDIF.
ENDIF.
PERFORM data_selection .
PERFORM update_alv_grid_display.
CALL FUNCTION 'MESSAGES_SHOW'.
ENDFORM.

Can you please let me know how I can find out the duplicate record.
you need to split the records from flat file structure into your internal table ans use a delete ADJACENT duplicates comparing fields
split flat_str into wa_f1 wa_f2 wa_f2 at tab_space.

Build daily Big flat file using SAP R/3 material master data

Hi all,
I must build an interface between SAP R/3 4.7 and an external application
Daily a flat file must be generated contaning Material data in full exchange mode
(volume 100 000 materials, <10 fields => ~ File of 30Mo)
No complex mapping, only format mapping (Ex : Price quantity...)
I don't know yet if I will use Idocs or BAPI to extract data. However, I'm sure I must use multiple inbound messages because of volume
After reading documentation, I consider the both following solutions to manage it :
1) A simple asynchronous interface with a file adapter in construction mode APPEND to build flat file progressively with arrival of SAP messages
=>
viable solution ?
Is there problem of concurrent access to file in such a solution ?
(idocs or bapi data will be processed ones following the others )
2) BPM Collecting and Bundling Messages - One Interface
(Unlimited Loop with Deadline trigger correspondint to time I must build flat file)
=>
Each SAP message is a small message but the outbound XML will represent a 30 Mo flat file => Is it a problem ??
Which solution do you advise ?
Is there another ?
Last question : (more general one)
In such scenario, I can't see XI-value added comparing a creation of flat file directly with R/3.
I will generate a big number of idocs, use a big part of XI memory ...
Can you give me your opinion about it ?
Thanks a lot
Best regards
Jean-Charles
Message was edited by:
Jean-Charles
Message was edited by:
Jean-Charles

Hi Jean-Charles,
>>>volume 100 000 materials, <10 fields => ~ File of 30Mo
You have a performance task...
>>>Is there problem of concurrent access to file in such a solution ?
No, the adapter framework has its own queueing, the message would be stored and executed later. If sequence is important, you can use EOIO as well.
>>>BPM Collecting and Bundling Messages - One Interface
BPM is not required in your case, you should avoid becoz of performance task. You can append at file adapter, no problem. Bundling and splitting is even possible at Integration Server without BPM.
>>>Each SAP message is a small message but the outbound XML will represent a 30 Mo flat file => Is it a problem ??
Avoid using BAD words
>>>Each SAP message is a small message but the outbound XML will represent a 30 Mo flat file => Is it a problem ??
Use the <a href="https://websmp110.sap-ag.de/quicksizing">Quick Sizer</a> for sizing the hardware.
Regards,
Udo

How can we load a flat file with very, very long lines into a table?

Hello:
We have to load a flat file with OWB. The problem is that each of the lines in the file might be up to 30.000 characters long (up to 1.000 units of information in each line, 30 characters long each)
Of course, our mapping should insert these units of information as independent rows in a table (1.000 rows, in our example).
We do not know how to go about it. We usually load flat files using table functions, but we am not sure that they will be able to cope with these huge lines. And how should we pivot those lines? Will the Pivot operator do the trick? Or maybe we should pivot those lines outside the database before loading them?
We are a bit lost. Any suggestion would be appreciated.
Regards
Edited by: [email protected] on Oct 29, 2008 8:43 AM
Edited by: [email protected] on Oct 29, 2008 8:44 AM

Yes, well, we could define a 1.000 column external table, and then map those 1.000 columns to the Pivot operator… perhaps it would work. But we have been investigating a little bit, and we think that we have found a better solution: there is a unix utility called “fold”. This utility can split our 30.000 character lines in 1.000 lines, 30 characters long each: just what we needed. Then we can load the resulting file using an external table.
We think this is a much better solution that handling 1.000 columns in the external table and in the Pivot operator.
Thanks for your help.
Regards
Edited by: [email protected] on Oct 29, 2008 10:35 AM

How can I load a flat file into a ZTABLE dynamically

I need to create a program which can Load a ZTABLE from a flat file structure (delimited and fixed options required). We have many ZTables where this will be required so I was hoping to do it dynamically somehow. Otherwise I will have to create one ABAP for every ZTable we have to load.
My Inputs should be
PARAMETERS: p_ztable TYPE ddobjname,         "Z Table Name
             p_infile(132) LOWER CASE,        "File Name
             p_delim(1).                      "Delimiter
I know that I can read the file by using gui_upload
    CALL FUNCTION 'GUI_UPLOAD'
    EXPORTING
      filename                = c_infile
      has_field_separator     = p_delim
    TABLES
      data_tab                = indata
    EXCEPTIONS
      file_open_error         = 1
      file_read_error         = 2
        OTHERS                  = 9.
I know that I can split the contents of this file (if a delimiter is used). However I will not know the actual field names of the table until runtime as to what to fields to split it into. In the example below I have the actual table (t_rec) and each of the fields (pernr, lgart, etc) in the code but each table I
need to load will be different – it will have a different # of fields as well.
FORM read_data_pc.
LOOP AT indata.
    PERFORM splitdata USING indata.
    APPEND t_rec.
    CLEAR t_rec.
ENDLOOP.
ENDFORM.
FORM splitdata USING p_infile.
   SPLIT p_infile AT p_delim INTO
        t_rec-pernr                 "Employee #
        t_rec-lgart                 "Wage Type
        t_rec-begda                 "Effective date
        t_rec-endda.                 "End date
ENDFORM.
Once I split the data into the fields then I can just look and insert the record.
Does anyone have any ideas? Specific code examples would be great if you do. Thx!!

Hi janice,,
Try this sample code where you can upload data from a flat file into the internal table.
REPORT z_test.
TABLES: mara.
FIELD-SYMBOLS : <fs> .
DATA : fldname(50) TYPE c.
DATA : col TYPE i.
DATA : cmp LIKE TABLE OF rstrucinfo WITH HEADER LINE.
DATA: progname LIKE sy-repid,
dynnum LIKE sy-dynnr.
DATA itab TYPE TABLE OF alsmex_tabline WITH HEADER LINE.
DATA: BEGIN OF ZUPLOAD1_T OCCURS 0 ,
         matnr like mara-matnr,
         ersda like mara-ersda,
         ernam like mara-ernam,
         laeda like mara-laeda,
      END OF ZUPLOAD1_T.
*DATA: ZUPLOAD1_T LIKE mara OCCURS 0 WITH HEADER LINE.
DATA: wa_data LIKE TABLE OF ZUPLOAD1_T WITH HEADER LINE.
selection-screen
SELECTION-SCREEN: BEGIN OF BLOCK blk WITH FRAME TITLE text-001.SELECTION-SCREEN : SKIP 1. PARAMETERS : p_file LIKE rlgrap-filename.SELECTION-SCREEN : SKIP 1.SELECTION-SCREEN : END OF BLOCK blk
. AT SELECTION-SCREEN ON VALUE-REQUEST FOR p_file.
F4 Value for File
CALL FUNCTION 'KD_GET_FILENAME_ON_F4'   EXPORTING
                      PROGRAM_NAME        = SYST-REPID
                      DYNPRO_NUMBER       = SYST-DYNNR
                      FIELD_NAME          = ' '
     static              = 'X'
                      MASK                = ' '
    CHANGING      file_name           = p_file   EXCEPTIONS     mask_too_long       = 1     OTHERS              = 2
          . IF sy-subrc <> 0.
MESSAGE ID SY-MSGID TYPE SY-MSGTY NUMBER SY-MSGNO
        WITH SY-MSGV1 SY-MSGV2 SY-MSGV3 SY-MSGV4.
ENDIF.
START-OF-SELECTION.
CALL FUNCTION 'ALSM_EXCEL_TO_INTERNAL_TABLE'
EXPORTING    filename                      = P_FILE
i_begin_col                   = 1
i_begin_row                   = 1
i_end_col                     = 5
i_end_row                     = 12507
     tables
   intern                        = ITAB
EXCEPTIONS
        INCONSISTENT_PARAMETERS       = 1
        UPLOAD_OLE                    = 2
       OTHERS                        = 3.          .
IF sy-subrc <> 0.
MESSAGE ID SY-MSGID TYPE SY-MSGTY NUMBER SY-MSGNO
        WITH SY-MSGV1 SY-MSGV2 SY-MSGV3 SY-MSGV4.
ENDIF.
CALL FUNCTION 'GET_COMPONENT_LIST'
EXPORTING
    program          = SY-REPID
    fieldname        = 'ZMARA'
       tables
       components       = CMP.
LOOP AT itab.    AT NEW row.
IF sy-tabix = 1.        APPEND ZUPLOAD1_T.
ENDIF.
ENDAT.
col = itab-col.
READ TABLE cmp INDEX col.
CONCATENATE 'ZUPLOAD1_T-' cmp-compname INTO fldname.
ASSIGN (fldname) TO <fs>.
<fs> = itab-COL.
APPEND ZUPLOAD1_T. ENDLOOP.
DELETE ZUPLOAD1_T where matnr eq space.
LOOP AT ZUPLOAD1_T INTO wa_data.
insert mara from wa_data .
    WRITE: / ZUPLOAD1_T-matnr, 20 ZUPLOAD1_T-ersda , 45 ZUPLOAD1_T-ernam, 55 ZUPLOAD1_T-laeda.
*HERE IAM JUST CHECKING I NEED TO UPDATE A ZTABLE
ENDLOOP.
insert ZMARA FROM table itab ACCEPTING DUPLICATE KEYS.
I have tried it for mara.Please let me know whether it was helful.
Regards,
Kannan

How to load a flat file into BW-BPS using Web Browser

Hello, i have a problem with the "How to do Paper". I want to upload a Excel CSV file , but the paper only describes a txt file Uplaod. Does anybody can help me ?Thanks !

You need to parse the line coming in from the flat file...
You can do this with generic types in your flat file structure (string).
Then you loop through the table of strings that is your flat file and parse the string so that it breaks up the line for each comma. There is an ABAP command called: SPLIT - syntax is as follows:
SPLIT dobj AT sep INTO
{ {result1 result2 ...} | {TABLE result_tab} }
[IN {BYTE|CHARACTER} MODE].
Regards,
Zane

Problem in the BDC program to upload the data from a flat file.

Hi,
I am required to write a BDC program to upload the data from a flat file. The conditions are as mentioned below:-
1) Selection Screen will be prompted to user and user needs to provide:- File Path on presentation server (with F4 help for this obligatory parameter) and File Separator e.g. @,#,$,%,... etc(fields in the file will be separated by using this special character) or fields may be separated by tab(tab delimited).
2) Finally after the data is uploaded, following messages need to be displayed:-
a) Total Number of records successfully uploaded.
b) Session Name
c) Number of Sessions created.
Problem is when each record is fetched from flat file, the record needs to be split into individual fields separated by delimiter or in case tab separated, then proceeding in usual manner.
It would be great if you provide me either the logic, pseudocode, or sample code for this BDC program.
Thanks,

Here is an example program, if you require the delimitor to be a TAB, then enter TAB on the selection screen, if you require the delimitor to be a comma, slash, pipe, whatever, then simply enter that value. This example is simply the uploading of the file, not the BDC, I assume that you know what to do once you have the data into the internal table.
REPORT zrich_0001.
TYPES: BEGIN OF ttab,
        rec TYPE string,
       END OF ttab.
TYPES: BEGIN OF tdat,
       fld1(10) TYPE c,
       fld2(10) TYPE c,
       fld3(10) TYPE c,
       fld4(10) TYPE c,
       END OF tdat.
DATA: itab TYPE TABLE OF ttab.
data: xtab like line of itab.
DATA: idat TYPE TABLE OF tdat.
data: xdat like line of idat.
DATA: file_str TYPE string.
DATA: delimitor TYPE string.
PARAMETERS: p_file TYPE localfile.
PARAMETERS: p_del(5) TYPE c.
AT SELECTION-SCREEN ON VALUE-REQUEST FOR p_file.
DATA: ifiletab TYPE filetable.
DATA: xfiletab LIKE LINE OF ifiletab.
DATA: rc TYPE i.
CALL METHOD cl_gui_frontend_services=>file_open_dialog
    CHANGING
      file_table = ifiletab
      rc         = rc.
READ TABLE ifiletab INTO xfiletab INDEX 1.
IF sy-subrc = 0.
    p_file = xfiletab-filename.
ENDIF.
START-OF-SELECTION.
TRANSLATE p_del TO UPPER CASE.
CASE p_del.
    WHEN 'TAB'.
      delimitor = cl_abap_char_utilities=>horizontal_tab.
    WHEN others.
      delimitor = p_del.
ENDCASE.
file_str = p_file.
CALL METHOD cl_gui_frontend_services=>gui_upload
    EXPORTING
      filename = file_str
    CHANGING
      data_tab = itab.
LOOP AT itab into xtab.
    CLEAR xdat.
    SPLIT xtab-rec AT delimitor INTO xdat-fld1
                                     xdat-fld2
                                     xdat-fld3
                                     xdat-fld4.
    APPEND xdat to idat.
ENDLOOP.
LOOP AT idat into xdat.
    WRITE:/ xdat-fld1, xdat-fld2, xdat-fld3, xdat-fld4.
ENDLOOP.
Regards,
Rich Heilman

Flat file load - filename in updaterule/transferrule

Hi there
I am working on a solution where i need to use a part of the file name to include it in the transfer structure or update rule. The filename contains a date which is to be used as a version concept.
Is there a table where i can select this information during loadtime?
Chris

hi Chris,
this is the 'how to' get the table name :
go to any infopackage with flat file upload, tab 'external data', cursor position on the box contains file path, press F1, will come out 'performance assistant' screen, click on 'technical info', in the next screen, see for Field Data : 'Transparent tabl' and 'Field name'. there table RSLDPSEL and field name FILENAME.
in this case, the table is transparent table(straight forward), if not transparent table then may take more effort or even unable to get the where the data filled.
with se16 view table RSLDPSEL, get another info LOGDPID may be neeeded, LOGDPID = Infopackge technical name.
since the file path name may contains "\", in your routine you may need abap statement "SPLIT AT \" to get the file name only.
can consider to make such name convention for file name and date always take current date (sy-datum) then can simplify the procedure ?

Flat File Split

Similar Messages

Maybe you are looking for