Processing of Flat File of 2.5GB size in OWB

Can any one throw some light on performance of OWB in processing an input file of size 2.5GB.
Thanks in advance

This will take some time due to IO. Use an external table to process the file and be patient.
Regards
Jörg

Similar Messages

Oracle best product to process the flat file

Hi,
Please help me by suggesting the best available oracle tool / product to read / write / process the flat file in oracle server.
Thanks
Kannan

SQL Loader, external table, utl file, ...
And as usual - DEFINE BEST.
Gints Plivna
http://www.gplivna.eu

Process for Flat file delta load from Application server to transactional??

I want to do a flat file extraction in delta loads from application server and that should be loaded into a transactional cubes scheduling a process chain.
Can any one help with ABAP code and step by step process for this?
Thank you
Devi

Hi Devi,
As per your explaination, you want to load a list of files from application server using process chain.
You can do this with below steps.
1) Create a Event and call event to run Process chain multiple times.
2) Put all filenames in a file say, source-File
3) Write a rooting to read filename from Source-File and trigger InfoPackage
4) At the end of the process chain create new customized program which will delete first line from Source-File so that it will take next file in next run as well as delete currently loaded file. Once this is done trigger event again to start process chain again.
Regards,
Ganesh

Problem with Active Sync Process in Flat File

I have a flat file (Employee.csv) as a resource. I populated that data into IDM using Active Sync.
Ive created another flat file that is empty (CopyEmployee.csv) and added that as resource to IDM.
By using the Active Sync process I want to populate the data from Employee.csv to CopyEmployee.csv.
I tried but unable to get the output. Is this really possible ?
Im using IDM 6.0 on SunApp 8.1

Nope, flatfile is a read only adapter, to the best of my knowledge. You can use it to feed IM, that's it. No recon, nothing else. That's coz IM doesn't maintain a state for it (just does a diff with an index file during ActiveSync).
You can think of maintaining a simulated resource on a file though.
Ankush

Processing complex flat file

I have a requirement to load complex (hierarchical) flat file to Oracle database. The flat file has multiple (different) records. Each record is fixed positional.Here is an example
1 123455 6777888
2566773 4444 test
466666 7844 tyeww
477777 7844 tyeww
5213
Records are identified by the first character in the line. (In this example we have record type 1, record type 2, record type 4 and record type 5). Also the records are hierarchical.
Record Type 1
---Record Type 2 *
------Record Type 4 *
---Record Type 5
When the data is loaded to database it should be normalized. That means I have to take some value from Record Type 1, Record Type 2 and Record Type 4.
Can ODI handle this? do I have to write some script? If so How?

OK,
The following technique will help you:
1) Create a database sequence
2) create a interface to load a temp table (I named MY_TEMP_TABLE) with 1 "varchar' column so long as the biggest record (I named col_record), 1 column to store the sequence (I named col_seq) and 1 column to storage the parent line (that one should accept null values) (I named col_parent)
3) Map the record to col_record at source
4) Map the sequence to the col_seq at staging area
5) at col_parent column, put the following code and map it to staging area:
case when substr(src.col_record, 1,1) 1
then (select max(col_sequence)
from MY_TEMP_TABLE mtt
where substr(mtt.col_record, 1,1) < substr(src.col_record, 1,1)
else
null
end
PS: src.col_record is the column from your text file
This code will create the right hierarchy to the file.
after that, you can you the first technique that I told you, dragging and drooping this MY_TEMP_TABLE so many time as records types at a new interface, each one with the right filter, and create the joins.
You could even, add one more column to load the record type and work with it instead the "substr" command in the above select and at filter in the new interface.
Did I miss something? Does it solve the problem?

Flat file as source and target in owb-error

hi,
I want to run an interface with source as flat file and target as rdbms table.
The mapping got deployed with warnings.
And when i run the mapping i get the below error.
ORA-04063: package body "Schema.project_name" has errors
ORA-06508: PL/SQL: could not find program unit being called: "schema.project_name"
ORA-06512: at line 1

Hi
There's nothing wrong with your theory, however you might get a DB lock depending on the locking strategy of your RDBMS and it's current configuration.
Now, assuming that DB locking is your issue (get a DBA to check when the dataflow stalls), there are 2 solutions.
1, change the DB lock settings
2, take a copy of the target table in a new dataflow and use that as the source in the orginal dataflow.
Michael

Loading Flat files with into multiple tables using OWB

Hi,
How to implement the following logic in OWB.
LOAD DATA
INFILE 'myfile.txt'
BADFILE 'myfile.bad'
DISCARDFILE 'myfile.dsc'
APPEND
Into TABLE_Awhen (1:1) = 'A'
(Col1 Position(1:1) CHAR,
Col2 Position(2:5) CHAR)
Into TABLE_Bwhen (1:1) = 'B'
(Col1 Position(1:1) CHAR,
Col2 Position(2:20) EXTERNAL INTEGER)
Into TABLE_C
when (1:1) = 'C'
(Col1 Position(1:1) CHAR,
Col2 Position(2:20) EXTERNAL INTEGER)
I am using 10g version of OWB.I tried using the splitter operator.
I am getting the following error when i use the splitter.
An invalid combination of operators prevents the generation of code in a single implementation language (PL/SQL code, or SQL*Loader code, or ABAP code). For example, you may have included a SQL*Loader only operator such as the Data Generator in a mapping with a PL/SQL implementation type. If you designed the mapping to generate PL/SQL code, an invalid combination or sequence of operators prevents the generation of PL/SQL code in any of the operating modes(set based, row based, row based target only). If the mapping contains an operator that generates only PL/SQL output, all downstream dataflow operators must also be implementable by PL/SQL. You can use SQL operators in such a mapping only after loading the PL/SQL output to a target. Detail is as follows:
PL/SQL set based operating mode: Operator trailer_source_txt does not support SQL generation.
PL/SQL row based operating mode: Operator trailer_source_txt does not support SQL and PL/SQL generation.
PL/SQL row based (target only) operating mode: Operator trailer_source_txt does not support SQL and PL/SQL generation.
Both SQL and PL/SQL handlers are not supported by trailer_source_txt as output
SQL*Loader: Operator SPLIT does not support SQL*Loader generation.
ABAP: Operator trailer_source_txt does not support ABAP generation.
Thanks in advance,
VInay

Hi
Splitter can be used ib PL/SQL mappings, but if you use a flat file in a mapping, than it will be an SQLLoader mapping. So I suggest to you to create a mapping which load your flat file into a table, and from this table you load the data into the three table with the spillet in a PL/SQL mapping. Create two mappings.
Or you can use an external table in a mapping with a splitter.
Ott Karesz
http://www.trendo-kft.hu

How to Process flat File in Oracle Apps through Concurrent Program

Hello Everyone,
My client has a request, to process a bank file (Lockbox) which is a flat file that will be copied on UNIX box and I will have to create a new concurrent request that will process this flat file and will update receipt information in Oracle Apps database tables.
Could you please suggest, if there are any other standard Oracle Apps functions (Example FND) available which can be used through Concurrent program that can be used to open a file from a particular directory and can be read from the flat file and after processing this file can be closed.
Please let me know, if you have a small example, that would help me a lot.
Thanks

There are base concurrent programs in Accts Receivable that do consume lockbox flat files. Pl see the AR Setup/User Guides at
http://download.oracle.com/docs/cd/B40089_10/current/html/docset.html
Srini

Unable to process flat-files without delimiters in ODI 11.1.1.7

Hi,
We have a requirement to process a flat file using ODI 11.1.1.7 (installed on Windows and mounted on Oracle 11g Database). The flat-file is a normal ASCII file coming from mainframes machine and has no delimiters (column as well as rows). The first 48 characters is ROW 1, from character 49 to 96 is ROW 2 and so on. I am unable to create a DataStore for this file in ODI. If I reverse engineer using the file RKM, it is creating a datastore with over 2000 columns which is not the case (actual file layout given below). The absence of a row delimiter is making ODI think that the entire file has only one single row.
I am also unable to create a DataStore manually. If I am not providing any value for the "row delimiter" it is throwing me an error. However, I am able to process this file using SQL*Loader (there is a FIX option available where we can specify the length of a row).
Is there a way to do it from within ODI? I tried to modify a KM to create a control file for SQL*Loader and execute it but the control file is not getting generated as expected.
File Layout:
Column 1 - Warrant Number (Position 1 - 9) - Number
Column 2 - Type (Position 10-10) - Number
Column 3 - Warrant Amount (Position 11-18) - Packed Decimal Signed
Column 4 - Issue Fund (Position 19-22) - String
Column 5 - Issue Sub Fund (Position 23-24) - String
Column 6 - Filler (Position 25-48) - String
Thanks
Srivatsan P

Hi LuizFilipe,
I tried also with your method but i am getting below error when i am trying to view data.
See com.borland.dx.dataset.DataSetException error code: BASE+62
com.borland.dx.dataset.DataSetException: Execution of query failed.
Chained exception:
java.sql.SQLException: ODI-40439: Could not read heading rows from file
at com.sunopsis.jdbc.driver.file.FileResultSet.<init>(FileResultSet.java:164)
at com.sunopsis.jdbc.driver.file.impl.commands.CommandSelect.execute(CommandSelect.java:57)
at com.sunopsis.jdbc.driver.file.CommandExecutor.executeCommand(CommandExecutor.java:33)
at com.sunopsis.jdbc.driver.file.FilePreparedStatement.executeQuery(FilePreparedStatement.java:135)
at com.borland.dx.sql.dataset.o.f(Unknown Source)
at com.borland.dx.sql.dataset.QueryProvider.e(Unknown Source)
at com.borland.dx.sql.dataset.JdbcProvider.provideData(Unknown Source)
at com.borland.dx.dataset.StorageDataSet.refresh(Unknown Source)
at com.borland.dx.sql.dataset.QueryDataSet.refresh(Unknown Source)
at com.sunopsis.graphical.frame.edit.AbstractEditFrameGridBorland.initialize(AbstractEditFrameGridBorland.java:628)
at com.sunopsis.graphical.frame.edit.AbstractEditFrameGridBorland.<init>(AbstractEditFrameGridBorland.java:869)
at com.sunopsis.graphical.frame.edit.EditFrameTableData.<init>(EditFrameTableData.java:50)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at oracle.odi.ui.editor.AbstractOdiEditor$1.run(AbstractOdiEditor.java:176)
at oracle.ide.dialogs.ProgressBar.run(ProgressBar.java:656)
at java.lang.Thread.run(Thread.java:662)
Well i created file datastore with selected below things
File format : fixed
Heading(number of lines) : 0
field separator : null
record separator : by default --> MS-DOS
text delimiter: (blank)
decimal separator : (blank)
then when i did reverse engineering, able to see the row of data and then i separate the data with c1,c2,c3,c4
please let me know did i miss something.
Thanks
Himanshu

Not able to display flat file's error line in output

The below Source Code is BDC for XD01 (update Customer Master Record) in CALL TRANSACTION method. This program is written in call transaction method cause as per users requirement, user wants error log to be printed as soon as the BDC process finished in the background mode.
When you execute this below program it will show a selection screen from where you can select your flat file from system.
After executing this code show output like below as soon as it finished processing the FLAT file.
1. Customer Update Summary
2. Successful record with customer number and name.
3. Error Records with 'record number' and 'reason for error'
(For testing you just have to copy-paste-save-active the source code)
And I have encountered an issue over here in 3rd section-Record Number of Error Records.
The Record Number is for display the line number of that flat file line which in which error has occurred.
The Record Number Entries showing only 0 instead of showing line number of error entry .
I have used Function Module FORMAT_MESSAGE to capture the errors. But not able to display line number of that flat file which is containing error.
So please suggest me what changes I should make in below source code to get proper output.
Flat file for your R&D
1000     1000     10     10     0001     Company     Chobey & Group1N     Choubey     Amit Choubey     New Alipore Road     Habijabi Apartment     Rastar Dhra      777777     kolkata     777777     kolikata     howrah     777777     IN     WB     1111111111     2222222222     3333333333     4444444444     5555555555     6666666666     0700     EAST     02     01     1060003     A1     Z1     ZZ13     CALS     1     A     10     1
1000     1000     10     10     0001     Company     Chobey & Group2N     Choubey     Amit Choubey     New Alipore Road     Habijabi Apartment     Rastar Dhra      777777     kolkata     777777     kolikata     howrah     777777     IN     WB     1111111111     2222222222     3333333333     4444444444     5555555555     6666666666     0700     EAST     02     01     1060003     A1     Z1     ZZ13     CALS     1     A     10     1
1000     1000     10     10     0001     Company     Chobey & Group3N     Choubey     Amit Choubey     New Alipore Road     Habijabi Apartment     Rastar Dhra      777777     kolkata     777777     kolikata     howrah     777777     IN     WB     1111111111     2222222222     3333333333     4444444444     5555555555     6666666666     0700     EAST     02     01     1060003     A1     Z1     ZZ13     CALS     .     A     10     1
1000     1000     10     10     0001     Company     Chobey & Group4N     Choubey     Amit Choubey     New Alipore Road     Habijabi Apartment     Rastar Dhra      777777     kolkata     777777     kolikata     howrah     777777     IN     WB     1111111111     2222222222     3333333333     4444444444     5555555555     6666666666     0700     EAST     02     01     1060003     A1     Z1     ZZ13     CALS     1     A     ..     1
1000     1000     10     10     0001     Company     Chobey & Group5N     Choubey     Amit Choubey     New Alipore Road     Habijabi Apartment     Rastar Dhra      777777     kolkata     777777     kolikata     howrah     777777     IN     WB     1111111111     2222222222     3333333333     4444444444     5555555555     6666666666     0700     EAST     02     01     1060003     A1     Z1     ZZ13     CALS     1     A     10     1
And Source Code
*& Report Z_TEST_SAI                                                  *
REPORT Z_TEST_SAI no standard page heading line-size 255.
TYPES: BEGIN OF t_cust,
        bukrs like rf02d-bukrs,
        vkorg like rf02d-vkorg,
        vtweg like rf02d-vtweg,
        spart like rf02d-spart,
        ktokd like rf02d-ktokd,
        anred like kna1-anred,
        name1 like kna1-name1,
        sortl like kna1-sortl,
        name2 like kna1-name2,
        name3 like kna1-name3,
        name4 like kna1-name4,
        stras like kna1-stras,
        pfach like kna1-pfach,
        ort01 like kna1-ort01,
        pstlz like kna1-pstlz,
        ort02 like kna1-ort02,
        pfort like kna1-pfort,
        pstl2 like kna1-pstl2,
        land1 like kna1-land1,
        regio like kna1-regio,
        telx1 like kna1-telx1,
        telf1 like kna1-telf1,
        telfx like kna1-telfx,
        telf2 like kna1-telf2,
        teltx like kna1-teltx,
        stceg like kna1-stceg,
        cityc like kna1-cityc,
        lzone like kna1-lzone,
        niels like kna1-niels,
        kukla like kna1-kukla,
        akont like knb1-akont,
        fdgrv like knb1-fdgrv,
        vzskz like knb1-vzskz,
        zterm like knb1-zterm,
        vkbur like knvv-vkbur,
        versg like knvv-versg,
        vsbed like knvv-vsbed,
        ktgrd like knvv-ktgrd,
        taxkd like knvi-taxkd,
END OF t_cust.
TYPES: BEGIN OF t_sucrec,
          cnum TYPE kna1-kunnr,                          "Customer Number
          cnam TYPE kna1-name1,                          "Customer Name
END OF t_sucrec.
TYPES: BEGIN OF t_errrec,
*        lineno TYPE i,                                  "Line Number
         lineno TYPE string,
       message TYPE string,                              "Error Message
END OF t_errrec.
DATA: v_file TYPE string,                                "Variable for storing flat file
it_cust TYPE STANDARD TABLE OF t_cust,                   "Internal table of Customer
wa_cust LIKE LINE OF it_cust,                            "Workarea of Internal table it_cust
it_sucrec TYPE STANDARD TABLE OF t_sucrec,               "Internal table of Success records
wa_sucrec LIKE LINE OF it_sucrec,                        "Workarea of Internal table it_sucrec
it_errrec TYPE STANDARD TABLE OF t_errrec,
wa_errrec LIKE LINE OF it_errrec,
it_bdctab LIKE bdcdata OCCURS 0 WITH HEADER LINE,        "Internal table structure of BDCDATA
it_messagetab LIKE bdcmsgcoll OCCURS 0 WITH HEADER LINE, "Tracing Error Messages
v_date LIKE sy-datum,                                    "Controlling of session date
v_index LIKE sy-index,                                   "Index Number
v_totrec TYPE i,                                         "Total Records
v_errrec TYPE i,                                         "Error Records
v_sucrec TYPE i,                                         "Success Records
v_sesschk TYPE c.                                        "Session maintenance
*& SELECTION-SCREEN
SELECTION-SCREEN: BEGIN OF BLOCK blk1 WITH FRAME TITLE text-001 NO INTERVALS.
PARAMETERS: p_file    TYPE rlgrap-filename.              "rlgrap-filename is a predefined structure
SELECTION-SCREEN: END OF BLOCK blk1.
SELECTION-SCREEN: BEGIN OF BLOCK blk2 WITH FRAME TITLE text-002 NO INTERVALS.
PARAMETERS: p_mode    LIKE ctu_params-dismode DEFAULT 'N',
            p_update LIKE ctu_params-updmode DEFAULT 'A'.
SELECTION-SCREEN END OF BLOCK blk2.
*& INITIALIZATION
INITIALIZATION.
v_date = sy-datum - 1.
*& AT SELECTION-SCREEN ON VALUE-REQUEST FOR p_file
AT SELECTION-SCREEN ON VALUE-REQUEST FOR p_file.
CALL FUNCTION 'F4_FILENAME'
EXPORTING
program_name = syst-cprog
dynpro_number = syst-dynnr
FIELD_NAME = ' '
IMPORTING
file_name = p_file.
*& START-OF-SELECTION
START-OF-SELECTION.
v_file = p_file.
CALL FUNCTION 'GUI_UPLOAD'
EXPORTING
filename = v_file
filetype = 'ASC'
has_field_separator = 'X'
TABLES
data_tab = it_cust
EXCEPTIONS
file_open_error = 1
file_read_error = 2
no_batch = 3
gui_refuse_filetransfer = 4
invalid_type = 5
no_authority = 6
unknown_error = 7
bad_data_format = 8
header_not_allowed = 9
separator_not_allowed = 10
header_too_long = 11
unknown_dp_error = 12
access_denied = 13
dp_out_of_memory = 14
disk_full = 15
dp_timeout = 16
OTHERS = 17.
IF sy-subrc = 0.
*MESSAGE ID sy-msgid
*TYPE sy-msgty
*NUMBER sy-msgno
*WITH sy-msgv1 sy-msgv2 sy-msgv3 sy-msgv4.
ENDIF.
*& END-OF-SELECTION
END-OF-SELECTION.
v_index = sy-index.
LOOP AT it_cust INTO wa_cust.
perform bdc_dynpro      using 'SAPMF02D' '0100'.
    perform bdc_field       using 'BDC_CURSOR'
                                  'RF02D-KTOKD'.
    perform bdc_field       using 'BDC_OKCODE'
                                  '/00'.
    perform bdc_field       using 'RF02D-BUKRS'
                                  wa_cust-bukrs.
    perform bdc_field       using 'RF02D-VKORG'
                                  wa_cust-vkorg.
    perform bdc_field       using 'RF02D-VTWEG'
                                  wa_cust-vtweg.
    perform bdc_field       using 'RF02D-SPART'
                                  wa_cust-spart.
    perform bdc_field       using 'RF02D-KTOKD'
                                  wa_cust-ktokd.
    perform bdc_dynpro      using 'SAPMF02D' '0110'.
    perform bdc_field       using 'BDC_CURSOR'
                                  'KNA1-TELTX'.
    perform bdc_field       using 'BDC_OKCODE'
                                  '=VW'.
    perform bdc_field       using 'KNA1-ANRED'
                                  wa_cust-anred.
    perform bdc_field       using 'KNA1-NAME1'
                                  wa_cust-name1.
    perform bdc_field       using 'KNA1-SORTL'
                                  wa_cust-sortl.
    perform bdc_field       using 'KNA1-NAME2'
                                  wa_cust-name2.
    perform bdc_field       using 'KNA1-NAME3'
                                  wa_cust-name3.
    perform bdc_field       using 'KNA1-NAME4'
                                  wa_cust-name4.
    perform bdc_field       using 'KNA1-STRAS'
                                  wa_cust-stras.
    perform bdc_field       using 'KNA1-PFACH'
                                  wa_cust-pfach.
    perform bdc_field       using 'KNA1-ORT01'
                                  wa_cust-ort01.
    perform bdc_field       using 'KNA1-PSTLZ'
                                  wa_cust-pstlz.
    perform bdc_field       using 'KNA1-ORT02'
                                  wa_cust-ort02.
    perform bdc_field       using 'KNA1-PFORT'
                                  wa_cust-pfort.
    perform bdc_field       using 'KNA1-PSTL2'
                                  wa_cust-pstl2.
    perform bdc_field       using 'KNA1-LAND1'
                                  wa_cust-land1.
    perform bdc_field       using 'KNA1-REGIO'
                                  wa_cust-regio.
    perform bdc_field       using 'KNA1-SPRAS'
                                  'EN'.
    perform bdc_field       using 'KNA1-TELX1'
                                  wa_cust-telx1.
    perform bdc_field       using 'KNA1-TELF1'
                                  wa_cust-telf1.
    perform bdc_field       using 'KNA1-TELFX'
                                  wa_cust-telfx.
    perform bdc_field       using 'KNA1-TELF2'
                                  wa_cust-telf2.
    perform bdc_field       using 'KNA1-TELTX'
                                  wa_cust-teltx.
    perform bdc_dynpro      using 'SAPMF02D' '0120'.
    perform bdc_field       using 'BDC_CURSOR'
                                  'KNA1-CITYC'.
    perform bdc_field       using 'BDC_OKCODE'
                                  '=VW'.
    perform bdc_field       using 'KNA1-STCEG'
                                  wa_cust-stceg.
    perform bdc_field       using 'KNA1-CITYC'
                                  wa_cust-cityc.
    perform bdc_field       using 'KNA1-LZONE'
                                  wa_cust-lzone.
    perform bdc_dynpro      using 'SAPMF02D' '0125'.
    perform bdc_field       using 'BDC_CURSOR'
                                  'KNA1-KUKLA'.
    perform bdc_field       using 'BDC_OKCODE'
                                  '=VW'.
    perform bdc_field       using 'KNA1-NIELS'
                                  wa_cust-niels.
    perform bdc_field       using 'KNA1-KUKLA'
                                  wa_cust-kukla.
    perform bdc_dynpro      using 'SAPMF02D' '0130'.
    perform bdc_field       using 'BDC_CURSOR'
                                  'KNBK-BANKS(01)'.
    perform bdc_field       using 'BDC_OKCODE'
                                  '=VW'.
    perform bdc_dynpro      using 'SAPMF02D' '0340'.
    perform bdc_field       using 'BDC_CURSOR'
                                  'RF02D-KUNNR'.
    perform bdc_field       using 'BDC_OKCODE'
                                  '=VW'.
    perform bdc_dynpro      using 'SAPMF02D' '0360'.
    perform bdc_field       using 'BDC_CURSOR'
                                  'KNVK-NAMEV(01)'.
    perform bdc_field       using 'BDC_OKCODE'
                                  '=VW'.
    perform bdc_dynpro      using 'SAPMF02D' '0210'.
    perform bdc_field       using 'BDC_CURSOR'
                                  'KNB1-VZSKZ'.
    perform bdc_field       using 'BDC_OKCODE'
                                  '=VW'.
    perform bdc_field       using 'KNB1-AKONT'
                                  wa_cust-akont.
    perform bdc_field       using 'KNB1-FDGRV'
                                  wa_cust-fdgrv.
    perform bdc_field       using 'KNB1-VZSKZ'
                                  wa_cust-vzskz.
    perform bdc_dynpro      using 'SAPMF02D' '0215'.
    perform bdc_field       using 'BDC_CURSOR'
                                  'KNB1-ZTERM'.
    perform bdc_field       using 'BDC_OKCODE'
                                  '=VW'.
    perform bdc_field       using 'KNB1-ZTERM'
                                  wa_cust-zterm.
    perform bdc_dynpro      using 'SAPMF02D' '0220'.
    perform bdc_field       using 'BDC_CURSOR'
                                  'KNB5-MAHNA'.
    perform bdc_field       using 'BDC_OKCODE'
                                  '=VW'.
    perform bdc_dynpro      using 'SAPMF02D' '0230'.
    perform bdc_field       using 'BDC_CURSOR'
                                  'KNB1-VRSNR'.
    perform bdc_field       using 'BDC_OKCODE'
                                  '=VW'.
    perform bdc_dynpro      using 'SAPMF02D' '0610'.
    perform bdc_field       using 'BDC_OKCODE'
                                  '=VW'.
    perform bdc_field       using 'BDC_CURSOR'
                                  'RF02D-KUNNR'.
    perform bdc_dynpro      using 'SAPMF02D' '0310'.
    perform bdc_field       using 'BDC_CURSOR'
                                  'KNVV-VERSG'.
    perform bdc_field       using 'BDC_OKCODE'
                                  '=VW'.
    perform bdc_field       using 'KNVV-AWAHR'
                                  '100'.
    perform bdc_field       using 'KNVV-VKBUR'
                                  wa_cust-vkbur.
    perform bdc_field       using 'KNVV-WAERS'
                                  'INR'.
    perform bdc_field       using 'KNVV-KALKS'
                                  '1'.
    perform bdc_field       using 'KNVV-VERSG'
                                  wa_cust-versg.
    perform bdc_dynpro      using 'SAPMF02D' '0315'.
    perform bdc_field       using 'BDC_CURSOR'
                                  'KNVV-VSBED'.
    perform bdc_field       using 'BDC_OKCODE'
                                  '=VW'.
    perform bdc_field       using 'KNVV-LPRIO'
    perform bdc_field       using 'KNVV-KZAZU'
                                  'X'.
    perform bdc_field       using 'KNVV-VSBED'
                                  wa_cust-vsbed.
    perform bdc_field       using 'KNVV-ANTLF'
                                  '9'.
    perform bdc_dynpro      using 'SAPMF02D' '0320'.
    perform bdc_field       using 'BDC_CURSOR'
                                  'KNVV-KTGRD'.
    perform bdc_field       using 'BDC_OKCODE'
                                  '=VW'.
    perform bdc_field       using 'KNVV-BOKRE'
                                  'X'.
    perform bdc_field       using 'KNVV-KTGRD'
                                  wa_cust-ktgrd.
    perform bdc_dynpro      using 'SAPMF02D' '1350'.
    perform bdc_field       using 'BDC_CURSOR'
                                  'KNVI-TAXKD(01)'.
    perform bdc_field       using 'BDC_OKCODE'
                                  '=VW'.
    perform bdc_field       using 'KNVI-TAXKD(01)'
                                  wa_cust-taxkd.
    perform bdc_dynpro      using 'SAPMF02D' '0324'.
    perform bdc_field       using 'BDC_CURSOR'
                                  'KNVP-PARVW(01)'.
    perform bdc_field       using 'BDC_OKCODE'
                                  '=UPDA'.
CALL TRANSACTION 'XD01' USING it_bdctab
MODE p_mode
UPDATE p_update
MESSAGES INTO it_messagetab.
IF sy-subrc = 0.
*& reading success records to corresponding internal table
READ TABLE it_messagetab WITH KEY msgtyp = 'S'.
IF sy-subrc = 0.
wa_sucrec-cnum = it_messagetab-msgv1.
wa_sucrec-cnam = wa_cust-name1.
APPEND wa_sucrec TO it_sucrec.
CLEAR wa_sucrec.
ENDIF.
ELSE.
*& reading error records to corresponding internal table
READ TABLE it_messagetab WITH KEY msgtyp = 'E'.
IF sy-subrc = 0.
CALL FUNCTION 'FORMAT_MESSAGE'
EXPORTING
id = sy-msgid
no = it_messagetab-msgnr
v1 = it_messagetab-msgv1
v2 = it_messagetab-msgv2
v3 = it_messagetab-msgv3
v4 = it_messagetab-msgv4
IMPORTING
msg = wa_errrec-message.
wa_errrec-lineno = v_index.
*******wa_errrec-lineno = v_index.
*******wa_errrec-message = it_messagetab-msgv1.
APPEND wa_errrec TO it_errrec.
CLEAR wa_errrec.
ENDIF.
ENDIF.
CLEAR : it_bdctab, it_messagetab.
REFRESH: it_bdctab, it_messagetab.
ENDLOOP.
DESCRIBE TABLE it_cust LINES v_totrec.
DESCRIBE TABLE it_errrec LINES v_errrec.
DESCRIBE TABLE it_sucrec LINES v_sucrec.
PERFORM disp_data.
SKIP 2.
IF v_sucrec > 0.
PERFORM disp_success_data.
ENDIF.
SKIP 2.
IF v_errrec > 0.
PERFORM disp_error_data.
ENDIF.
*& Form bdc_dynpro
*# text
*# -->P_0104 text
*# -->P_0105 text
FORM bdc_dynpro USING program
dynpro.
CLEAR it_bdctab.
it_bdctab-program = program.
it_bdctab-dynpro = dynpro.
it_bdctab-dynbegin = 'X'.
APPEND it_bdctab.
ENDFORM. " bdc_dynpro
*& Form bdc_field
*# text
*# -->P_0109 text
*# -->P_IT_cust_LIFNR text
FORM bdc_field USING fnam
fval.
CLEAR it_bdctab.
it_bdctab-fnam = fnam.
it_bdctab-fval = fval.
APPEND it_bdctab.
ENDFORM. " bdc_field
*& Form disp_data
*# text
*# --> p1 text
*# <-- p2 text
FORM disp_data .
ULINE (45).
WRITE : / sy-vline,
12 'FAMD CUSTOMER UPDATE SUMMARY'(004) COLOR 1,
45 sy-vline.
ULINE /(45).
WRITE : / sy-vline,
'Total Records Processed'(007),
28 '=',
30 v_totrec,
45 sy-vline,
/ sy-vline,
'Error Records'(005),
28 '=',
30 v_errrec,
45 sy-vline,
/ sy-vline,
'Successful Records'(006),
28 '=',
30 v_sucrec,
45 sy-vline.
ULINE /(45).
ENDFORM. " disp_data
*& Form disp_success_data
*# text
*# --> p1 text
*# <-- p2 text
FORM disp_success_data .
ULINE (45).
WRITE : / sy-vline,
14 'Successful Records'(012) COLOR 1,
45 sy-vline.
ULINE /(45).
WRITE : / sy-vline ,
'Customer Number'(010) COLOR 2,
17 sy-vline,
25 'Customer Name'(011) COLOR 2,
45 sy-vline.
ULINE /(45).
LOOP AT it_sucrec INTO wa_sucrec.
WRITE: / sy-vline ,
wa_sucrec-cnum,
17 sy-vline,
19 wa_sucrec-cnam,
45 sy-vline.
ENDLOOP.
ULINE /(45).
ENDFORM. " disp_success_data
*& Form disp_error_data
*# text
*# --> p1 text
*# <-- p2 text
FORM disp_error_data .
ULINE (85).
WRITE : / sy-vline,
35 'Error Records'(013) COLOR 1,
85 sy-vline.
ULINE /(85).
WRITE : / sy-vline,
'Record Number'(008) COLOR 2,
sy-vline,
37 'Reason for error'(009) COLOR 2,
85 sy-vline.
ULINE /(85).
LOOP AT it_errrec INTO wa_errrec.
WRITE : / sy-vline,
wa_errrec-lineno,
17 sy-vline,
wa_errrec-message,
85 sy-vline.
ENDLOOP.
ULINE /(85).
ENDFORM. " disp_error_data

Worked out n found the solution

Golden Gate for flat file

hi,
I have tried with GoldenGate for Oracle/ non-Oracle databases. Now, I am trying for flat file.
What i have done so far:
1. I have downloaded Oracle "GoldenGate Application Adapters 11.1.1.0.0 for JMS and Flat File Media Pack"
2. Kept it on the same machine where Database and GG manager process exists. Port for GG mgr process 7809, flat file 7816
3. Following doc GG flat file administrators guide Page 9 --> configuration
4. Extract process on GG manager process_
edit params FFE711*
extract ffe711
userid ggs@bidb, password ggs12345
discardfile E:\GoldenGate11gMediaPack\V26071-01\dirrpt\EXTFF.dsc, purge
rmthost 10.180.182.77, mgrport 7816
rmtfile E:\GoldenGate11gMediaPack\V26071-01\dirdat\ffremote, purge, megabytes 5
add extract FFE711, EXTTRAILSOURCE ./dirdat/oo*
add rmttrail ./dirdat/pp, extract FFE711, megabytes 20*
start extract FFE711*
view report ffe711*
Oracle GoldenGate Capture for Oracle
Version 11.1.1.1 OGGCORE_11.1.1_PLATFORMS_110421.2040
Windows (optimized), Oracle 11g on Apr 22 2011 03:28:23
Copyright (C) 1995, 2011, Oracle and/or its affiliates. All rights reserved.
Starting at 2011-11-07 18:24:19
Operating System Version:
Microsoft Windows XP Professional, on x86
Version 5.1 (Build 2600: Service Pack 2)
Process id: 4628
Description:
** Running with the following parameters **
extract ffe711
userid ggs@bidb, password ********
discardfile E:\GoldenGate11gMediaPack\V26071-01\dirrpt\EXTFF.dsc, purge
rmthost 10.180.182.77, mgrport 7816
rmtfile E:\GoldenGate11gMediaPack\V26071-01\dirdat\ffremote, purge, megabytes 5
CACHEMGR virtual memory values (may have been adjusted)
CACHEBUFFERSIZE: 64K
CACHESIZE: 1G
CACHEBUFFERSIZE (soft max): 4M
CACHEPAGEOUTSIZE (normal): 4M
PROCESS VM AVAIL FROM OS (min): 1.77G
CACHESIZEMAX (strict force to disk): 1.57G
Database Version:
Oracle Database 11g Enterprise Edition Release 11.1.0.7.0 - Production
PL/SQL Release 11.1.0.7.0 - Production
CORE 11.1.0.7.0 Production
TNS for 32-bit Windows: Version 11.1.0.7.0 - Production
NLSRTL Version 11.1.0.7.0 - Production
Database Language and Character Set:
NLS_LANG environment variable specified has invalid format, default value will b
e used.
NLS_LANG environment variable not set, using default value AMERICAN_AMERICA.US7A
SCII.
NLS_LANGUAGE = "AMERICAN"
NLS_TERRITORY = "AMERICA"
NLS_CHARACTERSET = "AL32UTF8"
Warning: your NLS_LANG setting does not match database server language setting.
Please refer to user manual for more information.
2011-11-07 18:24:25 INFO OGG-01226 Socket buffer size set to 27985 (flush s
ize 27985).
2011-11-07 18:24:25 INFO OGG-01052 No recovery is required for target file
E:\GoldenGate11gMediaPack\V26071-01\dirdat\ffremote, at RBA 0 (file not opened).
2011-11-07 18:24:25 INFO OGG-01478 Output file E:\GoldenGate11gMediaPack\V2
6071-01\dirdat\ffremote is using format RELEASE 10.4/11.1.
** Run Time Messages **
5. on Flat file GGSCI prompt-->_*
edit params FFR711*
extract ffr711
CUSEREXIT E:\GoldenGate11gMediaPack\GGFlatFile\V22262-01\flatfilewriter.dll CUSEREXIT passthru includeupdatebefores, params "E:\GoldenGate11gMediaPack\GGFlatFile\V22262-01\sample-dirprm\ffwriter.properties"
SOURCEDEFS E:\GoldenGate11gMediaPack\V26071-01\dirdef\vikstkFF.def
table ggs.vikstk;
add extract ffr711, exttrailsource ./dirdat/pp*
start extract ffr711*
view report ffr711*
Oracle GoldenGate Capture
Version 11.1.1.0.0 Build 078
Windows (optimized), Generic on Jul 28 2010 19:05:07
Copyright (C) 1995, 2010, Oracle and/or its affiliates. All rights reserved.
Starting at 2011-11-07 18:21:31
Operating System Version:
Microsoft Windows XP Professional, on x86
Version 5.1 (Build 2600: Service Pack 2)
Process id: 5008
Description:
** Running with the following parameters **
extract ffr711
CUSEREXIT E:\GoldenGate11gMediaPack\GGFlatFile\V22262-01\flatfilewriter.dll CUSE
REXIT passthru includeupdatebefores, params "E:\GoldenGate11gMediaPack\GGFlatFil
e\V22262-01\sample-dirprm\ffwriter.properties"
E:\GoldenGate11gMediaPack\GGFlatFile\V22262-01\ggs_Windows_x86_Generic_32bit_v11
_1_1_0_0_078\extract.exe running with user exit library E:\GoldenGate11gMediaPac
k\GGFlatFile\V22262-01\flatfilewriter.dll, compatiblity level (2) is current.
SOURCEDEFS E:\GoldenGate11gMediaPack\V26071-01\dirdef\vikstkFF.def
table ggs.vikstk;
CACHEMGR virtual memory values (may have been adjusted)
CACHEBUFFERSIZE: 64K
CACHESIZE: 1G
CACHEBUFFERSIZE (soft max): 4M
CACHEPAGEOUTSIZE (normal): 4M
PROCESS VM AVAIL FROM OS (min): 1.87G
CACHESIZEMAX (strict force to disk): 1.64G
Started Oracle GoldenGate for Flat File
Version 11.1.1.0.0
** Run Time Messages **
Problem I am facing_
I am not sure where to find the generated flat file,
even the reports are showing there is no data at manager process
I am expecting replicat instead of extract at Flatfile FFR711.prm
I have done this much what to do give me some pointers.....
Thanks,
Vikas

Ok, I haven't run your example, but here are some suggestions.
Vikas Panwar wrote:
extract ffe711
userid ggs@bidb, password ggs12345
discardfile E:\GoldenGate11gMediaPack\V26071-01\dirrpt\EXTFF.dsc, purge
rmthost 10.180.182.77, mgrport 7816
rmtfile E:\GoldenGate11gMediaPack\V26071-01\dirdat\ffremote, purge, megabytes 5
ggsci> add extract FFE711, EXTTRAILSOURCE ./dirdat/oo
ggsci> add rmttrail ./dirdat/pp, extract FFE711, megabytes 20
ggsci> start extract FFE711
You of course need data captured from somewhere to test with. You could capture changes directly from a database and write those to a trail, and use that as a source for the flat-file writer; or, if you have existing trail data, you can just use that (I often test with old trails, with known data).
In your example, you are using a data pump that is doing nothing more than pumping trails to a remote host. That's fine, if that's what you want to do. (It's actually quite common in real implementations.) But if you want to actually capture changes from the database, then change "add extract ... extTrailSource" to be "add extract ... tranlog". I'll assume you want to use the simple data pump to send trail data to the remote host. And I will assume that some other database capture process is creating the trail dirdat/oo
Also... with your pump "FFE711", you can create either a local or remote trial, that's fine. But don't use a rmtfile (or extfile). You should create a trail, either a "rmttrail" or "exttrail". The flat-file adapter will read that (binary) trail, and generate text files. Trails automatically roll-over, the "extfile/rmtfile" do not (but they do have the same internal GG binary log format). (You can use a 'maxfiles' to force them to rollover, but that's beside the point.)
Also, <ul>
<li> don't forget your "table" statements... or else no data will be processed!! You can wildcard tables, but not schemata.
<li> there is no reason that anything would be discarded in a pump.
<li> although a matter of choice, I don't see why people use absolute paths for reports and discard files. Full paths to data and def files make sense if they are on the SAN/NAS, but then I'd use symlinks from dirdat to the storage directory (on Unix/Linux)
<li> both windows and unix can use forward "/" slashes. Makes examples platform-independent (another reason for relative paths)
<li> your trails really should be much larger than 5MB for better performance (e.g,. 100MB)
<li> you probably should use a source-defs file, intead of a dblogin for metadata. Trail data is by its very nature historical, and using "userid...password" in the prm file inherently gets metadata from "right now". The file-writer doesn't handle DDL changes automatically.
</ul>
So you should have something more like:
Vikas Panwar wrote:
extract ffe711
sourcedefs dirdef/vikstkFF.def
rmthost 10.180.182.77, mgrport 7816
rmttrail dirdat/ff, purge, megabytes 100
table myschema.*;
table myschema2.*;
table ggs.*;For the file-writer pump:
+5. on Flat file GGSCI prompt+
extract ffr711
CUSEREXIT flatfilewriter.dll CUSEREXIT passthru includeupdatebefores, params dirprm\ffwriter.properties
SOURCEDEFS dirdef/vikstkFF.def
table myschema.*;
table ggs.*;
ggsci> add extract ffr711, exttrailsource ./dirdat/pp
ggsci> start extract ffr711
Again, use relative paths when possible (the flatfilewriter.dll is expected to be found in the GG install directory). Put the ffwriter.properties file into dirprm, just as a best-practice. In this file, ffwriter.properties, is where you define your output directory and output files. Again, make sure you have a "table" statement in there for each schema in your trails.
Problem I am facing_
I am not sure where to find the generated flat file,
even the reports are showing there is no data at manager process
I am expecting replicat instead of extract at Flatfile FFR711.prm
I have done this much what to do give me some pointers.....The generated files are defined in the ffwriter.properties file. Search for "rootdir" property, e.g.,
goldengate.flatfilewriter.writers=csvwriter
csvwriter.files.formatstring=output_%d_%010n
csvwriter.files.data.rootdir=dirout
csvwriter.files.control.ext=_data.control
csvwriter.files.control.rootdir=dirout
...The main problem you have is: (1) use rmttrail, not rmtfile, and (2) don't forget the "table" statement, even in a pump.
Also, for the flat-file adapter, it does run in just a "extract" data pump; no "replicat" is ever used. The replicats inherently are tied to a target database; the file-writer doesn't have any database functionality.
Hope it helps,
-m

Reading huge flat file in OSB 11gR1

Hi,
I want to read a flat file in OSB.The size of the flat file may be larger, upto 1 MB.
As per my knowledge, OSB provides following approaches to read a flat file-
1.JCA(creating a file adapter in jdev and importing artifacts in OSB)
2.MFL transformation
3.Java callout
Please let me know which is the best way to read the flat file.Also , is there any other way to do the same.
Thanks in advance.
Regards,
Seemant
Edited by: Seemant Srivastava on Feb 18, 2011 1:47 PM

Which option is best one to convert a flat file to XML - is it via File Adapter or MFL ? Well, it's a topic of debate and it usually depends on your choice. Manoj has explained it clearly above that why one may prefer File Adapter over MFL. It also depends on your familiarity with the product. If you are a Oracle developer dealing with BPEL/Mediator mostly then you will prefer going for File adapter in this situation, even with OSB, but if you are a OSB developer (since the time it was known as ALSB) you will prefer MFL over adapter.
It's just matter of choice & your comfort. Remember, in different-different cases, both solutions may result in different performance, so better test them from performance perspective and then choose.
Such flexibility of optional tags can only be handled by mfl.I don't think so. File adapter should also be able to handle this use case. Have you checked this -
http://download.oracle.com/docs/cd/E17904_01/integration.1111/e10231/nfb.htm#CHDDHEAI
Also, upto what size of input file is supported by mfl.It's a unanswerable question. It totally depends upon your system structure. I personally don't prefer huge files translation at OSB because it hurts the performance of OSB.
but I think such feature will not be supported when the file adapter artifacts are imported and used in OSB 11.1.1.3/11.1.1.4Correct. From OSB Dev guide -
25.2.1.1 Oracle JCA Adapter Limitations
Following are limitations when using some JCA adapters with Oracle Service Bus:
•Oracle JCA Adapter for AQ – Streamed payload is not supported with Oracle Service Bus.
•Oracle JCA Adapter for FTP and Files – Attachments (large payload support), pre- and post-processing of files, using a re-entrant valve for processing ZIP files, content streaming, and file chunked read are not supported with Oracle Service Bus.
http://download.oracle.com/docs/cd/E17904_01/doc.1111/e15866/jca.htm#BABBICIA
Regards,
Anuj

Large Flat Files

I have some large pipe delimited flat files (upto 1.5gb in size) from a Mainframe that i need to get into Oracle. I was just wondering if this is possible via Oracle Workbench.
Problem with using SQL*Loader was getting the files onto the database box because of company politics :)
any help or ideas much appreciated

Ninder,
The Migration Workbench won't be able to help you in your case. As you say the files are coming from a mainframe then I assume they come from DB2 running on the mainframe.
The migration workbenches - either the original 10.1.0.4 or the SQL*Developer version - do not currently support migrating DB2 on z/OS.
In any case, for migrating large amounts of data it was always recommended to use offline migration via SQL*Loader as the online migration was not really suitable.
Regards,
Mike

Flat file to XML Document

Hi,
I have a requirement where i have to process a flat file and create an XML document out of it. This XML is turn has to conform to schema. Can this be done in ODI? If yes, what is the knowledge module I should be using to achieve this? Further, how do I create the data model for the XML, and how do i specify the directory where the file needs to be written to? I hear that there is an IKM by the name "SQL to XML IKM" desinged to genereate XML from database. Can anyone please forward it to me? My mail address is [email protected] Any help is greatly appreciated.
Edited by: Sankash on 01-Apr-2009 07:19

Refer to this thread.
Re: ODI with XML
It will guide you thru the process at a high level.
Thanks
CJ
Please do remember to assign points (Helpful / Correct) accordingly to the thread before closing, that encourage people to help you more quickly and effectively.

Flat file as a source in a mapping

Dear all
i noticed that when you create a mapping with a Flat File Operator as a source, the OWB generates SQL* Loader code instead of PL/SQL package.
Is there any parameter you have to change in order to generate PL/SQL code or, if not how can you convert SQL* Loader code into PL/SQL?
Any help would be appreciated

Hi,
OWB generates SQL * Loader code for Flat File operator, if you try to change the configuration parameter to PL/SQL it results in error.
External tables would be a better choice as it reduces the processing time, disk space required and also makes it possible to use SQL and other complex transformations.

Processing of Flat File of 2.5GB size in OWB

Similar Messages

Maybe you are looking for