Header/Detail Flat file parsing
Hi,
How to define a native flat file schema for a file with following kind of layout i.e., Master record followed by Multiple detail record. I need to have a correlation between the Master and its Detail records.
MASTER_A1,MASTER_A2,MASTER_A3
DETAIL_A11,DETAIL_A12,DETAIL_A13
DETAIL_A21,DETAIL_A22,DETAIL_A23
MASTER_B1,MASTER_B2,MASTER_B3
DETAIL_B11,DETAIL_B12,DETAIL_B13
DETAIL_B21,DETAIL_B22,DETAIL_B23
DETAIL_B31,DETAIL_B32,DETAIL_B33
DETAIL_B41,DETAIL_B42,DETAIL_B43
MASTER_C1,MASTER_C2,MASTER_C3
DETAIL_C11,DETAIL_C12,DETAIL_C13
DETAIL_C21,DETAIL_C22,DETAIL_C23
DETAIL_C31,DETAIL_C32,DETAIL_C13
http://blogs.oracle.com/reynolds/2009/05/mastering_details_with_flat_fi.html
Regards,
Anuj
Similar Messages
-
Header in flat file destination (ssis)
Hi
I am creating a header in the flat file destination which has some hard coded values and record count which will get its value from a variable assisgned to a row count transformation in the package.
There is no problem in hard coded value but when i try to get the value of record count, i always get zero.
This is how i am doing it.
In the flat file destination, when we open flat file destination editor, i hard code the values which i want in the HEADER itself right there in the editor.
For the record count, i go properties of the flat file and select expression. In the expression editor, in property, i am selecting 'HeaderRowDelimiter' and drag that variable in the expression box.
Let me know if there is any suggestion.
ThanksThe value of the rowcount is only known AFTER the Data Flow Task is completely finished... and there header is writen before that.
Here are some examples:
http://agilebi.com/jwelch/2008/02/08/adding-headers-and-footers-to-flat-files/
http://social.msdn.microsoft.com/Search/en-US/sqlserver?query=add%20header%20and%20footer%20ssis&rq=meta:Search.MSForums.ForumID(00e50af7-5f43-43ad-af05-d98b73c1f760)+site:microsoft.com&rn=SQL+Server+Integration+Services+Forum
Please mark the post as answered if it answers your question | My SSIS Blog:
http://microsoft-ssis.blogspot.com -
The flat files are present in a unix server .
No other application can be loaded there
i have a lunix machine from where i have to access unix server data
which is the better way
1)directly reading the file from the unix server
2)loading the file in lunix and then reading
How this can be done in java ????
Now lets say we are reading the file in some way or the other ,How to do i parse it
File parsing
different file:
1)fixed format
file size 50 mb
eg
a)the data will be in fixed column size
0-60 -name
61-70 -department
71-80 -age
b)there will be section strip data for a particular section
Note :-Here i have to parse a particular section from the flat file the size of file is 50mb?
eg
section-Finance
name department age
Smith dep1 32
john dep2 40
turner dep3 56
section-marketing
name department age
antony dep1 60
black dep2 57
2)csv file
file size 15 kb
here the data size will be not be fixed but will have a seperator
name|department|age
Smith|dep1|32
john|dep2|40
turner|dep3|56
antony|dep1|60
black|dep2|57
here seperator is | it can be anything
Thanx in advance
MeghnaXml Convert:
http://industry.java.sun.com/solutions/products/by_product/0,2348,all-4398-99,00.html
Flat File to Xml Converion:
http://www.infoloom.com/gcaconfs/WEB/philadelphia99/lyons.HTM -
Open Hub - Header in flat file
Hi there Experts.
I am working with a opeb Hub destination, and so far everything has been straight forward.
But i need to include headers in the first line of the file.
By default the Open Hub i generating a Control File, so it is possible to look in this file, but the requirements from the customer is to have fieldnames inside the same file as the data.
Any help will be awarded.
IngridI have faced this. There is no delivered mechanism. One work around would be (after the extraction) to open the file using ABAP and paste the header. If one tries to figure (in the start routine or end routine) that this is the first record of the first package, one faces the problem with data types (header has a CHAR data types where the fields of the file may not all be CHAR). We eventually convinced the user to drop the requirement. As you mention, there are generated files with header information.
Not sure this helps. -
Parse multiple files in one flat file?
Hi all,
I'm currently working with flat file with this kind of structure:
"849000","1","2","3","4" <- begin of file
"849HD","","1939","12" <- header level
"849D1","39193","313","1" <- detail level
"849D2","","description","48,13" <- detail description level
"849RT","133,1","N4","203" <- totals level
The problem is that the file i have to pick up (the map is File => EDI)
can contain many structures (every estructure is an edi to generate)
example:
"849000","1","2","3","4" <- first file
"849HD","","1939","12"
"849D1","39193","313","1"
"849D2","","description","48,13"
"849RT","133,1","N4","203" <- end of first file
"849000","2","","","6" <- second file
"849HD","","92","23"
"849D1","99","912","1"
"849D2","","second description","3,11"
"849RT","61","2","UP","102" <- end of second file
- How can i parse this file in order to get a nested structure?
MT_file
MT_file/row (0.unbounded) <- that would contain 2 files
MT_file/row/849000
MT_file/row/849HD
MT_file/row/849D1
MT_file/row/849D2
MT_file/row/849RT
Because i believe content conversion (KeyFieldValue) is not effective since it will take the key as generate the segments all together without respecting the order
Any ideas?
Thanks!I'm not so sure about that, i mean i think that the KeyField won't we able to understand the hiercachy
example:
"849000","1","2","3","4" <- first file
"849HD","","1939","12"
"849D1","39193","313","1"
"849D2","","description","48,13"
"849RT","133,1","N4","203" <- end of first file
"849000","2","","","6" <- second file
"849HD","","92","23"
"849D1","99","912","1"
"849D2","","second description","3,11"
"849RT","61","2","UP","102" <- end of second file
with keys:
849000
849HD
849D1
849D2
849RT
i will probably get all in a group, and in doing that i'll loose the reference for the first and second file
resulting:
"849000","1","2","3","4" <- first file
"849000","2","","","6" <- second file
"849HD","","1939","12"
"849HD","","92","23"
"849D1","39193","313","1"
"849D1","99","912","1"
"849D2","","description","48,13"
"849D2","","second description","3,11"
"849RT","133,1","N4","203" <- end of first file
"849RT","61","2","UP","102" <- end of second file
Or am i missing something?
This is file => EDI, so the channel would be SENDER -
TO IGNORE A HEADER IN A FLAT FILE (SENDER) XI
Developing a solution XI in file-jdbc cenario and
the file is a flat file with header line and various
detail lines. I need to write the jdbc table ignoring
the header line of the flat file. How can We do this?
Someone can help me?
Thanks and regards,
Midervilson.Hi Midervilson,
If you are using a content conversion based on key fields in the sender file adapter and if your header and details are differentiated by a key field then you can ignore them in content conversion itself. i.e say if your header is identified by a key field value '1' and details are identified by a key field value '2' then just configure your file content conversion details alone and your header will be automatically ignored.
Otherway you can achive this is using mapping. while mapping do not map your header from source to JDBC structure.
hope this helps...
Regards
Anand
Message was edited by: Anand Torgal -
Flat file having multiple header lines , how to separate the same and pushit into my intrnal table
Hi Guys,
I got an issue , My flat file having multiple header lines
TPS27RPR MOME THRIFT (SAVING) PLAN SYSTEM Page 2
09:54 05 JAN 2014 List of Contributors w/Company Contributions Phase DAY Unit TRE
Company No: xxxxxxxx - xxxxxxxxxxxxx.
Employee/ Employee/
Employee Full Name/ Company Company Allocation
Number ID Number PS PN End Date Contribution Profit Total Balance Percentage
some lines of data and again
TPS27RPR MOME THRIFT (SAVING) PLAN SYSTEM Page 2
09:54 05 JAN 2014 List of Contributors w/Company Contributions Phase DAY Unit TRE
Company No: xxxxxxxx - xxxxxxxxxxxxx.
Employee/ Employee/
Employee Full Name/ Company Company Allocation
Number ID Number PS PN End Date Contribution Profit Total Balance Percentage
in this case how could I separate the header lines from the flat file and update the details to my internal table.
kindly help me on this issue.
Thanks and regards,
M.S.Amirtharajvijay.Hi Amirthraj,
If the flat file is static (if the content within it is same for all) then use the offset concept. First get the contents of the flat file into an internal table and later find the field that has to be in final internal table within the line, calculate the offset for the line and get the value into an workarea. Finally modify/append the work area into an internal table. Also wait for other experts answers for any other hassle free way to achieve the requirement.
With Regards,
Giriesh M -
How to Add Header and Footer in Flat file
HI All,
I am using ODI 11g.
MY interface is populating data from joining to 2 tables through sql query and Loads in to Flat file.
But i need to add Header and Footer in the Flat file.
In the below 3 lines
The 1st line is header ....2nd line is Data ....3rd line is Footer.
Any ideas how to work on this.
HSB9KT.KTPRTO.EBSDATA.D0000000 20120626020636
CFFE00263 SOLGSR014991340000527894536058173880207SOLNE06242012072420120000009048382650000247MRS
T00000000000000000000
Thanks,
LonyHi Drieux,
i am able to add Header and Footer in this.
But i want the header like Its like HSB9KT.KTPRTO.EBSDATA.D0000000 || SYSDATE (YYYYMMDDHHMMSS)
In my Out put i only see header as HSB9KT.KTPRTO.EBSDATA.D0000000 .
I have added a Variable where i am using SELECT to_char(sysdate,'YYYYMMDDHHMMSS') FROM DUAL query.
create header (HSB9KT.KTPRTO.EBSDATA.D0000000 #v_DATE)
/*$$SNPS_START_KEYSNP$CRDWG_TABLESNP$CRTABLE_NAME=FFT_SOLSNP$CRLOAD_FILE=/home/tarak/FFT_SOL.datSNP$CRFILE_FORMAT=FSNP$CRFILE_SEP_FIELD=0x0009SNP$CRFILE_SEP_LINE=0x000ASNP$CRFILE_FIRST_ROW=0SNP$CRFILE_ENC_FIELD=SNP$CRFILE_DEC_SEP=SNP$CRSNP$CRDWG_COLSNP$CRCOL_NAME=descrSNP$CRTYPE_NAME=STRINGSNP$CRLINE_OFFSET=1SNP$CRLENGTH=994SNP$CRPRECISION=994SNP$CR$$SNPS_END_KEY*/
Any idea why this sysdate part is not showing in Header. -
How to create sqlplus output spool to a flat file with a header record
Hi all,
I've requirement to spool data from a table to a flat file along with column headings in the first row.I'm getting data but I need header record in the first row also?
Thanks,
Mahender.Hi, Mahender,
If you give this SQL*Plus command before you start SPOOLing
SET PAGESIZE 50000then you can get the usual SQL*Plus column headings, and they won't repeat unless you have more than 50,000 rows of output.
When I can't do that, I use PROMPT
{spool}
SPOOL foo.txt
PROMPT empno ename job mgr ...
SELECT empno, ename, job, mgr ...
PROMPT will trim leading whitespace (unless you enclose it in quotes), but it will leave spacing between the columns alone.
You could also do a separate query, where you select some literals from dual:SPOOL foo.txt
SELECT ' empno ename job mgr ...'
FROM dual;
SELECT empno, ename, job, mgr ... -
How to remove Header and Footer from Flat File
Hi,
In a scenario , we are going to recieve a flat file which will contains header , footer and data. We have to have to load data in Oracle tables and remove header and footer.
once the data load is complete , we have to delete flat file from source directory.
Please help me in solving this issue.
Thanks,
KamleshYou have a header parameter in the file datastore. You can provide any number to skip rows from top of the file. For footer you can create a filter in staging area.
Filter coould be something like this.
where source_alias.source_column not like '%FOOTER_OR_ANY_VALUE_AS_PER_YOUR_REQ' -
SSIS : How to create Column Header dynamically using expression in Flat File Source
Hi Team,
I need to keep configured Header Names for columns, Is there is any way to set each column name from expression? or is there is any other way?Nope
But you could add a dummy row to your source to include column headers and then use options column headers in first row in flat file connection manager.
So suppose you've three columns column0,coulmn1,column2 and you want to make it as ID,Name,Datethen make source query as
SELECT 'ID' AS Col1,'Name' AS Col2,'Date' AS Col3, 0 AS ord
UNION ALL
SELECT Column1,Column2,Column3,1
FROM YourTable
ORDER BY Ord
then choose column headers in first row option
Please Mark This As Answer if it helps to solve the issue Visakh ---------------------------- http://visakhm.blogspot.com/ https://www.facebook.com/VmBlogs -
Flat file header diferent from remaining file (function module GUI_UPLOAD)
Hello gurus,
I need to upload a flat file but the header is the control record with the number of lines in the file, and has a diferent structure from remaining file records.
I'm using GUI_UPLOAD in ASCII mode for the flat file and my first line with the control record does not "fit" the structure of the remaining records.
Whats the best way I read diferent line structures in the flat file so I can have a counter from the first line and all other lines in an internal table?
Using GUI_UPLOAD twice with diferent internal tables will have performance degraded...
regards.
Sérgio.Hi,
Declare an itab as
ITAB TYPE STANDARD TABLE OF STRING
Variable for Tab limited
data :gc_con_tab TYPE c VALUE cl_abap_char_utilities=>horizontal_tab.
pass the control record to the internal table first then.
loop at it_final into wa_final.
concatenating gc_con_tab will add TABs in your output.
CONCATENATE wa_final-f1 wa_final-f2
into ITABl separated by gc_con_tab.
append ITAB.
endloop.
Then use GUI_DOWNLOAD to download ITAB.
Regards,
Shanmugavel chandrasekaran -
Variable(Dynamic) Header & Trailer in Flat File
Hi,
I am looking to create Header and trailer in flat file; What I am looking for is the best approach that is flexible to handle dynamic/variable header and footer for multiple systems needing header and footer (in different formats).
Thanks
TusharPackage parameter indicating the environment and a DerivedColumn transform that generates a column with T or P depending on the parameter value.
-
Flat file export with a Constant header record???
I have a mapping that export the records of a table in a Flat File.
But I need to agreggate a constant in the first line of the export file (Header Record)
For example AAABBBCCCDDDEEEFFFGGG
How Can I put a Constant Header record?Marcelo,
When you use flat file as a target you can set the header to be your field names. Now... it looks like this is not exactly what you are looking for.
Alternative is to use a pre mapping procedure to create your file with the one line in it, and have the mapping produce additional records that are appended to the first line. Whether you create a new file or append to an existing file is a property on the target file, if I am not mistaken (it would be a configuration property otherwise).
Thanks,
Mark. -
Step by Step details on how to load data from a flat file
hi can anyone explain how to load data from a flat file. Pls giv me step by step details. thnx
hi sonam.
it is very easy to load data from flat file. whn compared with other extrations methods...
here r the step to load transation data from a flat file.......
step:1 create a Flat File.
step:2 log on to sap bw (t.code : rsa1 or rsa13).
and observe the flat file source system icon. i.e pc icon.
step:3 create required info objects.
3.1: create infoarea
(infoObjects Under Modeling > infoObjects (root node)-> context menu -
> create infoarea).
3.2: create char /keyfig infoObject Catalog.(select infoArea ---.context menu --->create infoObject catalog).
3.3: create char.. infoObj and keyFig infoObjects accourding to ur requirement and activate them.
step:4 create infoSource for transaction data and create transfer structure and maintain communication structure...
4.1: first create a application component.(select InfoSources Under modeling-->infosources<root node>>context menu-->create applic...component)
4.2: create infoSource for transation data(select appl..comp--.context menu-->create infosource)
>select O flexible update and give info source name..
>continue..
4.4: *IMp* ASSIGN DATASOURCE..
(EXPAND APPLIC ..COMP..>EXPAND YOUR INFOSOURCE>CONTEXT MENU>ASSIGN DATASOURCE.)
>* DATASOURCE *
>O SOURCE SYSTEM: <BROWSE AND CHOOSE YOUR FLAT FILE SOURCE SYSTEM>.(EX:PC ICON).
>CONTINUE.
4.5: DEFINE DATASOURCE/TRANSFER STRUCTURE FOR IN FOSOURCE..
> SELECT TRANSFER STRUCTURE TAB.
>FILL THE INFOOBJECT FILLED WITH THE NECESSARY INFOOBJ IN THE ORDER OR SEQUENCE OF FLAT FILE STRUCTURE.
4.6: ASSIGN TRANSFER RULES.
---> NOW SELECT TRANSFER RULES TAB. AND SELECT PROPOSE TRANSFER RULES SPINDLE LIKE ICON.
(IF DATA TARGET IS ODS -
INCLUDE 0RECORDMODE IN COMMUNICATION STRUCTURE.)
--->ACTIVATE...
STEP:5 CREATE DATATARGET.(INFOCUBE/ODS OBJECT).
5.1: CREATE INFO CUBE.
-->SELECT YOUR INFOAREA>CONTEXT MENU>CREATE INFOCUBE.
5.2: CREATE INFOCUBE STRUCTURE.
---> FILL THE STRUCTURE PANE WILL REQUIRE INFOOBJECTS...(SELECT INFOSOURCE ICON>FIND UR INFOSOURCE >DOUBLE CLICK > SELECT "YES" FOR INFOOBJECT ASSIGNMENT ).
>MAINTAIN ATLEAST ON TIME CHAR.......(EX; 0CALDAY).
5.3:DEFINE AND ASSIGN DIMENSIONS FOR YOUR CHARACTERISTICS..
>ACTIVATE..
STEP:6 CREATE UPDATE RULES FOR INFOCUDE USING INFOSOURCE .
>SELECT UR INFOCUBE >CONTEXT MENU> CREATE UPDATE RULES.
> DATASOURCE
> O INFOSOURCE : _________(U R INFOSOURCE). AND PRESS ENTER KEY.......
>ACTIVATE.....UR UPDATE RULES....
>>>>SEE THE DATA FLOW <<<<<<<<----
STEP:7 SCHEDULE / LOAD DATA..
7.1 CREATE INFOPACKAGE.
--->SELECT INFOSOURCE UNDER MODELING> EXPAND UR APPLIC.. COMP..> EXPAND UR INFOSOURCE..> SELECT UR DATASOURCE ASSIGN MENT ICON....>CONTEXT MENU> CREAE INFOPACKAGE..
>GIVE INFOPACKAGE DISCREPTION............_________
>SELECT YOUR DATA SOURCE.-------> AND PRESS CONTINUE .....
>SELECT EXTERNAL DATA TAB...
> SELECT *CLIENT WORKSTATION oR APPLI SERVER >GIVE FILE NAME > FILE TYPE> DATA SAPARATER>
>SELECT PROCESSING TAB
> PSA AND THEN INTO DATATARGETS....
>DATATARGET TAB.
>O SELECT DATA TARGETS
[ ] UPDATE DATATARGET CHECK BOX.....
--->UPDATE TAB.
O FULL UPDATE...
>SCHEDULE TAB..
>SELECT O START DATA LOAD IMMEDIATELY...
AND SELECT "START" BUTTON........
>>>>>>>>>>
STEP:8 MONITOR DATA
> CHECK DATA IN PSA
CHECK DATA IN DATA TARGETS.....
>>>>>>>>>>> <<<<<<<<<----
I HOPE THIS LL HELP YOU.....
Maybe you are looking for
-
I have updated my iphone 5 to new ios 7 and I did not like it so I like to restore back to old ios 6.1.4. please let me know how to restore to ios 6.1.4.
-
Resource bundle in the task definition
Hi all, I'm trying to assign my own resource bundle to a human task definition like specified in the developer doc (http://download-west.oracle.com/docs/cd/B31017_01/integrate.1013/b28981/workflow.htm#BABDFDHC). The documentation is not clear at all
-
Does WebLogic 6.0 beta work with WebGain Studio 4.0 like 5.1 did? Thx--GR
-
Multiple contact persons for single vendor--LSMW
Hi experts, i want to upload multiple contact persons for single vendor. using lsmw direct input method or recording method is it possible..if possible can u let me know the procedure. Thx in advance, Neelu
-
Buttons in Fireworks - Studio 8
This may be basic, I am not sure. I need to copy a button I have designed with all the over states and I want the copy to be editable with different text. Every time I copy a button, when I edit the text, the text in the original button changes also.