Flat file in different node

Post Author: Mokimorris
CA Forum: Data Integration
Is there a way to access flat file which is not in Job Server / Local PC and is in any third unix machine.

Post Author: bhofmans
CA Forum: Data Integration
The way to accomplish this would be by sharing the folder where the file is located so that it can be accessed by the job server. If this is sharing between Windows and UNIX, you would need to use SAMBA or something similar.
Another way would be to use ftp and let the job ftp the file to the jobserver before processing it. There are built-in options to enable ftp in Data Integrator.

Similar Messages

InfoPackage to load multiple flat file for different plant

Hi
There is a requirements to load the data from multiple flat files by different plant into BI which source data is coming from other system. Each flat file has it own naming convention.
As data has to load via infopackage into the targeted cube. Then I have to create multiple infopackage in order to load the data according to the number of flat files that provided. However, new plant will be created due to expansion of bussiness. If that is the case, additional infopackage is required to create as and when there is a new plant.
Is there a way to create a infopackage to load the data from flat file by plant? Your advice is much appreciated.
Thanks and regards
Kang Ring

Hi,
The best way I can suggest is....
1. Create a folder in AL11 for your requirements.
2. In InfoPackage under Extraxction Tab you can have File Name , just side of this you can fine ABAP Routines ICon there you click on that and asks Name give some and then it will takes to your into ABAP Code there you can access the AL11 and then write code in such way that to take file irrespective of the File name, whatever the file is there in that Folder it ahoud take from that AL11 Folder path.
3. Load the Data using PC.
4.At the end of the PC Delete the File in AL11 folder , becasue we don't need it for next load.
Else
See the thread
Re: Automation of load from application server
Thanks
Reddy
Edited by: Surendra Reddy on Jun 4, 2010 11:03 AM

SSIS project - read multiple flat files with different formats

hi all,
i need to import multiple flat files with different formats into different tables of the sql server database and not able to figure out the best way out in ssis to do so...
please advise the possible methods in ssis to do so and if possible the process which can be dynamic as file names or columns might change in future.

Hi AK1987,
To import flat files with dynamic columns, we can use Script Task inside a Foreach Loop Container to parse the first row of the flat file to get the columns names and save them into a .NET variable, then, we can create “Create Table” script based on this
variable, and then store the script into a SSIS package variable. After that, we create a staging table based on the package variable, load the flat file data to the staging table. Eventually, we load data from the staging table to the destination table. For
the detail steps, please walk through the following blog:
http://www.citagus.com/citagus/blog/importing-from-flat-file-with-dynamic-columns/
Regards,
Mike Yin
TechNet Community Support

SSIS - Import Multiple flat files with different metadata

Hi ,
I have set of flat files with different metadata structure, I would like to load them into staging tables.
1. ) Can we load the flatfiles into the staging tables with out having multiple data flow task.
2.) If possible , can we programmatically select the staging table based on the metadata of the flatfile and load them.
Please advise.
Thanks
Thiya

Nope in SSIS a data flow task needs to have a fixed metadata. So if your file metadata varies then best option would be use OPENROWSET syntax to pull the data and populate into your staging table. You may also use
SELECT .. INTO StagingTable ... FROM OPENROWSET (...)
syntax to create staging table at runtime based on the file metadata
http://sqlmate.wordpress.com/2012/08/09/use-your-text-csv-files-in-your-queries-via-openrowset/
If you want to do this in SSIS you need to create data flow dynamically using script task and build the metadata
see
http://www.selectsifiso.net/?p=288
Please Mark This As Answer if it helps to solve the issue Visakh ---------------------------- http://visakhm.blogspot.com/ https://www.facebook.com/VmBlogs

Flat file with different layouts in a single file

Hello Friends,
Good Morning.
I am new to ODI - Oralce Data Integrator 11G.
I have a flat file where I had to load the flat file data to a oracle DB ( 11G ).
The flat file has 5 different layouts ( Data of 5 different suppliers coming as a single file and each layout is different ) and each layout corrresponds to one supplier information.
I had to load each layout data into a different table using a single source file ( That has all supplier information).
Please advice.
Thanks/Kumar

Ok below are the layouts.
BASE record
DS01 record
RM01 record
CR01 record
NW01 record
Few more question to you.
Are they always come in sequential like below ?
BASE record
DS01 record
RM01 record
CR01 record
NW01 record
BASE record
DS01 record
RM01 record
CR01 record
NW01 record
BASE record
DS01 record
RM01 record
CR01 record
NW01 record
Do you want only insert or insert/update as well ?

Using ssis import a multiple flat files with different mappings

I have an scenario for import a file
20 different flat file source and having XML mapping document in a table.
get files from that path(i have path and file name extension in table)
i have map this file based on the file name extension in dynamically
create a 20 different staging table based on mapping in dynamically
kindly help for this

Hi Karthick,
As Arthur said, if you don't want to hard code the data flow, you need to read XML from the SQL Server table and use Script Task to implement the dynamic columns mapping. To read XML from a SQL table, you can refer to:
http://blog.sqlauthority.com/2009/02/13/sql-server-simple-example-of-reading-xml-file-using-t-sql/
In the Script Task, we need to parse the first row of the flat file to get the columns information and save that to a .NET variable. Then, the variable will be used to create the query that will be used to create SQL table. For more information, please see:
http://www.citagus.com/citagus/blog/importing-from-flat-file-with-dynamic-columns/
Regards,
Mike Yin
TechNet Community Support

Too many flat files with different formats.

Hi gurus,
i have a headache problems. that is follows:
We will upload data from flat files,but you know there are about 100 .xls files and with different formats.
and as you know the datasource of the flat file must correspond to every fields of the flat file.
but you know there are 100 flat files and if we donot do any optimise,we will create about 100 transformation and 100 datasource to meet our requirements.
but that seems impossible.
is there any good idea to decrease the number of transfomation ?

Hi AK1987,
To import flat files with dynamic columns, we can use Script Task inside a Foreach Loop Container to parse the first row of the flat file to get the columns names and save them into a .NET variable, then, we can create “Create Table” script based on this
variable, and then store the script into a SSIS package variable. After that, we create a staging table based on the package variable, load the flat file data to the staging table. Eventually, we load data from the staging table to the destination table. For
the detail steps, please walk through the following blog:
http://www.citagus.com/citagus/blog/importing-from-flat-file-with-dynamic-columns/
Regards,
Mike Yin
TechNet Community Support

BizTalk Mapper - Looping multiple nodes to map to a single node in a single row (flat file)

Hi everybody,
I'm still new in developing BizTalk app and require some help in this one problem. Appreciate your time and input to help me on this.
Basically I have an XML document as input and a flat file as output. Example for input is as per below. The "Contact" node's maxOccurs here is set to unbounded and could be multiple. (phone, fax, website, telex ...)
<root>
<CustomerName>Company A</CustomerName>
<Contact>
<Type>Phone</Type>
<Locator>03566789</Locator>
<Type>Phone</Type>
<Locator>03566790</Locator>
<Type>Fax</Type>
<Locator>03566795</Locator>
<Type>Telex</Type>
<Locator>03566798</Locator>
<Type>Website</Type>
<Locator>www.companyA.com</Locator>
</Contact>
</root>
The expected output in XML would look like below. The final outcome would be a csv file. Strictly Phone 1, phone 2, fax and telex, the rest would be ignored.
<root>
<CustomerName>Company A</CustomerName>
<Phone1>03566789</Phone1>
<Phone2>03566790</Phone2>
<Fax>03566795</Fax>
<Telex>03566798</Telex>
</root>
Example of expected output result (csv file): CompanyName;Phone1;Phone2;Fax;Telex;
In our case here: Company A;03566789;03566790;03566795;03566798;
Another example could be: Company B;036778911;;036778912;; if only 1 phone number and 1 fax number provided.
I've used Table Looping and Table Extractor and nearly got the desired result except that it is represented in multiple rows instead of one: Example:
Company A;03566789;;;
Company A;;03566790;;;
Company A;;;03566795;;
Company A;;;;03566798;
Any idea how to do the mapping? I'm kind of stuck here and it sounds like an easy problem but i could not find any example to the solution that I need here. Table looping and table extractor is ok to map from single node flat file to multiple nodes but not
the reverse like in this example.
rgds,
sportivo

Hi,
Please refer to below links where similar issue has been answered.
http://social.msdn.microsoft.com/Forums/en-US/biztalkgeneral/thread/ecdff241-6795-4a95-bad7-48fca4410dfb
http://www.epinaki.com/2011/05/other-options-to-using-biztalk-table-looping-functoid-par-i/
I hope this helps you.
Thanks With Regards,
Shailesh Kawade
MCTS BizTalk Server
Please Mark This As Answer If This Helps You.
http://shaileshbiztalk.blogspot.com/

Loading Master Data from a Flat File

Hi Guru, I have flat files of different SAP tables (Transactional data tables and Master Data ), I want to upload this flat files to a DSO, my question is in the case of Master Data Files example SAP table USR03 ( User Address Data) in a Flat file format which is better to create a Master data text and attributes Datasource or create it as a transactional data Datasource to upload it to a DSO.

Hi
Master data datasource should be used to load master data like customer details. customer data is the master data which does not change frequently. customer infoobject may have some attributes like customer name, customer contact number, customer address, etc. we have to maintain these attributes and text data.
Transaction data is something which keeps changing very frequently like price of the material, sales, etc. for loading such data we go for transaction datasources.
As mentioned above too,the changes in the address are very rare, therefore youshould not use transactional datasource.
Regards, Rahul
Edited by: Rahul Pant on Feb 7, 2011 12:24 PM

Load from flat file confusion

hi
i have to load data using 2 flat files with different level of details
file 1 has data:::
Order...........ordertypes....amount$....Rule...
111.................z11.................100.........XYZ1
111.................z12.................100.........XYZ1
112.................z11.................100.........XYZ1
112.................z12.................100.........XYZ1
113.................z11.................100.........XYZ3
113.................z12.................100.........XYZ3
113.................z13.................100.........XYZ3
Now i have file2 data::
Rule..........Date..........Quantity
XYZ1........1.1.10..........111
XYZ3........9.1.10..........222
NOW user needs all this in 1 report!!
how to achieve this??
Create a ODS and store Order Number and Rule as KEY
and load data using 2 flat file loads at different times ??

Whether you should go with infoset or Infocube with some transformation logic depends on the what they want to see on report and how they want to see it. In your example, How would you want to combine the data from two files
Infoset gives you the below combination with quantity replicated on each and every order
Order...........ordertypes....amount$....Rule... Date Quantity
111.................z11.................100.........XYZ1......1.1.10......111
111.................z12.................100.........XYZ1......1.1.10......111
112.................z11.................100.........XYZ1......1.1.10......111
112.................z12.................100.........XYZ1......1.1.10......111
113.................z11.................100.........XYZ3......9.1.10......222
113.................z12.................100.........XYZ3......9.1.10......222
113.................z13.................100.........XYZ3......9.1.10......222
If you want to do some processing on the quantity when you combining, then Infoset might not be optimal
Order...........ordertypes....amount$....Rule... Date Quantity
111.................z11.................100.........XYZ1......1.1.10......111/4
111.................z12.................100.........XYZ1......1.1.10......111/4
112.................z11.................100.........XYZ1......1.1.10......111/4
112.................z12.................100.........XYZ1......1.1.10......111/4
113.................z11.................100.........XYZ3......9.1.10......222/3
113.................z12.................100.........XYZ3......9.1.10......222/3
113.................z13.................100.........XYZ3......9.1.10......222/3
I have to know what exactly they are trying to achieve by combining the data to suggest a method over another
Edited by: Uday Pothireddy on Nov 15, 2010 9:50 PM

3 flat files grouped into 1 flat file

Hi XI Gurus!
IS there a way to merge the contents of all 3 flat files (source) to 1 flat file without using BPM? And IF I have to use BPM, could you please give me a general idea on how I can go on about it? After merging, I have to map the fields to an IDOC (target). Thanks so much for this.
Regards,
SAPenthusiast

Hi,
If you are not looking out for some specific thing and want to add second file at the end of first file and so on, then write a shell script to merge all files in one and then use that shell script in file adapter with option RUN OS Command before processing or you can use output file directly with File to IDoc mapping with FCC.
Assumption of this approach -
1. I assume that your flat files have same structure so that when you use FCC, it will parse all fields.
If this is not the case and all flat files have different structures then you have to use BPM for merging it. As suggested above, you have to use BPM Collect pattern of SAP BAsis component of your ESR or you have to create something similar BPM.
For your scenario using BPM, you have to merge the files first and after merging it, that is output file of the scenario, you have to create another scenario for mapping merged file to Idoc.
In case of BPM
You will have 2 scenario - FIle to file (Merge flat file in One)
2nd will be file to Idoc.
Regards
Aashish Sinha

How to join 5 different tables using SQL to make it to a flat file structur

I am trying to load five differnt tables into one flat file structure table without cartesian product.
I have five different tables Jobplan, Jobtask(JT), Joblabor(JL), Jobmaterial(JM) and Jpsequence(JS) and the target table as has all the five tables as one table.
The data i have here is something like this.
jobplan = 1record
jobtask = 5 records
joblabor = 2 records
jobmaterial = 1 record
jpsequence = 3 records
The output has to be like this.
JPNUM     DESCRIPTION     LOCATION     JT_JPNUM     JT_TASK     JL_JPNUM     JL_labor     JM_JPNUM     JM_MATERIAL     JS_JPNUM     JS_SEQUENCE
1001     Test Jobplan     USA     NULL     NULL     NULL     NULL     NULL     NULL     NULL     NULL
1001     Test Jobplan     USA     1001     10     NULL     NULL     NULL     NULL     NULL     NULL
1001     Test Jobplan     USA     1001     20     NULL     NULL     NULL     NULL     NULL     NULL
1001     Test Jobplan     USA     1001     30     NULL     NULL     NULL     NULL     NULL     NULL
1001     Test Jobplan     USA     1001     40     NULL     NULL     NULL     NULL     NULL     NULL
1001     Test Jobplan     USA     1001     50     NULL     NULL     NULL     NULL     NULL     NULL
1001     Test Jobplan     USA     NULL     NULL     1001     Sam     NULL     NULL     NULL     NULL
1001     Test Jobplan     USA     NULL     NULL     1001     Mike     NULL     NULL     NULL     NULL
1001     Test Jobplan     USA     NULL     NULL     NULL     NULL     1001     Hammer     NULL     NULL
1001     Test Jobplan     USA     NULL     NULL     NULL     NULL     NULL     NULL     1001     1
1001     Test Jobplan     USA     NULL     NULL     NULL     NULL     NULL     NULL     1001     2
1001     Test Jobplan     USA     NULL     NULL     NULL     NULL     NULL     NULL     1001     3
Please help me out with this issue.
Thanks,
Siva
Edited by: 931144 on Apr 30, 2012 11:35 AM

Hope below helps you
CREATE TABLE JOBPLAN
( JPNUM NUMBER,
DESCRIPTION VARCHAR2(100)
INSERT INTO JOBPLAN VALUES(1001,'Test Jobplan');
CREATE TABLE JOBTASK
( LOCATION VARCHAR2(10),
JT_JPNUM NUMBER,
JT_TASK NUMBER
INSERT INTO JOBTASK VALUES('USA',1001,10);
INSERT INTO JOBTASK VALUES('USA',1001,20);
INSERT INTO JOBTASK VALUES('USA',1001,30);
INSERT INTO JOBTASK VALUES('USA',1001,40);
INSERT INTO JOBTASK VALUES('USA',1001,50);
CREATE TABLE JOBLABOR
( JL_JPNUM NUMBER,
JL_LABOR VARCHAR2(10)
INSERT INTO JOBLABOR VALUES(1001,'Sam');
INSERT INTO JOBLABOR VALUES(1001,'Mike');
CREATE TABLE JOBMATERIAL
( JM_JPNUM    NUMBER,
JM_MATERIAL VARCHAR2(10)
INSERT INTO JOBMATERIAL VALUES(1001,'Hammer');
CREATE TABLE JOBSEQUENCE
( JS_JPNUM    NUMBER,
JS_SEQUENCE NUMBER
INSERT INTO JOBSEQUENCE VALUES(1001,1);
INSERT INTO JOBSEQUENCE VALUES(1001,2);
INSERT INTO JOBSEQUENCE VALUES(1001,3);
SELECT   JP.JPNUM        AS JPNUM       ,
         JP.DESCRIPTION AS DESCRIPTION ,
         NULL            AS LOCATION    ,
         NULL            AS JT_JPNUM    ,
         NULL            AS JT_TASK     ,
         NULL            AS JL_JPNUM    ,
         NULL            AS JL_labor    ,
         NULL            AS JM_JPNUM    ,
         NULL            AS JM_MATERIAL ,
         NULL            AS JS_JPNUM    ,
         NULL            AS JS_SEQUENCE
FROM JOBPLAN JP
UNION ALL
SELECT   JP.JPNUM        AS JPNUM       ,
         JP.DESCRIPTION AS DESCRIPTION ,
         JT.LOCATION     AS LOCATION    ,
         JT.JT_JPNUM     AS JT_JPNUM    ,
         JT.JT_TASK      AS JT_TASK     ,
         NULL            AS JL_JPNUM    ,
         NULL            AS JL_labor    ,
         NULL            AS JM_JPNUM    ,
         NULL            AS JM_MATERIAL ,
         NULL            AS JS_JPNUM    ,
         NULL            AS JS_SEQUENCE
FROM JOBPLAN JP, JOBTASK JT
UNION ALL
SELECT   JP.JPNUM        AS JPNUM       ,
         JP.DESCRIPTION AS DESCRIPTION ,
         NULL            AS LOCATION    ,
         NULL            AS JT_JPNUM    ,
         NULL            AS JT_TASK     ,
         JL.JL_JPNUM     AS JL_JPNUM    ,
         JL.JL_labor     AS JL_labor    ,
         NULL            AS JM_JPNUM    ,
         NULL            AS JM_MATERIAL ,
         NULL            AS JS_JPNUM    ,
         NULL            AS JS_SEQUENCE
FROM JOBPLAN JP, JOBLABOR JL
UNION ALL
SELECT   JP.JPNUM        AS JPNUM       ,
         JP.DESCRIPTION AS DESCRIPTION ,
         NULL            AS LOCATION    ,
         NULL            AS JT_JPNUM    ,
         NULL            AS JT_TASK     ,
         NULL            AS JL_JPNUM    ,
         NULL            AS JL_labor    ,
         JM.JM_JPNUM     AS JM_JPNUM    ,
         JM.JM_MATERIAL AS JM_MATERIAL ,
         NULL            AS JS_JPNUM    ,
         NULL            AS JS_SEQUENCE
FROM JOBPLAN JP, JOBMATERIAL JM
UNION ALL
SELECT   JP.JPNUM        AS JPNUM       ,
         JP.DESCRIPTION AS DESCRIPTION ,
         NULL            AS LOCATION    ,
         NULL            AS JT_JPNUM    ,
         NULL            AS JT_TASK     ,
         NULL            AS JL_JPNUM    ,
         NULL            AS JL_labor    ,
         NULL            AS JM_JPNUM    ,
         NULL            AS JM_MATERIAL ,
         JS.JS_JPNUM     AS JS_JPNUM    ,
         JS.JS_SEQUENCE AS JS_SEQUENCE
FROM JOBPLAN JP, JOBSEQUENCE JS;
     JPNUM DESCRIPTION     LOCATION      JT_JPNUM    JT_TASK   JL_JPNUM JL_LABOR     JM_JPNUM JM_MATERIA   JS_JPNUM JS_SEQUENCE
      1001 Test Jobplan    NULL       NULL        NULL       NULL       NULL       NULL       NULL    NULL          NULL
      1001 Test Jobplan    USA        1001        10         NULL       NULL       NULL       NULL    NULL          NULL
      1001 Test Jobplan    USA        1001        20         NULL       NULL       NULL       NULL    NULL          NULL
      1001 Test Jobplan    USA        1001        30         NULL       NULL       NULL       NULL    NULL          NULL
      1001 Test Jobplan    USA        1001        40         NULL       NULL       NULL       NULL    NULL          NULL
      1001 Test Jobplan    USA        1001        50         NULL       NULL       NULL       NULL    NULL          NULL
      1001 Test Jobplan    NULL       NULL        NULL       1001       Sam        NULL       NULL    NULL          NULL
      1001 Test Jobplan    NULL       NULL        NULL       1001       Mike       NULL       NULL    NULL          NULL
      1001 Test Jobplan    NULL       NULL        NULL       NULL       NULL       1001       Hammer NULL          NULL
      1001 Test Jobplan    NULL       NULL        NULL       NULL       NULL       NULL       NULL    1001          1
      1001 Test Jobplan    NULL       NULL        NULL       NULL       NULL       NULL       NULL    1001          2
      1001 Test Jobplan    NULL       NULL        NULL       NULL       NULL       NULL       NULL    1001          3
{code}

Numbering nodes when loading hierarchy from a flat file.

Hi All,
I want to load a hierarchy from a flat file. When we number the nodes, do they have to be in a sequence or can I give them any number I want? I want to make the GL account number as the node number since it makes it easy to generate the node number that way. The way I see it, the important thing is the parent node number; so this approach should be fine. Please correct me if I am wrong.
Thanks,
Uday

Hi Uday,
   You need to have a running sequence numbers for Node Id field in the flat file. The file structure is as
NODEID IOBJNM      NODENAME TLEVEL LINK PARENTID
00000001 0GLACC     GL233344             00000001
00000002 0GLACC     GL435556             00000001
00000003 0GLACC     GL545666             00000001
00000004 0costcenter cc433232            00000002
00000005 0costcenter cc433233            00000002
00000006 0costcenter CC555551            00000003
The Parentid will calculated by the system automatically.
when you prepare you file csv please ensure that you have 8 characters for the Node ID " 00000003" other wise you will get error.
And if you want to add GL Account number to your hierarchy then you need to add GL A/c Number as external characteristics in hierarchy and add it to your flat file .
Hope it helps.
Cheers,
Balaji

Different formats of the flat file for the same target

In our deployment, we use plugin code to extract the csv files in the required format. The customers are on same version of datamart, but they are on different versions of source database - from 3.x to 4.5 depending on which version of application they are using. In 4.0, we introduced a new column email in the user table in the source database. Accordingly, plugin will add the field in the csv file. But not all the customers will get the upgraded version of plugin at the same time. So ETL code needs to decide which data flow to process depending on the format of the csv file to load data to the same target table. I made the email field in the target table nullable but it still expects the same format of the csv file with delimiter for null value.
Need help to achieve this. Can I read the structure of the flat file in DS or get the count of delimiters so that I can use a conditional to use different data flow based on the format of the flat files.
Can I make the email column in the flat file optional?
Thanks much in advance.

You can add an email column that maps to null in a query transform for the source that does not contain this column.
Or else you can define two different file formats that map to the same file. One with the column and one without

Unsorted Flat File into IDoc with multiple use of nodes

Hi Experts!
I am facing a little problem. I have a source flat file for a classification where some fields appear several times.
My source flat file looks like this:
item1; field1a
item2; fieldA
item3; fieldxa
item1; field1b
as you can see the item1 exists twice (further appearances are also possible).
Now i have to map the flat file into an IDoc structure
My target IDoc looks like this
Header
-- node1
attribute1
-- node2
the "field1a" and "field1b" has to be mapped into the "attribute1" in "node1". "node1" has to be duplicated for each time an "item1" appears (.. and if item2, item3 etc. appears twice, three ... four times...).
So how can i reach it that the node1 will be duplicated automatically when an item appears twice or more times? I know that it could be possible to work with "SplitByValue"... but for this i need all item1 in an straight order.... but i dont have them in a correct order.
I am looking forward to your suggestions.
Thank you in advance.
Udo

Complex sorting is not or not easy possible with the grafical mapping tool.
Use a sequence mapping. The first mapping is a simple XSLT which does the sort. The second mapping works as usual.
I have an example XSLT which I used for a different purpose:
<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet version="1.0"
xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
<xsl:template match="/">
    <ORDERLIST>
      <xsl:for-each select="ORDERLIST/ITEM">
        <xsl:sort select="ID"/>
        <xsl:copy-of select="."/>
      </xsl:for-each>
    </ORDERLIST>
</xsl:template>
</xsl:stylesheet>
Regards
Stefan

Flat file in different node

Similar Messages

Maybe you are looking for