Handling flat file destinations dynamically

After performing a join operation on two tables i get the below resultset
pid, fname, typename, pname, pcost
1, cad, bars, product-1, 100
2, har, witte, product-2, 120
3, nes, bars, product-3, 119
Now i need to create files with the obtained resultset like
Column 'fname' is the folder name and 'typename' should be the file in the particular folder.
For example the first record should be inserted into file name 'bars.txt' in the folder 'cad'
and third record should be created in file name 'bars.txt' in the folder 'nes'.
Could someone please help me with this?

You can do this using Foreach Loop Container (Collection - Foreach ADO Enumerator).
1. First create 5 variables for pid, fname, typename, pname, pcost
2. Create another variable for the directory where you want to create the folders (Let's say variable name is strDestPath)
3. Create another variable of type "Object" (Let's say variable name is objFnames)
3. Next, use an Execute SQL Task and in the SQL Statement, put your query and the Result Set should be  "Full Result set"
4. On the same task, go to Result Set and set the following
Result Name to 0 ; Variable Name to User::objFnames
5.Next, drop a Foreach Loop Container (FELC) and connect the Execute SQL Task created above to FELC
6.In FELC, go to Collection > Select “Foreach ADO Enumerator” as Enumerator
7.In FELC, select the ADO object source variable : User::objFnames
8.In FELC, go to Variable Mappings and map the variables as
Variable                       
Index
User::Pid         
0
User::fname
1
User::typename         
2
User::pname              
3
User::pcost                 
4
9.Next, drop a Script task inside the FELC
10.Open the script task > and select all your variables as ReadOnlyVariables
11.Next, click Edit Script
12. Add the using System.IO class (just below using Sytem.Windows.Forms) and use the following script
public void Main()
// TODO: Add your code here
Dts.TaskResult = (int)ScriptResults.Success;
String originalPath = (String)Dts.Variables["User::strDestPath"].Value + "\\";
String folderPath = originalPath + (String)Dts.Variables["User::fname"].Value;
MessageBox.Show(folderPath);
if (Directory.Exists(folderPath))
Console.WriteLine("That path exists already.");
//return;
DirectoryInfo yearDi = Directory.CreateDirectory(folderPath);
string Filename = folderPath + "\\" + (String)Dts.Variables["User::typename"].Value + ".txt";
StreamWriter Writer = new StreamWriter(Filename);
Writer.WriteLine(Dts.Variables["User::pname"].Value + " " + Dts.Variables["User::pcost"].Value);
Writer.Close();
Narsimha

Similar Messages

  • Dynamic Column Names in Flat File Destination

    Hello,
    Inside a Data Flow Task, I have an ADO.Net data source which executes a stored procedure that provides results in 5 columns.
    The requirement is to have it connect to a flat file destination, such that the column names is dependent on what data was pulled by the data source. There is a variable indicator which identifies the data that was pulled. For example:
    If the indicator is 0, then the columns names will be A,B,C,D,E. Otherwise, if the indicator is 1, then column names will be V,W,X,Y,Z.
    Any suggestions will be of great help.
    AJ

    If you only have two variations then use a branched execution (based on precedence constraints) and direct it to one DFT or another based on the result returned by the stored procedure.
    Otherwise use .net
    code to create one package or another dynamically.
    PS: I suggest not to bother using SSIS for such a simplistic scenario.
    Arthur My Blog

  • Dynamic Header Values in Flat File Destination

    Hi,
    I have to put 'T' or 'P' as part of a header in a flat file destination. The header is sourced via a work table which defines what the header looks like. One of the columns should contain 'T' for test or 'P' for production. When the package is executed in
    the test environment, 'T' should appear as part of the header. When the package is executed in the production environment, 'P' should appear as part of the header.
    Any ideas on how to achieve this is very much welcome.
    Thanks,
    AJ

    Package parameter indicating the environment and a DerivedColumn transform that generates a column with T or P depending on the parameter value.

  • Importing From Flat File with Dynamic Columns

    HI
    I am using ssis 2008,i have folder in which I have Four(4) “.txt” files each file will have 2 columns(ID, NAME). I loaded 4
    files in one destination, but today I receive one more “.txt” file here we have 3 columns (ID, NAME, JOB) how can I get a message new column will receive in source. And how can I create in extra column in my destination table dynamically …please help me

    Hi Sasidhar,
    You need a Script Task to read the names and number of columns in the first row of the flat file each time and store it in a variable, then create a staging table dynamically based on this variable and modify the destination table definition if one ore more
    new columns need to be added, and then use the staging table to load the destination table. I am afraid there is no available working script for your scenario, and you need some .NET coding experience to achieve your goal. Here is an example you can refer
    to:
    http://www.citagus.com/citagus/blog/importing-from-flat-file-with-dynamic-columns/ 
    Regards,
    Mike Yin
    TechNet Community Support

  • Reorder columns in Flat File Destination

    Hi Friend's
    Although, I did some google in order to find the solution for reording the columns before exporting to a text file using FLAT FILE DESTINATION, one of the work around which i found is editing the final package xml file and moving the DTS:FlatFileColumn
    fields as required.
    Is there any other solution in order to avoid this approach as I have some 40 fields to be displayed in text file and manually reordring these fields in xml would be cumbersome.
    I need to create  text file using FLAT FILE DESTINATION.

    SSIS Data flow doesn't support dynamic metadata, and any changes in metadata like re-arrange mappings should be done manually. if you want to have columns in a re-arrange design which could be re-mapped dynamically and simply it is better to looks for
    another way than data flow task, dynamic t-sql queries can be good alternative.
    http://www.rad.pasfu.com
    My Submitted sessions at sqlbits.com

  • [Flat File Destination [220]] Error: Failed to write out column name for column "Column 2"

    I am using SSIS to extract fixed width data into a flat file destination and I keep getting below error. I have tried almost everything in this forum but still no solution. can anyone help me out to solve this problem.
    [Flat File Destination [220]] Error: Failed to write out column name for column "Column 2".
    [SSIS.Pipeline] Error: component "Flat File Destination" (220) failed the pre-execute phase and returned error code 0xC0202095
    Thanks

    Hi Giss68,
    Could you check the Advanced tab of the Flat File Connection Manager to see whether the InputColumnWidth and the OutputColumnWidth properties of the Column2 has the same value? Please refer to the following link about the same topic:
    http://stackoverflow.com/questions/10292091/how-do-i-fix-failed-to-write-error-while-exporting-data-to-ragged-right-flat-fil 
    If it doesn’t work, please post the sample data and the advanced settings of Column2 for further analysis.
    Regards,
    Mike Yin
    If you have any feedback on our support, please click
    here
    Mike Yin
    TechNet Community Support

  • How to handle flat file dissembler in biz-talk from getting spited in to multiple files???

    Hi,
    How to handle flat file dissembler in biz-talk from getting spited into multiple files according to number of transaction sets???
    where the Map flow is from Flat file to X12 Standard files.
    Please advise...

    You can go through following links:
    Developing
    Custom Pipeline Components
    How
    to Develop BizTalk Custom Pipeline Components - Part1
    Custom
    BizTalk Pipeline Disassembler Component
    Creating
    a custom BizTalk 2010 pipeline component–Part I
    add one more which talks about extending the FFDASM @http://msdn.microsoft.com/en-us/library/ee267856(v=bts.10).aspx
    Regards,
    Rachit
    Vote, if you find it useful.

  • Flat File Destination Multiple Delimiters

    Hi Guys,
    This might been an easy one for you. I need to find out if the data from a table can be loaded into a text file with multiple delimiters or not using SSIS? For example, if I have 2 columns of data from a table,
    ID Name
    100 Mark
    I need to load this into a flat file and the output should be like this
    100,@Mark
    So basically, I need to have 2 delimiters in the flat file destination namely "," and "@". How can this be done.
    Thanks in advance.

    Can't you just fill in two delimiters in the Column Delimiter field of the Flat File Connection Manager (never tried it)?
    Alternatives:
    -In the source query add a @ to the second column: select column1,
    '@' + column2 as column2 from yourTable
    -Add a Derived Column with an expression that adds a @ in front of column2: "@" + [column2]
    Please mark the post as answered if it answers your question | My SSIS Blog:
    http://microsoft-ssis.blogspot.com |
    Twitter

  • Flat file destination error

    hi
    i am adding new field to my existing package which is char(10).
    getting this error:
    [Flat File Destination [139]] Error: Data conversion failed. The data conversion for column "MEMBER_ID" returned status value 4 and status text "Text was truncated or one or more characters had no match in the target code page.".
    i right click flat file destination->i only see external and input column over there,
    there is no output column.
    in flat file connection mamanger ->outputcolumn->its char(10)
    what needs to be change here?

    Hi coool_sweet,
    According to your description, you are trying to add a new source column to a flat file in an existing package.
    Based on the error message, the issue should be caused by text was truncated when load new column to column "Column1" in Flat File Destination. Because the length of new column data should be more than 10 (with some junk values) in the Source,
    while you are trying to convert the column to 10 as the character length.
    To fix this issue, we can increase the length of column "Column1" in Flat File Connection Manager. Alternatively, if you still want to convert the new column to a column with length 10, we can right-click the Source component to select “Show Advanced
    Editor”, then go to tab 'input and output properties' to expand the output column corresponds to the column "Column1", change TrancationRowDiposition property to RD_IgnoreFailure.
    Besides, the issue can also be caused by using some special characters in the new column. We can check the Unicode checkbox on the right hand side of Locale property in Flat File Connection Manager to fix this issue. For more details, please see the following
    thread:
    https://social.msdn.microsoft.com/Forums/sqlserver/en-US/d3605656-4a13-47b6-b96e-45379e2b2a9f/export-to-flat-file-with-unicode-chinese-characters?forum=sqlintegrationservices
    If there are any other questions, please feel free to ask.
    Thanks,
    Katherine Xiong
    Katherine Xiong
    TechNet Community Support

  • Flat file Destination VS Raw file Destination ??

    Team ,
    Can some help me understand few differences and similarities  between Flat File Destination and Raw File Destination ? Thanks in advance .
    Rajkumar Yelugu

    Hi,
    Please check these differences :-
    Flat File Destination: The Flat File Destination component writes data out to a text file in one of the standard flat-file formats: delimited, fixed width, fixed width with row delimiter.
    Raw File Destination: The Raw File Destination is all about raw speed. It is an entirely native format and can be exported and imported more rapidly than any other connection type, in part because the data doesn’t need to pass through a connection
    manager.
    durgesh sharma

  • Header in flat file destination (ssis)

    Hi
    I am creating a header in the flat file destination which has some hard coded values and record count which will get its value from a variable assisgned to a row count transformation in the package.
    There is no problem in hard coded value but when i try to get the value of record count, i always get zero.
    This is how i am doing it.
    In the flat file destination, when we open flat file destination editor, i hard code the values which i want in the HEADER itself right there in the editor.
    For the record count, i go properties of the flat file and select expression. In the expression editor, in property, i am selecting 'HeaderRowDelimiter' and drag that variable in the expression box.
    Let me know if there is any suggestion.
    Thanks

    The value of the rowcount is only known AFTER the Data Flow Task is completely finished... and there header is writen before that.
    Here are some examples:
    http://agilebi.com/jwelch/2008/02/08/adding-headers-and-footers-to-flat-files/
    http://social.msdn.microsoft.com/Search/en-US/sqlserver?query=add%20header%20and%20footer%20ssis&rq=meta:Search.MSForums.ForumID(00e50af7-5f43-43ad-af05-d98b73c1f760)+site:microsoft.com&rn=SQL+Server+Integration+Services+Forum
    Please mark the post as answered if it answers your question | My SSIS Blog:
    http://microsoft-ssis.blogspot.com

  • How to create a dynamic SSIS package for multiple flat file destinations

    Hi,
    I have to create a ssis package which has single data flow task and inside that I have 23 source (sql- select * from - statements)- destination (flat files, 23 distinct) connection.
    Now for each product I have to create separate SSIS package (i.e. if prod=abc then these read select * from abc_tables and 23 abc_ txt files)
    I want to do it dynamically, means only single package and inside that variables will take select * values for each source-dest connection (so i believe 23 variables) and same for destination flat files.
    Let me know. :)
    ANK HIT - if reply helps, please mark it as ANSWER or helpful post

    Sorry It seems you're contradicting yourself. you say I know my source and dest structure and the you're
    asking all I want is to have a dynamic structure
    what does that mean?
    and reading your next sentence
    I want to run a package for 5 products, instead of creating 5 ssis packages with 23 source- dest connection, I would
    like to have one with only 23 source- dest connections 
    What I feel is what you're looking for is to  have a looping structure to loop through each of the 5
    products.
    In that case what you could do is this
    1. Create a object variable in SSIS 
    2. Use a Execute SQL Task to populate the variable with all available products (I think you'll have a master table for that). Set ResultSet property to Full ResultSet and then in ResultSet tab map Object variable to 0 th index
    3. use a ForEachLoop container with ADO .NET recordset enumerator and map to object variable. Create a variable of datatype same as that of Product identifier field to get individual values out
    4. Inside loop create your data flow task with 23 source destination connection. In the query part use a parameter for product field and map it to the variable containing product value to get only data for the product.
    Please Mark This As Answer if it helps to solve the issue Visakh ---------------------------- http://visakhm.blogspot.com/ https://www.facebook.com/VmBlogs

  • How to handle flat file with variable delimiters in the file sender adapter

    Hi friends,
    I have some flat files in the ftp server and hope to poll them into XI, but before processing in XI, I hope to do some content conversion in the file sender adapter, according to the general solution, I just need to specify the field names, field seperator, end seperator, etc. But the questions is:
    The fileds in the test data may have different amount of delimiters (,), for example:
    ORD01,,,Z4XS,6100001746,,,,,2,1
    OBJ01,,,,,,,,,,4,3     
    Some fileds only have 1 ',' as the delimiter, but some of them have multiple ','.
    How can I handle it in the content conversion?
    Regards,
    Bean

    Hi Bing,
    Plz do refer the following blogs u will get an idea:
    File content conversion Blogs
    /people/venkat.donela/blog/2005/03/02/introduction-to-simplefile-xi-filescenario-and-complete-walk-through-for-starterspart1
    /people/venkat.donela/blog/2005/03/03/introduction-to-simple-file-xi-filescenario-and-complete-walk-through-for-starterspart2
    /people/arpit.seth/blog/2005/06/02/file-receiver-with-content-conversion
    /people/anish.abraham2/blog/2005/06/08/content-conversion-patternrandom-content-in-input-file
    /people/shabarish.vijayakumar/blog/2005/08/17/nab-the-tab-file-adapter
    /people/venkat.donela/blog/2005/03/02/introduction-to-simplefile-xi-filescenario-and-complete-walk-through-for-starterspart1
    /people/venkat.donela/blog/2005/03/03/introduction-to-simple-file-xi-filescenario-and-complete-walk-through-for-starterspart2
    /people/venkat.donela/blog/2005/06/08/how-to-send-a-flat-file-with-various-field-lengths-and-variable-substructures-to-xi-30
    /people/anish.abraham2/blog/2005/06/08/content-conversion-patternrandom-content-in-input-file
    /people/shabarish.vijayakumar/blog/2005/08/17/nab-the-tab-file-adapter
    /people/jeyakumar.muthu2/blog/2005/11/29/file-content-conversion-for-unequal-number-of-columns
    /people/shabarish.vijayakumar/blog/2006/02/27/content-conversion-the-key-field-problem
    /people/michal.krawczyk2/blog/2004/12/15/how-to-send-a-flat-file-with-fixed-lengths-to-xi-30-using-a-central-file-adapter
    /people/arpit.seth/blog/2005/06/02/file-receiver-with-content-conversion
    http://help.sap.com/saphelp_nw04/helpdata/en/d2/bab440c97f3716e10000000a155106/content.htm
    Regards,
    Vinod.

  • Handling flat files in 'pass through' mode

    Is it possible to handle text files in PI in a pass through mode, i.e. just picking them up from a file adapter, and using another file adapter to send them out e.g. via FTP? Our customer wants to use the traceability of PI to do this. There would be no need for a transform.
    I suspect the answer is no, because PI processes XML messages. However, I recall that it is possible to send IDocs through in pass through mode, so I wondered if it would be possible to do the same with flat files.
    BR,
    Tony.

    Hi
    yes it is possible , you can send even image file also .
    https://www.sdn.sap.com/irj/sdn/go/portal/prtroot/docs/library/uuid/6d967fbc-0a01-0010-4fb4-91c6d38c5816
    The specified item was not found.
    you just to mention the dummy interfaces ..
    How to send any data (even binary) through XI, without using the Integration Repository ---good one
    this will solve your problem
    Regard's
    Chetan Ahuja
    Edited by: Chetan Ahuja on Sep 17, 2008 12:49 PM

  • Flat File Destination: Broken Rows

    I am exporting data to a csv file. One of the columns has values with so mixed characters which end up confusing the system and breaking the rows before the real end of the row. What I mean is; on the flat file a resultant single row can be broken down into
    two or more rows as a result of the characters in the offending column. I work for the healthcare industry. Please see the example of what I’m talking about below. This is a real value from one of the rows in my table which I have and this value(amongst many
    more under tis column) is causing the affected row to be split into two rows. Where the letters
    OE: is coming up as a new row on my flat file. I want all this to be in the same row. My column delimiter is double dagger and my row delimiter is {CR}{LF}
    Example value:
    Column name: Comments
    Value:
    BIBA: blah blah,
    accidentally
    kicked
    brick
    wall
    to
    lateral
    aspect
    to
    R
    foot.
    reports
    hearing
    a
    crack.
    mobilised
    after
    injury.
    pain
    ++
    swelling
    +
    nil
    obvious
    deformity.
    minimal
    movement
    of
    the
    toes.
    LAS
    obs:
    entonox
    to
    good
    affect
    RR18 98%air
    P66reg 114/75
    5.3mmols
    T36.8
    pain 8-4/10
    PMH:
    Nil
    Allergies-
    nuts&prawns
    OE:
    swelling
    and
    pain
    to
    lateral
    and
    medial
    malleolus
    and 5th
    metatarsals.
    For
    xray
    Mpumelelo

    I think I have managed to get a solution to my problem.
    Instead of using csv file I have decided to use text file as my destination flat file format when exporting the data
    I have used Tab{t} as my column delimiter
    I have left the default system specified                {CR}{LF} for row delimiter
    On my flat file connection manager, I have gone to the properties and changed
    AlwaysCheckForRowDelimiters from True to False.
    After a good fight the above worked. No more broken or split rows :).
    If anyone has a better approach or suggestion it will be gladly welcome.
    Many thanks,
    Mpumelelo

Maybe you are looking for

  • PDF from Google Apps gmail/gdri​ve by Cloudprint​er failed on HP Officejet Pro 8600 Plus e-All-in-O​ne

    Dear forum contributors, 1. HP Officejet Pro 8600 Plus e-All-in-One 2. Linux OpenSuSE 12.2 64-bit 3. Failed , after the failed message is a very long URL to my gmail attachment. 4. This issue occured from the beginning, I have this printer only for o

  • Change 1 page orientation

    So, My app is suppose to have 2 pages... one called "Home" and other called "Almoco". The thing is, the 2nd page needs to be horizontal because of the image I want to add. Any idea? Thanks

  • Inconsistent Execution of Code

    The following class works fine when called from the testclass but shows no graphics when called from another class in my project. import java.awt.*; import java.applet.*; import java.io.*; import javax.swing.*; import java.awt.event.*; import javax.i

  • PO Confirmations Message

    Hi, We are trying to implement an extended classic scenario of EBP and we are using XI to send POs and receive conformation back in XML format. We have implemented as per the ServiceProcurementNEW.pdf document for configuring the XI and SRM settings.

  • Job SAP_PERIODIC_ORACLE_SNAPSHOT

    Hi All, We have checked that job SAP_PERIODIC_ORACLE_SNAPSHOT plays main role in Updating currect space statistics in DB02. But I do not find how this job is getting triggered, I expect some house keeping job (might be PERFORMANCE_COLLECTOR) is trigg