Multiple flat files with Comma delimiter and Pipe Delimiter in the sub folders.

Hi,
I have a directory C:\doc\Outcomes\Health  --(This is the main path). 
In the path above i have multiple subfolders like 
A
B
C
D
Folder A & B have 20 flat files each which are comma separated and pipe delimiter files. 
Folder C&D have 20 excel files each.
1) So, In SSIS while looping through the subfolders how do i limit to loop only excel files one time and flat files one time.
2) In folder A&B, how do i loop only Pipe delimiter files (neglecting comma saperated files). I want to loop only pipe delimiter files while using for each loop container.
Thanks 

Both are txt files, but the data inside the files is saperated by ',' and '|'. ( comma and pipe)
Thats ok 
If delimiters are not consistent you can use this method
http://visakhm.blogspot.in/2014/07/ssis-tips-handling-inconsistent-text.html
Please Mark This As Answer if it solved your issue
Please Mark This As Helpful if it helps to solve your issue
Visakh
My MSDN Page
My Personal Blog
My Facebook Page

Similar Messages

  • SSIS project - read multiple flat files with different formats

    hi all,
    i need to import multiple flat files with different formats into different tables of the sql server database and not able to figure out the best way out in ssis to do so...
    please advise the possible methods in ssis to do so and if possible the process which can be dynamic as file names or columns might change in future.

    Hi AK1987,
    To import flat files with dynamic columns, we can use Script Task inside a Foreach Loop Container to parse the first row of the flat file to get the columns names and save them into a .NET variable, then, we can create “Create Table” script based on this
    variable, and then store the script into a SSIS package variable. After that, we create a staging table based on the package variable, load the flat file data to the staging table. Eventually, we load data from the staging table to the destination table. For
    the detail steps, please walk through the following blog:
    http://www.citagus.com/citagus/blog/importing-from-flat-file-with-dynamic-columns/ 
    Regards,
    Mike Yin
    TechNet Community Support

  • SSIS - Import Multiple flat files with different metadata

    Hi ,
    I have set of flat files with different metadata structure, I would like to load them into staging tables. 
    1. ) Can we load the flatfiles into the staging tables with out having multiple data flow task.
    2.) If possible , can we programmatically select the staging table based on the metadata of the flatfile and load them.
    Please advise.
    Thanks
    Thiya

    Nope in SSIS a data flow task needs to have a fixed metadata. So if your file metadata varies then best option would be use OPENROWSET syntax to pull the data and populate into your staging table. You may also use 
    SELECT .. INTO StagingTable ... FROM OPENROWSET (...)
    syntax to create staging table at runtime based on the file metadata
    http://sqlmate.wordpress.com/2012/08/09/use-your-text-csv-files-in-your-queries-via-openrowset/
    If you want to do this in SSIS you need to create data flow dynamically using script task and build the metadata
    see
    http://www.selectsifiso.net/?p=288
    Please Mark This As Answer if it helps to solve the issue Visakh ---------------------------- http://visakhm.blogspot.com/ https://www.facebook.com/VmBlogs

  • Using ssis import a multiple flat files with different mappings

    I have an scenario for import a file
    20 different flat file source and having XML mapping document in a table.
    get files from that path(i have path and file name extension in table)
    i have map this file based on the file name extension in dynamically
    create a 20 different staging table based on mapping in dynamically
    kindly help for this
     

    Hi Karthick,
    As Arthur said, if you don't want to hard code the data flow, you need to read XML from the SQL Server table and use Script Task to implement the dynamic columns mapping. To read XML from a SQL table, you can refer to:
    http://blog.sqlauthority.com/2009/02/13/sql-server-simple-example-of-reading-xml-file-using-t-sql/ 
    In the Script Task, we need to parse the first row of the flat file to get the columns information and save that to a .NET variable. Then, the variable will be used to create the query that will be used to create SQL table. For more information, please see:
    http://www.citagus.com/citagus/blog/importing-from-flat-file-with-dynamic-columns/ 
    Regards,
    Mike Yin
    TechNet Community Support

  • Loading Pipe delmiter flat file with ',' (comma)

    Hi Guys,
    I have a field which contains ',' as its value. The value is 0315A Mgr, Cust Service for a infoobject called ZJOB_TIT.
    Also, I have wrote the following code in field routine since previously this field was throwing hexadecimal errors and the load is through flat file.
    DATA: v_gpurqtrnr TYPE /BIC/OIZJOB_TIT.
    MOVE SOURCE_FIELDS-/BIC/ZJOB_TIT TO v_gpurqtrnr.
    TRANSLATE v_gpurqtrnr TO UPPER CASE.
    IF v_gpurqtrnr CN
    ',<>?/{}[]\|;:"''ABCDEFGHIJKLMNOPQRSTUVWXYZ!@#$%^&*()_-+=123456789~` .'
      RESULT = space.
    ELSE.
      RESULT = v_gpurqtrnr.
    ENDIF.
    After loading the data, in preview of PSA - everything looks fine. Field ZJOB_TIT shows to have the value 0315A Mgr, Cust Service. But when I load to the cube - I get blank for the same field.
    I am not sure what more i can do to eradicate this problem.

    Hi
    PSA is the data before under going your tranformation or field routine. so you are able to see the data in that field . Now as per the code which you have written above it will surely be deleting all the characteristics which you have given under '.......' . Modify your code to eliminate only the comma and not the alpha numberic values.
    Rajesh

  • How to create flat file with fixed lenght records

    I need help to export an Oracle table to a flat file with fixed lenght and without columns separator.
    the fixed length is the more important demand.
    My table have 50 columns with varchar, date and number .
    Date and number columns may be empty, null o with values.
    Thanks a lot for any help.
    [email protected]

    Hi,
    You can use this trick:
    SQL>desc t
    Name                                      Null?    Type
    NAME                                               VARCHAR2(20)
    SEX                                                VARCHAR2(1)
    SQL>SELECT LENGTH(LPAD(NAME,20,' ')||LPAD(SEX,1,' ')), LPAD(NAME,20,' ')||LPAD(SEX,1,' ') FROM T;
    LENGTH(LPAD(NAME,20,'')||LPAD(SEX,1,'')) LPAD(NAME,20,'')||LPA
                                          21                    aF
                                          21                    BM
                                          21                    CF
                                          21                    DM
    4 rows selected.
    SQL>SELECT *  FROM t;
    NAME                 S
    a                    F
    B                    M
    C                    F
    D                    M
    4 rows selected.Regards

  • Flat File with unequal spacing as delimiter

    I am trying to import multiple flat files with unequal spacing as delimiter in sql server using SSIS 2012, however when I import the file, all the data in the file is imported into only one column. None of the predefined delimiters in SSIS works with the
    way the flat file was configured. I read that I can use the script component in SSIS 2012 to create the additional columns and somehow manipulate the file using the spacing to assign a new delimiter, such as a common, but I can't find any resources on it.
    Anyone have any experience/knowledge of this situation can provide me with an example or some resources that can help me to solve this situation?
    Example of file format:
    26   SM FRIES         0      1     0       0.00        0.0
    30   PEPSI               0    22     0       0.00        0.0

    Looks like a fixed width file to me seeing the data posted
    If you can determine the width of the columns then you can use fixed width (or ragged right) option in flat file connection manager and identify the lengths of the individual fields to get values out.
    See this example
    http://gaurangpatel.net/importing-fixed-width-file-into-database-using-ssis-package
    Please Mark This As Answer if it solved your issue
    Please Vote This As Helpful if it helps to solve your issue
    Visakh
    My Wiki User Page
    My MSDN Page
    My Personal Blog
    My Facebook Page

  • 1 xml file to multiple xml files with 200 records in each

    Hello Experts,
    I have below scenario.
    Please suggest me which might be the best approch to work on.
    1) XML file to XML file
    I will have 1 pretty huge XML file which i need to break in multiple XML files with 200 records each.
      1) first approch i can have bpm in which i can split the file according to my requirement.
      2) Second approch i can create 2 scenarios in which 1st scenario will pick up XML file and create multiple flat files with File content conversion. Second scenario will pick up all these flat files and create XML files.
    2) XML file to XML file
    Or i can have multiple files with 1 record per file and i need to merge in multiple XML files with 200 records in each.
    So its kind of 1:N or M:N scenarios.
    Please tell me which is might be better performance and design wise.
    Or if you have any idea in any other way i can do this, then please reply as soon as possbile.
    Please tell me if you have OS command for the same or some script to run or anything which i can implement.
    Thanks,
    Hetal

    what is your senario? is it File to File?
    You can use multi mapping concept without BPM. You can handle the 200 records per message logic in the multimapping.
    Regards,
    Praveen Gujjeti.

  • Infospoke  flat file with tab seperator

    Hello,
    I have a infospoke which creates a flat file with comma field separator. I would like to have tab character as a field seperator. How would I change it.

    Varun,
       as Ram told you can give ',' as Separator.
    all the best.
    Reagrds,
    Nagesh Ganisetti.

  • Create a flat file with multiple characters for enclosures

    Hello,
    we use OWB 11g2 (11.2.02).
    Now we try to create a flat file with multiple characters for enclosures. The manual wrote:
    "Enclosures (Left and Right): Some delimited files contain enclosures that denote text strings within a field. If the file contains enclosures, enter an enclosure character in the text box or select one from the list. The list displays commonenclosures. However, you may enter any character. The default for both the left and right enclosure is the double quotation mark ("). You can specify multiple characters and hexadecimal characters as field enclosures."
    But it will not work. The OWB use the first character from the left enclosure definition as left enclosure and the second one as right enclosure !?!
    Did anyone know this behavior? Is there a solution for this problem?
    Thanks and regards
    Norbert

    HI Raghu,
               Use the function module 'GUI_UPLOAD'.
               In that you have to specify the field_separator value = 'X' in export section.
    Regards,
    S.C.K

  • Loading Multiple Flat File in to BI System with one InfoPackage

    Greetings,
    i have developed a routine to Load Multiple Flate File in one InfoPackage, but it doesn't work. Only the Last File are loaded. Who can Help me?
    Loading 'R:\Verzeichnis \ing_wan_b_002.fall.csv' to 'R:\Verzeichnis \ing_wan_b_240*.fall.csv'
    Routine:
    DATA: VerzUndDateiname TYPE string value 'R:\Verzeichnis \ing_wan_b_000.fall.csv',
          VerzUndDateinameKomplett TYPE string,
          count TYPE i value 2,
          countN(3) TYPE n.
    WHILE count <= 240.
    countN = count.
    VerzUndDateinameKomplett = VerzUndDateiname.
    REPLACE '000' WITH zaehlerN
                  INTO VerzUndDateinameKomplett.
    p_filename = VerzUndDateinameKomplett.
    count = count + 1.
    ENDWHILE.
    Best Regards
    Jens
    Edited by: JB6493 on May 18, 2009 1:03 PM
    Edited by: JB6493 on May 18, 2009 1:07 PM

    Hello Jens,
    you have to process the InfoPackage 239 times. Your routine would be executed once during one processing of the InfoPackage. This would run the WHILE statement and end up with the last filename after 239 iterations.
    Either try to concatenate the files you want to load (if possible) and load them in one go. Alternatively you could try to use a process chain to run the InfoPackage 239 times.
    Kind regards,
    Christoph

  • Export data to flat file with no delimiter

    Hi I've 3 columns in a view that i want to export to a flat file. I don't want the flat file to contain any delimiter such as commas , tabs etc..
    I am using fixed width with the following setting
    Header row delimiter {CR}-{LF}
    Advance Options : 
    InputColumnwith  125
    InputColumnwith 187
    InputColumnwith  7
    When i preview that data it looks correct , divided up into 3 columns
    However when the data is exported to the flat file i get just one long string in the flat file on the first row.
    What do i need to do to get the end of line characters to be identified 
    Thanks In advance 

    Column delimiter is here
    Otherwise header row terminates a file line
    Arthur
    MyBlog
    Twitter

  • Flat File with multiple record types (OWB 10.2.0.2)

    Hi!
    I`m using OWB 10.2.0.2 and I`m trying to load a flat file with multiple record types, using SQL LOADER.
    In the flat file editor in the Record tab, I`ve set the type values and the corresponding record names like this:
    Type Value Record Name
    ======== ===========
    T TRAILER
    0 DETAILS
    1 DETAILS
    2 DETAILS
    When using this flat file in a mapping to load the data in a staging table, the generated code looks like this:
    INTO TABLE TRAILER
    TRUNCATE
    REENABLE DISABLED_CONSTRAINTS
    WHEN (1:1) = 'T'
    INTO TABLE DETAILS
    APPEND
    REENABLE DISABLED_CONSTRAINTS
    WHEN (1:1) = '0,1,2'
    The above clause (WHEN (1:1) = '0,1,2') is wrong as I expect one "INTO TABLE..." clause for each record type.
    Could this be a bug or am I doing something wrong?
    Thanks a lot for your help,
    Yorgos

    We`re using two target tables, one for the trailer record and the other for the details records.
    We are facing this problem from the moment we upgraded from OWB 10.1 to OWB 10.2.0.2, so we think it must be something with the way the sql loader code is generated in the new version.
    As our data sources are mainly flat files coming from mainframes, this is a huge problem for us. We even asked an expert in DW from Oracle to help us on this, but still haven`t found a solution.
    Is there any workaround for this or should we forget sql loader and go with an external tables + custom PL/SQL code solution?
    Your help is greatly appreciated Jean-Pierre.
    Regards,
    Yorgos

  • Target flat file with pipe seperated

    Hi
    I want to load the data in flat file , i don't see pipe "|" seperation values to create a flat file in this format. how can i achive and how can i also how can i create file name dynamically
    Thanks

    Just type in your desired character.
    Cheers
    David

  • Re: Upload a flat file with multiple field seperators

    HI Experts,
    In my reports program iam trying to  upload  the data with in flat file with multiple seperators
    but data is not uploading can ypu please suggest the solution.
    Is there any function module to upload with multiple field seperators
    Example of flat file:
    E1234     JOHN     BB     HYD     IN     EN
    E1234,BURE,BH,DSNR,IN,EN
    E1236 KIWI KW HYT IN EN
    E1237;BRAD;BW;MSRD;IN;EN
    Thanks & Regards,
    Raghusagar.G

    HI Raghu,
               Use the function module 'GUI_UPLOAD'.
               In that you have to specify the field_separator value = 'X' in export section.
    Regards,
    S.C.K

Maybe you are looking for