Multiple flat file in and multiple target tables

Hi,
How can we have multiple flat file into multiple targets.
I am trying to load data from multiple flat files into respective tables. But it gives error like
VLD-2411: Cannot handle two file structures
Make sure that only one file structure is used in a SQL*Loader map
Can anyone help.
Regards
Rakesh Kumar

I donot thing in one mapping you can take multiple sqlloader file.
If want to load data form multiple file use External table.

Similar Messages

How to extract data from multiple flat files to load into corresponding tables in SQL Server 2008 R2 ?

Hi,
          I have to implement the following scenario in SSIS but don't know how to do since I never worked with SSIS before. Please help me.
          I have 20 different text files in a single folder and 20 different tables corresponding to each text file in SQL Server 2008 R2 Database. I need to extract the data from each text file and
load the data into corresponding table in Sql Server Database. Please guide me in how many ways I can do this and which is the best way to implement this job. Actually I have to automate this job. Few files are in same format(with same column names
and datatypes) where others are not.
1. Do I need to create 20 different projects ?
               or
    Can I implement this in only one project by having 20 packages?
             or
    Can I do this in one project with only one package?
Thanks in advance.

As I said I don't know how to use object data type, I just given a shot as below. I know the following code has errors can you please correct it for me.
Public
Sub Main()
' Add your code here
Dim f1
As FileStream
Dim s1
As StreamReader
Dim date1
As
Object
Dim rline
As
String
Dim Filelist(1)
As
String
Dim FileName
As
String
Dim i
As
Integer
i = 1
date1 =
Filelist(0) =
"XYZ"
Filelist(1) =
"123"
For
Each FileName
In Filelist
f1 = File.OpenRead(FileName)
s1 = File.OpenText(FileName)
rline = s1.ReadLine
While
Not rline
Is
Nothing
If Left(rline, 4) =
"DATE"
Then
date1 (i)= Mid(rline, 7, 8)
i = i + 1
Exit
While
End
If
rline = s1.ReadLine
End
While
Next
Dts.Variables(
"date").Value = date1(1)
Dts.Variables(
"date1").Value = date1(2)
Dts.TaskResult = ScriptResults.Success
End
Sub

Loading from multiple flat files to same table using SQL Loader

Hi Gurus,
Can anyone please brief me the pros and cons of kicking of multiple sql loader sessions that reads multiple flat files but inserting it into just one table.
The table is not partitioned. Avg record counts for each flat file is about 5-6 million.
Oracle 11g,
OS: Linux
Regards
Cherrish Vaidiyan

Vaidiyan wrote:
Hi Gurus,
Can anyone please brief me the pros and cons of kicking of multiple sql loader sessions that reads multiple flat files but inserting it into just one table.Cherrish,
Pros -> Faster loading of more data
Cons -> Potential performance degradation
Test to see how much resource consuming this task would be and do a priority comparison of that multi-multi load task with other stuff that will be happening in the database in the multi-multi load time so you could decide how to share resources in that time.

Loading Multiple Flat File around 80+

Hi,
Loading multiple flat file in to BI system.
My issue: I got a scenerio of loading multiple i.e., 80+ flat file into BI system every month and my client want it to be done through process chain r automation with out manual intervention.
Can any one suggest me how can i achiev this one.
Regards,
Prabhakar.

Hi All,
We have developed a logic to upload multiple flat file at a time i.e., .CSV file. Please find below is the logic regarding that routine.
*+ program filename_routine.
Global code
$$ begin of global - insert your declaration only below this line -
Enter here global variables and type declarations
as well as additional form routines, which you may call from the
main routine COMPUTE_FLAT_FILE_FILENAME below
data : v_filename type string.
data : v_foldername type string.
data : v_uploadfile type string.
data: v_download_file type string.
data : v_ext(4) type c.
data : v_FILE_EXISTS type c.
data : v_file type DXFILE-FILENAME.
data : v_count(3) type c.
data : v_length type i.
TYPES: BEGIN OF TY_file,
        LINE(900),
      END OF TY_file.
DATA: GIT_file TYPE TABLE OF TY_file.
$$ end of global - insert your declaration only before this line   -
form compute_flat_file_filename
using    p_infopackage type rslogdpid
           p_datasource   type rsoltpsourcer
           p_logsys       type rsslogsys
changing p_filename     type RSFILENM
           p_subrc        like sy-subrc.
$$ begin of routine - insert your code only below this line        -
This routine will be called by the adapter,
when the infopackage is executed.
v_count = 1.
*---- As per the folder location of the files & system we have to change
*the path fo the folder in the below variable i.e., 'c:\............
v_foldername =
'C:\Documents and Settings\Prabhakar-HP\Desktop\test\'.
*---- As per the Prefix of the files like 'CSI_01_..' we have change the
*prefix if required in the below variable i.e., 'CSI_01_..', IF REQUIRED
*OR IF WANT TO CHANGE THE PREFIX TO SOME OTHER.
v_filename = 'CSI_01_'.
v_ext = '.csv'.
condense v_count.
concatenate   v_foldername
                v_filename
                v_count
                v_ext       into v_uploadfile.
v_file = v_uploadfile.
CALL FUNCTION 'DX_FILE_EXISTENCE_CHECK'
    EXPORTING
      FILENAME       = v_file
      PC             = 'X'
    IMPORTING
      FILE_EXISTS    = v_FILE_EXISTS
    EXCEPTIONS
      RFC_ERROR      = 1
      FRONTEND_ERROR = 2
      NO_AUTHORITY   = 3
      OTHERS         = 4.
IF SY-SUBRC <> 0.
    MESSAGE ID SY-MSGID TYPE SY-MSGTY NUMBER SY-MSGNO
    WITH SY-MSGV1 SY-MSGV2 SY-MSGV3 SY-MSGV4.
ENDIF.
while v_FILE_EXISTS = 'X'.
    v_file = v_uploadfile.
    CALL FUNCTION 'DX_FILE_EXISTENCE_CHECK'
      EXPORTING
        FILENAME       = v_file
        PC             = 'X'
      IMPORTING
        FILE_EXISTS    = v_FILE_EXISTS
      EXCEPTIONS
        RFC_ERROR      = 1
        FRONTEND_ERROR = 2
        NO_AUTHORITY   = 3
        OTHERS         = 4.
    IF SY-SUBRC <> 0.
      MESSAGE ID SY-MSGID TYPE SY-MSGTY NUMBER SY-MSGNO
      WITH SY-MSGV1 SY-MSGV2 SY-MSGV3 SY-MSGV4.
    ENDIF.
    if v_file_exists = 'X'.
      CALL FUNCTION 'GUI_UPLOAD'
        EXPORTING
          FILENAME                = v_uploadfile
          FILETYPE                = 'ASC'
          HAS_FIELD_SEPARATOR     = 'X'
        TABLES
          DATA_TAB                = GIT_file
        EXCEPTIONS
          FILE_OPEN_ERROR         = 1
          FILE_READ_ERROR         = 2
          NO_BATCH                = 3
          GUI_REFUSE_FILETRANSFER = 4
          INVALID_TYPE            = 5
          NO_AUTHORITY            = 6
          UNKNOWN_ERROR           = 7
          BAD_DATA_FORMAT         = 8
          HEADER_NOT_ALLOWED      = 9
          SEPARATOR_NOT_ALLOWED   = 10
          HEADER_TOO_LONG         = 11
          UNKNOWN_DP_ERROR        = 12
          ACCESS_DENIED           = 13
          DP_OUT_OF_MEMORY        = 14
          DISK_FULL               = 15
          DP_TIMEOUT              = 16
          OTHERS                  = 17.
      IF SY-SUBRC <> 0.
       MESSAGE ID SY-MSGID TYPE SY-MSGTY NUMBER SY-MSGNO
               WITH SY-MSGV1 SY-MSGV2 SY-MSGV3 SY-MSGV4.
      elseif sy-subrc = 0.
**----- As per the REQUIRMENT, want to change the down loading file
*path means we have to change the path in the below variable i.e.,
*'c:\.......
v_download_file =
'C:\Documents and Settings\Prabhakar-HP\Desktop\test\satheesh.CSV'.
        CALL FUNCTION 'GUI_DOWNLOAD'
          EXPORTING
            FILENAME                = v_download_file
            FILETYPE                = 'ASC'
            APPEND                  = 'X'
          TABLES
            DATA_TAB                = git_file
          EXCEPTIONS
            FILE_WRITE_ERROR        = 1
            NO_BATCH                = 2
            GUI_REFUSE_FILETRANSFER = 3
            INVALID_TYPE            = 4
            NO_AUTHORITY            = 5
            UNKNOWN_ERROR           = 6
            HEADER_NOT_ALLOWED      = 7
            SEPARATOR_NOT_ALLOWED   = 8
            FILESIZE_NOT_ALLOWED    = 9
            HEADER_TOO_LONG         = 10
            DP_ERROR_CREATE         = 11
            DP_ERROR_SEND           = 12
            DP_ERROR_WRITE          = 13
            UNKNOWN_DP_ERROR        = 14
            ACCESS_DENIED           = 15
            DP_OUT_OF_MEMORY        = 16
            DISK_FULL               = 17
            DP_TIMEOUT              = 18
            FILE_NOT_FOUND          = 19
            DATAPROVIDER_EXCEPTION = 20
            CONTROL_FLUSH_ERROR     = 21
            OTHERS                  = 22.
        IF SY-SUBRC <> 0.
MESSAGE ID SY-MSGID TYPE SY-MSGTY NUMBER SY-MSGNO
        WITH SY-MSGV1 SY-MSGV2 SY-MSGV3 SY-MSGV4.
        ENDIF.
        refresh git_file.
      ENDIF.
    endif.
    if v_file_exists = 'X'.
      if p_subrc = 0.
        v_count = v_count + 1.
        condense v_count.
        concatenate
             v_foldername
             v_filename
             v_count
             v_ext into v_uploadfile.
      endif.
    else.
    endif.
endwhile.
v_uploadfile = v_download_file.
p_filename = v_uploadfile.
p_subrc = 0.
$$ end of routine - insert your code only before this line         -
endform. +*
Reagrds,
Prabhakar.

Multiple flat files appended, is file locked until all processed?

Hi All,
Have a scenario in place where we are collecting multiple flat files on ECC and using append mode on the receiver file adapter to create a single file for final output. We are only interested in submitting the output to it's target after all files sent to PI have been appended.
Does PI 7.0 lock the appended file until all files have been appended so I am sure I end up with a complete single file on output? If not, what are my options to ensure a completely appended file is output? Would it be a viable option to pass the appending file through another sender file adapter using 'msecs to wait before modification check'?
Thanks!
Don
Edited by: donald.meredith on Dec 1, 2011 2:31 PM

How many files are you collecting from ECC and appending at receiver side and how much time it is taking to complete the process?
if its possible to schedule your business to read data from FTP after completing append then no worries, else sometimes it creates trouble.
The option you mentioned is reliable one , but you ended up creating one more interface.
Regards,
Raj

InfoPackage to load multiple flat file for different plant

Hi
There is a requirements to load the data from multiple flat files by different plant into BI which source data is coming from other system. Each flat file has it own naming convention.
As data has to load via infopackage into the targeted cube. Then I have to create multiple infopackage in order to load the data according to the number of flat files that provided. However, new plant will be created due to expansion of bussiness. If that is the case, additional infopackage is required to create as and when there is a new plant.
Is there a way to create a infopackage to load the data from flat file by plant? Your advice is much appreciated.
Thanks and regards
Kang Ring

Hi,
The best way I can suggest is....
1. Create a folder in AL11 for your requirements.
2. In InfoPackage under Extraxction Tab you can have File Name , just side of this you can fine ABAP Routines ICon there you click on that and asks Name give some and then it will takes to your into ABAP Code there you can access the AL11 and then write code in such way that to take file irrespective of the File name, whatever the file is there in that Folder it ahoud take from that AL11 Folder path.
3. Load the Data using PC.
4.At the end of the PC Delete the File in AL11 folder , becasue we don't need it for next load.
Else
See the thread
Re: Automation of load from application server
Thanks
Reddy
Edited by: Surendra Reddy on Jun 4, 2010 11:03 AM

SSIS project - read multiple flat files with different formats

hi all,
i need to import multiple flat files with different formats into different tables of the sql server database and not able to figure out the best way out in ssis to do so...
please advise the possible methods in ssis to do so and if possible the process which can be dynamic as file names or columns might change in future.

Hi AK1987,
To import flat files with dynamic columns, we can use Script Task inside a Foreach Loop Container to parse the first row of the flat file to get the columns names and save them into a .NET variable, then, we can create “Create Table” script based on this
variable, and then store the script into a SSIS package variable. After that, we create a staging table based on the package variable, load the flat file data to the staging table. Eventually, we load data from the staging table to the destination table. For
the detail steps, please walk through the following blog:
http://www.citagus.com/citagus/blog/importing-from-flat-file-with-dynamic-columns/
Regards,
Mike Yin
TechNet Community Support

Directory in application server has multiple flat files

Hi
How should we know that the Directory in application server has multiple flat files.
my requirement is, daily my program runs in batch at night. So, i have to know are there more than 1 flat file existing in the directory.
regards
mac

Try this FM,
CALL FUNCTION 'SUBST_GET_FILE_LIST'
    EXPORTING
      dirname      = ws_filepath
      filenm       = ws_fname
    TABLES
      file_list    = i_rsfillst
    EXCEPTIONS
      access_error = 1
      OTHERS       = 2.
Pass the path and
say ur file name starts with ALPXXXXX
then pass
ws_fname = 'ALP*' .
In the table i_rsfillst you will get all the file in directory.
Try this through a sample prorgam ,dont try through SE37, it will show ACCESS_ERROR.
Regards,
Message was edited by: sreejesh purapadiath
Message was edited by: sreejesh purapadiath

Loading Multiple Flat File in to BI System with one InfoPackage

Greetings,
i have developed a routine to Load Multiple Flate File in one InfoPackage, but it doesn't work. Only the Last File are loaded. Who can Help me?
Loading 'R:\Verzeichnis \ing_wan_b_002.fall.csv' to 'R:\Verzeichnis \ing_wan_b_240*.fall.csv'
Routine:
DATA: VerzUndDateiname TYPE string value 'R:\Verzeichnis \ing_wan_b_000.fall.csv',
      VerzUndDateinameKomplett TYPE string,
      count TYPE i value 2,
      countN(3) TYPE n.
WHILE count <= 240.
countN = count.
VerzUndDateinameKomplett = VerzUndDateiname.
REPLACE '000' WITH zaehlerN
              INTO VerzUndDateinameKomplett.
p_filename = VerzUndDateinameKomplett.
count = count + 1.
ENDWHILE.
Best Regards
Jens
Edited by: JB6493 on May 18, 2009 1:03 PM
Edited by: JB6493 on May 18, 2009 1:07 PM

Hello Jens,
you have to process the InfoPackage 239 times. Your routine would be executed once during one processing of the InfoPackage. This would run the WHILE statement and end up with the last filename after 239 iterations.
Either try to concatenate the files you want to load (if possible) and load them in one go. Alternatively you could try to use a process chain to run the InfoPackage 239 times.
Kind regards,
Christoph

Flat-File Recon And Provision to DB Apps Table

Hi,
I have a query relating to the process for recon with Oracle HRMS and creating/updating users in OIM and then approval to provision to target resource.
The following is detailed process that I am looking for -
1. Run Recon to get the users from flat file location.
2. Create users in OIM and based on information from flat file trigger approval process for creation of the users in target resource
Please tell me what is the process I need to follow to configure the flat file gtc and also to trigger the approval/provisioning process for target resource.
Thanks.

Hi,
To configure the flat file gtc, just go through Admin console guid there you will find all the required steps for flat file recon. Or you can refer the OIM Labs also.
Now to trigger the db provisioning with approval, you can crete access policy on ALL User group as all the user get created in that particular group. While you will create access policy there is option to check for Approval Required,just chyeck that option and your approval will start tirgger automatically.
On the other end you need to create an resourceobject, it resource, provisioning process and approval process for database provisioning.
Let me know if you have any query for the same.
Regards
Alabhya Goel

Using ssis import a multiple flat files with different mappings

I have an scenario for import a file
20 different flat file source and having XML mapping document in a table.
get files from that path(i have path and file name extension in table)
i have map this file based on the file name extension in dynamically
create a 20 different staging table based on mapping in dynamically
kindly help for this

Hi Karthick,
As Arthur said, if you don't want to hard code the data flow, you need to read XML from the SQL Server table and use Script Task to implement the dynamic columns mapping. To read XML from a SQL table, you can refer to:
http://blog.sqlauthority.com/2009/02/13/sql-server-simple-example-of-reading-xml-file-using-t-sql/
In the Script Task, we need to parse the first row of the flat file to get the columns information and save that to a .NET variable. Then, the variable will be used to create the query that will be used to create SQL table. For more information, please see:
http://www.citagus.com/citagus/blog/importing-from-flat-file-with-dynamic-columns/
Regards,
Mike Yin
TechNet Community Support

Multiple flat files with Comma delimiter and Pipe Delimiter in the sub folders.

Hi,
I have a directory C:\doc\Outcomes\Health --(This is the main path).
In the path above i have multiple subfolders like
A
B
C
D
Folder A & B have 20 flat files each which are comma separated and pipe delimiter files.
Folder C&D have 20 excel files each.
1) So, In SSIS while looping through the subfolders how do i limit to loop only excel files one time and flat files one time.
2) In folder A&B, how do i loop only Pipe delimiter files (neglecting comma saperated files). I want to loop only pipe delimiter files while using for each loop container.
Thanks

Both are txt files, but the data inside the files is saperated by ',' and '|'. ( comma and pipe)
Thats ok
If delimiters are not consistent you can use this method
http://visakhm.blogspot.in/2014/07/ssis-tips-handling-inconsistent-text.html
Please Mark This As Answer if it solved your issue
Please Mark This As Helpful if it helps to solve your issue
Visakh
My MSDN Page
My Personal Blog
My Facebook Page

Loading Flat files with into multiple tables using OWB

Hi,
How to implement the following logic in OWB.
LOAD DATA
INFILE 'myfile.txt'
BADFILE 'myfile.bad'
DISCARDFILE 'myfile.dsc'
APPEND
Into TABLE_Awhen (1:1) = 'A'
(Col1 Position(1:1) CHAR,
Col2 Position(2:5) CHAR)
Into TABLE_Bwhen (1:1) = 'B'
(Col1 Position(1:1) CHAR,
Col2 Position(2:20) EXTERNAL INTEGER)
Into TABLE_C
when (1:1) = 'C'
(Col1 Position(1:1) CHAR,
Col2 Position(2:20) EXTERNAL INTEGER)
I am using 10g version of OWB.I tried using the splitter operator.
I am getting the following error when i use the splitter.
An invalid combination of operators prevents the generation of code in a single implementation language (PL/SQL code, or SQL*Loader code, or ABAP code). For example, you may have included a SQL*Loader only operator such as the Data Generator in a mapping with a PL/SQL implementation type. If you designed the mapping to generate PL/SQL code, an invalid combination or sequence of operators prevents the generation of PL/SQL code in any of the operating modes(set based, row based, row based target only). If the mapping contains an operator that generates only PL/SQL output, all downstream dataflow operators must also be implementable by PL/SQL. You can use SQL operators in such a mapping only after loading the PL/SQL output to a target. Detail is as follows:
PL/SQL set based operating mode: Operator trailer_source_txt does not support SQL generation.
PL/SQL row based operating mode: Operator trailer_source_txt does not support SQL and PL/SQL generation.
PL/SQL row based (target only) operating mode: Operator trailer_source_txt does not support SQL and PL/SQL generation.
Both SQL and PL/SQL handlers are not supported by trailer_source_txt as output
SQL*Loader: Operator SPLIT does not support SQL*Loader generation.
ABAP: Operator trailer_source_txt does not support ABAP generation.
Thanks in advance,
VInay

Hi
Splitter can be used ib PL/SQL mappings, but if you use a flat file in a mapping, than it will be an SQLLoader mapping. So I suggest to you to create a mapping which load your flat file into a table, and from this table you load the data into the three table with the spillet in a PL/SQL mapping. Create two mappings.
Or you can use an external table in a mapping with a splitter.
Ott Karesz
http://www.trendo-kft.hu

Is it possible to load multiple .xlsx files into a SQL Server Table using SSIS and a Foreach Loop Container when each Excel spreadsheet is comprised of two different worksheets

So we have an Invoice .xlsx File from a 3rd party vendor. It contains two worksheets..."Enrolled" and "Engaged". The data and data columns in each worksheet is different. Is it possible to loop through multiple .xlsx files using SSIS
and a Foreach Loop Container for each spreadsheet, and then another Foreach Loop Container to control each worksheet, and pump the Excel data into a SQL Server Table first for "Enrolled" and then for "Engaged"? How can I control the Foreach
Loop Container in SSIS to process ONLY the "Enrolled" worksheet first? And then the "Engaged" worksheet next?
I know I have multiples out here and I apologize for that...but right now it seems as though I take three steps forward and then two back.
Any help would be GREATLY appreciated!
Thanks in advance!

If the structure of the Excel sheets does not change from file to file then you can by having one ForEach Loop processing always the "Enrolled" sheet and another always the "Engaged" this is doable because the Excel OLEDB connector allows
to pick individual sheets, it is problematic therefore when sheet names themselves change.
MSDN has an example: https://msdn.microsoft.com/en-ca/library/ms345182.aspx
Arthur
MyBlog
Twitter

How to load multiple flat files

Hi,
I am new to ODI 10, I have one requirement where I need load flat files containing the folder size11GB.
I want to load them all in a single instances with using of single data server, single physical schema and single logical schema.
How can we do this.
Also to execute this in package what steps and precautions do we need to follow.
Thx

Is the data in your files the same format?
If so, simply follow one of the many guides to looping around files that use a common structure to load them, you can do this in parallel if you want.
http://odiexperts.com/multiple-files-single-interface/
or
http://www.odigurus.com/2011/05/multiple-files-single-target-table.html

Multiple flat file in and multiple target tables

Similar Messages

Maybe you are looking for