Importing CSV files into Multiple Tables in One Database

 I have a web based solution using Microsoft SharePoint and SQL Server that is created to centralize dat collection and reporting of program metrics used in montly reviews.
A person from each program enters dat manual or by pushing the data using automated data import tools. The user then is able to generate reports and provide presentations to their stakeholders.
There are are programs that are located in US and 2 that are located in Japan. All, including programs in Japan use the MS Project with a plug-in tool that allows them to auto input data. When the user click the "Send To.." button, the data goes
in to multiple tables in one database.
Everything is set up exactly the same for every program; however, we have discovered becase of a firewall, along with some of the different settings that the MS Project has over in Japan, the 2 program users are not able to auto import their data.
The suggestion is to have the program users export the MS Project file into a CSV and email it. I will then take it and convert the data, such as the dates and place them on a Network Drive. There will be 2 folders, one for each program.
I feel it will be an easy process just to load the data from the Network Drive into the same tables that are created for auto import.
My Concerns/Questions:
1. Should I create 1 SSIS package or should there be 2, one for each program?
2. US and Japan program users send their data once a month. The converted files are going to be saved in the location marked with a date (ex:201402). How can i have it to where SSIS will automatically load the data every time i place new files in the designated
folders or when i update an exsisting file?
Can you provide an example with your suggestion please?
I greatly appreciate the assistance!
Y_Tyler

Hello Vikash,
Thank you! This will help me get started.
There will be 2 folders, one with files with Program A data and the other with files with Program B data. All will have the same fields, just coming from different programs. You stated that I will be able to to load both in one package. Would there be two
paths since there will be two separate folders?
Using the example you provided, i am confident that I can create the package using one path into one table but not sure how to get the files from 2 paths (if there is) into multiple tables.
Can you help clarify this for me?
Thank you!
Y_Tyler

Similar Messages

  • How to import csv file with multiple tables into sql server

    I have multiple csv files that has one sheet but has 130 headers with each header having different data. 
    I'd like to import each one of these header rows with data into its own file in sql server. 
    I know very basic SSIS and am but am not familiar with the scripting in it though which what I assume I'd have to use. 
    Each header in the csv file is structured as such(also see example pic):
    first header would be this:                             
          ITEM = ORG_V                              
          DATE = 2013-07-22 10:00 ~ 2013-07-22 10:15      
    column names
    data
    second header would be this:
    ITEM = TER_V
          DATE = 2013-07-22 10:00 ~ 2013-07-22 10:15
    column names
    data
    The headers can be at any random row number as well as the data size in each excel file differs but they all start with "ITEM ="
    and then in the next row "DATE ="
    I could also convert these to excel files if it makes this process easier. 

    Why don't you put a filter on D3, filter out the blanks, copy/paste to a new CSV file, save it, and import it.
    There's no way you're going to get SQL to do that kind of thing for you.  The language is for set based operations, not for complex data manipulation tasks.
    Knowledge is the only thing that I can give you, and still retain, and we are both better off for it.

  • Getting Issue while uploading CSV file into internal table

    Hi,
    CSV file Data format as below
         a             b               c              d           e               f
    2.01E14     29-Sep-08     13:44:19     2.01E14     SELL     T+1
    actual values of column   A is 201000000000000
                     and  columen D is 201000000035690
    I am uploading above said CSV file into internal table using
    the below coding:
    TYPES: BEGIN OF TY_INTERN.
            INCLUDE STRUCTURE  KCDE_CELLS.
    TYPES: END OF TY_INTERN.
    CALL FUNCTION 'KCD_CSV_FILE_TO_INTERN_CONVERT'
        EXPORTING
          I_FILENAME      = P_FILE
          I_SEPARATOR     = ','
        TABLES
          E_INTERN        = T_INTERN
        EXCEPTIONS
          UPLOAD_CSV      = 1
          UPLOAD_FILETYPE = 2
          OTHERS          = 3.
      IF SY-SUBRC <> 0.
        MESSAGE ID SY-MSGID TYPE SY-MSGTY NUMBER SY-MSGNO
                WITH SY-MSGV1 SY-MSGV2 SY-MSGV3 SY-MSGV4.
      ENDIF.
    am getting all columns data into internal table,
    getting problem is columan A & D. am getting values into internal table for both; 2.01E+14. How to get actual values without modifying the csv file format.
    waiting for your reply...
    thanks & regards,
    abhi

    Hi Saurabh,
    Thanks for your reply.
    even i can't double click on those columns.
    b'se the program needs be executed in background there can lot of csv file in one folder. No manual interaction on those csv files.
    regards,
    abhi

  • Question about reading csv file into internal table

    Some one (thanks those nice guys!) in this forum have suggested me to use FM KCD_CSV_FILE_TO_INTERN_CONVERT to read csv file into internal table. However, it can be used to read a local file only.
    I would like to ask how can I read a CSV file into internal table from files in application server?
    I can't simply use SPLIT as there may be comma in the content. e.g.
    "abc","aaa,ab",10,"bbc"
    My expected output:
    abc
    aaa,ab
    10
    bbb
    Thanks again for your help.

    Hi Gundam,
    Try this code. I have made a custom parser to read the details in the record and split them accordingly. I have also tested them with your provided test cases and it work fine.
    OPEN DATASET dsn FOR input IN TEXT MODE ENCODING DEFAULT.
    DO.
    READ DATASET dsn INTO record.
      PERFORM parser USING record.
    ENDDO.
    *DATA str(32) VALUE '"abc",10,"aaa,ab","bbc"'.
    *DATA str(32) VALUE '"abc","aaa,ab",10,"bbc"'.
    *DATA str(32) VALUE '"a,bc","aaaab",10,"bbc"'.
    *DATA str(32) VALUE '"abc","aaa,ab",10,"b,bc"'.
    *DATA str(32) VALUE '"abc","aaaab",10,"bbc"'.
    FORM parser USING str.
    DATA field(12).
    DATA field1(12).
    DATA field2(12).
    DATA field3(12).
    DATA field4(12).
    DATA cnt TYPE i.
    DATA len TYPE i.
    DATA temp TYPE i.
    DATA start TYPE i.
    DATA quote TYPE i.
    DATA rec_cnt TYPE i.
    len = strlen( str ).
    cnt = 0.
    temp = 0.
    rec_cnt = 0.
    DO.
    *  Start at the beginning
      IF start EQ 0.
        "string just ENDED start new one.
        start = 1.
        quote = 0.
        CLEAR field.
      ENDIF.
      IF str+cnt(1) EQ '"'.  "Check for qoutes
        "CHECK IF quotes is already set
        IF quote = 1.
          "Already quotes set
          "Start new field
          start = 0.
          quote = 0.
          CONCATENATE field '"' INTO field.
          IF field IS NOT INITIAL.
            rec_cnt = rec_cnt + 1.
            CONDENSE field.
            IF rec_cnt EQ 1.
              field1 = field.
            ELSEIF rec_cnt EQ 2.
              field2 = field.
            ELSEIF rec_cnt EQ 3.
              field3 = field.
            ELSEIF rec_cnt EQ 4.
              field4 = field.
            ENDIF.
          ENDIF.
    *      WRITE field.
        ELSE.
          "This is the start of quotes
          quote = 1.
        ENDIF.
      ENDIF.
      IF str+cnt(1) EQ ','. "Check end of field
        IF quote EQ 0. "This is not inside quote end of field
          start = 0.
          quote = 0.
          CONDENSE field.
    *      WRITE field.
          IF field IS NOT INITIAL.
            rec_cnt = rec_cnt + 1.
            IF rec_cnt EQ 1.
              field1 = field.
            ELSEIF rec_cnt EQ 2.
              field2 = field.
            ELSEIF rec_cnt EQ 3.
              field3 = field.
            ELSEIF rec_cnt EQ 4.
              field4 = field.
            ENDIF.
          ENDIF.
        ENDIF.
      ENDIF.
      CONCATENATE field str+cnt(1) INTO field.
      cnt = cnt + 1.
      IF cnt GE len.
        EXIT.
      ENDIF.
    ENDDO.
    WRITE: field1, field2, field3, field4.
    ENDFORM.
    Regards,
    Wenceslaus.

  • Uploading CSV file into internal table

    Hi,
    I want to upload a CSV file into internal table.The flat file is having values as below:
    'AAAAA','2003-10-11 07:52:37','167','Argentina',NULL,NULL,NULL,NULL,NULL,'MX1',NULL,NULL,'AAAA BBBB',NULL,NULL,NULL,'1',NULL,NULL,'AR ',NULL,NULL,NULL,'ARGENT','M1V','MX1',NULL,NULL,'F','F','F','F','F',NULL,'1',NULL,'MX','MMI ',NULL
    'jklhg','2004-06-25 08:01:57','456','hjllajsdk','MANAGUA   ',NULL,NULL,'265-5139','266-5136 al 38','MX1',NULL,NULL,'hjgkid GRÖBER','sdfsdf dfs asdfsdf 380 ad ased,','200 as ads, sfd sfd abajao y 50 m al sdf',NULL,'1',NULL,NULL,'NI ',NULL,NULL,NULL,'sdfdfg','M1V','dds',NULL,NULL,
    Here I can not even split at ',' because some of the values are having value like NULL and some have values with comma too,
    The delimiter is a quote and the separator is a comma here.
    Can anyone help on this?
    Thanks.
    Edited by: Ginger on Jun 29, 2009 9:08 AM

    As long as there can be a comma in a text literal you are right that the spilt command doesn't help. However there is one possibility how to attack this under one assumption:
    - A comma outside a text delimiter is always considered a separator
    - A comma inside a text delimiter is always considered a comma as part of the text
    You have to read you file line by line and then travel along the line string character by character and setting a flag or counter for the text delimiters:
    e.g.
    "Text","Text1, Text2",NULL,NULL,"Text"
    String Index  1: EQ " => lv_delimiter = 'X'
    String Index  2: EQ T => text literal (because lv_delimiter = 'X')
    String Index  3: EQ e => text literal (because lv_delimiter = 'X')
    String Index  4: EQ x => text literal (because lv_delimiter = 'X')
    String Index  5: EQ t => text literal (because lv_delimiter = 'X')
    String Index  6: EQ " => lv_delimiter = ' ' (because it was 'X' before)
    String Index  7: EQ , => This is a separator because lv_delimiter = ' '
    String Index  8: EQ " => lv_delimiter = 'X' (because it was ' ' before)
    String Index  9: EQ T => text literal (because lv_delimiter = 'X')
    String Index 10: EQ e => text literal (because lv_delimiter = 'X')
    String Index 11: EQ x => text literal (because lv_delimiter = 'X')
    String Index 12: EQ t => text literal (because lv_delimiter = 'X')
    String Index 13: EQ 1 => text literal (because lv_delimiter = 'X')
    String Index 14: EQ , => text literal (because lv_delimiter = 'X')
    String Index 15: EQ T => text literal (because lv_delimiter = 'X')
    Whenever you hit a 'real' separator (lv_delimiter = ' ') you pass the value of the string before that up to the previous separator into the next structure field.
    This is not an easy way to do it, but if you might have commas in your text literal and NULL values I gues it is probably the only way to go.
    Hope that helps,
    Michael

  • Looking for a Notes app that I can import CSV files into?

    As the title says, I'm looking for a good notes application for my iPhone that I can import CSV files into. I have tried both Appigo and Notespark, but I can't easily scroll through them, as I have 2,000+ notes. Is there any app that I can import my notes into and also scroll through quickly? I'd like something that works similarly to how you scroll through songs on the iPhone/iTouch, with the column on the right where you can skip to songs (notes, in this case) that start with a certain letter.
    Thanks!

    Hi Tx Tar Heel,
    I've been using Office2HD: https://itunes.apple.com/us/app/office2-hd/id364361728?mt=8
    Its cheaper than Numbers and it also works for Word and PowerPoint files too. I like the Dropbox integration. I can start on my Office docs (Word, Excel, PowerPoint) in the office and then edit those files with Office2HD when I'm out of the office. Files saves right back to Dropbox so that when I get back to the office the files are already updated. Not bad for a $7.99 app!
    Hope this helps!
    ~Joe

  • Loading data from .csv file into Oracle Table

    Hi,
    I have a requirement where I need to populate data from .csv file into oracle table.
    Is there any mechanism so that i can follow the same?
    Any help will be fruitful.
    Thanks and regards

    You can use Sql Loader or External tables for your requirement
    Missed Karthick's post ...alredy there :)
    Edited by: Rajneesh Kumar on Dec 4, 2008 10:54 AM

  • Loading data from .csv file into existing table

    Hi,
    I have taken a look at several threads which talk about loading data from .csv file into existing /new table. Also checked out Vikas's application regarding the same. I am trying to explain my requirement with an example.
    I have a .csv file and I want the data to be loaded into an existing table. The timesheet table columns are -
    timesheet_entry_id,time_worked,timesheet_date,project_key .
    The csv columns are :
    project,utilization,project_key,timesheet_category,employee,timesheet_date , hours_worked etc.
    What I needed to know is that before the csv data is loaded into the timesheet table is there any way of validating the project key ( which is the primary key of the projects table) with the projects table . I need to perform similar validations with other columns like customer_id from customers table. Basically the loading should be done after validating if the data exists in the parent table. Has anyone done this kind of loading through the APEX utility-data load.Or is there another method of accomplishing the same.
    Does Vikas's application do what the utility does ( i am assuming that the code being from 2005 the utility was not incorporated in APEX at that time). Any helpful advise is greatly appreciated.
    Thanks,
    Anjali

    Hi Anjali,
    Take a look at these threads which might outline different ways to do it -
    File Browse, File Upload
    Loading CSV file using external table
    Loading a CSV file into a table
    you can create hidden items in the page to validate previous records before insert data.
    Hope this helps,
    M Tajuddin
    http://tajuddin.whitepagesbd.com

  • Best way to import csv file into sql server

    Hello DBAs
    always thankful for your help and one question, what is the best way to import a csv file into sql server ?

    and there is a
    Bulk Insert Task for the Control Flow.
    http://www.youtube.com/watch?v=ACuCU6IqO50
    Please mark the post as answered if it answers your question | My SSIS Blog:
    http://microsoft-ssis.blogspot.com |
    Twitter

  • Inserting into multiple tables on one form using UIX

    We are developing an application using struts and UIX. We would like to be able to insert into three tables on one form(view). How can we accomplish this?

    This is more of an ADF question, you'll need to set up view object, than just drag it from the data control palette like normal.
    If it's not already set up as a view in the db (split into multiple tables), you'll need to create a view object that includes all the fields you want, based on multiple entity objects.

  • Error loading local CSV file into external table

    Hello,
    I am trying to load a CSV file located on my C:\ drive on a WIndows XP system into an 'external table'. Everyting used to work correctly when using Oracle XE (iinstalled also locally on my WIndows system).
    However, once I am trynig to load the same file into a Oracle 11g R2 database on UNIX, I get the following errr:
    ORA-29913: Error in executing ODCIEXTTABLEOPEN callout
    ORA-29400: data cartridge error
    error opening file ...\...\...nnnnn.log
    Please let me know if I can achieve the same functionality I had with Oracle XP.
    (Note: I cannot use SQL*Loader approach, I am invoking Oracle stored procedures from VB.Net that are attempting to load data into external tables).
    Regards,
    M.R.

    user7047382 wrote:
    Hello,
    I am trying to load a CSV file located on my C:\ drive on a WIndows XP system into an 'external table'. Everyting used to work correctly when using Oracle XE (iinstalled also locally on my WIndows system).
    However, once I am trynig to load the same file into a Oracle 11g R2 database on UNIX, I get the following errr:
    ORA-29913: Error in executing ODCIEXTTABLEOPEN callout
    ORA-29400: data cartridge error
    error opening file ...\...\...nnnnn.log
    Please let me know if I can achieve the same functionality I had with Oracle XP.
    (Note: I cannot use SQL*Loader approach, I am invoking Oracle stored procedures from VB.Net that are attempting to load data into external tables).
    Regards,
    M.R.So your database is on a unix box, but the file is on your Windows desktop machine? So .... how is it you are making your file (on your desktop) visible to your database (on a unix box)???????

  • SQL Server 2012 Management Studio:Importing XML file to new Table of new Database-XML parsing: unexpected end of input???

    Hi all,
    In the Notepad, I created an xml file (ZenQroducts.xml) :
    <Qroducts>
    <Qroduct>
    <SKU>1</SKU>
    <Desc>Book</Desc>
    </Qroduct>
    <Qroduct>
    <SKU>2</SKU>
    <Desc>DVD</Desc>
    </Qroduct>
    <Qroduct>
    <SKU>3</SKU>
    <Desc>Video</Desc>
    </Qroduct>
    In my SQL Server 2012 Management Studio, I executed the following code:
    --to create a new object Qroducts in a new database OPENXMLtesting
    CREATE DATABASE OPENXMLtesting
    GO
    CREATE TABLE Qroducts(
    sku INT Primary KEY,
    qroduct_desc VARCHAR(30));
    INSERT INTO Qroducts (sku, qroduct_desc)
    SELECT X.qroduct.query('SKU').value('.', 'INT'),
    X.qroduct.query('Desc').value('.', 'VARCHAR(30)')
    FROM (
    SELECT CAST(x AS XML)
    FROM OPENROWSET(
    BULK 'H:\ZenQroducts.xml',
    SINGLE_BLOB) AS T(x)
    ) AS T(x)
    CROSS APPLY x.nodes('Qroducts/Qroduct') AS X(qroduct);
    SELECT sku, qroduct_desc
    FROM Qroducts;
    I got the following message:
    Msg 9400, Level 16, State 1, Line 6
    XML parsing: line 13, character 12, unexpected end of input
    I have no ideas why I got this "XML parsing:line 13, character12, unexpected end of input" message. Please kindly help, advise me on where I made mistake and how I can resolve this problem, and respond in this Forum.
    Thanks in advance,
    Scott Chang
     

    Hi Manish, Thanks for your response.
    Yes, it is a duplicate with Qroducts/Qroduct instead of Products/Product. I did it, because I don't know how to remove the existing "Products" database.
    Again, I got the existing "Qroducts" database in this second trial!!??  I am comletely lost in doing "Importing XML file to SQL Table" now!!  I think that I have not touched T-SQL and XML programming, since 2008. I
    did not catch the new features of T-SQL 2008, XML, SQL/XML, XQuery, etc. for long time. Recently, I did not know what DECLARE, @x, SET,...were, and dived into the SQL/XML and XQuery programming. I knew the SQL Basics (SELECT, FROM, WHERE,..), creating
    the names of databases and tables in SQL Server 2008/2012 Management Studio (Express) before. This morning, I found an old T-SQL 2008 Prgrammer's Guide that has the new features of SQL Server 2008 for T-SQL, XML, XQuery, etc. I just starting reading
    it to know what DECLARE, @x, SET,... are. But, I still don't know where the existing databases "Products" and "Qroducts" (created by me) are - they are not in the Databases of SQL Server 2012 Management Studio I am using!!??  Could
    you please kindly point out where the "Products" and "Qroducts" databases are?
    Prashanth responded to my posted question too and he asked to to try his code: DECLARE @xml XML, SELECT @xml =' <Qroducts>.....SELECT product.value.....FROM @XML.nodes.....AS.... I just learned that his code is doing the thing we
    want to do and print the results below the SQL Query. This is not what I need in doing my SQL/XML programming. I am reading/studying the new (2008) features of T-SQL, XQuery 1.0: An XML Query Language (Second Edition), and Microsoft XQuery Language Reference
    (SQL Server 2012 Books Online) closely now. I hope that I can resume the T-SQL, XML Query, SQL/XML, XQuery in my SQL Server 2012 Management Studio soon.
    Please tell me  where the existing databases "Products" and "Qroducts" I created in my previous trials in my SQL Server 2012 Management Studio.  This is what I need to know and to delete them, before I do this kind of "Importing
    XML file to Table of SQL Server 2012 Management Studio"  programming again.
    Please kindly help, advise and respond again.
    Many Thanks,
    Scott Chang

  • Insert Insert XML file into multiple records in Oracle Database

    I would like to find out if it is possible to insert a single XML file into multiple records or tuples in a Oracle database table. I have a single XML file which is at multiple levels. The meta data for the levels are common and each level can have meta data of their own in addition. I do not have any meta data field which will uniquely determine whether the data belongs to Root level, Intermediate level or at the document level. Is there any way I can determine which level the meta data belongs to and thereby make a corresponding entry into the database table tuple? For example I could have an attribute called level which is going to be present only in the database table and not in the XML file. If level=1 then it corresponds to "Root" meta data, if level=2 then it corresponds to "Intermediate" level and if level=3 then it corresponds to meta data at document level. I need a way to calculate the value for level from the XML file and thereby insert the meta data element into a tuple in a single table in Oracle.

    Hi,
    extract your xml and then you can use insert all clause.
    here's very small example on 10.2.0.1.0
    SQL> create table table1(id number,val varchar2(10));
    Table created.
    SQL> create table table2(id number,val varchar2(10));
    Table created.
    SQL> insert all
      2  into table1 values(id,val)
      3  into table2 values(id2,val2)
      4  select extractValue(x.col,'/a/id1') id
      5        ,extractValue(x.col,'/a/value') val
      6        ,extractValue(x.col,'/a/value2') val2
      7        ,extractValue(x.col,'/a/id2') id2
      8  from (select xmltype('<a><id1>1</id1><value>a</value><id2>2</id2><value2>b</value2></a>') col from dual) x;
    2 rows created.
    SQL> select * from table1;
            ID VAL                                                                 
             1 a                                                                   
    SQL> select * from table2;
            ID VAL                                                                 
             2 b                                                                    Ants

  • How to import data from CSV file into a table by using oracle forms

    Hi,
    I have a CSV file and i want to insert in oracle database in a table by using a button in oracle forms.
    the user can select CSV file by using open dialog .
    can any one help me to find method to make import and select file from the client machine ?
    thx.

    1. create table blob
    CREATE TABLE IB_LOVE
      DOC          BLOB,
      CONTRACT_NO  VARCHAR2(20 BYTE)                NOT NULL
    )2. use the code below to insert:
           INSERT INTO ordmgmt.ib_love
                       (contract_no, doc
                VALUES (:control.contract_no_input, NULL
           lb$result :=
             webutil_file_transfer.client_to_db (:b2.file_name, v_file_blob_name, v_col_blob_name,
                                                 'CONTRACT_NO = ' || :control.contract_no_input);
           :SYSTEM.message_level := 25;
           COMMIT;
           :SYSTEM.message_level := 0;3. use the code below to download
         if :control.CONTRACT_NO_INPUT is not null then
              vboolean :=   webutil_file_transfer.DB_To_Client_With_Progress(
                               vfilename,  --filename                       
                              'IB_LOVE', ---table of Blob item                       
                              'DOC',  --Blob column name                       
                              'CONTRACT_NO = ' || :CONTROL.CONTRACT_NO_INPUT, ---where clause to retrieve the record                       
                              'Downloading from Database', --Progress Bar title                       
                              'Wait to Complete'); --Progress bar subtitle  client_host('cmd /c start '||vfilename);
              client_host('cmd /c start '||vfilename);                 
         else
              errmsg('Please choose contract no');
         end if;4. use the code below to open file dialog
    x:= WEBUTIL_FILE.FILE_OPEN_DIALOG ( 'c:\', '*.gif', '|*.gif|*.gif|', 'My Open Window' ) ;
    :b2.FILE_NAME:=X;

  • Importing a CSV file into a table for Fiori

    Hello,
    I create a fiori application with sap web ide  and i need to import a csv file from my desktop and show its content into a table in my fiori application
    Please could some one help me
    Ameni,
    Subject was edited by: Michael Appleby

    Hello Jamie
    Thank you ,
    i try this code , i putted in my detail.view.xml this code :<l:VerticalLayout> <u:FileUploader id="fileUploader" name="myFileUpload" uploadUrl="upload/" width="400px" tooltip="Upload your file to the local server" uploadComplete="handleUploadComplete"/> <Button text="Upload File" press="handleUploadPress"/> </l:VerticalLayout>
    and i put in detail.controller.js the 2 functions : handleUploadComplete and handleUploadPress
    And when i run and i press in the button to load the file but it doesn't work !
    Best Regards,
    Ameni,

Maybe you are looking for