Importing a Text File to Database

Hi There,
Years ago with limited Coldfusion experience I created a Cold fusion process (below) that deletes the contents of a database table and inserts the contents of a 7mb text file which is passed via FTP down to our web server at regular intervals. Admittingly the code below is probably clunky to the more experienced, however i have found this process to work for the last 4 years without a problem. Recently my web server (shared hosted environment) has been throwing this error when this page runs,
"Request aborted due to heavy system load.",
I asked my host if there has been any changes, extra sites recently added to the server that may be causing this congestion which they naturally replied "no". I even tried reducing the text file size to about 3mb but still got the same issue. Is there any other more efficient coldfusion code options I can try that may be quicker and prevent the error from being thrown?
Im on Coldfusion 7 and using MS SQL server, database and coldfusion are on differnt boxes in a shared environment.The text file can contain null values and is delimeted by a pipe character "|"
Any help appreciated
<cfsetting requesttimeout="600">
<cftransaction>
<cfquery name="delete" datasource="#client.dsn#" username="#client.username#" passWord="#client.password#">
delete from xxxxxx
</cfquery>
<cfhttp method="get" textqualifier=" " username="xxxxx" password="xxxxxx" firstrowasheaders="no" delimiter="|" name="test" url="http://xxxxxxxxxxxxxx/br_orders.txt">
Records Read <cfoutput>#test.recordcount#</cfoutput>
<cfloop query="test">
<cfquery name="insert" datasource="#client.dsn#" username="#client.username#" passWord="#client.password#">
INSERT INTO xxxxx
(OrderNo, Line, AccountNo, AccountNoShip, Shipment, PurchaseOrder, Completed, OrderDate, ShipmentDate, Product, QtyOrdered, QtyShipped, Price, Carrier, Connote, Delivery1, Delivery2, Delivery3, Delivery4, Delivery5, Delivery6, Delivery7, Delivery8, expected)
VALUES (
<cfqueryparam value='#test.column_1#' cfsqltype="cf_sql_varchar">,
<cfqueryparam value='#test.column_2#' cfsqltype="cf_sql_varchar">,
<cfqueryparam value='#test.column_3#' cfsqltype="cf_sql_varchar">,
<cfqueryparam value='#test.column_4#' cfsqltype="cf_sql_varchar">,
<cfqueryparam value='#test.column_5#' cfsqltype="cf_sql_varchar">,
<cfqueryparam value='#test.column_6#' cfsqltype="cf_sql_varchar">,
<cfqueryparam value='#test.column_7#' cfsqltype="cf_sql_varchar">,
<cfqueryparam value='#test.column_8#' cfsqltype="cf_sql_varchar">,
<cfqueryparam value='#test.column_9#' cfsqltype="cf_sql_varchar">,
<cfqueryparam value='#test.column_10#' cfsqltype="cf_sql_varchar">,
<cfqueryparam value='#test.column_11#' cfsqltype="cf_sql_float">,
<cfqueryparam value='#test.column_12#' cfsqltype="cf_sql_float">,
<cfqueryparam value='#test.column_13#' cfsqltype="cf_sql_float">,
<cfqueryparam value='#test.column_14#' cfsqltype="cf_sql_varchar">,
<cfqueryparam value='#test.column_15#' cfsqltype="cf_sql_varchar">,
<cfqueryparam value='#test.column_16#' cfsqltype="cf_sql_varchar">,
<cfqueryparam value='#test.column_17#' cfsqltype="cf_sql_varchar">,
<cfqueryparam value='#test.column_18#' cfsqltype="cf_sql_varchar">,
<cfqueryparam value='#test.column_19#' cfsqltype="cf_sql_varchar">,
<cfqueryparam value='#test.column_20#' cfsqltype="cf_sql_varchar">,
<cfqueryparam value='#test.column_21#' cfsqltype="cf_sql_varchar">,
<cfqueryparam value='#test.column_22#' cfsqltype="cf_sql_varchar">,
<cfqueryparam value='#test.column_23#' cfsqltype="cf_sql_varchar">,
<cfqueryparam value='#test.column_24#' cfsqltype="cf_sql_varchar">
</cfquery>
</cfloop>
<cfparam name="OZ" default=""><cfset OZ=#dateAdd("h",18,now())#>
<cfset OZ=#CreateODBCDateTime(OZ)#>
<cfquery datasource="#client.dsn#" username="#client.username#" password="#client.password#">
UPDATE
UpdateTimes
SET
       time=#OZ#
  WHERE
  id=<cfqueryparam value="1" cfsqltype="cf_sql_integer">
</cfquery>
</cftransaction>

Girish Sharma wrote:
"I would be really grateful."
But now OP will not really grateful to you; because;
1.You are not going to perform/import the data at his machine.
2.You have not given him the required control file.... (even without knowning anything)
3.You have given only SQL Loader word; so now "oaf... i have to read Doc.."
Regards
Girish SharmaHahahaha . Supper.. ROFL! :)

Similar Messages

  • Import a text file into database

    If anybody could help me with this, i would GREATLY appreciate it! I need to write a procedure to import a text file into an oracle database table. -Thanks!

    See my response in Re: import a text file into database.
    Cheers, APC

  • Problem in bulk import of text files in SQL Server

    Hi,
    I am using sql server 2012 x 64 and trying to import some text files (tab/ comma separated) and ms access files in sql server tempdb
    I am trying bulkinsert on text file but it is inserting all data in a single line in sql while in text file I have 1000K lines,
    Then I tried importing other data using openrowset from access file but I am getting following error
    Cannot create an instance of OLE DB provider "MSDASC" for linked server "(null)".
    can any one please provide some assistance . If I am able to upload the data from flat text file then it will be enough as I am getting data in tsv or csv format
    Nitin

    If this is a one-off, I think you should try the import/export wizard in SQL Server Management Studio instead. This may be easier to get working.
    If you want to this import on a regular basis with BULK INSERT, we need to see the CREATE TABLE statement for your table and a sample file. Best if you can upload the file on Skydrive, Dropbox or similar, as the exact bytes matters.
    To connect to an Access database, you use the ACE provider. Never heard of MSDASC before.
    Erland Sommarskog, SQL Server MVP, [email protected]

  • Help needed in reading text file to database table

    Hello experts,
    i have to insert the values from the column of this text file and insert them in my database table.
    I have done a few file to table insertions but i'm having some trouble with this one.
    Any suggestions plz
    thanks
    liab_report      1.00                                                                                                                              Page: 1
    CDC:00537 / Mon Jun-21-2010                           LIABILITY REPORT                               Mon Jun-21-2010 22:06:26
    DRAW    1; SET    1;  November 7, 2009
                         TOTAL       PAID ON         TOTAL    EXPIRED ON         TOTAL    FRAC   OUTSTANDING
                       WINNERS      06/21/10          PAID      06/21/10       EXPIRED   ROUND
      DIVISION          AMOUNT        AMOUNT        AMOUNT        AMOUNT        AMOUNT  AMOUNT        AMOUNT
        Div1              0.00          0.00          0.00          0.00          0.00    0.00          0.00
        Div2         701040.00          0.00     660146.00          0.00      40894.00    0.00          0.00
        Div3        1444128.00          0.00    1330056.00          0.00     114072.00    0.00          0.00
        Div4        4711900.00          0.00    3889700.00          0.00     822200.00    0.00          0.00
                    6857068.00          0.00    5879902.00          0.00     977166.00    0.00          0.00
    DRAW    2; SET    1;  November 14, 2009
                         TOTAL       PAID ON         TOTAL    EXPIRED ON         TOTAL    FRAC   OUTSTANDING
                       WINNERS      06/21/10          PAID      06/21/10       EXPIRED   ROUND
      DIVISION          AMOUNT        AMOUNT        AMOUNT        AMOUNT        AMOUNT  AMOUNT        AMOUNT
        Div1              0.00          0.00          0.00          0.00          0.00    0.00          0.00
        Div2         817817.00          0.00     817817.00          0.00          0.00    0.00          0.00
        Div3        1687405.00          0.00    1611742.00          0.00      75663.00    0.00          0.00
        Div4        3402100.00          0.00    3034200.00          0.00     367900.00    0.00          0.00
                    5907322.00          0.00    5463759.00          0.00     443563.00    0.00          0.00
    DRAW    3; SET    1;  November 21, 2009
                         TOTAL       PAID ON         TOTAL    EXPIRED ON         TOTAL    FRAC   OUTSTANDING
                       WINNERS      06/21/10          PAID      06/21/10       EXPIRED   ROUND
      DIVISION          AMOUNT        AMOUNT        AMOUNT        AMOUNT        AMOUNT  AMOUNT        AMOUNT
        Div1              0.00          0.00          0.00          0.00          0.00    0.00          0.00
        Div2         779933.00          0.00     769804.00          0.00      10129.00    0.00          0.00
        Div3        1605548.00          0.00    1525104.00          0.00      80444.00    0.00          0.00
        Div4        4891700.00          0.00    4256800.00          0.00     634000.00    0.00        900.00
                    7277181.00          0.00    6551708.00          0.00     724573.00    0.00        900.00

    Plz clarify whether u want to load text file as a file into database or value of this text file into database. If values are to be loaded from this text file, U can better format the text file and use SQL loader to load the file into database. By formatting the database, i mean remove the unnecessary headings and characters, kee only the values to be loaded idelimited by ' '(space) or ','(comma). Create a control file and load it into the target table.

  • Error while importing a text file for a list of values

    Hi Experts,
    While importing a text file for a list of values in designer, I am getting error message as below:
    "Invalid setting in Excel key of the Engines section of the Windows Registry. (3433)".
    I am using BO XI R3. Universe Designer version : 12.1.0.882
    Any suggestion ?
    Regards

    plzzzzzzzzzzzzzzzzz.............sme1 help me out

  • Reading Text file or Database efficiently

    Hi.
    I have an option to read EMP names randomly from either text file or database . Can anyone tell me which one will be fast. I have to read more than 5000 names dynamically for my application and pick one name randomly.
    Many thanks, Ravi

    Hi quitte ,
    Thanks for replying. I just want to pick names randomly from file or database to name each and every customer in my application dynamically.
    For this, I created one table with Unique names and would like to design a code such that each customers will be named dynamically.
    Any Suggestions or Advise?
    Ravi

  • How to extract data from  text file to database table

    Hi ,
    I am trying to upload  data in text file to database table  using GUI_UPLOAD function .what would be the program for that.
    thanks in advance.

    Hi,
    I don't think you have a standard sap program to upload data from file to database table...
    Instead you can create a custom program like this..
    DATA: T_FILEDATA(1000) OCCURS 0 WITH HEADER LINE.
    DATA: T_ZTABLE LIKE ZTABLE OCCURS 0 WITH HEADER LINE.
    CALL FUNCTION 'GUI_UPLOAD'
      EXPORTING
        filename                      = 'C:\TEST.TXT'
      tables
        data_tab                      = T_FILEDATA
    EXCEPTIONS
       FILE_OPEN_ERROR               = 1
       FILE_READ_ERROR               = 2
       NO_BATCH                      = 3
       GUI_REFUSE_FILETRANSFER       = 4
       INVALID_TYPE                  = 5
       NO_AUTHORITY                  = 6
       UNKNOWN_ERROR                 = 7
       BAD_DATA_FORMAT               = 8
       HEADER_NOT_ALLOWED            = 9
       SEPARATOR_NOT_ALLOWED         = 10
       HEADER_TOO_LONG               = 11
       UNKNOWN_DP_ERROR              = 12
       ACCESS_DENIED                 = 13
       DP_OUT_OF_MEMORY              = 14
       DISK_FULL                     = 15
       DP_TIMEOUT                    = 16
       OTHERS                        = 17
    IF sy-subrc <> 0.
    MESSAGE ID SY-MSGID TYPE SY-MSGTY NUMBER SY-MSGNO
            WITH SY-MSGV1 SY-MSGV2 SY-MSGV3 SY-MSGV4.
    ENDIF.
    LOOP AT T_FILEDATA.
      T_ZTABLE = T_FILEDATA.
      APPEND T_ZTABLE.
    ENDLOOP.
    MODIFY ZTABLE FROM TABLE T_ZTABLE.
    COMMIT WORK..
    Thanks,
    Naren

  • "Import from Text File" & "Upload from Clipboard" icon not appearng in F110

    Hi Experts,
    While making the payment to the vendors through TN F110, in "PARAMETER" screen just next to 'STATUS' screen, there is a vendor field for selection in range and just ahead of this there is an icon for multiple vendor selection ( Bold arrow mark ). If I click on that multiple vender selection icon ( on bold arrow mark ), an another screen opens for multiple selection for vendors. In this screen 'Enter, Copy, Delete etc' icons are appearing at the bottom of the screen but "Import from Text File" & "Upload from Clipboard" icon are not appearing there.
    Can you please let me know how to bring that icons there?
    Points would be awarded.
    Rgds,
    Shailesh

    Hi Ravi,
    I have not yet created anything and have just entered the parameter. In the multiple screen for vendors there are 5 fields for individual and 5 for range selection. Even you can put more than five range selections or individual selections by clicking on the page button but there I am not finding these two icons.
    Can you please check at your end and let me know if you are getting or what to do to bring those two icons in F110.
    Rgds,
    Shailesh

  • When I import a text file(comma separated )into a numbers spread sheet all the data goes into one column. Why does the text not go into separate columns based on the commas.

    When I import a text file(comma separated) into a numbers spreadsheet all the data goes into one column instead of individual columns based on the comma separators.  Excel allows you to do this during the import..  Is there a way to accomplish this in numbers without opening it in Excel and the importing into Numbers.

    Your user info says iPad. This is the OS X Numbers forum. Assuming you are using OS X… Be sure the file is named with a .csv suffix.
    (I don't have an iPad, so I don't know the iOS answer.)

  • How to import a text file created in Pages (iWork '08) to Photoshop Elements

    I am totally a novice at working with Photoshop Elements.  I need help.  I want to create my business card in Photoshop Elements with a background image and then a text layer over the background layer.  The logo and text was created in Pages (iWork '08).  Can I import this text file to Photoshop Elements and if so how do I do that?  I cannot create the same logo in Elements unless someone can tell me how to draw a circle with a transparent background so that just the outline of the circle shows and the background comes through.  Thanks for any help you can offer.  I have OS X Snow Leopard 10.6.4 and Photoshop Elements 6.

    In PSE, to draw an empty circle use the elliptical marquee tool and hold down shift while you drag to constrain it to a circle. That will give you an empty selection, then go to Edit>Stroke (outline) selection. A window appears where you can choose the width and color of the stroke.
    However, I'm not understanding why you would want to create your business card in PSE rather than in Pages or equivalent. Is there a special reason for this?
    EDIT As for the text file, it would be simpler just to retype it in PSE, since it can't be very much text. Remember that PSE doesn't do all the fancy things with text that Pages can, although it can do some things (like warping) that Pages can't.
    You could activate the text tool and use copy/paste, but you'll lose most of the formatting.

  • Importing a text file with variable values into web reporting

    Hello,
    I'm looking for a possibility to import values from a text file into a variable selection in Web reporting.
    With BEx reporting in Excel this is possible, by clicking on the multiple selection button in the variable popup screen, and subsequently selecting the "Import from Text file" button.
    However, this function seems not to be available for web-reporting...
    It would be great if someone could help me out with this!
    Thanks & regards,
    Arvid

    Hi,
    we could resolve this issue, so i thought it may also be helpful for others:
    In our example we used a file with numbers for materials. This file is stored somewhere in a directory, to which SAP BI must have authorization to read.
    The file looks something like this:
    4711
    4712
    4713
    4714
    The file is named "import.txt" and lays in directory "/usr/sap/EC6/files/bi/"
    *&  Include           ZXRSRU01
    * global variables
    Data: intern_range LIKE LINE OF i_t_var_range,
          l_s_range    TYPE rsr_s_rangesid,
          line(100)    TYPE c,
          p_file(128)  TYPE c,
          length_rangelow  type i,
          tmp_rangelow     like l_s_range-low.
    * internal tables for selection-transfer from transaction
    * Data: BEGIN OF it_file occurs 0,
    *        it_p_file(128) TYPE c,
    *      END of it_file.
    IF i_step = 1.
    ** variables can be changed or set before pop-up appears
      CASE i_vnam.
    * take material from external file to selection-list
         WHEN 'ZSD_UPMA'.
    ** call of transaction, with which the path can be set
    *CALL TRANSACTION 'ZBW_VARIABLE' using it_file
    *MODE 'A'      " call should be visible, so that variable can be set
    *UPDATE 'S'.   " first transaction, then processing
    ** Der Pfad, der in dem Selektionsbild eingegeben wird, wird an die Variable übergeben
    ** Der Set-Parameter ist in Report ZSD_SELECT_VARIABLE
    *  get parameter id 'VAR' field p_file.
    p_file = '/usr/sap/EC6/files/bi/import.txt'.
    * further handling of variable in BI
          OPEN DATASET p_file FOR INPUT IN TEXT MODE ENCODING DEFAULT.
          IF sy-subrc = 0.
            READ DATASET p_file INTO line.
            WHILE sy-subrc = 0.
              IF line(2) <> '//'.
                l_s_range-sign = 'I'.
                l_s_range-opt  = 'EQ'.
                l_s_range-low  = line.
    * fill with leading Zeros
    * in variable tmp_rangelow the value from l_s_range-low is taken
        tmp_rangelow    = l_s_range-low.
    * read the length
        length_rangelow = strlen( tmp_rangelow ).
    * in our case: material has 18 characters
        while length_rangelow lt 18.
          CONCATENATE '0' tmp_rangelow INTO tmp_rangelow.
          length_rangelow = length_rangelow + 1.
        endwhile.
    * initialize l_s_range-low
        clear l_s_range-low.
    * set with filled values
        l_s_range-low = tmp_rangelow.
    * transfer to structure
                APPEND l_s_range TO e_t_range.
              ENDIF.
              READ DATASET p_file INTO line.
            ENDWHILE.
          ENDIF.
          CLOSE DATASET p_file.
        ENDCASE.
    ELSEIF i_step = 2.
    ** in step 2 all variable values from pop-up input can be processed or
    ** User Exit variables can be derived
    * UserExit Ende.
    ENDIF.
    Hope i could help!
    Best regards,
    Tobias

  • How to import a excel file into database?

    hi all,
    How can I import a excel file into database?
    What I need to do first?
    Thanks,
    Amy

    1. Create a conrtol file in your disc(with notepad) ex
    test_loader.ctl
    LOAD DATA
    INFILE='C:\Testcsv.csv'
    into table test_table
    FIELDS TERMINATED BY ','
    TRAILING NULL COLS
    (ID,COL1,COL2,COL3)
    The table descrption is
    scott@ORCL> desc test_table;
    Name Null? Type
    ID NUMBER
    COL1 VARCHAR2(10)
    COL2 VARCHAR2(10)
    COL3 VARCHAR2(10)
    scott@ORCL>
    2 .Create a file testcsv.csv from your excel file you want to load with " save as" csv comma seperated value
    in your hard disc c:\
    3. in dos command mode write
    C:\>sqlldr scott/tiger control=c:\testcsv.ctl log=c:\testcsv.log
    Thats it!

  • How can we import a text file in Labview,Is this process require CIN utility

    Hi,
    I am working in physics lab where i am working on labview and want to import a text file in labview, if anyone knows ,please mail me at [email protected]

    Hi Ajay,
    See attached V 7.0 example...
    Hope it helps,
    Nick
    Attachments:
    Read_txt_File.vi ‏20 KB

  • I'm creating an epk and I need to add a bio to the menu.  Can I import the text file or do I need to retype the bio somewhere?  What is the best way to proceed?

    I'm creating an epk and I need to add a bio to the menu.  Can I import the text file or do I need to retype the bio somewhere?  What is the best way to proceed?

    Electronic Press Kit.
    Usually in the form of video outtakes, movie trailers, cast and crew interviews. Basically, the stuff that gets bunged onto commercial DVDs as extras or ends up on late night TV "Making Of..." shows.
    To answer the OP, you want a button on the menu that leads to a still image, or series of them, set up as a slide show. You can copy/paste the text in DVD Studio and then add formatting via the Text menu. (Highlight the text and right click to access formatting controls). Or do it in a graphics application using the appropriate video image preset.

  • Importing a text file and displaying it

    Ok, my goal is to import a text file which i have done. I want to import it into a 2D array. There are 26 rows and 2 columns. Basically just a translator program. This is what i have to load the text file into java. But i am having trouble displaying the array with the data loaded into it. any ideas?
    public static void loadData(String a[][])
         throws java.io.IOException
         String file = "C:\\print.txt";
         String inLine;
         int row;
         FileReader fr = new FileReader(file);
         BufferedReader br = new BufferedReader(fr);
         for (row = 0; row<=26;row++)
              a[row][0] = br.readLine();
              a[row][1] = br.readLine();     
         br.close();     
         System.out.println(a[row][0]);
         }     

    my goal for this program is to have a user enter a phrase to be translated. I have the text file that i am loading into java that has the english word and then the translated word. I want to load that into the 2D array and display it. Then use a stringtokenizer for the translate part. But the problem i have right now is trying to display the data that was loaded into the array. any ideas?

Maybe you are looking for

  • Guidelines, Form Fields and Tab Order

    I've been fighting a form for about two weeks now with Acrobat 9 Pro. I used guidelines to make sure I could get 2 buttons and a field aligned properly. I have 92 instances of this arrangement. When I reopened the file after I had worked on it for a

  • Error in Inbound JMS adapter (ESB project)

    Hello all, I am trying out a sample ESB application, where an ESB service is listening on a JMS topic (OEMS database persistence) for new employee notifications. Upon receiving the message, the service will invoke a stored procedure to insert a recor

  • Condition JEXC- Manual Excise Value, is missing in pricing procedure A M RM

    Dear Gurus, During PO creation following massage occurred, Condition JEXC- Manual Excise Value, is missing in pricing procedure A M RM0000. scenario: - JEXC condition type not picking the value automatically against Tax Code. Pl. help, Regards,

  • Printing causes internet to temporarily disconnect.

    Printer: HP Laserjet 3600n My printer is connected to my router via an Ethernet cable. Every time a computer it our network prints something on this printer, the internet is temporarily cut. The computers don't show that they are disconnected from th

  • Original iPhone with Motorola t505

    I purchased a Motorola t505 bluetooth handsfree car kit with an infinixx A2DP adapter. I am pleased to say that it works quite well in my car, including the FM transmission. This unit announces the number of the caller when a call comes in. My only g