Validating CSV File BEFORE importing and moving using an SSIS package

I'm trying to come up with a process (I'm used to doing this kind of stuff in T-SQL) in SSIS to grab a collection of .csv files from a folder, validate them, then depending on if they pass validation, import them into my database and move them into an archive
folder. If the .csv does not pass validation, then it does not get imported, and instead gets placed into an error folder. The validation requirements is that the Column2 of my .csv contains a WKT text field and the file is considered valid if every record
can be successfully converted into a Geometry/Geography type field. 
Column0 Column1 Column2
abcd efgh LINESTRING (-71.4555487 41.6079686, -71.4550113 41.6088851)
ijkl mnop LINESTRING (-70.0748669 48.6634506, -70.0499 48.6548479)
qrst uvwx LINESTRING (-70.3159285 48.4199802, -70.3168512 48.4187551)

FOr that what you can do is dump the records to staging table and use STisValid function
see
http://technet.microsoft.com/en-us/library/bb933890.aspx
finally take the count of failed ones and if cnt > 0 set boolean value as true/false. Then for true cases move data to final table and for false cases (invalid values), move it to error folder
So package will look like below
1. Data flow task to transfer data to staging
2. execute sql task to call a procedure which does validation on staging data and returns a bit result. Set it to SSIS variable using output variable from procedure
3. Connect to data flow task to do transfer from staging to destination. the precedence constraint option would be Expression And Constraint with expression as 
@BooleanVariable == True
and constraint as OnSuccess
4. Connect to file system task to archive the file. the precedence constraint option would be Expression And Constraint with expression as 
@BooleanVariable == False
and constraint as OnSuccess
Please Mark This As Answer if it helps to solve the issue Visakh ---------------------------- http://visakhm.blogspot.com/ https://www.facebook.com/VmBlogs

Similar Messages

  • Unable to drag and drop mp4 movies into itunes but the file will import and play using home sharing

    Strange problem with my brothers MacBook.
    The movie files were converted to mp4 format ready to be played on an iphone and I imported them into my iTunes on an iMac with no problems.
    Files were also saved onto a flash drive which I passed on to my brother.
    My brother was unable to import files by drag and drop and also by "add to library", we even copied and pasted files from the flash drive to the desktop but that also didn't work.
    Using "home sharing the movies imported and played with no problems.
    Any ideas why this is happening?

    This appears to be a long-standing bug:
    https://bugzilla.mozilla.org/show_bug.cgi?id=634720
    and the accepted workaround is stated in comment 11.
    Oddly, it's possible to do the reverse, i.e. drag multiple eml files from Explorer to a TB folder.

  • Moving a csv file in iPad and into Contacts, I think

    Ok, I've created a csv file from Outlook Express conacts/email address and wanted to import it into Ipad and since it does'nt look like Apple's Mail program on the iPad excepts this like on a Mac, all I can figure out that it must be contacts that Mail uses for addressing emails, so if this is the case then how does one go about importing a csv file into Contacts?

    Apple uses the Contacts app on their computers and other devices for all contact functions including email and phone numbers. Your best bet may be to export your contacts to the vCard format. A vCard can easily be added to your iPhone or iCloud Contacts. To add a vCard file to your iPhone email it to your iPhone then open it to be prompted to add it to Contacts.
    Another option would be to look into adding iCloud syncing to your computer. Once setup your contacts will automatically sync between iCloud, your computer and iPhone.

  • How to write .CSV file to import device placeholder into Lion Server Profile Manager?

    I'm now using Mac Mini with Lion Server (10.7.4). I've already setup a server with Profile Manager to manage some iOS devices.
    Now I need to add many devices into Profile Manager. From some articles, (for example http://my.safaribooksonline.com/book/-/9780132778879/chapter-4dot-managing-accou nts/ch04lev1sec3), I found that I can use .CSV file to import many device placeholders at a time. So I create a .CSV file "devices.csv" with the content as the following:
    iPhone001,12345ABCD12,,,
    Then I import this file to "Devices" in Profile Manager, but in vain. It says that the placeholder is invalid.
    If there's any one could provide some exemples? Thanks a lot.

    Hi Nien-Yi Ho,
    A little late, but perhaps someone else can benefit ...
    The way I did this, is:
    - create an empty Excel spreadsheet
    - add 5 headers in the first row: DeviceName, SerialNumber, IMEI, MEID, UDID
    - in the next rows add your devices and specs (for the iPhones and iPads I only added DeviceName, SerialNumber and IMEI specs)
    - IMPORTANT: save the file as a Windows CSV file.
    If you import the CSV file now, all should go well.

  • File Error: The Specified file is open and in use by this or another app...

    I have imported a string of pictures and dragged them into the FCP timeline. I render them and it renders a certain percentage then it stops and says "File Error: The Specified file is open and in use by this or another application."
    After reading this, I quit out of FCP, restarted my computer, opened FCP, nothing other program, tried to render again and same thing happened. It renders part of it then the error message pops up, then I can render again and it will do another percent then it will pop again and so on.
    Suggestions, or does anyone know why this happened? Thank you very much for your time.
    A

    okay - don't know where you've got your scratch disc set but hopefully it's to another drive other than your boot.
    The same drive where your render files should be located, ie some drive other than your boot drive.
    Can you copy the jpegs over to that drive into a folder.
    Delete them from your tl, delete them from your proj. Save your project. Close fcp.
    If they're on your boot drive - delete them.
    Reopen fcp and Reimport them from the new location.
    Sorry for all the steps - but we have no idea how you have configured fcp or your level of experience. Trying to cover all the bases ...

  • FILE ERROR: The Specified file is open and in use...

    FILE ERROR: The Specified file is open and in use by this or another application.
    I get this error when I import a .MOV created by Live Type. All applications, except FCP, are closed. I can reboot the computer and get the same thing. It happens when I try and "render all". Thanks.

    It's probably either spotlight looking at the files, or some anti-virus program thinking your render is a virus.

  • Edit CSV files present in document library using Microsoft Excel

    Hi all
    I have a document library with CSV files uploaded. I have used the below mapping in DOCICON.xml present in \14\TEMPLATE\XML folder.
    <Mapping Key="csv" Value="csv16.gif" EditText="Microsoft Excel" OpenControl="SharePoint.OpenDocuments"/>
    And I modified mime type with "application/vnd.ms-excel" for csv in IIS7. Now, when i click on the file  popup is getting displayed with 2 Radio Buttons.
    1) Read Only
    2)Edit
    When i select "Read Only", the file is opening in Microsoft Excel in readonly mode But when i select "Edit", the file is opening in Note pad and its editable.
    Now, My requirement is to edit the csv file in Microsoft Excel not in NotPad.
    Can Any one help me to acheive this!! Thanks in advance.

    Try below:
    http://nickgrattan.wordpress.com/2011/01/05/sharepoint-opening-csv-files-with-microsoft-excel/
    Run the “Internet Information Services (IIS) Manager” application from the Start/Administrative tools menu.
    Select the server in the left-hand pane.
    Select “MIME Types” in the list of options in the middle pane.
    Then locate the .CSV entry (it should already exist) and change the MIME type to: application/vnd.ms-excel, and click
    OK.

  • Flash CS6 cant open play SWF files without importing and destroying them, how can I get around this?

    Flash CS6 can't open play SWF files without importing and destroying them, how can I get around this?
    I'm just trying to preview an swf file in flash like I have with all previous versions.

    What if my SWF loads external content from an online server?
    Not only does the current flash player prohibit such activity, it doesn't even pop open an error anymore saying there was an error connecting to an online source.
    Normally, I would simply drag the SWF into Flash and all connections would go through.  I could see traces, errors, and experience no issues.
    Now I can't even do that.  So what then?  You have crippled a fundamental use of the program, but THANK GOD we have that deco brush that nobody asked for.
    And for the record, the nature of my work benefits from not necessarily allowing the Flash Player to connect to online content.  The error it pops up?  That's simply another method of error checking that I require.  Updating the Flash Player options is not an option.
    Why would you even remove this key feature from Flash anyways?  It's been there for years ... has the ratio of people importing SWFs (a rather useless gesture in an increasing OOP world) really outweighed the people using Flash as a testing environment that much?

  • How can i upload files in servlets and also using jsp

    means that uploading files(.doc) or any files by using servlets and jsps

    yawmark wrote:
    saichand wrote:
    means that uploading files(.doc) or any files by using servlets and jsps
    [http://www.google.com/search?q=How+can+i+upload+files+in+servlets+and+also+using+jsp]
    ~Good topic titles are indeed brilliant search keywords. It's sad to see that it won't even come in the mind of the topicstarters to Google it before posting the topic.

  • Is it necessary that before import and export we set ORACLE_SID?

    Is it necessary that before import and export we set ORACLE_SID?

    if you set the variable you can do it so :
    ORACLE_SID=db1
    export ORACLE_SID
    exp joel/perez file=.............
    if you do not set the variable you can do it so :
    exp joel/perez@service_name file=.............
    Joel P�rez

  • After Install this yosemite my MBP getting heated than before ..and also used lil bit more ram

    After Install this yosemite my MBP getting heated than before ..and also used lil bit more ram

    Calibrating the battery is always a good option. It is far too soon for battery life to drop down as much as you mentioned. Once you do the calibration see what happens and if there are no improvements get applecare to send you a new one.

  • Use PL/SQL procedure to guard against malformed CSV files before upload

    Hi all,
    In my CSV upload utility, I've implemented error checking procedures, but they are invoked only AFTER data has been uploaded to temp table. This doesn't catch the following sample scenarios:
    1. The CSV is malformed, with some rows shifted due to fields and separators missing here and there.
    2. User has chosen a wrong CSV to upload (most likely number of fields mismatch)
    I'm wondering if it is a good idea to have procedure to read in the CSV, scan each line, count the number of fields (null fields but with delimiters showing the field exist is ok) for each record. If every single record matches the required number of fields for that table, then the CSV is ok, and the insert from external table to temp table is allow. This will ensure at least the CSV file has a valid matrix size for the target table, but rest of error checking is left until after temp table is populated.
    One of my concerns is, I specify "missing field values are null" in the external table parameters, which is necessary since not all fields are required. Does this specification causes a row with missing trailing separators still considered valid? If so then the stored procedure must be programmed to omit such a case.
    What do you think? Many thanks.

    Hi, Cuauhtemoc Amox
    Thank you for your advice. I have set web adi  using PL SQL interface.
    I have decided to go futher
    i have a procedure like that
    procedure delete_old_data ( p_id in number, p_description in varchar2)
    as
    begin
                   begin
                   if g_flag ='N' then
                   delete from xx_test;
                    g_flag='Y';
                    else null;
                    end if;
                    end;
    insert into xx_test (p_id,p_descriptiom);
    end;
    G_FLAG is a global variable with default value ='N'  in my package. When web_adi upload
    first row from excel sheet, then procedure delete all data from table and change g_flag to 'Y'.
    All other rows uploads succesfully; it  works fine.
    But when user change data in excel sheet and try to upload second time. DELETE_OLD_DATA procedure doesnt work in proper way.
    I have found what problem is. when user use same template and try to upload data several times there is same SESSION_ID in database.
    i.e. g_flag ='Y' when user try to upload second time. But when user log off and create template again it is work properly.
    My question is: How i can different upload attempts? May be there is some id for each upload proccess.
    if i can use this ID in my procedure user will have opportunity to use same template.
    Thank you

  • How to import data from CSV file into a table by using oracle forms

    Hi,
    I have a CSV file and i want to insert in oracle database in a table by using a button in oracle forms.
    the user can select CSV file by using open dialog .
    can any one help me to find method to make import and select file from the client machine ?
    thx.

    1. create table blob
    CREATE TABLE IB_LOVE
      DOC          BLOB,
      CONTRACT_NO  VARCHAR2(20 BYTE)                NOT NULL
    )2. use the code below to insert:
           INSERT INTO ordmgmt.ib_love
                       (contract_no, doc
                VALUES (:control.contract_no_input, NULL
           lb$result :=
             webutil_file_transfer.client_to_db (:b2.file_name, v_file_blob_name, v_col_blob_name,
                                                 'CONTRACT_NO = ' || :control.contract_no_input);
           :SYSTEM.message_level := 25;
           COMMIT;
           :SYSTEM.message_level := 0;3. use the code below to download
         if :control.CONTRACT_NO_INPUT is not null then
              vboolean :=   webutil_file_transfer.DB_To_Client_With_Progress(
                               vfilename,  --filename                       
                              'IB_LOVE', ---table of Blob item                       
                              'DOC',  --Blob column name                       
                              'CONTRACT_NO = ' || :CONTROL.CONTRACT_NO_INPUT, ---where clause to retrieve the record                       
                              'Downloading from Database', --Progress Bar title                       
                              'Wait to Complete'); --Progress bar subtitle  client_host('cmd /c start '||vfilename);
              client_host('cmd /c start '||vfilename);                 
         else
              errmsg('Please choose contract no');
         end if;4. use the code below to open file dialog
    x:= WEBUTIL_FILE.FILE_OPEN_DIALOG ( 'c:\', '*.gif', '|*.gif|*.gif|', 'My Open Window' ) ;
    :b2.FILE_NAME:=X;

  • After the upgrade to Mozilla 32 using csv files to import data does not work anymore. What should I do?

    Hello,
    After the latest upgrade to Mozilla 32 a previously recorded imacros scirpt in which the data was being imported into mozilla from a csv file keeps erroring out. This same problem occurred with the with the upgrade to Mozilla 30 and was fixed when Mozilla was upgraded to 31. Can you please advise on how to correct the problem so that csv files can be used in conjunction with imacros scripts. I would like to point out that imacros still works without the use of csv files, but when you introduce a csv to the script the imacros does not work.
    Thank you,
    Eugene

    hello eugene, please directly contact the imacros extension developers through the means provided at their [http://forum.iopus.com/viewforum.php?f=11 forum] - they can likely give you more detailed guidance and are the only ones who can fix bugs or make necessary adjustments in the addon for new firefox versions. thank you...

  • Renaming master files before import using metadata information

    In order to be sure that none of my master files has the same names I am using the following naming format:
    <camera model> YYYYMMDD HHMMSS
    Example: Nikon D2X 20070218 213445
    Until now I have used Photoshop CS2 in order to rename all my master files but it would be easier if is possible to automate the process. The date and time is taken from the metadata in each file. Which is the specific time the photo is taken and not just the "file creation date and time".
    Has anyone found a tool/automation that could do this?

    Karsten
    Have a look at DIM:
    http://www.alanlight.com/dim/Dim.htm
    In order to be 'future proof' I import and rename all images into a Finder folder structure created by the above. I then import them into Aperture - so I have the same structure in the Finder and in Aperture. I keep al RAW images in a different structure from JPEG images.
    DIM will do all you need
    Brian

Maybe you are looking for