Deleting old flat files inside a SSIS package - SSIS 2005

Hi,
after each flat file import, I need to move the acquired file into an history folder and deleting old files of 1 week ago.
How can I cancel these files, acting inside my SSIS 2005 pckg?
Many thanks

While in foreach loop, after a successful import just move the file with the File System Task. Deleting old files is a little trickier, you can also loop thru the files and delete those older than based on name (if the date in the name of the file) or use
File Properties Task to get file's age to a variable and then use a precedence constraint to decide on deleting or not
conditionally.
Your question is general. Many devs did this before, just Bing/Google examples out.
Arthur My Blog

Similar Messages

  • Ola Hallengren maintenance solution TSQL script not deleting old backup files from my local SQL server

    I have looked on Ola's site (https://ola.hallengren.com/frequently-asked-questions.html) and I did see this:
    DatabaseBackup is not deleting old backup files. What could the problem be?
    Verify that the SQL Server and SQL Server Agent service accounts have full control of the backup directory.
    If that directory is a network share, verify that the SQL Server and SQL Server Agent service accounts have full control of the network share.
    If you are using a proxy account, verify that the account is a member of the sysadmin server role and that it has full control of the backup directory and network share.
    Verify that the file is not locked in the file system; for example, a backup or antivirus software could be locking the file.
    DatabaseBackup has been designed not to delete transaction log backups that are newer than the most recent full or differential backup. This could explain why transaction log backups are not being deleted.
    I followed these instructions and then I right clicked the "sp_delete_backuphistory" job under "SQL Server Agent > Jobs" folder and clicked the "Start job at step ..." option. The job successfully ran, but my .bak backup
    files were still on located on my separate drive partition I use specifically for these backups. 
    The SQL Server and the SQL Server Agent are both using the same AD account. 
    Would any of you SQL gurus out there know how to resolve this?
    Thank you 

    The sp_delete_backuphistory job only deletes the historical info from the backup & restore tables in the msdb database. It does not do anything to the files on disk.
    https://msdn.microsoft.com/en-us/library/ms188328.aspx
    The code to actually purge the backup files from disk is inside the DatabaseBackup stored procedure. It is controlled by using the @CleanupTime parameter for that stored proc. Verify the value for that parameter is low enough to delete your files on disk. The
    value is in hours, and I believe the default is 48 hours. The SQL agent job name usually starts with "DatabaseBackup...".

  • Flat files data comma separated using SSIS.

    Hi,
    I have multiple flat files which come in comma separated columns. See example below :
    Customer Data
    CustID,FName,LName,Disease,Email,Phone
    12345,Xyz,Smit,Bronchitis, Asthma and fever,[email protected],80000000
    12346,Abc,Doe,fever Headache,[email protected],90000000
    12347,Klu,joe,Sugar, cough and fever,[email protected],12345678
    Please look at the ID's 12345 and 12347. The disease column has a internal comma space between. How do i remove the comma spaces in the disease column, so that it can be loaded from flat file to sql table using SSIS. ?
    Please help !
    Thanks

    Here is a full solution base on my post above (first option)
    1. create temp table (Give it a unique name):
    create table #T (Txt NVARCHAR(MAX))
    GO
    2. Insert all the data into temporary table. Each line in the text file, is a value for one column in a row in the table.
    -- I will jump to the table and use simple insert.
    -- If you have problem with step 1 then please inform us (this is simple bulk insert basically)
    insert #T (Txt) values
    ('1234435,Xyz,Stemit,Brfsdonchitis, Asthma and fever,[email protected],80000000'),
    ('12346,Agjdfjbc,Doge,fevhhhher Headsxdshhache,[email protected],90000000'),
    ('123447,Klu,joe,Sugar, cough and fever,[email protected],12345678')
    GO
    the result should be like this:
    Txt
    1234435,Xyz,Stemit,Brfsdonchitis, Asthma and fever,[email protected],80000000
    12346,Agjdfjbc,Doge,fevhhhher Headsxdshhache,[email protected],90000000
    123447,Klu,joe,Sugar, cough and fever,[email protected],12345678
    I use a SPLIT Function named Split_CLR_Fn. This is a CLR Split function that get input <string to split> and <string as delimiter,> and it return table with 2 columns ID, SplitData
    For example if you use: SELECT * from Split_CLR_Fn('text1,text2,text3,',') then you get result:
    ID SplitData
    1 Text1
    2 Text2
    3 Text3
    ** You can find in the internet several good functions, I HIGHLY RECOMMENDED NOT TO USE T-SQL FUNCTIONS but CLR FUNCTION. Check thi link to understand why:
    http://sqlperformance.com/2012/07/t-sql-queries/split-strings
    ** This is the best function that I know about and I use it, but I change the code a bit to return 2 columns and not just the SplitData as in this blog: http://sqlblog.com/blogs/adam_machanic/archive/2009/04/28/sqlclr-string-splitting-part-2-even-faster-even-more-scalable.aspx
    That's it :-) we are ready for the solution which is very simple
    Solution 1 (BAD solution but easy to write):
    select
    (select SplitData from Split_CLR_Fn(Txt,',') where ID = 1) CustID,
    (select SplitData from Split_CLR_Fn(Txt,',') where ID = 2) FName,
    (select SplitData from Split_CLR_Fn(Txt,',') where ID = 3) LName,
    STUFF((select ',' + SplitData from Split_CLR_Fn(Txt,',') where ID > 3 and ID < (select MAX(ID) from Split_CLR_Fn(Txt,',')) - 1 for XML path('')), 1 , 1,'') Disease,
    (select SplitData from Split_CLR_Fn(Txt,',') where ID = (select MAX(ID) from Split_CLR_Fn(Txt,',')) - 1) Email,
    (select SplitData from Split_CLR_Fn(Txt,',') where ID = (select MAX(ID) from Split_CLR_Fn(Txt,','))) Phone
    from #T
    GO
    Solution 2: better in this case since the format is constant (this is the solution I wrote about above)
    ;With MyCTE as (
    select
    Txt,
    SUBSTRING(Txt, 1, CHARINDEX(',', Txt, 1) - 1) as CustID
    , SUBSTRING(
    Txt
    ,CHARINDEX(',', Txt, 1) + 1 -- I start from the end of preview len
    , CHARINDEX(',', Txt, CHARINDEX(',', Txt, 1)+1)- CHARINDEX(',', Txt, 1) - 1
    ) as FName
    , SUBSTRING(
    Txt
    ,CHARINDEX(',', Txt, CHARINDEX(',', Txt, 1)+1)+1 -- I start from the end of preview len
    , CHARINDEX(',', Txt, CHARINDEX(',', Txt, CHARINDEX(',', Txt, 1)+1)+1) - CHARINDEX(',', Txt, CHARINDEX(',', Txt, 1)+1) - 1
    ) as LName
    , RIGHT(Txt, CHARINDEX(',', REVERSE(Txt), 1) - 1) as Phone
    , RIGHT(LEFT(Txt, Len(Txt) - Len(RIGHT(Txt, CHARINDEX(',', REVERSE(Txt), 1) - 1)) - 1), CHARINDEX(',', REVERSE(LEFT(Txt, Len(Txt) - Len(RIGHT(Txt, CHARINDEX(',', REVERSE(Txt), 1) - 1)) - 1)), 1) - 1) as Email
    from #T
    select CustID,FName,LName, Phone, Email, SUBSTRING(Txt, Len(CustID) + Len(FName) + Len(LName) + 4, Len(Txt) - Len(Email) - LEN(Phone) - Len(CustID) - Len(FName) - Len(LName) - 5) as Disease
    from MyCTE
    I hope that this is useful :-)
      Ronen Ariely
     [Personal Site]    [Blog]    [Facebook]

  • HT4007 I have just transferred all my raw, jpeg & tiff files into LR4 and using to opportunity to delete old raw files to free up disc space. How can i completely remove a raw file from my msc disc??

    I have just transferred all my raw, jpeg & tiff files into LR4 and using the  opportunity to delete old raw files to free up disc space. How can i completely remove a raw file from my mac disc??

    I'd be careful here: LR references files. It can copy them, like from an SD card to your hard drive, but it doesn't necessarily make duplicates. So when you say you "transferred" those files "into" LR it just means you referenced existing files. And if you delete those they're gone, unless you've made copies. LR does not import files by copying them into a managed library like Aperture does.

  • HT2731 i hav iphone and i restart it and delete all the files inside how can i start again my iphoe it say i need a valid sim with no pin lock to activate it again pls help

    i hav iphone and i restart it and delete all the files inside how can i start it again it say i need a valid sim with no pin lock to activate iphone again

    ICCID unknown is a critical failure, usually caused by hacking or jailbreaking the phone. It can not be fixed. The phone will need to be replaced.

  • When migrating ssis packages from 2005 to 2012 Activex Script Task component doesn't exist in 2012 version

    Hello,
    We are migrating the ssis packages from 2005 to 2012.
    I'm unable to convert Activex Script Task from 2005 vesrsion to 2012 because in 2012 version Activex Script Task doesn't exist.
    Can anyone please let me know what is the alternative way to convert Activex Script Task from 2005 vesrsion to 2012?

    Hi Vinay9738,
    Have you tried to upgrade the SSIS 2005 packages to SSIS 2012 packages by using the SSIS Package Upgrade wizard? In certain cases, ActiveX script in SSIS won’t work and we need to either modify the script or replace the ActiveX Script with stock SSIS
    tasks. You can find the mapping between some most common patterns used in DTS ActiveX Script and SSIS native tasks from the following link:
    http://help.pragmaticworks.com/dtsxchange/scr/ActiveX%20Script%20Task.htm
    Here is also a useful link about how to convert ADODB object of ActiveX Script to SSIS tasks:
    http://help.pragmaticworks.com/dtsxchange/scr/FAQ%20-%20How%20to%20convert%20ADODB%20object%20of%20ActiveX%20Script%20to%20native%20SSIS%20Task.htm
    Regards,
    Mike Yin
    TechNet Community Support

  • Reading flat files with variable names in SSIS

    I have a ssis package that reads a flat file from a network drive (using a flat file connection manager) and loads the data into sql server tables. I have this working on a fixed file name, however in reality the file name will not be the
    same from run to run.      Essentially, I need to check a folder each day and if there is a file there with a certain prefix (for example, 'datafile'), I need to process this file(s). In other words, if there is a file in the folder
    called datafilexyz, my process needs to read it in and process it.    If there are files named datafileabc, datafiledef, and testfile123, I need to read in and process the datafileabc and datafiledef flat files.    
    I'm not sure how to make this work.     I haven't had any SSIS training, just what I can find on the internet and looking at existing SSIS packages (I haven't found any that do what I'm trying to do here) so I'm kind of lost.   
    Any help is appreciated.  

    this is working well.    How can I, after loading each flat file, move the flat file to an archive folder?    I'm trying to use a file system task, but doing something wrong.    I created and reference
    an archive folder connection manager in the destination connection, and reference the original folder connection manager for the source connection, but get 'sourcepath is not valid on operation movefile'.    I think a file system task is what
    is needed (within the foreach loop after the data load for each flat file), but I'm not doing something correctly.
    Sounds fine except for one thing. Did you assign the filename variable to connection string property of the source file connection manager used by file system task?
    Also it should be existing file option you should choose for
    source connection and existing folder for destination.
    Please Mark This As Answer if it solved your issue
    Please Mark This As Helpful if it helps to solve your issue
    Visakh
    My MSDN Page
    My Personal Blog
    My Facebook Page

  • How to relink files inside an avchd package ?

    I messed up a library and lost a project, then I found a copy of it in the backup folder, now when I open the project it asked me to relink files, but a part of them are inside an avchd package. How to relink those files ?
    Thanks

    you mean to normally import them or is there a specific reimport function ?
    Thanks

  • Trouble burning cd - delete old/temp files?

    Hi there - I am having issues burning cds. Burning stops mid way due to an unknown error. I thought in the past, I rectified this by deleting old files, but cannot remember where to find them. Any suggestions on this? I am using the most recent version of iTunes, and have successfully run the cd burner diagnostics, with successful test results. Thanks!

    Yes...in iTunes and I also tried burning via Sonic, and got the same result

  • My Macbook Pro won't auto delete old TM files

    My wife's Macbook pro won't auto delete old files on the TM disc. I think it has to do with using a time capsule. I'd rather stop that and just use the WD Passport but how do I do that?

    Hi..
    My power cord has been trying to go out on me for a few weeks and tonight it just went capooey
    If there's a power issue on your Mac, it's doubtful uninstalling and reinstalling software will make any difference.
    Try Troubleshooting MagSafe adapters
    If the adapter is ok, try Resetting the System Management Controller (SMC)

  • Data convertion while exporting data into flat files using export wizard in ssis

    Hi ,
    while exporting data to flat file through export wizard the source table is having NVARCHAR types.
    could you please help me on how to do the data convertion while using the export wizard?
    Thanks.

    Hi Avs sai,
    By default, the columns in the destination flat file will be non-Unicode columns, e.g. the data type of the columns will be DT_STR. If you want to keep the original DT_WSTR data type of the input column when outputting to the destination file, you can check
    the Unicode option on the “Choose a Destination” page of the SQL Server Import and Export Wizard. Then, on the “Configure Flat File Destination” page, you can click the Edit Mappings…“ button to check the data types. Please see the screenshot:
    Regards,
    Mike Yin
    TechNet Community Support

  • Deleting Old Scratch Files Immediately

    I've got QMaster working OK with Compressor so far. However, I had started a very large batch render on a Friday, only to come back to find the scratch disk full, with some files done and some not done. After investigating, I found out that the scratch disk (170GB free when I left) was full, and that none of the files were cleaned up after the successful jobs were finished, leaving a bunch of useless files on the drive, and causing the remaining files in the batch to die. I've got about 750GB of video to compress and send to a client, and I'd rather not have to do it one small batch at a time (24GB largest single file).
    So, since this is not a matter of deleting files that are 1 day old (the lowest setting in the QMaster CP AFAIK), is there an easy way to clean out files that are no longer needed after a successful completion?
    (Send a e-mail message on completion that will trigger a shell script? I can do that, but I'd really rather not have to, especially since the computers in the cluster are not my department's machines.)
    p.s. I'd love to be able to avoid QMaster copying the originals to the scratch drive as well, say by sharing the volume, but that doesn't seem avoidable, and since it's automated, I'll live with it

    You can try moving your cluster storage directory to a volume with more space. Where is your destination for the output files? The only time I see files in the cluster storage directory is when a batch failed. Running out of disk space would definitely cause a failure.

  • Delete old application files  after update?

    I downloaded the 3.4.1 update and want to delete the 3.4 application files to free up more space on my laptop's hard drive.  Is this OK?
    Also, what aboout deleting older versions of catalogs?  Thanks.

    When you update, this removes the previous application files and replaces them with new ones. It will leave an empty folder.
    If you no longer use a particular catalog you can delete it, but make sure you really don't want the catalog any more. Installing an update doesn't change the catalog only the application.
    The only time a new catalog is created by LR (rather than by yourself), is when upgrading (rather than updating) from LR 2 to 3 and in this case LR creates a new catalog from your old one, leaving the old one intact. Once you are happy with everything you can remove the LR 2 catalog. Personally I don't. I kept the LR 1 catalog until LR 3 came out and will keep the LR 2 catalog until LR 4 comes out, but I am cautious like that and have loads of hard drives, so space isn't a problem. I just move the old catalog to a backup location...you never know what is going to happen and I may for some completely unknown reason need it some day..

  • How we will delete old recording files in ES2  ?.

    Hi ,
    The error ::
    2010-02-26 16:25:16,811 ERROR [STDERR] Feb 26, 2010 4:25:16 PM com.adobe.idp.auditworkflow.dsc.service.storage.ProcessRecordingStorageImpl manageRecordingFileSizes
    WARNING: deleting oldest recording(s) due to space limitation.
    I  am seeing above error many times in log file. Then googled i got  http://forums.adobe.com/thread/506674?tstart=0 thread. But  i know this one.
    Is there any page in adminui  ,  where i can delete all old recording  at a shot.
    Is there any  way  where i can improve  buffer size of server ?   Becuase some times  i am getting error like below :
    2010-02-26 16:12:12,558 WARN  [com.adobe.idp.dsc.workmanager.workload.policy.MemoryPolicy] Memory threshold exceeded: maximum limit 95.0, current value 106.0
    2010-02-26 16:12:12,559 WARN  [com.adobe.idp.dsc.workmanager.workload.policy.WorkPolicyEvaluationBackgroundTask] Policies violated for statistics on 'adobejb_server1:wm_default'. Cause: com.adobe.idp.dsc.workmanager.workload.policy.WorkPolicyViolationException: Work policy violated. Will attempt to recover in 60000 ms
    Please help me out !!

    Here's the help topic:
    http://help.adobe.com/en_US/livecycle/9.0/workbenchHelp/000375.html#1548929
    Here's how to configure the number of recordings to keep:
    http://help.adobe.com/en_US/livecycle/9.0/workbenchHelp/001482.html#1608026
    scott

  • I accidentally deleted all the files inside Quicktime, and now when I try to import to iMovie it won't let me...how do I fix this?

    How do I get them back? It's from the Quicktime that already came with my Mac...please help me...It won't let me import movies to iMovie!!

    What installer software package files? Could you tell me how to get there?
    Afraid I'm not a mind reader. You have to download the installer packages from whatever source provides those installer packages. For instance, you would download the DivX installer package file from DivX. The Perian codec installer package would found at the Perian web site. QT "Pro" codec components installers may be include with older Motion or FCP Suite software or from the Apple site, etc., etc., etc. At this point all I know for sure is that that you have part of the DivX package installed and should reinstall the entire package if you plan to use it for anything.
    Everyone's codec component configuration may be different depending on what codecs the need to use. Perhaps it would be easier if I provide an image showing the codec components installed on my Snow Leopard configured laptop:
    This is a very basic set-up since I do most of my video work on my iMac which has FCP installed on it. In this case you can see:
    a) AC3MovieImport.component (Apple)
    b) AppleIntermediaCodec.component (Apple)
    c) AppleMPEG2Codec.component (Apple)
    d) Elgato Turbo.component (Elgato)
    e) EyeTV MPEG Support.component (Elgato)
    f) Flip4Mac WMV Advanced.component (Telestream)
    g) Flip4Mac WMV Export.component (Telestream)
    h) Flip4Mac WMV Import.component (Telestream)
    i) Perian.component (Perian)
    j) x264Encoder.component (MacUpdate)
    The first three codecs were installed by my System Installer w/iMovie package (i.e., pre-installed on system when purchased or added afterwards by iMovie). The Elgato codec installers came with with my Turbo H.264 and EyeTV hardware devices. (I you don't have such devices, then you don't needcan't use these codecs.) The WMV codecs were purchased from Telestream. The Perian codec was a free download from the Perian web site and is used for DivX, XviD, 3ivX, DTS, AC3, MKV, FLV, and other audio/video/file type support. (I.e., I don't use the DivX package.) The X.264 codec is a leftover from my old Leopard system and not really used much anymore since I now prefer using HandBrake GUI for most X.264 custom encodes.
    At this point, would assume you need to rerun the Snow Leopard/iMovie Installer if you purchased the system with system pre-installed or purchased the it as a system upgrade. Another alternative would be to find another Snow Leopard OS platform and simply copy the missing Apple iMovie codecs from a working system to your non-working system. All other codecs are up to you. Only you know what third-party codecs were installed on your system. And if you don't remember, you will probably find out what you are missing every time you try to play a video containing an unsupported codec.
    Since you seem to be very inexperienced here, you may wish to consider joining a local user group or seeking some one-on-one assistance from someone in your local area.

Maybe you are looking for