SSIS package to load Source Control Explorer details into a table

Hi Guys,
We are maintaining our code in Source control explorer,TFS.
We have created a table in our database which will have columns like names, latest, last_checkin.
I like to read from TFS those values and load it into database.
We need to create reports based on the locations and send it to our client.
Any ideas or path i have to take in order to achieve this, will be very grateful.
Cheers!!!!!

Why would you use SSIS to read from TFS?
And then create a report?
Wrong design to my taste.
You need a lot of .Net code as you need TFS APIs used this way.
var server = RegisteredTfsConnections.GetProjectCollection(new Uri("http://hostname:8080/"));
var projects = TfsTeamProjectCollectionFactory.GetTeamProjectCollection(server);
var versionControl = (VersionControlServer)projects.GetService(typeof(VersionControlServer));
var newestDate = DateTime.MinValue;
Item newestItem = null;
var items = versionControl.GetItems("$/theproject/trunk/setup/*.msi");
foreach (var item in items.Items)
if (item.ItemType != ItemType.File)
continue;
if (item.CheckinDate > newestDate)
newestItem = item;
newestDate = item.CheckinDate;
newestItem.DownloadFile("C:\\temp\\" + Path.GetFileName(newestItem.ServerItem));
this still can be done in a Script Task, but defeats the purpose of using ETL.
Arthur My Blog

Similar Messages

  • SSIS package takes longer time when inserting data into temp tables

    querying records from one  server  and  inserting them into temp tables is taking longer time.
    are there any setting in package which  enhance the performance .

    will local temp table (#temp ) enhance the performance  ..
    If you're planning to use # tables in ssis make sure you read this
    http://consultingblogs.emc.com/jamiethomson/archive/2006/11/19/SSIS_3A00_-Using-temporary-tables.aspx
    Please Mark This As Answer if it helps to solve the issue Visakh ---------------------------- http://visakhm.blogspot.com/ https://www.facebook.com/VmBlogs

  • Show/Hide Directories in Source Control Explorer

    Is there a way to hide/ignore the directories that I do not need in Source Control Explorer?
    Thanks in advance!

    hi John
    no that is not possible.
    what you can do is
    Cload a folderwhich means it will not downloaded or compared etc.
    Please use Mark as Answer if my post solved your problem and use Vote As Helpful if a post was useful.

  • Identify missing files in source control explorer

    Is there a way to identify which files are in the workspace and not in the TFS source control solution? The "Compare" option in the "Source Control Explorer" is grayed out. Tried using the power tools
    but only the Analyzer is installed. The TFPT.exe is not installed. There is a solution in Stack Overflow however none of it works. Thanks in advance.

    Hi Clem,  
    Thanks for your post.
    In Source Control Explorer, right-click on source folder path and select Add Items to Folder…, then check which file(s) list in the opened dialog and can add to source control.
    We are trying to better understand customer views on social support experience, so your participation in this interview project would be greatly appreciated if you have time. Thanks for helping make community forums a great place.
    Click
    HERE to participate the survey.

  • Windows Explorer integration for Source Control Explorer

    Hi together,
    we've setupped an Online TFS to organize our Powershell Scripts in our Team. After that we've installed the Source Control Explorer for checking items in and out. From within the Source Control Explorer this works flawless.
    Some time ago i've worked with TortoiseSVN for Source Control and there were a Windows Explorer Integration which was super easy to use for making new local repositorys, checkin and checkout, is there something similar for TFS? Because we only write Powershell
    Scripts, we don't need Visual Studio and it would be easier to work directly in the Windows Explorer.
    Thanks for any tips and tricks!
    Rolf

    Hi Rolf,  
    Thanks for your post.
    You’re using VSO Account?
    What’s the version of your VS? if you’re using VS 2013, you can install the
    TFS 2013 Power Tools on your client. If you installed the TFS 2013 Power Tools on your VS 2013 client, after you mapped and got the source file to local workspace folder using Source Control Explorer, you can right-click on the local workspace folder and
    select Team Foundation Server menu to perform check in/out. 
    We are trying to better understand customer views on social support experience, so your participation in this interview project would be greatly appreciated if you have time. Thanks for helping make community forums a great place.
    Click
    HERE to participate the survey.

  • SSIS Package to load multiple excel sheets using foreach loop container

    I am creating a SSIS package which is accessing an excel file and then using a Foreach loop container to load each tab to sql server. However when I try to configure the Excel source, I get the following message
    "The argument system_comobject cannot be converted into parameter type. Additional Information - Object must implement iconvertible (mscorlib)"
    Any help to resolve would be much appreciated.
    thanks
    Scott

    If the title is only using column D and the rest of the columns are empty on sheet1/row1
    AND
    if only sheet1 as headers,
    THEN you can try something like the following
    do NOT select row 1 as header or anything.
    In your foreachloop > dataflow, you will start with EXCEL connection then Conditional Split inside which you will define one case for which expression should be like If column D has value like "Title expression" && Column A == "Column A Header
    name" THEN yay or nay. You might want to look into
    FINDSTRING function to assist with this (if you choose to use FINDSTRING, you are looking for !isNull for this case..If it finds the string you are looking for, it will pass an int value (title + header) if it doesn't find the string, it will pass
    NULL (data)) or you can say If Column A, B, C,E,F,G...is NULL and D is NOT NULL && Column A = Column A Header string, then Case 1 else default.
    The default output would be the real data.
    So when sheet 1 is processing, you will have Case 1 output with 2 rows (title + header) and Default output with the rest of the rows (real data).
    For other sheets, you will have Case 1 output with 0 rows (no title + header) and Default output with the data rows.
    The above sln assumes that the data in each sheet is similar in datatypes and length perspective, you have defined input/output datatypes properly, and etc.

  • Loading data from .csv file into Oracle Table

    Hi,
    I have a requirement where I need to populate data from .csv file into oracle table.
    Is there any mechanism so that i can follow the same?
    Any help will be fruitful.
    Thanks and regards

    You can use Sql Loader or External tables for your requirement
    Missed Karthick's post ...alredy there :)
    Edited by: Rajneesh Kumar on Dec 4, 2008 10:54 AM

  • Loading data from .csv file into existing table

    Hi,
    I have taken a look at several threads which talk about loading data from .csv file into existing /new table. Also checked out Vikas's application regarding the same. I am trying to explain my requirement with an example.
    I have a .csv file and I want the data to be loaded into an existing table. The timesheet table columns are -
    timesheet_entry_id,time_worked,timesheet_date,project_key .
    The csv columns are :
    project,utilization,project_key,timesheet_category,employee,timesheet_date , hours_worked etc.
    What I needed to know is that before the csv data is loaded into the timesheet table is there any way of validating the project key ( which is the primary key of the projects table) with the projects table . I need to perform similar validations with other columns like customer_id from customers table. Basically the loading should be done after validating if the data exists in the parent table. Has anyone done this kind of loading through the APEX utility-data load.Or is there another method of accomplishing the same.
    Does Vikas's application do what the utility does ( i am assuming that the code being from 2005 the utility was not incorporated in APEX at that time). Any helpful advise is greatly appreciated.
    Thanks,
    Anjali

    Hi Anjali,
    Take a look at these threads which might outline different ways to do it -
    File Browse, File Upload
    Loading CSV file using external table
    Loading a CSV file into a table
    you can create hidden items in the page to validate previous records before insert data.
    Hope this helps,
    M Tajuddin
    http://tajuddin.whitepagesbd.com

  • Load text or txt file into a table when a button is clicked by the user

    Can anyone please help me,
    I have a form where a user comes and uploads a text file(Unicode UTF-8) and clicks a button(Upload) ,
    when the button is clicked - it should load the text file data into the database table for the corresponding columns.
    Note: in text file columns are sapreted by tab(\t)
    & First row of text file contains column names.
    & Table name should be that text file name
    Can anyone please suggest me a possible solution or an approach.
    Thanks,
    Rathore
    Edited by: Rathore on May 14, 2010 1:49 AM

    Hi Legends and experts
    please reply give me some solutions

  • Loading data from multiple files to multiple tables

    How should I approach on creating SSIS package to load data from multiple files to multiple tables. Also, Files will have data which might overlap so I might have to create stored procedure for it. Ex. 1st day file -data from au.1 - aug 10 and 2nd day
    file might have data from aug.5 to aug 15.  So I might have to look for max and min date and truncate table with in that date range.

    thats ok. ForEachLoop would be able to iterate through the files. You can declare a variable inside loop to capture the filenames. Choose fully qualified as the option in loop
    Then inside loop
    1. Add execute sql task to delete overlapping data from the table. One question here is where will you get date from? Does it come inside filename?
    2. Add a data flow task with file source pointing to file .For this add a suitable connection manager (Excel/Flat file etc) and map the connection string property to filename variable using expressions
    3. Add a OLEDB Destination to point to table. You can use table or view from variable - fast load option and map to variable to make tablename dynamic and just set corresponding value for the variable to get correct tablename
    Please Mark This As Answer if it helps to solve the issue Visakh ---------------------------- http://visakhm.blogspot.com/ https://www.facebook.com/VmBlogs

  • Step by Step Instructions of loading Excel into SSIS via OLE Data Source Microsoft.ACE instead of Excel connection manager

    Hi,
    I have a very large Excel workbook which I need to load into SSIS. Using the default Excel connection manager SSIS seems to load the entire work sheet into memory before performing any operation even after setting IMEX=1 within the connection string.
    Can anyone provide me with step by step instructions on how to use  OLE Data Source Microsoft.ACE please?
    I've found the link; -
    https://www.connectionstrings.com/ace-oledb-12-0/
    Which looks very useful.
    Since I currently have Excel 2010 as my data source I tried; -
    Data Source=C:\PoC\POC_Data_Source\my.xlsx;Provider=Microsoft.ACE.OLEDB.12.0;Extended Properties="Excel 12.0 Xml;HDR=YES ;IMEX=1";
    The connection test successfully. However SSIS still seems to be loading the entire work sheet into memory before starting the data load.
    Kind Regards,
    Kieran.
    Kieran Patrick Wood http://www.innovativebusinessintelligence.com http://uk.linkedin.com/in/kieranpatrickwood http://kieranwood.wordpress.com/

    Thanks again Koen,
    I am developing remotely on a development server. So when I run SSDT I am running it within the development server box. My current technology / environment mix means I can't run SSDT from my local desktop.
    I am pretty sure that scanning of the rows is an issue causing performance, e.g. if I just click on the data source component SSDT freezes for several minutes, unless I just have a cut down version of the Excel file, say first 5,000 rows, then the response
    of SSDT is acceptable.
    Also when I have SSDT pointing to the full spreadsheet and I click around in the associated Data Flow I notice devenv.exe* 32 slowing grabbing 4gb of RAM and releasing it according to Windows Task Manager. 
    Kind Regards,
    Kieran.
    Kieran Patrick Wood http://www.innovativebusinessintelligence.com http://uk.linkedin.com/in/kieranpatrickwood http://kieranwood.wordpress.com/

  • Error executing SSIS packages from Agent

    Hi All,
         I designed SSIS package that loads data from Oracle table to SQl server table along with loading data into Diension and fact tables using SSIS packages only. First I created all SSIS packages that loads data into Staging tables from
    Oracle table in next level loading data into Dimension table and final level is loading data into Fact table. At lasts I created One master package there I'm calling all packages (Execute package task) differently.
    Configuration type: Parent package configuration for All child packages. XML config for parent package.
    When I execute SSIS package from BIDS data is loading up to Fact table. When I execute SSIS packages through Sql server agent ( Using File System Deployment)When I execute SSIS package from Sql server agent it is throwing an error message like 
    Message
    Executed as user: . Microsoft (R) SQL Server Execute Package Utility  Version 11.0.2100.60 for 32-bit  Copyright (C) Microsoft Corporation. All rights reserved.    Started:  10:45:10 PM  Error: 2013-10-17 22:45:12.60    
    Code: 0xC00220E6     Source: Master Data      Description: There is no project to reference.  End Error  Error: 2013-10-17 22:45:12.61     Code: 0xC0024107     Source:
    Master Data      Description: There were errors during task validation.  End Error  DTExec: The package execution returned DTSER_FAILURE (1).  Started:  10:45:10 PM  Finished: 10:45:13 PM  Elapsed: 
    2.59 seconds.  The package execution failed.  The step failed.
    I'm uploaded config file also while at the time of scheduling job.
    Regards,
    Praveen
    Regards, Praveen

    Hi Praveen,
    The issue may occur if the SQL Server Agent Service Account or Proxy account doesn’t have sufficient permissions. Please create a SQL Server Agent Proxy account that has the permissions to execute the package and perform update operations on the SQL Server
    fact tables.
    Reference:
    http://technet.microsoft.com/en-us/library/ms175834.aspx
    Regards,
    We
    are trying to better understand customer views on social support experience, so your participation in this
    interview project would be greatly appreciated if you have time.
    Thanks for helping make community forums a great place.

  • Issue with Dataflow Task in SSIS Package

    Hello Everyone,
    I have recently created an SSIS package to Load data from Flat file to SQL Server Table, it was working fine. Now I have bought a new laptop and copied the projects into this new system and changed all the connection managers(Flat files) accordingly. 
    Problem is, when I previously ran the package(With File1) in old laptop and check the distinct count on a table it was perfect, but when I ran the same package with same File on New Laptop, the Distinct count gives me an issue with particular record.
    Eg: Distinct Count with old laptop Run , 10,20,100
          Distinct Count with old New Run , 10,20,100  , 100
    There is a Space being generated for a particular record, when I run the package on new laptop with same File as old one.
    Any suggestions on this, I could not exactly figured out the Issue.
    Please Help.
    Thanks
    Thanks, Please Help People When they need..!!! Mark as answered if your problem is solved.

    You need to tell the responders here how exactly to reproduce your issue and provide the file.
    I do not know how the "distinct count" works.
    Seems to me it appears to be a database issue not SSIS.
    Arthur
    MyBlog
    Twitter
    When I perform the above query on old laptop, this gives me a result set of
    10
    20
    100
    When I perform the Same query on New laptop, this gives me a result set of
    10
    20
    100
    '100  ' (No Quotes please see the spaces generated)
    Thanks, Please Help People When they need..!!! Mark as answered if your problem is solved.
    Can you give us an idea of tasks you're having after the file source?
    Particulary see steps where you are doing some data manipulations.
    Please Mark This As Answer if it solved your issue
    Please Mark This As Helpful if it helps to solve your issue
    Visakh
    My MSDN Page
    My Personal Blog
    My Facebook Page
    I am using simple Flatfile source and oledb destination in here. this is causing issue, all the other tasks after this are loading perfectly except this.
    Thanks, Please Help People When they need..!!! Mark as answered if your problem is solved.

  • How to create a dynamic SSIS package for multiple flat file destinations

    Hi,
    I have to create a ssis package which has single data flow task and inside that I have 23 source (sql- select * from - statements)- destination (flat files, 23 distinct) connection.
    Now for each product I have to create separate SSIS package (i.e. if prod=abc then these read select * from abc_tables and 23 abc_ txt files)
    I want to do it dynamically, means only single package and inside that variables will take select * values for each source-dest connection (so i believe 23 variables) and same for destination flat files.
    Let me know. :)
    ANK HIT - if reply helps, please mark it as ANSWER or helpful post

    Sorry It seems you're contradicting yourself. you say I know my source and dest structure and the you're
    asking all I want is to have a dynamic structure
    what does that mean?
    and reading your next sentence
    I want to run a package for 5 products, instead of creating 5 ssis packages with 23 source- dest connection, I would
    like to have one with only 23 source- dest connections 
    What I feel is what you're looking for is to  have a looping structure to loop through each of the 5
    products.
    In that case what you could do is this
    1. Create a object variable in SSIS 
    2. Use a Execute SQL Task to populate the variable with all available products (I think you'll have a master table for that). Set ResultSet property to Full ResultSet and then in ResultSet tab map Object variable to 0 th index
    3. use a ForEachLoop container with ADO .NET recordset enumerator and map to object variable. Create a variable of datatype same as that of Product identifier field to get individual values out
    4. Inside loop create your data flow task with 23 source destination connection. In the query part use a parameter for product field and map it to the variable containing product value to get only data for the product.
    Please Mark This As Answer if it helps to solve the issue Visakh ---------------------------- http://visakhm.blogspot.com/ https://www.facebook.com/VmBlogs

  • SSIS Package compiled successful, in SQL Server Integration Service package executed sucessful, But fail to run in MS SQL Job Scheduler

    Hi Everyone,
    I having a problem to transfer data from MS SQL 2005 to IBMAS400. Previously my SSIS was running perfectly but there is some changes I need to be done in order for the system to work well. Considers my changes are minimal & just for upgrades (but I did
    include DELETE statements to truncate AS400 table before I insert fresh data from MS SQL table to the same AS400 table), so I compile my SSIS package & it run successfully & I passed it into MS SQL Integrated Service as 1 of the packages & manually
    executed the package & the result is the same, that mean it was successful again but when I try to run it in a MS SQL Job Scheduler, the job failed with these message shown below as extracted from the job View history. 
    Date today
    Log Job History (MSSQLToAS400)
    Step ID 1
    Server MSSQLServer
    Job Name MSSQLToAS400
    Step Name pumptoAS400
    Duration 00:00:36
    Sql Severity 0
    Sql Message ID 0
    Operator Emailed
    Operator Net sent
    Operator Paged
    Retries Attempted 0
    Message
    Executed as user: MSSQLServer\SYSTEM. ... 9.00.4035.00 for 32-bit  Copyright (C) Microsoft Corp 1984-2005. All rights reserved.    
    Started:  today time  
    Error: on today time     
    Code: 0xC0202009     Source: SSISMSSQLToAS400 Connection manager "SourceToDestinationOLEDB"     
    Description: SSIS Error Code DTS_E_OLEDBERROR.  An OLE DB error has occurred. 
    Error code: 0x80004005.  An OLE DB record is available.  
    Source: "IBMDA400 Session"  
    Hresult: 0x80004005  
    Description: "CWBSY0002 - Password for user AS400ADMIN on system AS400SYSTEM is not correct ".  End Error  
    Error: today     
    Code: 0xC020801C     
    Source: Data Flow Task OLE DB Destination [5160]     
    Description: SSIS Error Code DTS_E_CANNOTACQUIRECONNECTIONFROMCONNECTIONMANAGER.  The AcquireConnection method call to the connection manager "DestinationClearData" failed with error code 0xC0202009.  There may be error messages posted before
    this with more information on why the AcquireConnection method ca...  The package execution fa...  The step failed.
    So I hope somebody can shed some hints or tips for me to overcome time problem of mine. Thanks for your help in advance. As I had scoured thoroughout MSDN forums & found none solution for my problem yet. 
    PS: In the SQL Integrated Services when I deployed the package I set the security of the packages to Rely on server... 
    Hope this will help.

    Hi Ironmaidenroxz,
    From the message “Executed as user: MSSQLServer\SYSTEM”, we can see that the SQL Server Agent job ran under the Local System account. However, a Local System account doesn’t have the network rights natively, therefore, the job failed to communicate with
    the remote IBMAS400 server.
    To address this issue, you need to create a proxy account for SQL Server Agent to run the job. When creating the credentials for the proxy account, you can use the Windows domain account under which you executed the package manually.
    References:
    How to: Create a Credential
    How to: Create a Proxy
    Regards,
    Mike Yin
    TechNet Community Support

Maybe you are looking for