Mirroring, Replication or SSIS package

Im wanting to build a staging server for the ETL work to be carried out on my databases before extracting into my Datawarehouse.
Whats the best way to get a nightly copy my source database onto my staging server.
I don't need everything in the source databases, a lot of the tables are redundant so I would like the option of only pulling across the tables I need.
Mirroring looks like I need to mirror the full database, replication seems a bit of a sledge hammer for what I need, and ETL process has a bit of work but this may be the best.
I may be wrong on these any you guys recommend whats the best tool.
Thanks

Another method could be having a SSIS which transfers the only needed columns to the staging server.
Best Regards,Uri Dimant SQL Server MVP,
http://sqlblog.com/blogs/uri_dimant/
MS SQL optimization: MS SQL Development and Optimization
MS SQL Consulting:
Large scale of database and data cleansing
Remote DBA Services:
Improves MS SQL Database Performance
SQL Server Integration Services:
Business Intelligence

Similar Messages

  • How do I automatically backup SQL Agent jobs and SSIS packages on the mirror daily?

    I have seen this question asked before but I could not find a satisfactory answer. What is the best solution to get your SQL Agent jobs/schedules/etc. and your SSIS packages on the mirror server? Here's the details:
    Server A is the principal with 2 DBs mirrored over to server B. Everything is fine and dandy, DBs are synched and all good. In Disaster Recovery testing, we need to bring up server B, which now will serve as the principal. Server A is inaccessible. Now,
    we need all our jobs that are setup in server A to be in server B, ready to go, but disabled. We also need all our SSIS packages properly stored. Yes, we store our packages in the MSDB in server A.
    Now, I can see a few answers coming my way.
    1- Backup the MSDB to server B. When you bring server B up as principal in DR, restore the MSDB. All your jobs,schedules,steps, SSIS packages will be there. Is this possible? Can this be done on server B without rendering it incapable of serving as the principal
    for the mirrored DBs? My fear with this option is that there may be information in the MSDB itself about the mirroring state of the DBs? Or is all that in the Master DB? Have you tried this?
    2- Right click each job, script them out, re-create them on server B... No, thank you very much. :) I am looking for an AUTOMATED, DAILY backup of the jobs and SSIS packages. Not a manual process. Yes, we do change jobs and packages quite often and doing
    the job twice on two servers is not an option.
    3- Use PowerShell.. Really? Are we going back to scripting at the command prompt like that, that fast?
    Since I fear option number 3 will be the only way to go, any hints on scripts you guys have used that you like that does what I need to do?
    Any other options?
    Any help GREATLY appreciated. :-) I can be sarcastic but I am a good guy..
    Raphael
    rferreira

    I would go with option number 3. Once  you have a script simple run it....
    param([string]$serverName,[string]$outputPath) 
    function script-SQLJobs([string]$server,[string]$outputfile) 
        [reflection.assembly]::LoadWithPartialName("Microsoft.SqlServer.Smo") | Out-Null 
        $srv = New-Object Microsoft.SqlServer.Management.Smo.Server("$server") 
        $db = New-Object Microsoft.SqlServer.Management.Smo.Database 
        $scrp = New-Object Microsoft.SqlServer.Management.Smo.Scripter($srv) 
        $scrp.Options.ScriptDrops = $FALSE 
        $scrp.Options.WithDependencies = $TRUE 
        $jobs = $srv.JobServer.get_Jobs() 
        $jobs=$jobs | Where-Object {$_.Name -notlike "sys*"}     
        foreach($job in $jobs) 
            $script=$job.Script() 
            $script >> $outputfile 
            "GO" >> $outputfile 
    ---script-SQLJobs "SQLSRV12" "C:\Jobs\test.txt" 
    Best Regards,Uri Dimant SQL Server MVP,
    http://sqlblog.com/blogs/uri_dimant/
    MS SQL optimization: MS SQL Development and Optimization
    MS SQL Blog:
    Large scale of database and data cleansing
    Remote DBA Services:
    Improves MS SQL Database Performance

  • SSIS package is taking more time on production server than test server

    Same SSIS package was taking 18 hours on production and was just taking 6-7 hours on test server. When I increased the frequency of the Tlog backup file then it dropped to 11 hours. But still the difference of 3-4 hours is there.
    I had doubt that production server might have online transaction causing this issues, however this was proved wrong when I shutdown production sites and then also same output.
    Any idea where I am missing..
    Santosh Singh

    Thanks for taking time and trying to understand the explanation I had shared, please find comments inline:
    Are they performing server or database maintenance in the same window of time when the packages run. These include backups, index re-organize/rebuilt, partition switching, DBCC checks, statistics update, etc.?
    Comments: I am DBA too for them so I know better what is going on. No such jobs are executing during this window.
    What is the recovery model being used in production Full, Simple, Bulk-logged?
    Comments: Full
    Are the indexes and statistics being maintained in the source system you are extracting data from?
    Comments:Yes
    Have SQL Server configurations been modified such as Max Degree of Parallelism and Parallelism thresholds?
    Comments: Yes it was earlier, then I changed that back to Default while comparing Test environment which has everything default.
    Are there any special trace flags used to start up SQL Server in production?
    Comments: No only Trace 1118 for Tempdb enabled.
    Is SQL Server Auditing, Compression or Encryption being used in production?
    Comments: Only Backup compression has been enabled, no Auditing and no Encryption
    Is there Replication, Mirroring, Log-Shipping, Change Data Capture, Change Tracking being used in production?
    Comments: No Replication, Mirroring, Log-Shipping, Change Data Capture, Change Tracking being used in production. Earlier we had log shipping, however as of now log shipping as well. Its long time ago we removed log shipping.
    Are there other applications running along with SQL Server in the same Windows Server?
    Comments: No only sql server.
    Has SQL Server max memory changed from the default value? How much RAM is left for Integration Services and Windows? Check if paging is occurring.
    Comments: This is cluster setup for two instances on each node, so yes default memory is not still i had to adjust, however i didn't find high memory usage or memory issues in all log like sql, windows. 28 GB left for Integration Services and Windows
    and 50 GB-50 GB between two sql server instances. Paging looks normal too.
    Are there query hints being used in the SELECT queries?
    Comments:  Query we have bad, however this is our last target. The same query is executing in 6 hours in Test server and why it is taking now 11 hours is  the question of the hour for me.
    Thanks for all help again.
    Please let me know if I am missing anything.
    Santosh Singh

  • Sp_replmonitorhelppublication in a SSIS package

    Hi
    The SSIS package has a step where it calls for a DB snapshot replication (SQL Agent Job) to start after it has updated the database on server A.
    I want SSIS to wait till this snapshot generation and subscription has finished before going to the next task. The following task requires access to the tables that are being replicated, therefore need to be available.
    Will appreciated if someone could advise on how to use sp_replmonitorhelppublication and SP_replmonitorhelpsubscription to get the status for replication and only allow the SSIS package to progress once replication is finished.
    Thanks
    DBA

    Hi Hailey, you can use a Execute SQL Task to run specific T-SQL script to get the status whether the replication is finished, and assign a value to a SSIS variable. Then, the variable can be used in For Loop Container to determine whether go to the next
    step to process the package:
    http://www.bidn.com/blogs/mikedavis/ssis/158/make-an-ssis-package-delay-or-wait-for-data
    Regards, Leo

  • Error While executing a SSIS package which contains a script task through SQL Server Agent job

    Hi,
    I have a SQL Server 2012 SSIS package with a script task along with other tasks [data flow, execute sql tasks ]. When I manually executed the job through BIDS, its completed successfully. 
    Then I have automated the execution of the package through SQL Server Agent Job. But when I executed the package through SQL Agent job, it runs successfully for all the tasks except script task. When it comes to execute the Script Task, it is getting failed
    with the below error message.
    "Error: 2012-08-29 12:45:14.67
       Code: 0x00000001
       Source: Script Task 
       Description: Exception has been thrown by the target of an invocation.
    End Error
    DTExec: The package execution returned DTSER_FAILURE (1).
    Started:  12:45:10 PM
    Finished: 12:45:14 PM
    Elapsed:  4.353 seconds
    I have installed the SSIS on the 64-bit environment and SSIS service is running. Also I tried to run the job through 32 bit [job option] but I am getting the above error in all cases.
    Any help will be greatly appreaciated !
    Thanks,
    Navin
    - naveen.reddy

    Hi Arthur,
    My script task access the excel files in a network share, refresh them all and save them. When I execute the ETL manually or thru DTEXEC, it is executing successfully. I am facing the issue when I am executing thru SQL Agent Job only. Logging also showing
    the same error.
    "Error: 2012-08-23 12:45:14.67
       Code: 0x00000001
       Source: Script Task 
       Description: Exception has been thrown by the target of an invocation.
    End Error
    DTExec: The package execution returned DTSER_FAILURE (1).
    Started:  12:45:10 PM
    Finished: 12:45:14 PM
    Elapsed:  4.353 seconds
    - naveen.reddy

  • Excel Connection Manager suddenly can no longer connect to Excel from SSIS package

    <p>During the last 3 months, I used 32 bit SSIS 2008 (v 10.50.2500.0) on an WinXp machine (v 5.1.2600 SP3 Build 2600) to create a package that writes multiple query results (reports) to an excel spreadsheet. Each report is written to a different
    tabbed sheet in the same spreadsheet. </p><p>The package was&nbsp;working just fine until May 9 2013, when suddenly every single&nbsp;Data Flow task with an Excel Destination&nbsp;displayed error icons and raised the&nbsp;following
    error message when opening the Advanced Editor:</p><p>SSIS Error Code DTS_E_OLEDBERROR. An OLE DB error has occurred. Error code: 0x80004005 Description: "Unspecified error". <br />Error at &lt;DataFlow task name&gt;: SSIS
    error Code DTS_E_CANNOTACQUIRECONNECTIONFROMCONNECTIONMANAGER. The AcquireConnection method call to the connection manager &lt;Excel connection Manager&gt; failed with error code 0xC0202009. There may be error messages posted before this with more
    information on why the Acquire Connection method call failed. Exception from HRESULT: 0Xc020801c (Microsoft.SQlServer.DTSPipelineWrap)</p><p>Originally the error message reported that the Error Code was "0x8000FFFF", but I can't seem
    to reproduce that error code since. Now I am only getting the above error message.</p><p>From the time I created the original package (when it worked fine) until now, I have been using the same computer, writing to a (32 bit) 2010 Excel file in
    a local folder. The filename and location have not changed. The ConnectionString has not changed. The Connnection String I am using is Provider=Microsoft.ACE.OLEDB.12.0;Data Source=C:\Conversion\Conversion Results_dt01.xlsx;Extended Properties="EXCEL
    12.0 XML;HDR=YES;". </p><p>Now an attempt to create and test a NEW Connection Manager yields me this error. "Test connection failed because of an error in initializing provider. Unspecified error.",&nbsp; whereas before I had
    no problems creating a new connection manager.</p><p>I checked a second SSIS package, which I also built during the past 3 months, which loads multiple sheets from multiple Excel files to a SQL Server 2008 database holding table. It uses the same
    ACE OLE DB 12.0 connection to load the Excel speadsheets and this package continues to work fine on the same workstation.</p><p>In desparation, wondering if the Microsoft ACE OLE DB 12.0 driver had become corrupt (even though the second package
    was raising no errors), I did download and install the Microsoft ACE OLE DB 12.0 driver.&nbsp; That did not solve my problems with the first package.</p><p>I need to get this package running again ASAP as it is targeted for UAT testing in a
    matter of days. Any help with this issue would be greatly appreciated!</p>

    I submitted this issue above and now I am having the same problem again. I am using the same local computer, the same operating system (WinXp v 5.1.2600 SP3 Build 2600) as originally described. I am working with the same SSIS package that
    writes multiple query results (reports) to an excel spreadsheet. This package has been successfully executed multiple times, but now I am suddenly getting the same error as follows:
    Error at 1ConversionContacts [Connection manager "Excel Connection Manager"]: SSIS Error Code DTS_E_OLEDBERROR.  An OLE DB error has occurred. Error code: 0x80004005.
    An OLE DB record is available.  Source: "Microsoft Access Database Engine"  Hresult: 0x80004005  Description: "Unspecified error".
    Error at DFlow Write soft deleted duplicate claims to Excel sheet [Excel Destination - write duplicate claim nbrs that have been removed to Excel [61]]: SSIS Error Code DTS_E_CANNOTACQUIRECONNECTIONFROMCONNECTIONMANAGER.  The AcquireConnection method
    call to the connection manager "Excel Connection Manager" failed with error code 0xC0202009.  There may be error messages posted before this with more information on why the AcquireConnection method call failed.
    The package will not run because of these multiple errors (multiple because it occurs on every single excel destination in the package).
    I have Run64BitRuntime set to false on the project. I have triple checked that the excel file name (which I created and can access) exists with the correct name in the correct path, which is local on my machine. The file is the same file in the same
    location as when the package worked.  I am running the SSIS service under my own network login, as I was when the package worked. I have been consistently getting this same error whenever I change my network password without updating the password
    in the SSIS service, but I have triple verified that the service is using my latest password. If I try to open and edit the Excel Destination in any DataFlow task I get the same error. I can successfully create a new Excel Connection Manager pointing
    to the same excel file. However if I try to assign the newly created Excel Connection Manager in the data flow, I cannot activate the drop down to choose the Excel sheet without getting an 'Unspecified error' message. I then swapped out
    the file with its template version, and ensured the name was updated as correct. I still get the same errors.
    Oh, and the SSIS package is extracting data from a SQL Server 2008 database with an ADO.net Connection Manager using Windows authentication. I have no problems with the SQL Server connection, it has always worked and still does.  It is only
    the connection to excel output file which is locally stored on my machine.
    Has anyone got any suggestions for me?

  • Issue with creation of SSIS package - reading from XML

    Good Morning,
    having a slight difficulty here and more than this, am afraid solution is pretty straight forward. I am trying to read an XML-file and save the rows to a CSV file (late ron it should be sent to SQL-database).
    Am having this input XML
    <?xml version="1.0" encoding="utf-16"?>
    <DataSet>
    <xs:schema id="NewDataSet" xmlns="" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:msdata="urn:schemas-microsoft-com:xml-msdata">
    <xs:element name="NewDataSet" msdata:IsDataSet="true" msdata:UseCurrentLocale="true">
    <xs:complexType>
    <xs:choice minOccurs="0" maxOccurs="unbounded">
    <xs:element name="ROW">
    <xs:complexType>
    <xs:sequence>
    <xs:element name="COLUMN" type="xs:short" minOccurs="0" />
    </xs:sequence>
    </xs:complexType>
    </xs:element>
    </xs:choice>
    </xs:complexType>
    </xs:element>
    </xs:schema>
    <diffgr:diffgram xmlns:msdata="urn:schemas-microsoft-com:xml-msdata" xmlns:diffgr="urn:schemas-microsoft-com:xml-diffgram-v1">
    <NewDataSet>
    <ROW diffgr:id="ROW1" msdata:rowOrder="0">
    <COLUMN>1</COLUMN>
    </ROW>
    <ROW diffgr:id="ROW2" msdata:rowOrder="1">
    <COLUMN>2</COLUMN>
    </ROW>
    </NewDataSet>
    </diffgr:diffgram>
    </DataSet>
    but when running the SSIS-package, I get this
    no errors are shown, but also no rows are written/processed.
    Why is nothing read here?

    by the way, when I split the schema and the XML data, having those 2 files
    data.xml
    <?xml version="1.0" encoding="utf-8"?>
    <NewDataSet>
    <ROW >
    <COLUMN>1</COLUMN>
    </ROW>
    <ROW >
    <COLUMN>2</COLUMN>
    </ROW>
    </NewDataSet>
    schema.xsd
    <?xml version="1.0"?>
    <xs:schema attributeFormDefault="unqualified" elementFormDefault="qualified" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:msdata="urn:schemas-microsoft-com:xml-msdata">
    <xs:element name="NewDataSet">
    <xs:complexType>
    <xs:sequence>
    <xs:element minOccurs="0" maxOccurs="unbounded" name="ROW">
    <xs:complexType>
    <xs:sequence>
    <xs:element minOccurs="0" name="COLUMN" type="xs:string" />
    </xs:sequence>
    </xs:complexType>
    </xs:element>
    </xs:sequence>
    </xs:complexType>
    </xs:element>
    </xs:schema>
    then result is fine

  • How to find a certain usage of a specific value inside an SSIS package

    Hi,
    In T-SQL, there are many tools to search for instances of a specific value in all the objects such as tables and SPs.
    The only way I can think of in SSIS, is to search for a value in the XML of the DTSX, but it is not that user friendly.
    Are there any existing tools or other ways for a value search that can assist ?
    Thanks,
    Dani

    Since a package is just a plain text XML file thus  you are able to perform a text search in the file.
    http://sqlblog.com/blogs/jamie_thomson/archive/2012/06/12/using-find-replace-with-regular-expressions-inside-a-ssis-package.aspx
    Best Regards,Uri Dimant SQL Server MVP,
    http://sqlblog.com/blogs/uri_dimant/
    MS SQL optimization: MS SQL Development and Optimization
    MS SQL Consulting:
    Large scale of database and data cleansing
    Remote DBA Services:
    Improves MS SQL Database Performance
    SQL Server Integration Services:
    Business Intelligence

  • SSIS Package resources

    Hello, can someone please guide me to some resources on creating SSIS packages that read data from files, load data into SQL server tables and interface with BPC using the SAP provided extensions to SSIS.
    Thanks.

    Hey Zack -
    This is pertaining to Version 5, but the same procedures translate to later versions...
    http://www.sdn.sap.com/irj/sdn/go/portal/prtroot/docs/library/uuid/2027ebee-a928-2a10-4485-b88c926f5e24
    Good luck!
    Sean

  • SSIS package fails with the error as given below. But its warning messages are incorrect.

    My SSIS package loads data from access database to SQL Server database.
    It's a straight copy. It fails with the below warnings and error message as shown below:
    Error messages:
    [OLE DB Source [113]] Error: There was an error with OLE DB Source.Outputs[OLE DB Source Output].Columns[RetentionID] on OLE DB Source.Outputs[OLE DB Source Output]. The column status returned was: "Text was truncated or one or more characters had no
    match in the target code page.".
    [OLE DB Source [113]] Error: The "OLE DB Source.Outputs[OLE DB Source Output].Columns[RetentionID]" failed because truncation occurred, and the truncation row disposition on "OLE DB Source.Outputs[OLE DB Source Output].Columns[RetentionID]"
    specifies failure on truncation. A truncation error occurred on the specified object of the specified component.
    [SSIS.Pipeline] Error: SSIS Error Code DTS_E_PRIMEOUTPUTFAILED.  The PrimeOutput method on OLE DB Source returned error code 0xC020902A.  The component returned a failure code when the pipeline engine called PrimeOutput(). The meaning of the failure
    code is defined by the component, but the error is fatal and the pipeline stopped executing.  There may be error messages posted before this with more information about the failure.
    There are 3 warning messages even before I run the package. Warnings are all about the mismatch between the source and destination column sizes.
    I cross checked the source and destination column sizes. The destination column sizes are fairly high than the source column sizes.
    It worked well all the while and when we are going to pre-prod it is throwing errors. Any help is highly appreciated.
    Thank you.

    It worked well all the while and when we are going to pre-prod it is throwing errors. Any help is highly appreciated.
    Please check if the dev destination table has same column data types and size as compared with pre prod table
    Thanks, hsbal

  • How to Create a SQL Agent Job For A SSIS Package with Sql Server Authentication

    Hi ALl,
    I have a SSIS package which basically has a data flow task in which i pull the data from one server and copy it into another server and my source server is the one where i dont have windows authentication and i have to only use a sql server authentication
    . This package runs fine if i click the server connection properties type the password and save it.
    Now, my task is to set up a sql agent job which basically uses a proxy account and takes this package from the file system and runs it.But when i try to run this package, its failing with an error saying 
    "Login Failed For rpt5user" where rpt5user is the username for my sql server authentication of the source connection.
    Can someone please help me with any suggestions on how to do this?
    I have heard that we can achieve it by using xml config file which i have never used and i am trying to google around but for no luck.
    So, If someone can please throw any suggestions or ideas on this it would be great.
    Thanks

    You need to add password as a config item and set it from the file source or sql table
    see this as an example
    http://blogs.msdn.com/b/runeetv/archive/2009/12/22/ssis-package-using-sql-authentication-and-dontsavesensitive-as-protectionlevel.aspx
    Please Mark This As Answer if it solved your issue
    Please Mark This As Helpful if it helps to solve your issue
    Visakh
    My MSDN Page
    My Personal Blog
    My Facebook Page

  • Getting error while trying to upload the data in excel from SSIS package through sql agent job

    We are getting below errors.
    Error:
    The Microsoft Jet database engine cannot open the file '\\serversdev\Documents\QC Files\Prod.xls'. It is already opened exclusively by another
    user, or you need permission to view its data.
    Please suggest ASAP
    Regards,
    Ramu
    Ramu Gade

    Hi Dikshan Gade,
    According to your description, you want to upload data from excel to database, when you call ssis package through SQL Server Agent job, you got the error message.
    To troubleshoot the problem, please refer to the following steps:
    Validate that the account has permissions on the file and the folder.
    Verify that the file and the data source name (DSN) are not marked as Exclusive.
    Make sure SQL Server Agent Services service account has the permission to access the database.
    We can check SQL Server Agent’s activity logs, Windows Event logs and SSIS logs to get more clues. Also the tool Process Monitor is helpful to track the cause of registry or file access related issues. For more information about the issue, please refer to
    the following KB article:
    http://support.microsoft.com/kb/306269
    If you have any more questions, please feel free to ask.
    Thanks,
    Wendy Fu
    Wendy Fu
    TechNet Community Support

  • Multiple OLE DB connection managers to the same database instance in a SSIS package

    Hi,
    While migrating DTS package to SSIS package, I came across one which had multiple OLE DB connection managers all connecting to the same Oracle schema in the same instance.
    There are that many execute SQL task corresponding to each of these connection managers.
    Each of  these SQL task call different Oracle stored procedures and executes them in the same schema in the same instance.
    My question is, is there any particular reason or advantage in having multiple connection managers?
    Can't we have one single connection manager and use the same in all of these SQL tasks?
    Does multiple connection managers provide any added advantage like parallelism or something?
    In this particular case, there are no actual workflows, rather each of these SQL tasks are stand alones. But there are other packages which has SQL tasks in workflow but still each of them calling separate connection managers all calling the same schema
    in the same instance.
    Thanks for your time.
    Cordially, Biju George

    Well like I said, each of  these SQL task calls "different" Oracle stored procedures even though executes them in the same schema in the same instance. I tried a sample testing by creating two separate packages, one with only one single
    OLE DB connection referred in all of the SQL tasks and in the other one with mutiple OLE DB connections to the same schema in the same instance each of which is referred in each of the SQL task in one to one mapping.
    Now the first one created one session in Oracle but the later created as many sessions as there where OLE DB connection managers. The first one took longer to complete than the second one since second one created independent sessions and executed those procedures
    sort of parallely.
    I guess that can be called an advantage of creating multiple connection managers to the same schema in the same instance.
    Cordially, Biju George

  • How to create a dynamic SSIS package for multiple flat file destinations

    Hi,
    I have to create a ssis package which has single data flow task and inside that I have 23 source (sql- select * from - statements)- destination (flat files, 23 distinct) connection.
    Now for each product I have to create separate SSIS package (i.e. if prod=abc then these read select * from abc_tables and 23 abc_ txt files)
    I want to do it dynamically, means only single package and inside that variables will take select * values for each source-dest connection (so i believe 23 variables) and same for destination flat files.
    Let me know. :)
    ANK HIT - if reply helps, please mark it as ANSWER or helpful post

    Sorry It seems you're contradicting yourself. you say I know my source and dest structure and the you're
    asking all I want is to have a dynamic structure
    what does that mean?
    and reading your next sentence
    I want to run a package for 5 products, instead of creating 5 ssis packages with 23 source- dest connection, I would
    like to have one with only 23 source- dest connections 
    What I feel is what you're looking for is to  have a looping structure to loop through each of the 5
    products.
    In that case what you could do is this
    1. Create a object variable in SSIS 
    2. Use a Execute SQL Task to populate the variable with all available products (I think you'll have a master table for that). Set ResultSet property to Full ResultSet and then in ResultSet tab map Object variable to 0 th index
    3. use a ForEachLoop container with ADO .NET recordset enumerator and map to object variable. Create a variable of datatype same as that of Product identifier field to get individual values out
    4. Inside loop create your data flow task with 23 source destination connection. In the query part use a parameter for product field and map it to the variable containing product value to get only data for the product.
    Please Mark This As Answer if it helps to solve the issue Visakh ---------------------------- http://visakhm.blogspot.com/ https://www.facebook.com/VmBlogs

  • How do I get SQL Agent to perform an FTP connection through a SSIS package?

    I have created a number of SSIS packages that I am running through SQL Agent in various jobs.  I have set up a Credential / Proxy for the job steps and for the most part everything runs no problem.  The issue I have is with the step where I make
    a connection to a FTP server and download files.  In BIDS the step works fine but in SQL Agent the step times out.  I presume that it has to do with the account that it is being run under but that is what I thought the Credential / Proxy was supposed
    to handle (as it does for all the other steps).  I am running on SQL Server 2008 R2.  I have created a test job that has only one step which is to connect to the FTP server and download some text files.  I have tried utilizing package configuration
    files and have tried various ProtectionLevels to no avail.
    Anyone have any thoughts or direction for me to take?  Will continue to research and will post if I resolve this.
    Thanks, Andrew

    On the dev environment the ProtectionLevel is set to EncryptSensitiveWithUserKey and that is what I tried first on the prod server.  Subsequently I also tried utilizing a package config file (with ProtectionLevel then of DontSaveSensitive) and have
    also tried working with the ServerStorage.
    For deployment I copied the relevant dtsx files to a solution on the prod server.  I verified that the packages all had tested connections in the Connection Manager.  I have run all the packages in the BIDS on the prod server and they all worked. 
    I have created a test package from scratch on the prod server to test the ftp connection.  This test package runs from BIDS fine.  I have set the SQL Agent file to access through the file system (no luck) and have saved the package to the SSIS Packages
    and had the job step access from there (no luck either).

Maybe you are looking for