Running same SSIS Packages for Different Destination Databases
I have created SSIS packages which are supposed to separate data from a single source database into different client databases at a time.
For e.g. my Source db contains data for 10 clients. SSIS pkgs uses ClientName as input parameter to call StoredProcs to get data for given client and push that data into that client's database. So I have 1 Source database and 10 Client databases which stores
their own data.
In Package Configuration I kept SourceConn and DestClientConn as two connection string. So I will have 10 different .dtsconfig file for all these client in every env. like DEV, TEST, UAT, PROD. OR keep 1 .dtsconfig and change DestClientConn before running
Packages
Is there any other effective way to handle this dynamic configuration in SSIS?
you just need single config file and you'll have 1 sourceconn property and 10 destination property mapped inside it.
And I dont think you need 10 procedures for this. I think what you need is a single data flow task with OLEDB source to connect to your source. then add a conditional split and define 10 outputs based on client field value. Then join each of the output to
a OLEDB destination using the 10 destination connection to get different client data on different database.
Please Mark This As Answer if it helps to solve the issue Visakh ---------------------------- http://visakhm.blogspot.com/ https://www.facebook.com/VmBlogs
Similar Messages
-
Can you run multiple APEX listeners for different instances on the same server
Please confirm that we can run multiple APEX listeners for different instances on the same server? Is it an xml setup configuration or do we need to do more??
Thank You,
Tony Miller
SmartDog Services
Austin, TXYes.. Just exactly what I was looking for...
Thanks Jari!!
Thank You,
Tony Miller
SmartDog Services
Austin, TX -
Getting an error running an SSIS package from SQL server agent
This is the first time I'm running an SSIS package in SQL server agent.
I'm running a SQL server agent job and getting this error message:
Am I using the correct options? (options list below)
Message
Executed as user: SITEX\stxdatasqlexec. Microsoft (R) SQL Server Execute Package Utility Version 10.50.1600.1 for 32-bit Copyright (C) Microsoft Corporation 2010. All rights reserved. Started: 1:42:00 PM Error: 2014-04-17
13:42:01.01 Code: 0xC0011007 Source: {B72BF421-785A-44EC-9570-D2A40B046CA7} Description: Unable to load the package as XML because of package does not have a valid XML format. A
specific XML parser error will be posted. End Error Error: 2014-04-17 13:42:01.02 Code: 0xC0011002 Source: {B72BF421-785A-44EC-9570-D2A40B046CA7} Description: Failed
to open package file "C:\Users\halexandrovich\Documents\Nbr_Rolls_base.dtsx" due to error 0x80070003 "The system cannot find the path specified.". This happens when loading a package and the file cannot be opened or loaded correctly
into the XML document. This can be the result of either providing an incorrect file name was specified when calling LoadPackage or the XML file was specified and has an incorrect format. End Error Could not load package "C:\Users\halexandrovich\Documents\Nbr_Rolls_base.dtsx"
because of error 0xC0011002. Description: Failed to open package file "C:\Users\halexandrovich\Documents\Nbr_Rolls_base.dtsx" due to error 0x80070003 "The system cannot find the path specified.". This happens when loading a
package and the file cannot be opened or loaded correctly into the XML document. This can be the result of either providing an incorrect file name was specified when calling LoadPackage or the XML file was specified and has an incorrect format. Source:
{B72BF421-785A-44EC-9570-D2A40B046CA7} Started: 1:42:00 PM Finished: 1:42:01 PM Elapsed: 0.047 seconds. The package could not be found. The step failed.
Here are the options I'm using:
package source - file system
data source destination = my SQL server db
execution option - use 32 bit runtime
command line entry:
- /FILE "C:\Users\halexandrovich\Documents\Nbr_Rolls_base.dtsx" /CONNECTION DestinationConnectionOLEDB;"\"Data Source=esxdspr-sql01.sitex.com;Initial Catalog=SuesTbls;Provider=SQLNCLI10;Integrated Security=SSPI;Auto Translate=false;\""
/CONNECTION SourceConnectionFlatFile;"\\sitex.com\dfs\Production\SFTPData\LPSEFTProcess\FromAADFS\Rpt_Nbr_Rollsperbatch_20140331_20_0.txt"
/X86 /CHECKPOINTING OFF /REPORTING E
I checked the filename so it must be the XML format it's complaining about.Thank you, pituach.
I tried running the package using just my c drive without partition, C:\Nbr_Rolls.dtsx, but still getting the same
error. I do not see any option on this site to post my package. I know it's not the path that's the issue. Our support desk already fixed that because I was getting a different message before.
Message
Executed as user: SITEX\stxdatasqlexec. Microsoft (R) SQL Server Execute Package Utility Version 10.50.1600.1 for 32-bit Copyright (C) Microsoft Corporation 2010. All rights reserved. Started: 11:23:00 AM Error: 2014-04-18
11:23:00.44 Code: 0xC0011007 Source: {2BEE62EB-DD08-4867-9F5C-660167964650} Description: Unable to load the package as XML because of package does not have a valid XML format. A
specific XML parser error will be posted. End Error Error: 2014-04-18 11:23:00.45 Code: 0xC0011002 Source: {2BEE62EB-DD08-4867-9F5C-660167964650} Description: Failed
to open package file "C:\Nbr_Rolls.dtsx" due to error 0x800C0006 "The system cannot locate the object specified.". This happens when loading a package and the file cannot be opened or loaded correctly into the XML document. This can be the result of
either providing an incorrect file name was specified when calling LoadPackage or the XML file was specified and has an incorrect format. End Error Could not load package "C:\Nbr_Rolls.dtsx" because of error 0xC0011002. Description: Failed
to open package file "C:\Nbr_Rolls.dtsx" due to error 0x800C0006 "The system cannot locate the object specified.". This happens when loading a package and the file cannot be opened or loaded correctly into the XML document. This can be the result of
either providing an incorrect file name was specified when calling LoadPackage or the XML file was specified and has an incorrect format. Source: {2BEE62EB-DD08-4867-9F5C-660167964650} Started: 11:23:00 AM Finished: 11:23:00 AM
Elapsed: 0.125 seconds. The package could not be found. The step failed. -
i have searched all over the google and here on MSDN(I have read all the threads related to my problem ) , i am new to SISS and doing my clg project ,
I have this package which loads data from dim tables to a fact table , my Package runs without any problem in BIDS (2012)
I have installed SQl Server integration services on SQL and service is also running in services.msc,
the problem comes when i execute my package using my C# application i am using MSDN Method to execute a SSIS package programmatically given here.
"http://msdn.microsoft.com/en-us/library/ms136090.aspx"
For loading my FactTable i have used a Derived columns and some lookups ,
Now if i don't use derived columns and looks ups and just use source and destination component then my package runs fine from my C# application but if i use derived columns or looks ups etc it gives the following error
"Error in Microsoft.SqlServer.Dts.Runtime.TaskHost/SSIS.Pipeline: To Run a SSIS package outside of SQL Server Data Tools you must Install Derived Column of Integration services of higher"
I am searching and trying to resolve this issue from previous 2 days but without any luck please help or guide me what is solution to this problem ,,i will be very thankful
I have also found that integration services that are running on my system have version 10.0
but the BIDS have version Version 11.0.3402.0
IS this causing problem ? if yes then what i have to do ?Hi BlaxButt,
The package is developed in SSDT installed by SQL Server 2012, so it is a SSIS 2012 package. However, the Integration Services you have installed is SQL Server 2008 R2 version. To run the package outside SSDT, you need SSIS 2012 installed. The reason why
the package runs fine with only Source and Destination components is that such a simple package can be executed by the DTExec utility installed by SQL Server 2012 Data base Engine or Client Tools (SQL Server Import and Export Wizard). To run a package that
uses other tasks/components outside SSDT/BIDS, the SSIS runtime is also required except the DTExec utility. To obtain the SSIS 2012 runtime, we have to install SSIS 2012 on the server where the package runs.
Reference:
http://stackoverflow.com/questions/19989099/getting-error-running-ssis-package-on-non-ssis-server
Regards,
Mike Yin
TechNet Community Support -
Hello Team,
I am trying to execute a SSIS package from web page. When i try to do that i am getting following error.
"To run a SSIS package outside of SQL Server data tools you must install <task name used in package> of Integration service or highter."
In my machine Integration Services are installed and its service is also in running state.
Please help me on this.
Thanks,
Ramesh
Thanks, Ramesh ArigeThe SSIS package developed using SSIS 2008 Server R2 and Integrations Services 10.0 is exists in my machine. Is this wrong configuration, please help me on this.
I am using the below code copied from CodeProject
Thank you so much for responding.
Ramesh
Thanks, Ramesh Arige
Which way are you using from the provided blog? Using 1) C# Code or 2) C# and Stored Procedure?
Cheers,
Vaibhav Chaudhari
MCP, MCTS, MCSA (SQL Server 2012) -
Same preparedstatement object for different queries.. ?
Actually I m getting same preparedstatement object for many queries in a for loop..In this scenario its not working properly..Suppose like dis..
connection con.setAutoCommit(false);
PreparedStatement prepStmt = con.prepareStatement(
"UPDATE DEPT SET MGRNO=? WHERE DEPTNO=?");
prepStmt.setString(1,mgrnum1);
prepStmt.setString(2,deptnum1);
prepStmt.addBatch();
PreparedStatement prepStmt = con.prepareStatement(
"UPDATE EMP SET EMPNO=? WHERE DEPTNO=?");
prepStmt.setString(1,empno1);
prepStmt.setString(2,deptnum2);
prepStmt.addBatch();
Whether we can take same preparedstatement object for different query..??Its not giving any error but its not updating all the records but only the last one...
Any solution..kriti123 wrote:
Actually I m getting same preparedstatement object for many queries in a for loop..In this scenario its not working properly..Suppose like dis..
Ignore all of the above.
You are using a batch.
There is a specific idiom for that. You create the statement once and then keep calling the appropriate methods on it to load and then execute the batch, and check errors.
When done you close it. If you need commit you are going to need to deal in blocks.
If you have a large volume of data then database import tools are very likely to be substantially faster. -
We have SSIS installed on a machine that is not part of a cluster but it is accessible by the cluster.
Our job we have running on the cluster is failing with the following error:
Error: 2014-01-31 09:14:37.52 Code: 0xC000F427 Source: SCR - DP1 Connections
Description: To run a SSIS package outside of SQL Server Data Tools you must install SCR - DP1 Connections of Integration Services or higher. End Error
Any advice or information on how to resolve this would be great.
Many Thanks.Hi NessaBella,
Integration Services service is not a cluster-aware service, and does not support failover from one cluster node to another. Therefore, in a clustered environment, Integration Services should be installed and started as a stand-alone service on each node
in the cluster.
Based on the error message, it seems that SSIS is not installed on the cluster node on which the job was running. Although SSIS is installed on a machine that is not part of the cluster and can be accessed by each cluster node, the SSIS service installed
on a remote server cannot be used a cluster node. So, please install the shared feature SQL Server Integration Services on each cluster node. Besides, if certain packages need to run in 32-bit mode in certain jobs and the SQL Server installed is 64-bit version,
you also need to install BIDS/SSDT on the cluster node to get the 32-bit runtime of SSIS.
References:
Integration Services (SSIS) in a Cluster
Loading and Running a Remote Package Programmatically
Regards,
Mike Yin
TechNet Community Support -
How to Run 2012 SSIS packages on 2008R2 Version
HI Friends,
I have scheduled a job to run 2012 SSIs packages on 2008R2. I am getting this error can you please Give any ideas how can I run them without converting!
ERROR Is :
Package migration from version 6 to version 3 failed with error 0xC001700A "The version number in the package is not valid. The version number cannot be greater than current version number."Hello,
There is backward compatibility in SSIS but no forward compatibility, meaning, when 2008 R2 is developed they were aware of all versions less than 2008 R2 and included execution scenarios , this is backward compatibility
while 2008 R2 is being developed , 2012 is not yet developed. so we cannot execute a 2012 package on 2008 R2 Server where vice versa is possible
Correct.
The "backwards compatibility" is implemented by SSIS upgrading older versions of packages to the current version of SSIS when executing. If the upgrade fails, you're out of luck.
MCSE SQL Server 2012 - Please mark posts as answered where appropriate. -
Can i use the same email address for different users?
we are a small department with one email address, can i use the same email address for different users?
No. Each user needs to have his unique ID.
Mylenium -
Is it possible to have same file size for different dpi?
I changed one.TIFF file (300dpi, 1024X1332) to .jpg files of four different dpi. But when I checked the four result jpg files, I found out that they are all in same file size and quality.( I also have checked the property of the files in the Windows.)
I think more DPI means more data and more file size. Am I wrong?
I use Photoshop CS 5.1(64bit, WINDOWS) - which is part of my Adobe Master Collection CS5.5.
TIFF(300dpi, 1024X1332) ->
1. JPG(72dpi, 1024X1332) : 306KB
2. JPG(108dpi, 1024X1332) : 306KB
3. JPG(144dpi, 1024X1332) : 306KB
4. JPG(600dpi, 1024X1332) : 306KB
I tested a few times more with different files. and same result comes out.(same file size for different dpi)
Thanks in advance.Yes absolutely. Great observation. PPI does not control the number of pixels, just how big they are.
Now, if you change the PPI in the Image Size dialog with Resample checked.. then, that is a different story. In that case you will be changing the pixel dimension of an image (i.e.,changing the total number of pixels making up the image) while keeping its print size.
In your test files, you will notice all the print sizes are different, because all you were telling Photoshop to do was change the size of the pixels (if or when the image is ever printed), which is really just a bit of metadata of the file. -
Can we use same number range for different fields
can we use same number range for different fields. how can we see the number range defined for a particular field
Hai,
Yes you can have same number range for different fields by using the same domain to the data elements that are user to your two fields.
To see the range:
Go to SE11,Type table Name open it-->double click on the DataElement adjasent to the field name ,You will get Domian ,Double click on Domain You will get Domain.There See Value range option .Value range cam also be defined by Value Table which will also at the Doamin level.
<b>Reward points if it helps you.</b>
Regds,
Rama.Pammi -
How to create a dynamic SSIS package for multiple flat file destinations
Hi,
I have to create a ssis package which has single data flow task and inside that I have 23 source (sql- select * from - statements)- destination (flat files, 23 distinct) connection.
Now for each product I have to create separate SSIS package (i.e. if prod=abc then these read select * from abc_tables and 23 abc_ txt files)
I want to do it dynamically, means only single package and inside that variables will take select * values for each source-dest connection (so i believe 23 variables) and same for destination flat files.
Let me know. :)
ANK HIT - if reply helps, please mark it as ANSWER or helpful postSorry It seems you're contradicting yourself. you say I know my source and dest structure and the you're
asking all I want is to have a dynamic structure
what does that mean?
and reading your next sentence
I want to run a package for 5 products, instead of creating 5 ssis packages with 23 source- dest connection, I would
like to have one with only 23 source- dest connections
What I feel is what you're looking for is to have a looping structure to loop through each of the 5
products.
In that case what you could do is this
1. Create a object variable in SSIS
2. Use a Execute SQL Task to populate the variable with all available products (I think you'll have a master table for that). Set ResultSet property to Full ResultSet and then in ResultSet tab map Object variable to 0 th index
3. use a ForEachLoop container with ADO .NET recordset enumerator and map to object variable. Create a variable of datatype same as that of Product identifier field to get individual values out
4. Inside loop create your data flow task with 23 source destination connection. In the query part use a parameter for product field and map it to the variable containing product value to get only data for the product.
Please Mark This As Answer if it helps to solve the issue Visakh ---------------------------- http://visakhm.blogspot.com/ https://www.facebook.com/VmBlogs -
Error while running the SSIS package from SQL DB to excel file - export option
hi all,
I have 4.6 million records in my sql db and i want to copy this into a excel file. for this i went to the db and rt click and export to command and started the ssis package running. but after
few minutes, it throws me an error that "error in transferring data into excel file."
Can anyone help me why this happened and the resolution ?
help is appreciated!
Copying to `excel1_Wbook` (Error)
Messages
Error 0xc0202009: Data Flow Task 1: SSIS Error Code DTS_E_OLEDBERROR. An OLE DB error has occurred. Error code: 0x8007000E.
(SQL Server Import and Export Wizard)
Error 0xc0047022: Data Flow Task 1: SSIS Error Code DTS_E_PROCESSINPUTFAILED. The ProcessInput method on component "Destination - excel1_Wbook" (217) failed with error code 0xC0202009 while processing input "Destination Input" (228).
The identified component returned an error from the ProcessInput method. The error is specific to the component, but the error is fatal and will cause the Data Flow task to stop running. There may be error messages posted before this with more information
about the failure.
(SQL Server Import and Export Wizard)
Error 0xc02020c4: Data Flow Task 1: The attempt to add a row to the Data Flow task buffer failed with error code 0xC0047020.
(SQL Server Import and Export Wizard)
Error 0xc0047038: Data Flow Task 1: SSIS Error Code DTS_E_PRIMEOUTPUTFAILED. The PrimeOutput method on Source - excel1_Wbook returned error code 0xC02020C4. The component returned a failure code when the pipeline engine called PrimeOutput(). The
meaning of the failure code is defined by the component, but the error is fatal and the pipeline stopped executing. There may be error messages posted before this with more information about the failure.
(SQL Server Import and Export Wizard)You need to split your data and create (at least 5) worksheet targets.
For example if you have a ROW_NUMBER column you can use for instance a
Conditional Split for something like:
ROW_NUMBER % 5 == 0 for Case 1 (excel 1)
ROW_NUMBER % 5 == 1 for Case 2 (excel 2)
ROW_NUMBER % 5 == 2 for Case 3 (excel 3)
ROW_NUMBER % 5 == 3 for Case 4 (excel 4)
ROW_NUMBER % 5 == 4 for Case 5 (excel 5) -
How to pass parameters to sql agent job run configured ssis package
Hi all,
I have a big problem at my small project.
I build my SSIS package that get its variables values from a configuration file..
and when i build a SQL agent job to run this package in a schedule i set the values of variables in it .. but in run-time the package still get its parameters from the configuration file !??
any help please ?>SQL agent job to run this package in a schedule i set the values of variables in it
One way, setup a configuration table for the package. Let the package read the values for the variables from there.
Kalman Toth Database & OLAP Architect
SQL Server 2014 Design & Programming
New Book / Kindle: Exam 70-461 Bootcamp: Querying Microsoft SQL Server 2012 -
Not able to run the SSIS package in SQL Server Agent Job in SQL 2012
Hi,
I scheduled a job (SQL SERVER AGENT) which will call my SSIS package. This package will execute 5 packages in loop.
When I execute this job I am getting error as below
============================================================
Started : 12:30:19 PM
Error : 2014-01-23 12:30:20.37
Code : 0xC00220EC
Source : Execute Package Task
Description : Error 0x80070005. Failed to create an instance of empty child package.
The Distributed Component Object Model (DCOM) configuration or the installation of SQL Server Integration Services,
may be corrupted on your machine. End Error
Error : 2014-01-23 12:30:20.37
Code : 0xC00220DE
Source : Execute Package Task
Description : Error 0x80070005 while loading package file "D:\SSIS\RetailMobileApp\Staging_Pkg\stg_Category_Master.dtsx". Access is denied.End Error
DTExec : The package execution returned DTSER_FAILURE (1).
Started : 12:30:19 PM
Finished : 12:30:20 PM
Elapsed : 1.123 seconds.
The package execution failed.
The step failed.
============================================================
In the STEPS I have selected as follows:
Type : SQL Server Integration Services Package
Run as: SQL Server Agent Service Account
Package Source: File System
What config I need to check here ? This job is not able to access the package stored in the folder.
SridharThanks for your reply. I had created credentials and used this while creating the Proxy account. With the Proxy account I am able to execute the Job Successfully. Concern here is I used my NT account to create this Proxy. Suppose If I left from the
current project if some one is coming to this project later they need to change the credential with their NT account and run the JOB? Is this right way or we an create any other account in common? If so how?
Sridhar
Maybe you are looking for
-
MacBook Keys not working even after KeyBoard Firmware update
My friend's sister bought me a MacBook from the US last October. Fortunately, Apple announced the release of Leopard also on the same month and I got a free upgrade to leopard. I installed leopard about 3 months back. My keyboard was working fine whe
-
To exit the command window after weblogic app server startup
Hi, The startWeblogic.cmd starts the weblogic V 10.0 server without exiting the prompt. Since our requirement wanted the startWeblogic script to exit the window after server startup, we set the parameter SET doExitFlag=true in startweblogic.cmd file
-
Solaris 10 v440 install failure.
Hi, Have a v440 which passes full diag routine. with brand new solaris 10 dvd received from sun, with a freshly formatted 9gb hdd. When trying to bbot from the dvd, "boot cdrom", i get the following cycling through. Hardware watchdog enabled svc.conf
-
Hi I am experiencing an error when sending an IDOC message to SAP from BizTalk. The IDOC reaches SAP with no issues however we do not receive the standard GUID response message back and this error is reported by BizTalk - Error Description: An inter
-
Why didn't the contents of my iBook app on my iPhone transfer to my new iPad 3 when I synced the iPad in iTunes on my computer.? HELP PLEASE