Dynamic source and destination tables

Question

Hi all
I've got to import 142 tables from csv into SQL 2008 on a regular basis.
I was looking at building a 142-part SSIS package to do this, then thought there must be a dynamic way of doing it.
Is there any way of dynamically changing the source and and destination tables?
The csv filenames will remain identical, the SQL tables will be the same names but with "_Staging" at the end of them (e.g. SRSection.csv will always go into SRSection_Staging).
I can then write MERGE statements to update the main tables from the staging data.
Any help on this would be greatly appreciated.
I get the the feeling I would need a FOREACH LOOP container but I'd really aprreciate a step-by-step guide if you can.

Answer 1

Please check this :- http://sql-bi-dev.blogspot.com/2010/07/dynamic-database-connection-using-ssis.html  
STEP1:
To begin, Create two tables as shown below in on of the environment:
-- Table to store list of Sources
CREATE TABLE SourceList (
   ID [smallint],
   ServerName [varchar](128),
   DatabaseName [varchar](128),
   TableName [varchar](128),
   ConnString [nvarchar](255)
GO
-- Local Table to store Results
CREATE TABLE Results(
   TableName  [varchar](128),
   ConnString [nvarchar](255),
   RecordCount[int],
   ActionTime [datetime]
GO
STEP 2:
Insert all connection strings in SourceList table using below script:
INSERT INTO SourceList
SELECT 1 ID,
'(local)' ServerName,
--Define required Server
'TestHN' DatabaseName,--Define DB Name
'TestTable' TableName,
'Data Source=(local);Initial Catalog=TestHN;Provider=SQLNCLI10.1;Integrated Security=SSPI;Auto Translate=False;' ConnString
Insert as many connections as you want.
STEP 3:
Add new package in your project and rename it with ForEachLoopMultipleServers.dtsx. Add following variable:
Variable
Type
Value
Purpose
ConnString
String
Data Source=(local);
Initial Catalog=TestHN;
Provider=SQLNCLI10.1;
Integrated Security=SSPI;
Auto Translate=False;
To store default connection string
Query
String
SELECT '' TableName,
N'' ConnString,
0 RecordCount,
GETDATE() ActionTime
Default SQL Query string.
This can be modified at runtime based on other variables
SourceList
Object
System.Object
To store the list of connection strings
SourceTable
String
Any Table Name.
It can be blank.
To store the table name of current connection string.
This table will be queried at run time
STEP 4:
Create two connection managers as shown below:
Local.TestHN: For local database which has table SourceList. Also this will be used to store the result in Results table.
DynamicConnection: This connection will be used for setting up dynamic connection with multiple servers.
Now click on DynamicConnection in connection manager and click on ellipse to set up dynamic connection string. Map connection String with variable
User::ConnString.
STEP 5:
Drag and drop Execute SQL Task and rename with "Execute SQL Task - Get List of Connection Strings". Now click on properties and set following values as shown in snapshot:
Result Set: Full Result Set
Connection: Local.TestHN
ConnectionType: Direct Input
SQL Statement: SELECT ConnString,TableName FROM SourceList
Now click on Result Set to store the result of SQL Task in variable User::SourceList.
STEP 6:
Drag and drop ForEach Loop container from toolbox and rename with "Foreach Loop Container - DB Tables". Double click on ForEach Loop container to open Foreach Loop Editor. Click on Collection  and select
Foreach ADO Enumerator as Enumerator. In Enumerator configuration, select User::SourceList as ADO object source variable as shown below:
STEP 7: Drag and drop Script Task inside ForEach Loop container and double click on it to open Script Task Editor. Select
User::ConnString,User::SourceTable as
ReadOnlyVariables and User::Query as
ReadWriteVariables. Now click on Edit Script button and write following code in Main function:
public void Main()
try
String Table = Dts.Variables["User::SourceTable"].Value.ToString();
String ConnString = Dts.Variables["User::ConnString"].Value.ToString();
MessageBox.Show("SourceTable = " + Table +
"\nCurrentConnString = " + ConnString);
//SELECT '' TableName,N'' ConnString,0 RecordCount,GETDATE() ActionTime
string SQL = "SELECT '" + Table +
"' AS TableName, N'" + ConnString +
"' AS ConnString, COUNT (*) AS RecordCount, GETDATE() AS ActionTime FROM " + Dts.Variables["User::SourceTable"].Value.ToString() +
" (NOLOCK)";
      Dts.Variables["User::Query"].Value = SQL;
      Dts.TaskResult = (int)ScriptResults.Success;
catch (Exception e)
      Dts.Log(e.Message, 0,
null);
STEP 8:
Drag and drop Data Flow Task and double click on it to open Data Flow tab. Add OLE DB Source and Destination. Double click on OLE DB Source to configure the properties. Select
DynamicConnection as OLE DB connection manager and
SQL command from variable as Data access mode. Select variable name as User::Query. Now click on
columns to genertae meta data.
Double click on OLE DB Destination to configure the properties. Select Local.TestHN as
OLE DB connection manager and Table or view - fast load as
Data access mode. Select [dbo].[Results] as Name of the table or the view. now click on
Mappings to map the columns from source. Click OK and save changes.
Finally DFT will look like below snapshot:
STEP 9: We are done with package development and its time to test the package.
Right click on the package in Solution Explorer and select execute. The message box will display you the current connection string.
 Once you click OK, it will execute Data Flow Task and load the record count in Results table. This will be iterative process untill all the connection are done. Finally package will execute successfully.
You can check the data in results table:
Here is the result:
SELECT *
FROM SourceList
SELECT *
FROM Results
Regards, Pradyothana DP. Please Mark This As Answer if it solved your issue. Please Mark This As Helpful if it helps to solve your issue. ========================================================== http://www.dbainhouse.blogspot.in/