Component has not input columns and/or input columns have invalid id when creating package programmatically

Question

I am developing an ETL tool which has to create SSIS packages dynamically (and later execute them without being edited in Data Tools). One of my proto type scenarios is a simple data flow from a flat file to a flat file - without anything in between.
However there is a problem with the columns of the flat file destination - it says it doesn't have any.
Saving the package to XML and opening the destination component in the advanced editor (tab "Input and Output Properties") really shows that are no input columns defined. One click on the tab "Column Mappings" however solves the
problem. The columns and the mapping are generated, the package can run.
I already tried to place "ReinitializeMetaData()" calls nearly anywhere in the code, for both components, unfortunately without success:
destinationComponentWrapper.AcquireConnections(null);
destinationComponentWrapper.ReinitializeMetaData();
destinationComponentWrapper.ReleaseConnections();
The actual error during validation is (When the line for column mapping in the code below is commented out):
"0xC020201B - The number of input columns for Flat File Destination.Inputs[Flat File Destination Inputs] cannot be zero"
I am also trying to insert a column mapping - this leads to a com error: 0xC020201B
In this case I am wondering why Data Tools tell me that there are no input columns yet I can iterate over them - they just don't have a valid ID (0x8000004B / -2147483573) which is then causing the 0xC020201B.
See also "PROBLEM HERE" marker in the code below.
How can I make the destination component recognize the columns programmatically, as when I click on the "Column Mappings" tab of the advanced editor of the component in Data Tools?
Here is the complete proto type code:
using Microsoft.SqlServer.Dts.Pipeline.Wrapper;
using Microsoft.SqlServer.Dts.Runtime;
using Microsoft.SqlServer.Dts.Runtime.Wrapper;
using System;
using System.IO;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using Dts = Microsoft.SqlServer.Dts;
namespace msdatapipe
class Program
static void Main(string[] args)
Dts.Runtime.Application application = new Dts.Runtime.Application();
Dts.Runtime.Package package = new Dts.Runtime.Package();
package.Name = "Data Commander DTS Package";
// ADD PIPELINE TASK
Dts.Runtime.Executable pipelineExecutable = package.Executables.Add("STOCK:PipelineTask");
Dts.Runtime.TaskHost pipelineTaskHost = (Dts.Runtime.TaskHost)pipelineExecutable;
pipelineTaskHost.Name = "Data Commander Pipeline Task";
Dts.Pipeline.Wrapper.MainPipe pipelineTask = (Dts.Pipeline.Wrapper.MainPipe) pipelineTaskHost.InnerObject;
// ADD SOURCE COMPONENT TO THE PIPELINE TASK
String sourceFilePath = "C:\\cmi\\tmp\\data.csv";
Dts.Runtime.ConnectionManager sourceConnectionManager = package.Connections.Add("FLATFILE");
sourceConnectionManager.ConnectionString = sourceFilePath;
sourceConnectionManager.Name = "Source Connection";
sourceConnectionManager.Description = "Source Connection Manager";
sourceConnectionManager.Properties["ColumnNamesInFirstDataRow"].SetValue(sourceConnectionManager, "True");
sourceConnectionManager.Properties["Format"].SetValue(sourceConnectionManager, "Delimited");
sourceConnectionManager.Properties["TextQualifier"].SetValue(sourceConnectionManager, "\"");
sourceConnectionManager.Properties["CodePage"].SetValue(sourceConnectionManager, "65001");
sourceConnectionManager.DelayValidation = false;
// START --- MANUALLY READ COLUMNS FROM TEXT FILE AND ADD TO SOURCE CONNECTION MANAGER COLUMNS
IDTSConnectionManagerFlatFile100 sourceConnectionInterface = sourceConnectionManager.InnerObject as IDTSConnectionManagerFlatFile100;
using (StreamReader reader = new StreamReader(sourceFilePath))
String firstSourceFileLine = reader.ReadLine();
String[] columnNames = firstSourceFileLine.Split(',');
for (int i = 0; i < columnNames.Length; ++i)
columnNames[i] = columnNames[i].Substring(1, columnNames[i].Length - 2);
IDTSConnectionManagerFlatFileColumns100 sourceColumns = sourceConnectionManager.Properties["Columns"].GetValue(sourceConnectionInterface) as IDTSConnectionManagerFlatFileColumns100;
IDTSConnectionManagerFlatFileColumn100 column = null;
foreach (String columnName in columnNames)
column = sourceColumns.Add();
column.DataType = DataType.DT_TEXT;
column.ColumnDelimiter = ",";
column.TextQualified = true;
(column as IDTSName100).Name = columnName;
Console.WriteLine(columnName);
column.ColumnDelimiter = "\r\n";
// END --- MANUALLY READ COLUMNS FROM TEXT FILE AND ADD TO SOURCE CONNECTION MANAGER COLUMNS
sourceConnectionInterface.RowDelimiter = "\r\n";
Dts.Pipeline.Wrapper.IDTSComponentMetaData100 sourceComponent = pipelineTask.ComponentMetaDataCollection.New();
sourceComponent.Name = "Flat File Source";
sourceComponent.ComponentClassID = "DTSAdapter.FlatFileSource";
sourceComponent.ValidateExternalMetadata = true;
CManagedComponentWrapper sourceComponentWrapper = sourceComponent.Instantiate();
sourceComponentWrapper.ProvideComponentProperties();
// LINK CONNECTION TO THE SOURCE COMPONENT
sourceComponent.RuntimeConnectionCollection[0].ConnectionManager = Dts.Runtime.DtsConvert.GetExtendedInterface(sourceConnectionManager);
sourceComponent.RuntimeConnectionCollection[0].ConnectionManagerID = sourceConnectionManager.ID;
// Connect to the data source, and then update the metadata for the source.
//sourceComponentWrapper.Validate();
//sourceComponentWrapper.ProvideComponentProperties();
sourceComponentWrapper.AcquireConnections(null);
sourceComponentWrapper.ReinitializeMetaData();
sourceComponentWrapper.ReleaseConnections();
// ADD DESTINATION CONNECTION MANAGER
Dts.Runtime.ConnectionManager destinationConnectionManager = package.Connections.Add("FLATFILE");
destinationConnectionManager.ConnectionString = "C:\\cmi\\tmp\\data_out.csv";
destinationConnectionManager.Name = "DatComConOut";
destinationConnectionManager.Description = "Data Commander Connection Manager";
destinationConnectionManager.DelayValidation = false;
destinationConnectionManager.Properties["Format"].SetValue(destinationConnectionManager, "Delimited");
destinationConnectionManager.Properties["TextQualifier"].SetValue(destinationConnectionManager, "\"");
destinationConnectionManager.Properties["CodePage"].SetValue(destinationConnectionManager, "65001");
// ADD DESTINATION COMPONENT
Dts.Pipeline.Wrapper.IDTSComponentMetaData100 destinationComponent = pipelineTask.ComponentMetaDataCollection.New();
destinationComponent.Name = "Flat File Destination";
destinationComponent.ComponentClassID = "DTSAdapter.FlatFileDestination";
destinationComponent.ValidateExternalMetadata = true;
CManagedComponentWrapper destinationComponentWrapper = destinationComponent.Instantiate();
destinationComponentWrapper.ProvideComponentProperties();
destinationComponent.RuntimeConnectionCollection[0].ConnectionManager = Dts.Runtime.DtsConvert.GetExtendedInterface(destinationConnectionManager);
destinationComponent.RuntimeConnectionCollection[0].ConnectionManagerID = destinationConnectionManager.ID;
// CREATE PATH
Dts.Pipeline.Wrapper.IDTSPath100 pipelinePath = pipelineTask.PathCollection.New();
pipelinePath.AttachPathAndPropagateNotifications(sourceComponent.OutputCollection[0], destinationComponent.InputCollection[0]);
// DESTION CONNECTION MANAGER COLUMNS
IDTSConnectionManagerFlatFile100 destinationConnectionInterface = destinationConnectionManager.InnerObject as IDTSConnectionManagerFlatFile100;
IDTSConnectionManagerFlatFileColumns100 destinationColumns = destinationConnectionManager.Properties["Columns"].GetValue(destinationConnectionManager) as IDTSConnectionManagerFlatFileColumns100;
// START --- MANUALLY READ COLUMNS FROM DESTINATION COMPONENT AND ADD THEM TO DESTIONATION CONNECTION MANAGER COLUMNS
foreach (IDTSInput100 input in destinationComponent.InputCollection)
IDTSVirtualInput100 virtualInput = input.GetVirtualInput();
IDTSConnectionManagerFlatFileColumn100 column = null;
foreach (IDTSVirtualInputColumn100 virtualInputColumn in virtualInput.VirtualInputColumnCollection)
column = destinationColumns.Add();
column.DataType = DataType.DT_TEXT;
column.ColumnDelimiter = ",";
column.TextQualified = true;
(column as IDTSName100).Name = virtualInputColumn.Name;
column.ColumnDelimiter = "\r\n";
// END --- MANUALLY READ COLUMNS FROM DESTINATION COMPONENT AND ADD THEM TO DESTIONATION CONNECTION MANAGER COLUMNS
// NEED TO CREATE NEW META DATA AFTER COLUMNS
destinationComponentWrapper.AcquireConnections(null);
destinationComponentWrapper.ReinitializeMetaData();
destinationComponentWrapper.ReleaseConnections();
// MAP INPUT COLUMNS
foreach (IDTSInput100 input in destinationComponent.InputCollection)
IDTSVirtualInput100 virtualInput = input.GetVirtualInput();
foreach (IDTSVirtualInputColumn100 virtualInputColumn in virtualInput.VirtualInputColumnCollection)
Console.WriteLine("VIC " + virtualInputColumn.Name + "(" + virtualInputColumn.ID + ")");
destinationComponentWrapper.SetUsageType(input.ID, virtualInput, virtualInputColumn.LineageID, DTSUsageType.UT_READONLY);
foreach (IDTSExternalMetadataColumn100 metadataColumn in input.ExternalMetadataColumnCollection)
if (virtualInputColumn.Name.Equals(metadataColumn.Name)) {
// PROBLEM HERE:
// An unhandled exception of type 'System.Runtime.InteropServices.COMException' occurred in msdatapipe.exe
// Additional information: Exception HRESULT: 0xC0047072
// at this point is virtualInputColumn.ID = -2147483573
destinationComponentWrapper.MapInputColumn(input.ID, virtualInputColumn.ID, metadataColumn.ID);
// if the code is commented out and the program rerun there is a package validation error instead: 0xC020201B
// 0xC020201B - The number of input columns for Flat File Destination.Inputs[Flat File Destination Inputs] cannot be zero
// NEED TO CREATE META DATA AFTER COLUMNS HAVE BEEN ADDED TO THE DESTIONATION MANAGER
// AND USAGE TYPE HAS BEEN SET OR WE'LL GET A VS_NEEDSNEWMETADATA
destinationComponentWrapper.AcquireConnections(null);
destinationComponentWrapper.ReinitializeMetaData();
destinationComponentWrapper.ReleaseConnections();
// SAVING DTSX PACKAGE FOR DEBUGGING
application.SaveToXml("C:\\Users\\CMI\\documents\\visual studio 2010\\Projects\\SSIS Tutorial\\SSIS Tutorial\\generated.dtsx", package, null);
Microsoft.SqlServer.Dts.Runtime.DTSExecResult packageResults = package.Execute();
Console.WriteLine("package execution result: " + packageResults.ToString());
// Basic check for errors
foreach (DtsError error in package.Errors)
Console.WriteLine("ErrorCode : 0x{0:X}", error.ErrorCode);
Console.WriteLine(" SubComponent : {0}", error.SubComponent);
Console.WriteLine(" Description : {0}", error.Description);
Console.WriteLine("press the return key hard to continue...");
Console.Read();

Answer 1

I suggest you revise your code against examples given in http://blogs.msdn.com/b/mattm/archive/2008/12/30/samples-for-creating-ssis-packages-programmatically.aspx
Arthur My Blog