Is there a way to Insert Data into a Lookup Column Type on a SharePoint List Destination in SSIS?

Question

Greetings.
I have successfully worked out inserting SQL data (2008 R2) into my 2010 SharePoint list (New, Update, Delete) by creating an SSIS Data Flow Task as outlined here:
http://fsugeiger.blogspot.com/2010/01/synchronise-sql-table-with-sharepoint.html
However, the problem I am running into is inserting data into the SharePoint Columns that are "Lookup" column types. I verified that all of the values I am copying from SQL into the SharePoint lookup column exist in the customn list it is pointing to. It
is important to have this column be a lookup column as it links to another custom list that has many more columns of related information.
I have read and re-read the SharePoint SSIS Adapters 2011.docx from
http://sqlsrvintegrationsrv.codeplex.com/ and the only section that seems to apply is this:
"Looking Up Values in a SharePoint List
If you have to look up a value in a SharePoint list, you can use the Lookup transformation in your data flow, and use the SharePoint List source to load the lookup table. You may have to add a Derived Column transformation or a Script component that splits
data in the lookup column on the ";#" delimiter to separate the ID value from the description.
If you are replacing values in your data with the values that you look up in the list, then loading the changed data back into SharePoint, you only have to include the ID from the lookup column. SharePoint ignores the description if you include it."
I am not sure if the above statement means that I should be passing the assocaited ID's other than the actual data into the SharePoint List destination. If that is the case, that will not really work as the lookup contains hundreds of rows. Not too mention
I have several of these lookup column types pointing to several different lists.
Any guidance in how I can put data into a SharePoint Lookup column type via Data Flow Task would be so much appreaciated.
Thank you.
My errors are:
Error: 0x0 at Data Flow Task, SharePoint List Destination: Error on row ID="1": 0x1 - Unspecified error, such as too many items being updated at once (batch), or an invalid core field value.
Error: 0xC0047062 at Data Flow Task, SharePoint List Destination [1903]: Microsoft.Samples.SqlServer.SSIS.SharePointListAdapters.PipelineProcessException: Errors detected in this component - see SSIS Errors at Microsoft.Samples.SqlServer.SSIS.SharePointListAdapters.SharePointListDestination.ProcessInput(Int32
inputID, PipelineBuffer buffer) at Microsoft.SqlServer.Dts.Pipeline.ManagedComponentHost.HostProcessInput(IDTSManagedComponentWrapper100 wrapper, Int32 inputID, IDTSBuffer100 pDTSBuffer, IntPtr bufferWirePacket)
Error: 0xC0047022 at Data Flow Task, SSIS.Pipeline: SSIS Error Code DTS_E_PROCESSINPUTFAILED. The ProcessInput method on component "SharePoint List Destination" (1903) failed with error code 0x80131500 while processing input "Component Input" (1912). The identified
component returned an error from the ProcessInput method. The error is specific to the component, but the error is fatal and will

Answer 1

I have found a solution to my problem and thought I would share it here in case there are others who are struggling with the above scenario. If you have a better way, I would love to hear about it since my way is a bit tedious.
In a nutshell, in order to have an SSIS package put data from an OLE DB Source into a SharePoint List Destination Lookup Column, you need to pass the ID of the value that is being looked up, not the value that is in the “master” OLE DB source.
Rough explanation, OLE DB Source value for column “Approp” is “4005” --> SQL matches “4005” with the ID in the new lookup table (“4005” = ID “5” as defined in the SharePoint lookup list) --> “5” gets passed into SharePoint List destination lookup
column --> SharePoint displays “4005” and successfully links to the lookup list.
Funny thing (not really), the error(s) outlined in my original post are not related in getting data into a SharePoint Lookup column as I am now successful in getting data into the system but I am still getting the same above error(s). I think it has to do
with the ID column in the SharePoint list destination. What I can’t seem to figure out is why since I am not linking any data to that ID column (at least on new records). I am however linking it on Update and Delete and the errors mentioned above disappear
and things work well.
There are three tasks that need to get done in order to get data from SQL into a SharePoint lookup column assuming you have already set up your SharePoint lookup lists:
1. Create new lookup table(s) in SQL that has the IDs from the SharePoint Lookup list and the values coming from the “master” OLD DB Source. You can see the ID column in SharePoint by toggling it on in a view.
2. Create a SQL command that JOINs all the databases and tables so that the ID is passed and not the value into the SharePoint lookup column
3. Change the “Data access mode” to “SQL Command” instead of the “Table or view” in the OLE DB Source and paste your command into the “SQL command text:” area.
Other helpful info is that you may also need to add additional columns in the new lookup tables in SQL for the scenarios when the data is not unique. You can see this two times in my SQL command example for Units and JobTitles:
SELECT
pps.SSNm,
pps.file_updated,
pps.Employee_id,
/* pps.CheckDistNm,*/
Check_Distribution_id = COALESCE( d.ID, 0 ),
pps.Job_nbr,
pps.SeqNm,
pps.action_eff_dt,
Fund_id = COALESCE( f.id, 0 ),
Appropriation_id = COALESCE( ap.id, 0 ),
ActionCode_id = COALESCE( ac.id, 0 ),
SpecNumber_id = COALESCE( jt.ID, 0 ),
pps.Employee_id,
/* pps.Fund,
pps.Approp,
pps.Unit,*/
Unit_id = COALESCE( u.ID, 0 ),
PosNm,
PosCode,
pps.LastName,
pps.FirstName,
pps.MI
FROM
x_PPS.aReportVw.pps_screens_active AS pps
LEFT OUTER JOIN dbo.DistributionNumbers AS d ON
pps.CheckDistNm = d.Check_Distribution
LEFT OUTER JOIN dbo.Units AS u ON
pps.Fund = u.Fund AND
pps.Approp = u.Approp AND
pps.Unit = u.Unit
LEFT OUTER JOIN dbo.Appropriations AS ap ON
pps.Approp = ap.Approp
LEFT OUTER JOIN dbo.Funds AS f ON
pps.Fund = f.Fund
LEFT OUTER JOIN dbo.ActionCodes AS ac ON
pps.ActionCode = ac.ActionCode
LEFT OUTER JOIN dbo.JobTitles AS jt ON
pps.SpecNm = jt.SpecNumber AND
pps.JurisClass = jt.JurisClass