SSIS Data Flow task using SharePoint List Adapter Setting SiteUrl won't work with an expression
Hi,
I'm trying to populate the SiteUrl from a variable that has been set based on a query to a SQL table that has a URL field. Here are the steps I've taken and the result.
Created a table with a url in it to reference a SharePoint Task List.
Created a Execute SQL Task to grab the url putting the result set in a variable called SharePointUrl
Created a For Each container and within the collection I use the SharePointUrl as the ADO object source variable and select rows in the first table.
Still in the For Each container within the Variable mappings I have another Package Variable called PassSiteUrl2 and I set that to Index 0 or the value of the result set.
I created a script task to then display the PassSiteUrl2 variable and it works great I see my url
This is where it starts to suck eggs!!!!
I insert a Data Flow Task into my foreach loop.
I Insert a SharePoint List Adapter into my Data Flow
Within my SharePoint List Adapter I set my list to be "Tasks", My list view to be "All Tasks" and then I set the url to be another SharePoint site that has a task list just so there is some default value to start with.
Now within my Data Flow I create an expression and set the [SharePoint List Source].[SiteUrl] equal to my variable @[User::PassSiteUrl2].
I save everything and run my SSIS package and it overlays the default [SharePoint List Source].[SiteUrl] with blanks in the SharePoint List Adapter then throws the error that its missing a url
So here is my question. Why if my package variable displays fine in my Control Flow is it now not seen or seen as blanks in the Data Flow Expression. Anyone have any ideas???
Thanks
Donald R. Landry
Thanks Arthur,
The scope of the variable is at a package level and when I check to see if it can be moved Package level is the highest level. The evaluateasexpression property is set to True. Any other ideas?
I also tried to do the following. Take the variable that has the URL in it and just assign it to the description of the data flow task to see if it would show up there (the idea being the value of my @[User::PassSiteUrl] should just show in the
description field when the package is run. That also shows up blank.
So i'm thinking its my expression. All I do in the expression is set [SharePoint List Source].[SiteUrl] equal to @[User::PassSiteUrl] by dragging and dropping the variable into the expression box. Maybe the expression should be something
else or is their a way to say @[User::PassSiteUrl] = Dts.Variables("User::PassSiteUrl2").Value.ToString()
In my script task I use Dts.Variables("User::PassSiteUrl2").Value.ToString() to display
the value in the message box and that works fine.
Donald R. Landry
Similar Messages
-
I want to use my Personal Laserwriter, but it won't work with my imac
I have a personal laserwriter printer. It work great with my older mac - Performa 580CD. However when I tried setting it up with my imac - using a MacOS 9.X driver and usb-serial adaptor - it only works with OS 9.x softeware.
So I have a laser printer that I cannot use. I'd like to get it up and running for my MacBoo Pro, too. Any advice?The personal LaserWriter NTR and the Personal LaserWriter 320 are Postscript level 2 printers that can run with the built-in Mac OS X Drivers.
What is missing is a way to make the connection to recent machines that do not have a Printer/Modem/LocalTalk port. You can use the Mac's Ethernet port to carry AppleTalk packets (in addition to the standard TCP/IP packets) and use a converter to bridge from Ethernet to LocalTalk.
Two converters are available on the used market: The Farallon EtherMac iPrint LT and the Asante Asantetalk. The Asante may also be available new, but at a cost near US$100 new, you could get a more capable used Ethernet-capable printer for the same money.
If you have an older Mac that has a Printer/Modem/LocalTalk port and an Ethernet port and is still running OS 9 or earlier, you can download a package called LocalTalk Bridge to run in the background and do the conversion.
The other Personal LaserWriters would require a bridge AND a third-party driver. -
SQL Query using a Variable in Data Flow Task
I have a Data Flow task that I created. THe source query is in this file "LPSreason.sql" and is stored in a shared drive such as
\\servername\scripts\LPSreason.sql
How can I use this .sql file as a SOURCE in my Data Flow task? I guess I can use SQL Command as Access Mode. But not sure how to do that?Hi Desigal59,
You can use a Flat File Source adapter to get the query statement from the .sql file. When creating the Flat File Connection Manager, set the Row delimiter to a character that won’t be in the SQL statement such as “Vertical Bar {|}”. In this way, the Flat
File Source outputs only one row with one column. If necessary, you can set the data type of the column from DT_STR to DT_TEXT so that the Flat File Source can handle SQL statement which has more than 8000 characters.
After that, connect the Flat File Source to a Recordset Destination, so that we store the column to a SSIS object variable (supposing the variable name is varQuery).
In the Control Flow, we can use one of the following two methods to pass the value of the Object type variable varQuery to a String type variable QueryStr which can be used in an OLE DB Source directly.
Method 1: via Script Task
Add a Script Task under the Data Flow Task and connect them.
Add User::varQuery as ReadOnlyVariables, User::QueryStr as ReadWriteVariables
Edit the script as follows:
public void Main()
// TODO: Add your code here
System.Data.OleDb.OleDbDataAdapter da = new System.Data.OleDb.OleDbDataAdapter();
DataTable dt = new DataTable();
da.Fill(dt, Dts.Variables["User::varQuery"].Value);
Dts.Variables["QueryStr2"].Value = dt.Rows[0].ItemArray[0];
Dts.TaskResult = (int)ScriptResults.Success;
4. Add another Data Folw Task under the Script Task, and join them. In the Data Flow Task, add an OLE DB Source, set its Data access mode to “SQL command from variable”, and select the variable User::QueryStr.
Method 2: via Foreach Loop Container
Add a Foreach Loop Container under the Data Flow Task, and join them.
Set the enumerator of the Foreach Loop Container to Foreach ADO Enumerator, and select the ADO object source variable as User::varQuery.
In the Variable Mappings tab, map the collection value of the Script Task to User::QueryStr, and Index to 0.
Inside the Foreach Loop Container, add a Data Flow Task like step 4 in method 1.
Regards,
Mike Yin
TechNet Community Support -
Does Apple list available printers that work with AirPort Express?
Just bought an AirPort Express hoping that I could print wirelessly with my HP 1350 All-in-One, and it won't work. Found out after the fact that my printer doesn't support USB wireless printing.
I'm looking for a basic color printer to work with AirPort Express for wireless printing by other family members, and I'll continue to use my HP 1350 with USB for scanning and OCR functions.
Does Apple provide a list of recommended printers that will work with AirPort Express?
Kevin
PowerBook G4 1.5 Ghz Mac OS X (10.4.8) 1 GB DDR SDRAMNo Apple doesn't any more.
I have been maintaining a community created list.
http://tech.ifelix.net/1013.html
This is an unofficial list of USB printers which are are compatible with the AirPort Extreme Base Station and Airport Express. Printers in bold have been added to the original list that Apple provided.
Printers in dark red are in fact the only printers I have personally tested.
Please also note that due to changes in the firmware of both the Airport Extreme base station and the introduction of the Airport Express some printers which are in this list may no longer in fact be compatible. -
How to use SSIS 2014 to read from Sharepoint List ?
Hi there,
I have designed a simple entry form for users using List in Sharepoint 2013.
I need to use the information to merge into my datawarehouse.
Question :
1. How can I setup SSIS (SQL 2014) to get the source from List in Sharepoint?
I have tried to download the script as suggested in codeplex but it seems not working with SQL 2014 as there is no SSIS toolbar displayed.
Any help is much appreciated
Thank you and Best RegardsHi SylviaO,
The current
SharePoint List Adapter that extracting and loading SharePoint Data in SQL Server Integration Services supports x86/x64 systems using SQL 2005 / SQL 2008 / SQL 2008 R2/ SQL 2012. It doesn’t support SQL 2014. Maybe we need wating the update.
Besides, the
OData Source component can also be used to read from SharePoint lists. So we can download and install the 64 bit ODataSourceForSQLServer2014-amd64.msi or 32 bit ODataSourceForSQLServer2014-x86.msi from the
Microsoft® SQL Server® 2014 Feature Pack.
The following blog about using the SSIS OData Source Connector to read data from SharePoint lists is for your reference:
http://whitepages.unlimitedviz.com/2014/03/using-the-odata-source-connector-with-sharepoint-online-authentication/
If there are any other questions, please feel free to ask.
Thanks,
Katherine Xiong
Katherine Xiong
TechNet Community Support -
I am trying to compare a DB2 date format to a date variable from SSIS. I have tried to set the variable as datetime and string. I have also casted the SQL date as date in the data flow task. I have tried a number of combinations of date formats, but no luck
yet. Does anyone have any insights on how to set a date (without the time) variable and be able to use it in the data flow task SQL? It has to be an easy way to accomplish that. I get the following error below:
An invalid datetime format was detected; that is, an invalid string representation or value was specified. SQLSTATE=22007".
Thanks!Hi Marcel,
Based on my research, in DB2, we use the following function to convert a string value to a date value:
Date(To_Date(‘String’, ‘DD/MM/YYYY’))
So, you can set the variable type to String in the package, and try the following query:
ACCOUNT_DATE BETWEEN '11/30/2013' AND Date(To_Date(?, ‘DD/MM/YYYY’))
References:
http://stackoverflow.com/questions/4852139/converting-a-string-to-a-date-in-db2
http://www.dbforums.com/db2/1678158-how-convert-string-time.html
Regards,
Mike Yin
TechNet Community Support -
Data Flow Task problems after installing SQL Server 2014 SSIS
I can't create any Data Flow Task. Once I create (tested and working) Connection (it doesn't
matter if it is Excel, Flat File or other) and try to use it with a Source Assistant, I get the following messages:
===================================
The component could not be added to the Data Flow task.
Could not initialize the component. There is a potential problem in the ProvideComponentProperties method. (Microsoft Visual Studio)
===================================
Error at Data Flow Task [SSIS.Pipeline]: Component ", clsid {C4D48377-EFD6-4C95-9A0B-049219453431}" could not be created and returned error code 0x80070005 "Access is denied.". Make sure that the component is registered correctly.
Error at Data Flow Task [ [1]]: The component is missing, not registered, not upgradeable, or missing required interfaces. The contact information for this component is "".
===================================
Exception from HRESULT: 0xC0048021 (Microsoft.SqlServer.DTSPipelineWrap)
Program Location:
at Microsoft.SqlServer.Dts.Pipeline.Wrapper.CManagedComponentWrapperClass.ProvideComponentProperties()
at Microsoft.DataTransformationServices.Design.PipelineTaskDesigner.AddNewComponent(String clsid, Boolean throwOnError, Boolean select)
===================================
The component could not be added to the Data Flow task.
Could not initialize the component. There is a potential problem in the ProvideComponentProperties method. (Microsoft Visual Studio)
The component could not be added to the Data Flow task.
Could not initialize the component. There is a potential problem in the ProvideComponentProperties method. (Microsoft Visual Studio)
===================================
Error at Data Flow Task [SSIS.Pipeline]: Component ", clsid {C4D48377-EFD6-4C95-9A0B-049219453431}" could not be created and returned error code 0x80070005 "Access is denied.". Make sure that the component is registered correctly.
Error at Data Flow Task [ [1]]: The component is missing, not registered, not upgradeable, or missing required interfaces. The contact information for this component is "".
Error at Data Flow Task [SSIS.Pipeline]: Component ", clsid {C4D48377-EFD6-4C95-9A0B-049219453431}" could not be created and returned error code 0x80070005 "Access is denied.". Make sure that the component is registered correctly.
Error at Data Flow Task [ [1]]: The component is missing, not registered, not upgradeable, or missing required interfaces. The contact information for this component is "".
===================================
Exception from HRESULT: 0xC0048021 (Microsoft.SqlServer.DTSPipelineWrap)
Program Location:
at Microsoft.SqlServer.Dts.Pipeline.Wrapper.CManagedComponentWrapperClass.ProvideComponentProperties()
at Microsoft.DataTransformationServices.Design.PipelineTaskDesigner.AddNewComponent(String clsid, Boolean throwOnError, Boolean select)
Exception from HRESULT: 0xC0048021 (Microsoft.SqlServer.DTSPipelineWrap)
at Microsoft.SqlServer.Dts.Pipeline.Wrapper.CManagedComponentWrapperClass.ProvideComponentProperties()
at Microsoft.DataTransformationServices.Design.PipelineTaskDesigner.AddNewComponent(String clsid, Boolean throwOnError, Boolean select)See examples here:
http://www.sqlusa.com/bestpractices/ssis-wizard/
What is your data source?
What is your data destination?
Kalman Toth Database & OLAP Architect
SQL Server 2014 Database Design
New Book / Kindle: Beginner Database Design & SQL Programming Using Microsoft SQL Server 2014 -
Unable preview the data of oracle source in Data flow task in SSIS
I am working on a SSIS package DATA FLOW TASK to get the data from Oracle Source.My oracle source query is as belows
select Employees from EmployeDW WHERE StartDate =('15-MAY-14')
There is no error in execution of package but there is no data in Destination table.When i am PREVIEW the data source i dont find any data but when i executed above query in Oracle i am getting the values.Do i need to make any chages in oracle data source Side.
Can you guys help me please.What is the source connectoin used and its configuration? Can you please share the screenshots? That will help.
Happy to help! Thanks. Regards and good Wishes, Deepak. http://deepaksqlmsbusinessintelligence.blogspot.com/ -
Hi All,
I got a client requirement to create reports using SharePoint List as data source. The report should show reflection depends on values changed (I mean animation).
I have heard about the power view/power pivot which does this kind of animations in reports.
Can someone please guide me on creating reports which shows animations
In power view/power pivot using SharePoint List as data source in SharePoint 2010.
Thanks in advance.
MercuryManHi MercuryMan,
Yes, Power View, a feature of SQL Server 2012 Reporting Services Add-in for Microsoft SharePoint Server 2010 or SharePoint 2013 Enterprise Edition, is an interactive data exploration, visualization, and presentation experience.
It provides multiple views featuring tiles, slicers, a chart filter, and a number of visualizations, including cards, small multiples, and a bubble chart. So, we can use Power View to do intuitive ad-hoc reporting for business users such as data analysts, business
decision makers, and information workers.
Currently, Power View report only supports two types of data models: PowerPivot Worksheet, and data models based on Analysis Services Tabular Model or Multidimensional Cube.
In your scenario, you can create PowerPivot worksheets using SharePoint List as data source, deploy the PowerPivot worksheet to a SharePoint Library or PowerPivot Gallery, and then generate Power View reports based on the PowerPivot worksheets on the SharePoint
site.
To use SharePoint List as data source in Excel PowerPivot, you can refer to the following resource:
http://blogs.technet.com/b/excel_services__powerpivot_for_sharepoint_support_blog/archive/2013/07/11/excel-services-using-a-sharepoint-list-as-a-data-source.aspx
http://technet.microsoft.com/en-us/library/hh230322.aspx
To create a Power View report based on PowerPivot model, you can refer to the following links:
http://technet.microsoft.com/en-us/library/hh231522.aspx
http://technet.microsoft.com/en-us/library/hh759325.aspx
Regards,
Mike Yin
If you have any feedback on our support, please click
here
Mike Yin
TechNet Community Support -
SQL 2005 SP2 - Cannot open Data Flow Task in SSIS
I have just installed Service Pack 2 on my SQL 2005 Standard Edition.
However, now all my SSIS packages will not allow me to open my Data FLow Tasks. I get the following error:
TITLE: Microsoft Visual Studio
Cannot show the editor for this task.
ADDITIONAL INFORMATION:
The task returned an unsupported control editor type. (Microsoft.DataTransformationServices.Design)
If I try to create a new Data Flow task I get:
TITLE: Microsoft Visual Studio
Failed to create the task.
ADDITIONAL INFORMATION:
The designer could not be initialized. (Microsoft.DataTransformationServices.Design)
I have tried to install the latest hotfixes after this but they had no effect.
Can anybody help me???? Please?I have had this same issue, where tasks would open fine in an SSIS package until SP2 was installed and then I get the same issue as noted above, i.e. :
TITLE: Microsoft Visual Studio
Cannot show the editor for this task.
ADDITIONAL INFORMATION:
The designer could not be initialized. (Microsoft.DataTransformationServices.Design)
If anyone has some ideas on this, it would be greatly appreciated. -
Why do we want use 'Data Flow Task' to 'Data Flow Tast' in Control Flow?
I found an example in my company's SSIS package folders. In Control Flow, it has one data flow connected to the other. Both of them are importing data from flat file and then exported to database. I think these two are kind of at the same level and cannot
see any reasons to conncect them together.
Thanks & Happy Thxgiving,
GavinHi Gavin,
Just as Arthur said, if there is no relationship between those two data flow tasks, we don’t have to connect them together. If they connect together, maybe there are some relationship between them.
According to your description, both of the data flow tasks are importing data from flat file and then exported to database. It seems that there is no direct relationship between them. Another possibility is that there is a precedence constraint between them.
The precedence constraint can be based on a combination of the first execution results and the evaluation of expressions. We can check the issue by double-check the connection string between them.
The following screenshot is for your reference:
If there are any other questions, please feel free to ask.
Thanks,
Katherine Xiong
Katherine Xiong
TechNet Community Support -
Dynamically pick the table names in data flow task SSIS
Hi All,
I want to create a SSIS package which loads the data to a table on the other server every day. I have around 250 tables to load everyday and source and destination table names are available in a metadata table, table names have to read from the metadata
table and data should be loaded. Is there a way that we can configure the source and destination table names dynamically in Data flow task?
I am newbie to SSIS can any help with the solution for this problem.You can do that, not a big deal. The underlying problem is say suppose you constructed a ETL based on some x source and y destination and have put x(3 columns) as source and y destination(3 columns).. As you said if we have choice of dynamically pick the
table names.. ETL might fail when you face below situation
source x(3 columns) destination y (4 columns) and there will be no mapping as it is dynamic. Sometimes mapping also fails even if the source and destination have same number of columns. If you still want to do... follow below steps:
Create two variables:
1.variable1 , datatype string
2.variable datatype string
take one execute sql task, pick your source table names dynamically as you desired from metadata table \
"SELECT sourcetblname as Res FROM @metadata WHERE ID=1" in sql statement and then go to name the result name as Res (I meant same as table alias) and map it to variablename1
And in variable2 go to expression and write "Select * from "+@[User::variable].. and this will be your constructed dynamic command for oledb destination.
And connect that execute sql task to (Data flow task)oledb source and choose data access mode as sql command with variable,and choose variable2. below is the diagram.
- please mark correct answers -
SSIS Data flow item does not show on the Tools Choose Tools Item
I would like to add share point to my data source for my SSIS project.
After I installed Share Point Source and Destination list, I went to Tools Choose Tools Item, but SSIS Data Flow tab does not show. I went to SSIS Tools box, but the components are not there.
I would like to know are there any way to add Share Point Data source using SSIS.
Your help and information is great appreciated,
I use SQL Server 2012 and VS 2010.
Regards,
Souris,Hi Souris,
I have installed the SharePointListAdaptersSetup-Beta-2012-01-28.msi in my SQL Server 2012 test environment, the SharePoint List Source and Destination adapters is listed in the Common type items of the SSIS Toolbox. To narrow down the root cause, please
try the following steps:
Make sure the SharePointListAdapters.dll exists in the C:\Program Files (x86)\Microsoft SQL Server\110\DTS\PipelineComponents folder.
Make sure it is installed to GAC successfully, the SharePointListAdapters.dll should exist in C:\Windows\Microsoft.NET\assembly\GAC_MSIL\SharePointListAdapters\v4.0_1.2012.0.0__f4b3011e1ece9d47 folder.
Refresh the Toolbox and check the Common type items.
Since the SharePoint List Adapter is a third-party component, I would suggest you ask a question in its dedicated discussion plate:
http://sqlsrvintegrationsrv.codeplex.com/discussions
Regards,
Mike Yin
TechNet Community Support -
Excel Spreadsheet as Data Source & Copying that Data to an Existing SharePoint List
Hello,
I have an Excel spreadsheet that I get daily and cut and paste into an existing SharePoint list. Is there a way to make that Excel spreadsheet a data source and copy the data to an existing SharePoint list with less manual involvement?
Tools at my disposal include: SharePoint 2010 Designer, InfoPath, and Access.
I thought there might be a REST method I could use via InfoPath but I'm not aware of any offhand.
Goal is to have a method to upload/overwrite an Excel file (stored in a SharePoint library), every day, and then automate the process of copy it to a SharePoint list (actually overwriting the old data in the SharePoint list).
Any help would be greatly appreciated.Hi,
We can create a console application and use C# code to read the data from the Excel file and using SharePoint .Net Client Object Model to add the data into a SharePoint list, then create a windows job to execute the console application.
The following articles for your reference:
How to: Create a Console Application
https://msdn.microsoft.com/en-us/library/office/ms438026(v=office.14).aspx
C# How To Read .xlsx Excel File With 3 Lines of Code
http://www.codeproject.com/Tips/801032/Csharp-How-To-Read-xlsx-Excel-File-With-Lines-of
How to: Create, Update, and Delete List Items
https://msdn.microsoft.com/en-us/library/office/ee539976%28v=office.14%29.aspx?f=255&MSPPError=-2147217396
Schedule a task
http://windows.microsoft.com/en-au/windows/schedule-task#1TC=windows-7
Best Regards
TechNet Community Support
Please remember to mark the replies as answers if they help, and unmark the answers if they provide no help. If you have feedback for TechNet Support, contact
[email protected] -
We have an SSIS package that runs on clustered MSSQL 2012 Enterprise Nodes that is failing. We use a job to executer the package.
Environmental information:
Product - Microsoft SQL Server Enterprise: Core-based Licensing (64-bit)
Operating System - Microsoft Windows NT 6.1 (7601)
Patform - NT x64
Version - MSSQL Version 11.0.3349.0
Package is set to 32 -bit. All permissions verified. Runs in lower environments, same MSSQL version. All environments are clustered. In the failing environment, all nodes are at the same service pack. I have not verified if all
nodes in the failing environment have SSIS installed. Data access is installed. We have other simpler packages that run in this environment, just not this one. Time to ask the community for help!
Error:
Source: Data Flow Task - Data Flow Task (SSIS.Pipeline) Description: The version of Lookup is not compatible with this version of the DataFlow. End Error Error: Code: 0xC0048020
Description: Component "Conditional Split, clsid {7F88F654-4E20-4D14-84F4-AF9C925D3087}" could not be created and returned error code 0x80070005 "Access is denied.". Make sure that the component is registered correctly. End Error
Description: The component is missing, not registered, not upgradeable, or missing required interfaces. The contact information for this component is "Conditional Split;Microsoft Corporation; Microsoft SQL Server; (C) Microsoft Corporation; All Rights
Reserved; http://www.microsoft.com/sql/support;0". End Error
(Left out shop specific information. This is the first error in the errors returns by the job history for this package. )
Thanks in advance.Hi DeveloperMax,
According to your description, the error occurs when you execute the package with Agent job on clustered MSSQL 2012 Enterprise Nodes.
As per my understanding, I think this issue can be caused by you use SQL Server Agent to schedule a SQL Server Integration Services package in a 64-bit environment. And the SSIS package is referencing some 32-Bit DLL or 32-Bit drivers which are available
only in 32-bit versions, so the job failed.
To fix this issue, we should use the 32-bit version of the DTExec.exe utility to schedule the 64-bit SQL Server Agent to run a package. To run a package in 32-bit mode from a 64-bit version of SQL Server Agent, we can go to the Job Step dialog box, then
select “32 bit runtime” in the Advanced tab.
Besides, we should make sure that SQL Server Integration Services is installed on the failing environment.
If there are any other questions, please feel free to ask.
Thanks,
Katherine Xiong
Katherine Xiong
TechNet Community Support
Maybe you are looking for
-
My father has a MacBook 5.1 w/ OS X 10.5.8. He needs a new printer and I purchased a HP all-in-one, but, it requires OS X 10.6. What are my options?
-
How to get return value from java and read by other application?
i want to read return value from java and the other application read it. for example: public class test_return { test_return(){ public int check(){ return 1; public static void main(String args[]){ new test_return().check()
-
I want to buy a new camcorder and am considering one by JVC. This one, plus a few others such as Sony use AVCHD. I gather this would be need converting before any editing can be done in imovie?? I cant find a simple answer to this question. It seems
-
How do I add a grass texture to a selection in a photograph?
Usung Photoshop CS5 Extended in Windows 7 on a PC, I would like to make a selection and add a grass texture to a field I have painted. I went to Filter-TEXTURE-Texturizer-but do not see a grass texture I can add to the selction like I can Canvas etc.
-
Center Image Component's Source
In my flex app, I have an image control within a VBOX and I set the size to 100%by100%. When an image loads (as usual) it is placed in the top left hand corner of the component. How can I keep the image component's source centered while still keeping