For Each Loop Load Dynamic Excel Files

Hi All,
SSIS 2005
I want to load excel file .xls one at a time files that have different names without having to select the file in the connection manager, every time. So a user can easily place the excel file in the folder and right the sql job and it loads the file data.
The data flow tasks all work when I manually select the excel file, but I am unable to get this to work dynamically.
I have setup a for each loop with multiples data flow tasks and sql tasks to load my data from file.
Variables:
@filename d:\new file\*.xls
For Each Loop:
ForEachFile Enumerator
Folder: d:\new file\
Files: *.xls
Retrieve File Name: Fully Qualified
Variable Mappings:
User::filename (index 0)
Connection Managers:
Excel Connection Manager: Expressions, ConnectionString @[User::filename]
Error Message: Error at new buyer(package name) the connection string format is not valid. it must consist of one or more components of the form X=Y, seperated by semicolons.
Can anyone assist thanks.

You still need the file name variable.
ExcelFilePath is the fully qualified file name of your Excel file. ie. it contains both the path and file name.
e.g. D:\inbox\myExcelFile.xls
All the steps you mentioned are correct, except you need to set the property of ExcelFilePath of the Excel connection to the variable that you map to in the ForEach Loop:
Connection Managers:
Excel Connection Manager: Expressions, ExcelFilePath @[User::filename]
Hope this helps.
~ J.

Similar Messages

Package Help needed For each Loop container

Hello all,
I need a bit of advice, I`m a newbee to SSIS so I`m learning. Here`s what I need. I have a EDI 837 file which needs to be converted and inserted into a sql table. These are multiple files and I need to maintain them with a monthly load. So I have a source
folder where these files will be dumped. I created a Package which will grab the files process the EDI aspect inserts into the data into table. This is the story on a high level.
Here`s what I did.
Created a For each loop container.
I have a stored proc which takes in a 837 files and gets the fields what I want and it inserts it into a staging table from there I have a data flow task which pipes it into my production table.
Inside the loop first task is to exec the Stored Proc but my question is once a file is picked up from for each loop I need the file to be inserted into a table which I am referencing in my store proc. All the EDI conversion takes place in my SP and it references
the table..
My question is the For each loop container picks up the files but where does the data goes?? Can someone help me out
FM

The foreach loop container (file enumerator) only returns the filename or filepath which you can use as a parameter in an Execute SQL Task or within an expression on a connection manager (which can be used in a Data Flow). So it doesn't do anything with
the file. You have to add a task within the loop to do something with the file (Execute SQL Task/Data Flow Task/File System Task/etc.).
Please mark the post as answered if it answers your question | My SSIS Blog:
http://microsoft-ssis.blogspot.com |
Twitter

Loading accdb file using for each loop in to single target table

I have 3 accdb file as below in single folder, in control flow I have created a for each loop to loop through all the below file. In dataflow task I have created oledb source using connection manager to point to first file a1.accdb. I am trying
to load all the file into single below mentioned target table, here It is only loading first file it is not looping through other two file.
This I have tried to load .txt files and csv files it is working perfectly but similarly when I try to load access db file table it is not working, can any one of you help me?
1. a1.accdb contains only one table a1tab ( col1 col2 col3 )
2. a2.accdb contains only one table a2tab ( col1 col2 col3 )
3. a3.accdb contains only one table a3tab ( col1 col2 col3 )
I have a target table with similar structure as source
Target table <TargetTable> Col1 Col2 Col3
aak

Thanks Arthur, please find below response
How do you handle the different table names? Cant we handle it in a similar manner, how we handle the .csv/.txt files with different name and similar structure for sources(multiple files) and target (single table)
Is it good to drop the idea of for each loop and create a separate task for each accdb file? to be loaded to same target table( performance wise etc.. any justification would be helpful)
You must handle the connection string dynamically?
Can you help me with the logic for handling connection string dynamically
aak

Get the Excel file creatoin Date through For Each Loop each excel file in SSIS

I had For Each Loop
in Collection Path had 2 files it had Creation date of 25/01/2014
So I want Update in Table that date as File_Receive_Date column
How Can I do ??
in variable I am using VFile_FULL_Name(=Filename with Path by attach to Foreach loop collection&connection manager)
which other System Variable to use in SSIS ??
I have created vRef_Date but in Expression =====??? what should I do through only SSIS ??
Thanks
Madhu

The easiest approach is https://filepropertiestask.codeplex.com/ can capture the file creation date, but it is not working for all SSIS versions
I recommend using .net FILE IO libraries to get the created date which needs to be done in a Script Task.
The ForLoop approach will not let you achieve this goal.
Arthur
MyBlog
Twitter

For Each Loop: Truncation error on excel field

In my SSIS package, I'm using a WMI file watcher combined with a ForEach loop to process excel files and load them into a SQL Server database as they are deposited in a drop box folder. Recently I've been getting truncation errors on one of my fields (Comments).
I know that this is because Excel is scanning the first 8 rows to determine column length and there are several records that have a comments field that is greater than what excel determined.
I've tried going to the excel editor in my advanced editor and setting the data type length of my external and output columns to a larger length (500 W_STR vs 255 W_STR) but I get a warning saying that the data type is not valid. I've tried adding IMEX=1
to my extended properties in my excel connection string but no dice. Still getting truncation errors.
I was wondering if anyone had any thoughts or suggestions? I'm always getting new files so the comments field will potentially always encounter this type of error. This SSIS package runs on a production server so making a Registry change to the excel drivers
regarding the scanning of the first 8 rows is not an option for me.
Migince

I wound up making the registry change for # of rows scanned by excel and that seemed to solve the issue.
I tried utilizing an excel template but every time my for each loop scanned for a new file, the metadata was updated with whatever column length was in the first 8 records of my new file. If you've got a good example of building an excel template to set
the column length metadata in my excel source (For Each Loop) I would love to take a look at it. Good information to know.

Excel Sheet Looping: For each loop from variable Enumerator

Hi,
Is it possible to do excel sheets looping using For each loop Enumerator as "For Each loop from Variable enumerator" ? If yes, please help.
Also,
I have an excel file with 10 sheets, but want to loop only 5 sheets. Please help..how can i achieve this.
Actually i tried with
1> stored the 5 sheets name into a variable (say 'SheetsName') of type string using script task
2> For each loop container
enumerator - 'For each ADO Schema Rowset enumerator'
connection: ADO
schema: 'Tables'
variable mapping: 'User::SheetsName'
3> Within For each loop container added a data flow task
4> Inside DFT - excel source -
Data access Mode: 'Table Name or view name variable'
Variable Name: 'SheetsName'
but this loops through all the 10 sheets irrespective of only 5 sheets names stored in the variable "SheetsName".
Please help...
Please Mark as Answer if my post solved your problem or Vote As Helpful if this helps. Blogs: www.sqlserver2005forum.blogspot.com

>> Is it possible to do excel sheets looping using For each loop Enumerator as "For Each loop from Variable enumerator" ? If yes, please help.
I don't think so. To loop through all worksheets of Excel For ADO.NET Schema Rowset Enumerator is the only option.
Please refer:
https://msdn.microsoft.com/en-us/library/ms345182(v=sql.110).aspx
>> I have an excel file with 10 sheets, but want to loop only 5 sheets
So you have configured For Each loop container which loops throgh all sheets and returns you sheet name to variable "SheetName".
Drag and drop one Sequence Container inside ForEach Loop Container and connect it to Data Flow Task.
Right click on connector and use expression: (which will check if current sheet is one of 5 required sheets)
@User::SheetName == "FirstSheet" || @User::SheetName == "SecondSheet" || .....
Below blog demonstrates kind of simillar steps:
http://www.bidn.com/blogs/MikeDavis/ssis/559/ssis-skip-certain-files-in-a-for-each-loop
-Vaibhav Chaudhari

How to use for each loop in XSLT when my source is a multilayout file and db

How do I use a for each loop in XSLT when my source is multilayout file and db .
My multilayout file is basically contain 2 kind of data one for employee and another for dependent.
Now my requirement is I want to read each line of data whether it is it is employee or dependent do a join with db and write it in my target file.
eg : File content
1 RichaKumari 311289 TCS INDIA
2 KiarnKumar 456782 BRO RichaKumari 311289 INDIA
2 Ravi 456882 BRO RichaKumari 311289 INDIA
eg : db Content
311289 RichaKumari TCS INDIA Bangalore [email protected]
Now in Final File I need something like this :
1 RichaKumari 311289 TCS INDIA
2 KiarnKumar RichaKumari 311289 TCS INDIA
2 Ravi RichaKumari 311289 TCS INDIA
here 1 and 2 are the identifier which will decide which layout to follow .

Hi,
I think you'll need two nested for-each's... Something like...
<xsl:for-each select="$root/row[layout = 1]">
<xsl:variable name="employee" select="."/>
... write employee ...
<xsl:for-each select="$root/row[layout = 2 and dependentid = $employee/id]">
... write dependent ...
</xsl:for-each>
</xsl:for-each>
Cheers,
Vlad

SQL Agent Permssions With SSIS for each loop not looping through files

Hi I am Having trouble getting a ssis package to iterate through a file directory using a proxy account set up in Sql Server.
So I have a package that loops over csv files. If I run the package manually or set the sql agent job to run as Sysadmin the pakage runs fine all files are processed.
However if I use the proxy account. the package completes but no files are processed. there are no failures, there is no error handling set up in the control flow. the for each loop container simply does not recognise any files when run under a proxy.
Note: the proxy is set up with credentials to access the folder. the credential is also set up with the server admin server role... still nothing
i'm not sure is this is a ssis or permissions problem?
Any comments will be appreciated

Can you check this and see if you've configured all the steps correctly
http://www.mssqltips.com/sqlservertip/2163/running-a-ssis-package-from-sql-server-agent-using-a-proxy-account/
Also enable logging in package and see if you're getting any error messages obtained in output table/file based on the logging option you chose.
Please Mark This As Answer if it helps to solve the issue Visakh ---------------------------- http://visakhm.blogspot.com/ https://www.facebook.com/VmBlogs

Stuck - load the Excel file dynamically with different columns and worksheet names

I have a situation where I want to load the Excel file dynamically, and the excel file have different columns or even worksheet name. Any idea how I could
approach this? I believe there's no way to modify the meta data (specifically the mapping) in the data flow.

Hi Chimumu,
The SSIS stock adapters don't supporting dynamic column mapping, to achieve your goal, you need to use Script Task/Component to read the Excel sheet name and the columns in the worksheet, and then map to the output columns of the script component. You can
also refer to the following blogs:
http://micktechblog.blogspot.com/2011/06/ssis-excel-import-with-unknown-number.html
http://wikiprogrammer.wordpress.com/2011/04/08/dynamic-column-mapping-in-ssis-part-1/
Regards,
Mike Yin
TechNet Community Support

Need help with Dynamic Excel File Name please.

I am try to output an excel file with dynamic date.
Here what I done.
I am using SQL 2012.
Create Execute SQL Task Connect Type: Excel
Create Data Flow Task set to DelayValidation: True
Create OLE DB Sourc
Create Data Converstion
Excel Destination
Excel Connection, Expression, select ExcelFilePath
@[User::sXLFilePath] + @[User::sFileName] + RIGHT("0" + (DT_WSTR, 2) DATEPART("DD", GETDATE()), 2)+ RIGHT("0" + (DT_WSTR, 2) DATEPART("MM", GETDATE()), 2) + RIGHT((DT_WSTR,
4) DATEPART("YYYY", GETDATE()), 2) +".csv"
C:\ExcelOutPut\SOX_CAM_SQL_Report_010215.xls
What I try to accomplish is output the file with each day append to it, date must be DDMMYY.
I google it and found many samples, tested it, and none of them is work for me.
Any suggestions or some examples to share is greatly appreciate.
I am new to SSIS. I found one poster have similar issue and inside the posted below, there was one suggestion to create variable and connection string but how do I bind that variable to Excel Connection manger.
Please help.
Thank you so much in advance.
Ex: SOX_CAM_SQL_Report _020215.csv
SOX_CAM_SQL_Report _030215.csv
--Similar issue:
https://social.msdn.microsoft.com/Forums/en-US/bda433aa-c8f8-47c9-9e56-efd20b8354ac/creating-a-dynamic-excel-file?forum=sqlintegrationservices
Suggestion in the above posted but where can bind this to Excel Connection Manger.
Please help provide step by step. Thanks.
"Provider=Microsoft.Jet.OLEDB.4.0;Data Source=c:\\temp\\" + "ExcelTarget" + (DT_WSTR,4)DATEPART("yyyy",GETDATE()) +
".xls" + ";Extended Properties=\"EXCEL 8.0;HDR=YES\";"
And yes, as you were intimating, the delay validation on the dataflow should be set.

Hi NguyenBL,
According to your description, you created ssis package to export data from database to excel, when the package runs, you want to create new excel and name the file with time stamp. If that is the case, we can achieve the goal by following steps:
Create a script task used to create excel files.
Create a data flow task to export data from database to excel.
Add OLE DB source to data flow task.
Add Excel Destination to data flow task.
Create connection manager for OLE DB and Excel.
Click Excel Connection Manager, in Properties window, click (…) button next to Expressions, then set ExcelFilePath with expression like below:
"C:\\ETL Lab\\CreateNewExcel\\ExportData_"+REPLACE((DT_STR, 20, 1252)(DT_DBTIMESTAMP)@[System::StartTime], ":", "")+".xls"
For detail information, please refer to the document:
https://sqljourney.wordpress.com/2013/01/12/ssis-create-new-excel-file-dynamically-to-export-data/
If you have any more questions, please feel free to ask.
Thanks,
Wendy Fu
Wendy Fu
TechNet Community Support

Creating a dynamic excel file

Is it possible that i can create a dynamic excel file (destination)
ex, i want to create a Dyanamic Excel destination file with a filename base on the date
this will run on jobs. Is this possible?
11172006.xls, 11182006.xls

This example is useful for loading data from an OLEDB source into a dynamically created Excel file.
NOTE:
This is the core functionality. Things like logging, checkpointing, documentation, etc., are at the user's discretion.
Steps:
1. Click on package properties. Set "DelayValidation" property to True.
The package will not validate tasks, connections, until they are executed.
2. Create a package level variable "XLFileRootDir" as string and set it to the root
directory where you want the excel file to be created.
Example: C:\\Project\Data\
3. Create an Excel connection in the connection manager. Browse to the target directory
and select the destination XL filename or type it in. It doesn't matter if the file doesn't exist.
4. Go to the Excel connection properties and expand the expressions ellipse (The button
with "..." on it).
Under the property drop down, select 'ExcelFilePath' and click on the ellipse to
configure the expression:
@[User::XLFileRootDir] + (DT_WSTR, 2) DATEPART("DD", GETDATE()) + (DT_WSTR, 2) DATEPART("MM", GETDATE()) + (DT_WSTR, 4) DATEPART("YYYY", GETDATE()) +".xls"
This should create an xl file like 01132007.xls.
5. Add a SQL task to package and double click to edit.
In the general tab, set 'ConnectionType' to 'Excel'.
For 'SQLStatement', enter the create table SQL to create destination table.
For example:
CREATE TABLE `Employee List` (
`EmployeeId` INTEGER,
`EmployeeName` NVARCHAR(20)
Copy the create table command. It will come in handy later.
6. Add a Data Flow task. In the data flow editor, add an OLEDB source and an Excel destination.
Configure the source to select EmployeeId and EmployeeName from a table.
7. Connect this to Excel destination. In the destination editor, select the Excel connection in the
manager, choose 'table or view' for data access mode and for 'name of the Excel sheet' click on
new button and paste the create table command from Step 5.
Map the columns appropriately in the mappings tab and you are done.
Let me know if you have any questions.
I was able to get this to work (dynamic Excel file name with date), but I want to process multiple
data files per day, so I must add hour and minute to the file names "..._2014.xls". But as soon as I add the hour/min to the 'ExcelFilePath' of the Connection Manager the Excel Destination task fails because it can't find the file.
So can you please help me how do i do this ?
Thank you

Need to read each sheet of an excel file and pass it to XI

Hi
Actually i need to read each sheet of an excel file and pass it to XI and will create one xml file .I am able to read the excel file by writting a custom module and it is generating the xml output from XI.But my requierment is to read the excel file and create the xml with the sheet name of the excel.suppose sheet name is sheet1, the xml file name will be sheet1.xml.if sheet name is sheet2 the xml file name will be sheet2 and so on....how can i pass this parameter from NWDS to SAP XI s i am passing the string which contains the sheet data.

If I understood correct you are already reading the XLS and creating the XML (not sure if this is a separate executing body out side of XI). Are you using any sort of Object Modal in Java to read the XML? I am aware of Visual Basic object modal for Excel but not sure if one is available in Java from Microsoft.
The Object modal that you are using must be providing an iterator to loop through each of the sheet in the Excel workbook and you should be able to create the XMLs as you desired.
I would be glad to look at you module if you are doing it once the XI file adapter has picked up the excel file.
VJ

Loading an Excel file into a Z-Table

Hi Gurus,
Can somebody guide me how to load an excel file into a Z-Table.

Hi,
You need to write a ABAP code for this....
The following is the code for loading data into an internal table:
types: begin of ttab,
       rec(1000) type c,
       end of ttab.
types: begin of tdat,
       fld1(10) type c,
       fld2(10) type c,
       fld3(10) type c,
       end of tdat.
data: itab type table of ttab with header line.
data: idat type table of tdat with header line.
data: file_str type string.
parameters: p_file type localfile.
at selection-screen on value-request for p_file.
call function 'KD_GET_FILENAME_ON_F4'
       exporting
            static    = 'X'
       changing
            file_name = p_file.
start-of-selection.
file_str = p_file.
call function 'GUI_UPLOAD'
       exporting
            filename                = file_str
       tables
            data_tab                = itab
       exceptions
            file_open_error         = 1
            file_read_error         = 2
            no_batch                = 3
            gui_refuse_filetransfer = 4
            invalid_type            = 5
            no_authority            = 6
            unknown_error           = 7
            bad_data_format         = 8
            header_not_allowed      = 9
            separator_not_allowed   = 10
            header_too_long         = 11
            unknown_dp_error        = 12
            access_denied           = 13
            dp_out_of_memory        = 14
            disk_full               = 15
            dp_timeout              = 16
            others                  = 17.
delete itab index 1.
loop at itab.
    clear idat.
    split itab-rec at cl_abap_char_utilities=>horizontal_tab
                          into idat-fld1
                               idat-fld2
                               idat-fld3.
    append idat.
endloop.
loop at idat.
    write:/ idat-fld1, idat-fld2, idat-fld3.
endloop.
From this internal store it onto the database.....
Sharma one more thing you must have posted this question in ABAP fourum for better results....
Hope it helps,
Thanks,
Happy Life,
Aravind

For Each Loop: Enumerator Configuration In the For each loop Editor Help needed

Hi All,
I have a package that runs and updates a table with everytime someone adds a flat file in the source folder. One of the requirement is to upload files with the following
ACA Flat files will have a naming convention of “ACA_ccyy_<julian date>” : How can I add this?
Under Files this is how I have setup the qualifies "ACA_*.*"
Is there anyway to pass the entire naming convention in the file section? If so how can I ?
Please help?
FM

As you have ACA_*.* set in Files Filter, it will return all the files from the folder which start with ACA_.
You can have one dummy Sequence Container inside FE Loop Container. Connect it to your starting task and do right click on Precedence Constraint and write the expression the way you want by using SSIS functions. If the expression is evaluated to True only
then it will go to the next task.
In this way you can skip Non-ACA files.
Please refer:
SSIS Skip Certain Files in a For Each Loop
- Vaibhav

For Each Loop Issue In SSIS

Hi All,
I have a SSIS Package which unzips the folder and then imports the files and then moves *.zip folders into processed location. If this task is executed as a part of total package its not moving the *.zip folders into processed location, but when we execute
for each loop alone this *.zip folders are moved into processed location. I am unable to figure out the issue here.
ZippedFile variable holds fully qualified file name..
Rohan
Rohan

Can you put a OnPreExec breakpoint on the file system task (Right click on the file system task > Edit Breakpoint... > Select the first (OnPreExec breakpoint) and when the pkg pauses @ the breakpoint, chk the value of the ZippedFle and FilePathDestination
variable values in your Local window (should be available at the bottom of the SSIS window as one of the tabs and you will need to expand the variable tree and find these two variables and their values). Are you seeing the appropriate values here?

For Each Loop Load Dynamic Excel Files

Similar Messages

Maybe you are looking for