How can I load a .xlsx File into a SQL Server Table using a Foreach Loop Container in SSIS?
I know I've REALLY struggled with this before. I just don't understand why this has to be soooooo difficult.
I can very easily do a straight Data Pump of a .xlsX File into a SQL Server Table using a normal Excel Connection and a normal Excel Source...simply converting Unicode to DT_STR and then using an OLE DB Destination of the SQL Server Table.
If I want to make the SSIS Package a little more flexible by allowing multiple .xlsX spreadsheets to be pumped in by using a Foreach Loop Container, the whole SSIS Package seems to go to hell in a hand basket. I simply do the following...
Put the Data Flow Task within the Foreach Loop Container
Add the Variable Mapping Variable User::FilePath that I defined as a Variable and a string within the FOreach Loop Container
I change the Excel Connection and its Expression to be ExcelFilePath ==> @[User::FilePath]
I then try and change the Excel Source and its Data Access Mode to Table Name or view name variable and provide the Variable Name User::FilePath
And that's when I run into trouble...
Exception from HRESULT: 0xC02020E8
Error at Data Flow Task [Excel Source [56]]:SSIS Error Code DTS_E_OLEDBERROR. An OLE DB error has occured. Error code: 0x80004005.
Error at Data Flow Task [Excel Source [56]]: Opening a rowset for "...(the EXACT Path and .xlsx File Name)...". Check that the object exists in the database. (And I know it's there!!!)
I don't understand by adding a Foreach Loop Container to try and make this as efficient as possible has caused such an error unless I'm overlooking something. I have even tried delaying my validations and that doesn't seem to help.
I have looked hard in Google and even YouTube to try and find a solution for this but for the life of me I cannot seem to find anything on pumping a .xlsX file into SQL Server using a Foreach Loop Container.
Can ANYONE please help me out here? I'm at the end of my rope trying to get this to work. I think the last time I was in this quandry, trying to pump a .xlsX File into a SQL Server Table using a Foreach Loop Container in SSIS, I actually wrote a C# Script
to write the contents of the .xlsX File into a .csv File and then Actually used the .csv File to pump the data into a SQL Server Table.
Thanks for your review and am hoping and praying for a reply and solution.
Hi ITBobbyP,
If I understand correctly, you want to load data from multiple sheets in an .xlsx file into a SQL Server table.
If in this scenario, please refer to the following tips:
The Foreach Loop container should be configured as shown below:
Enumerator: Foreach ADO.NET Schema Rowset Enumerator
Connection String: The OLE DB Connection String for the excel file.
Schema: Tables.
In the Variable Mapping, map the variable to Sheet_Name, and change the Index from 0 to 2.
The connection string for Excel Connection Manager is the original one, we needn’t make any change.
Change Table Name or View name to the variable Sheet_Name.
If you want to load data from multiple sheets in multiple .xlsx files into a SQL Server table, please refer to following thread:
http://stackoverflow.com/questions/7411741/how-to-loop-through-excel-files-and-load-them-into-a-database-using-ssis-package
Thanks,
Katherine Xiong
Katherine Xiong
TechNet Community Support
Similar Messages
-
So we have an Invoice .xlsx File from a 3rd party vendor. It contains two worksheets..."Enrolled" and "Engaged". The data and data columns in each worksheet is different. Is it possible to loop through multiple .xlsx files using SSIS
and a Foreach Loop Container for each spreadsheet, and then another Foreach Loop Container to control each worksheet, and pump the Excel data into a SQL Server Table first for "Enrolled" and then for "Engaged"? How can I control the Foreach
Loop Container in SSIS to process ONLY the "Enrolled" worksheet first? And then the "Engaged" worksheet next?
I know I have multiples out here and I apologize for that...but right now it seems as though I take three steps forward and then two back.
Any help would be GREATLY appreciated!
Thanks in advance!If the structure of the Excel sheets does not change from file to file then you can by having one ForEach Loop processing always the "Enrolled" sheet and another always the "Engaged" this is doable because the Excel OLEDB connector allows
to pick individual sheets, it is problematic therefore when sheet names themselves change.
MSDN has an example: https://msdn.microsoft.com/en-ca/library/ms345182.aspx
Arthur
MyBlog
Twitter -
How can I load a pdf-file into a byte[]?
Hello,
I would like to load a PDF-File into a byte-Array. How can I do this? The standard Streams shouldn't work because this can't be converted into characters. Can it? So, what does work?
Any idea?
Bye,
WolfangWhy would someone not do:
ByteArrayOutputStream baos = new ByteArrayOutputStream();
InputStream is = new FileInputStream("foo.pdf");
byte[] buf = new byte[4096];
for(int len=-1;(len=is.read(buf))!=-1;)
baos.write(buf,0,len);
is.close();
baos.flush();
baos.close();
buf = baos.toByteArray(); // Here are you bytes!You don't need to know the exact size, let ByteArrayOutputStream do it for you. Streams are generally for bytes, Readers/Writers generally for characters. Why exactly did you say: ``The standard Streams shouldn't work because this can't be converted into characters?'' -
How can I load a .TXT file into a dynamic text box?
I am sure that many people know how to load a .txt file into
a dynamic text box. But I do not. I want to be able to reference a
txt file from the server into the text box. So that when I change
the text file it changes in the flash movie without even editing
the flash file itself. Thank you.http://www.oman3d.com/tutorials/flash/loading_external_text_bc/
I think this is the simplest way to go :) -
How can I load a flat file into a ZTABLE dynamically
I need to create a program which can Load a ZTABLE from a flat file structure (delimited and fixed options required). We have many ZTables where this will be required so I was hoping to do it dynamically somehow. Otherwise I will have to create one ABAP for every ZTable we have to load.
My Inputs should be
PARAMETERS: p_ztable TYPE ddobjname, "Z Table Name
p_infile(132) LOWER CASE, "File Name
p_delim(1). "Delimiter
I know that I can read the file by using gui_upload
CALL FUNCTION 'GUI_UPLOAD'
EXPORTING
filename = c_infile
has_field_separator = p_delim
TABLES
data_tab = indata
EXCEPTIONS
file_open_error = 1
file_read_error = 2
OTHERS = 9.
I know that I can split the contents of this file (if a delimiter is used). However I will not know the actual field names of the table until runtime as to what to fields to split it into. In the example below I have the actual table (t_rec) and each of the fields (pernr, lgart, etc) in the code but each table I
need to load will be different – it will have a different # of fields as well.
FORM read_data_pc.
LOOP AT indata.
PERFORM splitdata USING indata.
APPEND t_rec.
CLEAR t_rec.
ENDLOOP.
ENDFORM.
FORM splitdata USING p_infile.
SPLIT p_infile AT p_delim INTO
t_rec-pernr "Employee #
t_rec-lgart "Wage Type
t_rec-begda "Effective date
t_rec-endda. "End date
ENDFORM.
Once I split the data into the fields then I can just look and insert the record.
Does anyone have any ideas? Specific code examples would be great if you do. Thx!!Hi janice,,
Try this sample code where you can upload data from a flat file into the internal table.
REPORT z_test.
TABLES: mara.
FIELD-SYMBOLS : <fs> .
DATA : fldname(50) TYPE c.
DATA : col TYPE i.
DATA : cmp LIKE TABLE OF rstrucinfo WITH HEADER LINE.
DATA: progname LIKE sy-repid,
dynnum LIKE sy-dynnr.
DATA itab TYPE TABLE OF alsmex_tabline WITH HEADER LINE.
DATA: BEGIN OF ZUPLOAD1_T OCCURS 0 ,
matnr like mara-matnr,
ersda like mara-ersda,
ernam like mara-ernam,
laeda like mara-laeda,
END OF ZUPLOAD1_T.
*DATA: ZUPLOAD1_T LIKE mara OCCURS 0 WITH HEADER LINE.
DATA: wa_data LIKE TABLE OF ZUPLOAD1_T WITH HEADER LINE.
selection-screen
SELECTION-SCREEN: BEGIN OF BLOCK blk WITH FRAME TITLE text-001.SELECTION-SCREEN : SKIP 1. PARAMETERS : p_file LIKE rlgrap-filename.SELECTION-SCREEN : SKIP 1.SELECTION-SCREEN : END OF BLOCK blk
. AT SELECTION-SCREEN ON VALUE-REQUEST FOR p_file.
F4 Value for File
CALL FUNCTION 'KD_GET_FILENAME_ON_F4' EXPORTING
PROGRAM_NAME = SYST-REPID
DYNPRO_NUMBER = SYST-DYNNR
FIELD_NAME = ' '
static = 'X'
MASK = ' '
CHANGING file_name = p_file EXCEPTIONS mask_too_long = 1 OTHERS = 2
. IF sy-subrc <> 0.
MESSAGE ID SY-MSGID TYPE SY-MSGTY NUMBER SY-MSGNO
WITH SY-MSGV1 SY-MSGV2 SY-MSGV3 SY-MSGV4.
ENDIF.
START-OF-SELECTION.
CALL FUNCTION 'ALSM_EXCEL_TO_INTERNAL_TABLE'
EXPORTING filename = P_FILE
i_begin_col = 1
i_begin_row = 1
i_end_col = 5
i_end_row = 12507
tables
intern = ITAB
EXCEPTIONS
INCONSISTENT_PARAMETERS = 1
UPLOAD_OLE = 2
OTHERS = 3. .
IF sy-subrc <> 0.
MESSAGE ID SY-MSGID TYPE SY-MSGTY NUMBER SY-MSGNO
WITH SY-MSGV1 SY-MSGV2 SY-MSGV3 SY-MSGV4.
ENDIF.
CALL FUNCTION 'GET_COMPONENT_LIST'
EXPORTING
program = SY-REPID
fieldname = 'ZMARA'
tables
components = CMP.
LOOP AT itab. AT NEW row.
IF sy-tabix = 1. APPEND ZUPLOAD1_T.
ENDIF.
ENDAT.
col = itab-col.
READ TABLE cmp INDEX col.
CONCATENATE 'ZUPLOAD1_T-' cmp-compname INTO fldname.
ASSIGN (fldname) TO <fs>.
<fs> = itab-COL.
APPEND ZUPLOAD1_T. ENDLOOP.
DELETE ZUPLOAD1_T where matnr eq space.
LOOP AT ZUPLOAD1_T INTO wa_data.
insert mara from wa_data .
WRITE: / ZUPLOAD1_T-matnr, 20 ZUPLOAD1_T-ersda , 45 ZUPLOAD1_T-ernam, 55 ZUPLOAD1_T-laeda.
*HERE IAM JUST CHECKING I NEED TO UPDATE A ZTABLE
ENDLOOP.
insert ZMARA FROM table itab ACCEPTING DUPLICATE KEYS.
I have tried it for mara.Please let me know whether it was helful.
Regards,
Kannan -
I have written a routine that loads a Hex file, checks for record marker,length,address etc then loads the data into a table on front panel. However, my routine only loads single record hex files. I need to be able to load multiple record files, I need to identify the ':' colon record marker start for each record in the hex file, I just can't see how to loop my routine so that all records with ':' prefix marker are loaded.
Jeff,
I'm not exactly sure of what you are trying to do. Is it possible you could post your code, or a screen shot of the diagram in question, so that I can get a better understanding, and possibly offer some assistance?
I am familiar with hex files, but not with the : marker separating records. This may be able to be solved with simple text parsing, and as I am not familiar with how you are retrieving and/or displaying the files, I require more information.
Thanks -
How can I upload a pdf file into the application server?
Hi,
I have the otf data, which i have converted into pdf using the funcation module "Convert_otf". Is it possible to upload the pdf file in application server?Dear Srishti,
Use OPEN DATASET in BINARY MODE.
following link will help you.
http://scn.sap.com/thread/1480434
thanks,
vidyasagar -
Data Pump .xlsx into a SQL Server Table and the whole 32-Bit, 64-Bit discussion
First of all...I have a headache!
Found LOTS of Google hits when trying to data pump a .xlsx File into a SQL Server Table. And the whole discussion of the Microsoft ACE 64-Bit Driver or the Microsoft Jet 32-Bit Driver.
Specifically receiving this error...
An OLE DB record is available. Source: "Microsoft Office Access Database Engine" Hresult: 0x80004005 Description: "External table is not in the expected format.".
Error: 0xC020801C at Data Flow Task to Load Alere Coaching Enrolled, Excel Source [56]: SSIS Error Code DTS_E_CANNOTACQUIRECONNECTIONFROMCONNECTIONMANAGER. The AcquireConnection method call to the connection manager "Excel Connection Manager"
failed with error code 0xC0202009.
Strangely enough, if I simply data pump ONE .xlsx File into a SQL Server Table utilizing my SSIS Package, it seems to work fine. If instead I am trying to be pro-active and allowing for multiple .xlsx Files by using a Foreach Loop Container and a variable
@[User::FileName], it's erroring out...but not really because it is indeed storing the rows onto the SQL Server Table. I did check all my Delay
Why does this have to be sooooooo difficult???
Can anyone help me out here in trying to set-up a SSIS Package in a rather constrictive environment to pump a .xlsx File into a SQL Server Table? What in God's name am I doing wrong? Or is all this a misnomer? But if it's working how do I disable the error
so that is stops erroring out?Hi ITBobbyP,
According to your description, when you import data of .xlsx file to SQL Server database, you got the error message.
The error can be caused by the following reasons:
The excel file is locked by other processes. Please kindly resave this file and name it to other file name to see if the issue will be fixed.
The ACE(Access Database Engine) is not up to date as Vaibhav mentioned. Please download the latest ACE and install it from the link:
https://www.microsoft.com/en-us/download/details.aspx?id=13255.
The version of OFFICE and server bitness is not the same. To solve the problem, please refer to the following document:
http://hrvoje.piasevoli.com/2010/09/01/importing-data-from-64-bit-excel-in-ssis/
If you have any more questions, please feel free to ask.
Thanks,
Wendy Fu
Wendy Fu
TechNet Community Support -
[Forum FAQ] How to calculate the total count of insert rows within a Foreach Loop Container in SSIS?
Introduction
We need to loop through all the flat files that have the same structure in a folder and import all the data to a single SQL Server table. How can we obtain the total count of the rows inserted to the destination SQL Server table?
Solution
We can use Execute SQL Task or Script Task to aggregate the row count increment for each iteration of the Foreach Loop Container. The following steps are the preparations before we add the Execute SQL Task or Script Task:
Create a String type variable FilePath, two Int32 type variables InsertRowCnt and TotalRowCnt.
Drag a Foreach Loop Container to the Control Flow design surface, set the Enumerator to “Foreach File Enumerator”, specify the source folder and the files extension, and set the “Retrieve file name” option to “Fully qualified”.
On the “Variable Mappings” tab of the container, map the variable FilePath to the collection value.
Drag a Data Flow Task to the container, in the Data Flow Task, add a Flat File Source, a Row Count Transformation, and an OLE DB Destination, and join them. Create a Flat File Connection Manager to connect to one of the flat files, and then configure the
Flat File Source as well as the OLE DB Destination adapter. Set the variable for the Row Count Transformation to “User::InsertRowCnt”.
Open the Property Expressions Editor for the Flat File Connection Manager, and set the expression of “ConnectionString” property to
“@[User::FilePath]”.
(I) Execute SQL Task Method:
In the Control Flow, drag an Execute SQL Task under the Data Flow Task and join them.
Create one or using any one existing OLE DB Connection Manager for the Execute SQL Task, set the “ResultSet” option to “Single row”, and then set the “SQLStatement” property to:
DECLARE @InsertRowCnt INT,
@TotalRowCnt INT
SET @InsertRowCnt=?
SET @TotalRowCnt=?
SET @TotalRowCnt=@InsertRowCnt+@TotalRowCnt
SELECT TotalRowCnt=@TotalRowCnt
On to parameter 1.
On the “Result Set” tab of the Execute SQL Task, map result 0 to variable “User::TotalRowCnt”.
(II) Script Task Method:
In the Control Flow, drag a Script Task under the Data Flow Task and join them.
In the Script Task, select variable InsertRowCnt for “ReadOnlyVariables” option, and select variable TotalRowCnt for “ReadWriteVariables”.
Edit the Main method as follows (C#):
public void Main()
// TODO: Add your code here
int InsertRowCnt = Convert.ToInt32(Dts.Variables["User::InsertRowCnt"].Value.ToString()
int TotalRowCnt = Convert.ToInt32(Dts.Variables["User::TotalRowCnt"].Value.ToString());
TotalRowCnt = TotalRowCnt + InsertRowCnt;
Dts.Variables["User::InsertRowCnt"].Value = TotalRowCnt;
Dts.TaskResult = (int)ScriptResults.Success;
Or (VB)
Public Sub Main()
' Add your code here
Dim InsertRowCnt As Integer =
Convert.ToInt32(Dts.Variables("User::InsertRowCnt").Value.ToString())
Dim TotalRowCnt As Integer =
Convert.ToInt32(Dts.Variables("User::TotalRowCnt").Value.ToString())
TotalRowCnt = TotalRowCnt + InsertRowCnt
Dts.Variables("User::TotalRowCnt").Value = TotalRowCnt
Dts.TaskResult = ScriptResults.Success
End Sub
Applies to
Microsoft SQL Server 2005
Microsoft SQL Server 2008
Microsoft SQL Server 2008 R2
Microsoft SQL Server 2012
Microsoft SQL Server 2014
Please click to vote if the post helps you. This can be beneficial to other community members reading the thread.Hi ITBobbyP,
If I understand correctly, you want to load data from multiple sheets in an .xlsx file into a SQL Server table.
If in this scenario, please refer to the following tips:
The Foreach Loop container should be configured as shown below:
Enumerator: Foreach ADO.NET Schema Rowset Enumerator
Connection String: The OLE DB Connection String for the excel file.
Schema: Tables.
In the Variable Mapping, map the variable to Sheet_Name, and change the Index from 0 to 2.
The connection string for Excel Connection Manager is the original one, we needn’t make any change.
Change Table Name or View name to the variable Sheet_Name.
If you want to load data from multiple sheets in multiple .xlsx files into a SQL Server table, please refer to following thread:
http://stackoverflow.com/questions/7411741/how-to-loop-through-excel-files-and-load-them-into-a-database-using-ssis-package
Thanks,
Katherine Xiong
Katherine Xiong
TechNet Community Support -
How can I load an external SWF into a movie clip that's inside other movie clip?
Hi.
I creating my first flash (actionscript 3.0) website but I'm
stuck with a visual effect I want to create.
I have a window on my website called contentWindow. Every
time you click a button this window is supposed to leave the stage,
load the requested content and return to the stage.
The sliding window is a movie clip with 83 frames, 21 to
enter the stage, 21 to leave the stage again, 20 for nothing (its
just to simulate the loading time) and 21 to return to the stage.
Now my goal is, when the user clicks on a navigation button,
the window exits the stage, loads an external SWF with the content,
and then returns to the stage.
I've the "window" movie clip with an instance name of
"contentWindow". Inside there is another movie clip with an
instance name of "contentLoader". The content that the user
requested should appear inside the "contentLoader".
Now, when the contentWindow leaves the stage, I get this
error message:
quote:
TypeError: Error #1009: Cannot access a property or method of
a null object reference.
at rwd_fla::MainTimeline/trigger()
If I switch
"contentWindow.contentLoader.addChild(navLoader);" for
"contentWindow.addChild(navLoader);" it works fine, but the
external SWF doesn't move with the window.
How can I load an external SWF into a movie clip that's
inside other movie clip?Hi,
Recently, I have been putting together a flash presentation.
And I am just wondering if the following might help you, in your
communication with the said swf file:
loader.contentLoaderInfo.addEventListener(Event.COMPLETE,
onComplete);
function onComplete(event:Event):void
event.target.content.thinggy_mc.y -= 100;
Not the best example, but this allows you to target a mc
within an external swf file. I think if you look up this code, you
will have an answer ;)
Kind Regards,
Boxing Boom -
How to import the data in a .xls or .xlsx file into a oracle database table
Hi,
Please tell me how to import the data in a .xls or .xlsx file into a oracle database table in Oracle 10gR2 using Oracle Warehouse Builder 10gR2.....can we do something through Non-Oracle->ODBC?Yes, it is possible, look at this thread
[SQLServer access from AIX Warehouse builder|http://forums.oracle.com/forums/thread.jspa?messageID=2502982]
If your server (with target DB and OWB runtime) is on Windows OS this configuration will be simpler - you can use single server.
And additional link on OWB blog (with 11g transparent gateway)
[http://blogs.oracle.com/warehousebuilder/2008/01/11g_heterogeneous_agent.html]
(configuring nonoracle connection with 10g generic connectivity very similar to 11g gateway)
Also look at
http://asktom.oracle.com/pls/asktom/f?p=100:11:0::::P11_QUESTION_ID:4406709207206#18830681837358
Regards,
Oleg -
How do you load exs. format files into EXS 24 sampler?
How do you load EXS24 format files into the EXS24 sampler? the only format I can load directly is aiff and wav. can't find any posts addressing this question? please help
You don't load them in, you need to put them in:
*System/users/your user name/library/application support/logic/Sampler instruments*
Restart Logic if it's running, they will then appear in the EXS library. -
How can I turn garage band files into mp3s?
On my older Macbook i have always been able to easily turn my .band files into mp3s under the Advanced heading in itunes, but with my Macbook Pro, that is not an option. The option has become "Create AAC version". How can I turn my .band files into mp3s so that I can load them onto my web page and my students can pick them up as mp3s and put them on their devices?
The option has become "Create AAC version".
Your iTunes importing preferences are currently set to AAC. If you change your Importing preferences to mp3, the option should then become "Create mp3 version". See the following document for some more details:
iTunes: How to convert a song to a different file format -
How can i turn a photoshop file into a website with knowing code? can i upload Muse or DreamWeaver?
How can i turn a photoshop file into a website with knowing code? can i upload Muse or DreamWeaver?
I created a few images (pages for a site) on photoshop and want to turn them into HTML code for a website- Can i Just upload the file into one of these programs and make a file with the correct code imediatly?
I was reading about using EDGE Reform but i did not understand if i can create the code with it, in the end it says it still needs to be sent to website developer after?You need to look at Dreamweaver as a code editor. You need to learn to create good, clean HTML code with style sheets to make websites with Dreamweaver.
There are tutorials that will take you from a layered .PSD file to HTML. Here's a good one:
http://net.tutsplus.com/tutorials/site-builds/from-psd-to-html-building-a-set-of-website-d esigns-step-by-step/
Adobe's Muse is a semi-WYSIWYG website layout creator for small websites (1-5 pages). But there is no way you can go from a .PDS file to Muse, you have to start the site in Muse.
I have seen a few Muse sites and they're OK, but there is some code that Muse creates that is kind of odd. If you want a good, clean website that will work for you or your business, Dreamweaver is the best choice. Alternatively, you could hire a pro.
-Mark -
How can I convert a pdf file into a usable xcel file
how can I convert a pdf file into a xcel file. the pdf file file is layed out in colums, justified and separated in various types - i need to be able to set colum limits manualy, thanks for your input/
Without further information, I would suggest using either Acrobat or the Export PDF service.
Maybe you are looking for
-
Java.io.File causes "access denied" exception in a signed applet
Hi, New to these forums and not entirely where it's appropriate to post this issue, so I'll stick it here for now until told otherwise. The problem: My applet throws the following exception. INFO: Exception Message: access denied (java.io.FilePermiss
-
PLease I need help. My ipad mini wont fully charge. It says 100 % charged but its still shows the lightning bolt
-
Calling SQL stored procedure dynamically
Hallo, could anybody tell me how to call HANA SQL stored procedure dynamically? The coding looks like this: v_ProcedureName := '"' || KpiNamespace || '::' || KpiName || '"'; v_SQL := 'CALL ' || :v_ProcedureName || '( lt_outKPI )'; EXEC :v_SQL; For th
-
Reading the boot sector from an applet
I want to be able to read the boot sector of a floppy on a Wintel machine from an applet. I realize that I'll need to sign the applet to relax the security restrictions. I suppose that I'll need to use JNI and do the actual disk accesses in C. If tha
-
Error : No available resource. Wait-time expired.
Hi, We have a big problem of connection pooling in SunOne. After out program execute a lot of sql command in a short time, SunOne 7.0 throws an Exception ---- No available resource. Wait-time expired. We modify the connection pool setting, but it doe