SSIS 2012 - Importing multiple files with different columns (where columns are stored in a table)

Hi everyone,
I have been tasked to upgrade an existing DTS package into an SSIS package.
The old package uses 3 tables.
Table 1) Has a list of file names
Table 2) Has a list of columns per file
Table 3) Is the data destination.
So what is supposed to happen is:
Step 1) the package should load the files into a resultset
Step 2) for each file in the resultset, the package should get the related columns
Step 3) the package should now map the columns to the destination table (variable column numbers here)
Step 4) the import should take place.
I have gotten to step 2 but am not sure how to dynamically map the columns to the destination table.
Any pointers anyone?
Thanks in advance!

Hi Tonyhektic,
What’s the nature of Table 2, does it only store the column names for each file? I don’t know how it is implemented in DTS package, however, in SSIS package, dynamic columns mapping is not supported. To work around this issue, we need to use Script component
to do the columns mapping. You can refer to Sorna’s script above or have a look at the following blogs:
http://munishbansal.wordpress.com/2009/06/09/dynamic-columns-mapping-%E2%80%93-script-component-as-destination-ssis/
http://blog.quasarinc.com/ssis/best-solution-to-load-dynamically-change-csv-file-in-ssis-etl-package/
Regards,
Mike Yin
TechNet Community Support

Similar Messages

Creating pdf-file from multiple files with different orientation

How do I put together two files with different orientation in the page set up so that the lying A4-format is standing up in the new pdf-file?

lophelia wrote:
How do I put together two files with different orientation in the page set up so that the lying A4-format is standing up in the new pdf-file?
First I've underline a Phrase in your Question did you mean underlying A4-format?
Are both PDF Files created with A4-Format? If yes then:
Did you shift orientation to Portriat Format for the item(s) that you need, such as a Chart or an excel Document? If so:
Then when the documents are meged They should print correctly.

Batch Rename Multiple files with different names

Hi,
Is there any way to batch rename multiple files with individual names? I.e
IMG_123 changed to RSP45AS
IMG_124 changed to MOL157A
IMG_125 changed to AGKH135
IMG_126 changed to MNOLH13
IMG_127 changed to ASFBLUG
Etc.
Are they any programs or scripts or plug ins that would do that job?
Thanks

HI there Onemorewave,
It looks like there is a batch remname feature included in OS X 10.10 Yosimte if you are planing on updating.
Rename files, folders, and disks - Mac Help
Rename multiple items
Select the items, then Control-click one of them.
In the shortcut menu, select Rename Items.
In the pop-up menu below Rename Folder Items, choose to replace text in the names, add text to the names, or change the name format.
Replace text: Enter the text you want to remove in the Find field, then enter the text you want to add in the “Replace with” field.
Add text: Enter the text to you want to add in the field, then choose to add the text before or after the current name.
Format: Choose a name format for the files, then choose to put the index, counter, or date before or after the name. Enter a name in the Custom Format field, then enter the number you want to start with.
Click Rename.
Note: To batch rename, you would want to choose the "Format" option.
-Griff W

When I take a screenshot (command, shift, 4), I get an error: "you don't have permission to save files in the location where screenshots are stored". Help!

I used to be able to take screenshots just fine.
Now, when I try to take a screenshot (using command + shift + 4) I get an error stating that my screenshot cannot be saved, because I do not have permission to save where they are stored. HELP!

If not running Lion, please do this:
With the Finder active, press the following key combinations, in this order:
Shift-Command-H
Command-3
Command-I
The Info window for your home folder will open. Click the lock icon to authenticate. In the Sharing & Permissions section, you should have read & write access. If not, make it so. Regardless of whether you made any changes, select "Apply to enclosed items" from the action (gear icon) menu.
If running Lion, do this:
Boot from your recovery partition by holding down the key combination command-R at startup. Release the keys when you see a gray screen with a spinning dial.
When the recovery desktop appears, select Utilities > Terminal from the menu bar.
In the Terminal window, enter “resetpassword” (without the quotes) and press return. A “Reset Password” window opens.
Select your boot volume if not already selected.
Select your username from the menu labeled “Select the user account” if not already selected.
Under “Reset Home Directory Permissions and ACLs”, click the “Reset” button.
Select Apple menu > Restart from the menu bar.

Creating Multiple Files with different name at Receiver side

Gurus -
Can anybd tell me how to generate multiple files at the Receiver side in the given target directory with different file names?
One source message will contain number "N" using which we have to generate files "N" number of times at the Receiver side.
Kindly help.

hi,
are you following this blog?
provided you have SP>14 and no IDOC/HTTP involved.
and have you got the error you mentioned when testing with sample payload?
from where did you take the payload? if is from SXMB_MONI make sure you dont have <MESSAGE> tags in the payload,generally systems adds <MESSAGE> tags when you do multi mapping.
get the payload from SXMB_MONI right click ->view source payload(this is raw data) and test it.
it should be ok now.
and refer my discussion in the thread ,as I got he same error.
Split mapping created no messages -Mluti Mapping

9iFS: import multiple files with common attributes

Hello,
Customer wants to apply attributes to a set of contributed documents, all at once.
They also want to recursively apply partial attributes to sets of folders / documents.
Did I miss this fundamental feature? Does it require customization?

The problem is that I cannot change the pages size before I transform them to PDF files. The crop tool will not do, because I have to change the pages proportionally. With this tool I can crop only the page, the image will maintain its original size...
Thanks anyway :-)

To create multiple files with same content but with different names

Hi SapAll.
here i have got a tricky situation on Idoc to File Scenario.
in my interface of an Idoc to file ,there is requirement to create multiple files with different file names but with same content based on one Idoc Segment.
which means there will be one Zsegment with two fields in the idoc,where one field with (content refers to the name which file name should start with .so lets say if this segment is repeated for 3 times then PI should create 3 files in the same directory with same content but with different file names (from the filed).
so here for now iam using one reciever file communication channel.
can any body give me the quick answer.
regards.
Varma

What do you mean by different names?
when i make proper setting in the Receiver Channel....on how to create the filename (what to append) like add Timestamp, counter, date, messageid.....even in this case you will ahve file with different names and that too from same File channel.
You can perform multi-mapping in XI/ PI and then your File channel will place the files in the target folder with relevant names. You cannot use Dynamic Configuration with Multi-Mapping!
If you intend to use different File channels, then do the configuration as required (normal)...even over here you can follow multi-mapping.
Do not use a BPM!
Regards,
Abhishek.

SSIS - Import Multiple flat files with different metadata

Hi ,
I have set of flat files with different metadata structure, I would like to load them into staging tables.
1. ) Can we load the flatfiles into the staging tables with out having multiple data flow task.
2.) If possible , can we programmatically select the staging table based on the metadata of the flatfile and load them.
Please advise.
Thanks
Thiya

Nope in SSIS a data flow task needs to have a fixed metadata. So if your file metadata varies then best option would be use OPENROWSET syntax to pull the data and populate into your staging table. You may also use
SELECT .. INTO StagingTable ... FROM OPENROWSET (...)
syntax to create staging table at runtime based on the file metadata
http://sqlmate.wordpress.com/2012/08/09/use-your-text-csv-files-in-your-queries-via-openrowset/
If you want to do this in SSIS you need to create data flow dynamically using script task and build the metadata
see
http://www.selectsifiso.net/?p=288
Please Mark This As Answer if it helps to solve the issue Visakh ---------------------------- http://visakhm.blogspot.com/ https://www.facebook.com/VmBlogs

Using ssis import a multiple flat files with different mappings

I have an scenario for import a file
20 different flat file source and having XML mapping document in a table.
get files from that path(i have path and file name extension in table)
i have map this file based on the file name extension in dynamically
create a 20 different staging table based on mapping in dynamically
kindly help for this

Hi Karthick,
As Arthur said, if you don't want to hard code the data flow, you need to read XML from the SQL Server table and use Script Task to implement the dynamic columns mapping. To read XML from a SQL table, you can refer to:
http://blog.sqlauthority.com/2009/02/13/sql-server-simple-example-of-reading-xml-file-using-t-sql/
In the Script Task, we need to parse the first row of the flat file to get the columns information and save that to a .NET variable. Then, the variable will be used to create the query that will be used to create SQL table. For more information, please see:
http://www.citagus.com/citagus/blog/importing-from-flat-file-with-dynamic-columns/
Regards,
Mike Yin
TechNet Community Support

SSIS project - read multiple flat files with different formats

hi all,
i need to import multiple flat files with different formats into different tables of the sql server database and not able to figure out the best way out in ssis to do so...
please advise the possible methods in ssis to do so and if possible the process which can be dynamic as file names or columns might change in future.

Hi AK1987,
To import flat files with dynamic columns, we can use Script Task inside a Foreach Loop Container to parse the first row of the flat file to get the columns names and save them into a .NET variable, then, we can create “Create Table” script based on this
variable, and then store the script into a SSIS package variable. After that, we create a staging table based on the package variable, load the flat file data to the staging table. Eventually, we load data from the staging table to the destination table. For
the detail steps, please walk through the following blog:
http://www.citagus.com/citagus/blog/importing-from-flat-file-with-dynamic-columns/
Regards,
Mike Yin
TechNet Community Support

BO xi 3.1 u2013 schedule a template for multiple reports with different column

Hi,
I need to migrate the existing reports from SQL Server Reporting Services to BO, and the column order in the report is very important to the clients since clients have existing code to ingest these reports into their database. Now I want to schedule a single template with different parameters for all reports in BO, or create a single template with lot of parameters for all reports. My question is how I might use one template to create multiple reports with different column orders in the reports?
For example, client A has the following column order in their report: firstName, LastName, DateBirth, SSN. Client B has the following column order in their report: SSN, DateBirth, LastName, FirstName. Can I use one template to create these multiple reports?
Thanks!

Hi,
The only approach I can think of is to create a template report which uses variables
For each column you would need to variable
v_columnAName and v_columnAValue
v_columnAName would have a if statement in it
=if([client]="clientA" or [client]="clientC";NameOf([firstName]);if([client]="clientB";NameOf([SSN]);NameOf([lastName]));
v_columnA would have a if statement in it
=if([client]="clientA" or [client]="clientC";[firstName];if([client]="clientB";[SSN],[lastName]));
This would only work when you had a small set of clients.
This might be more managable if it was done in the universe
Regards
Alan

SSIS 2012 is intermittently failing with below "Invalid date format" while importing data from a source table into a Destination table with same exact schema.

We migrated Packages from SSIS 2008 to 2012. The Package is working fine in all the environments except in one of our environment.
SSIS 2012 is intermittently failing with below error while importing data from a source table into a Destination table with same exact schema.
Error: 2014-01-28 15:52:05.19
   Code: 0x80004005
   Source: xxxxxxxx SSIS.Pipeline
   Description: Unspecified error
End Error
Error: 2014-01-28 15:52:05.19
   Code: 0xC0202009
   Source: Process xxxxxx Load TableName [48]
   Description: SSIS Error Code DTS_E_OLEDBERROR. An OLE DB error has occurred. Error code: 0x80004005.
An OLE DB record is available. Source: "Microsoft SQL Server Native Client 11.0" Hresult: 0x80004005 Description: "Invalid date format".
End Error
Error: 2014-01-28 15:52:05.19
   Code: 0xC020901C
   Source: Process xxxxxxxx Load TableName [48]
   Description: There was an error with Load TableName.Inputs[OLE DB Destination Input].Columns[Updated] on Load TableName.Inputs[OLE DB Destination Input]. The column status returned was: "Conversion failed because the data value overflowed
the specified type.".
End Error
But when we reorder the column in "Updated" in Destination table, the package is importing data successfully.
This looks like bug to me, Any suggestion?

Hi Mohideen,
Based on my research, the issue might be related to one of the following factors:
Memory pressure. Check there is a memory challenge when the issue occurs. In addition, if the package runs in 32-bit runtime on the specific server, use the 64-bit runtime instead.
A known issue with SQL Native Client. As a workaround, use .NET data provider instead of SNAC.
Hope this helps.
Regards,
Mike Yin
If you have any feedback on our support, please click
here
Mike Yin
TechNet Community Support

Design a single template for multiple reports with different column order

Hi,
I need to migrate the existing reports from SQL Server Reporting Services to Crystal Report 2008, and the column order in the report is very important to the clients since clients have existing code to ingest these reports into their database. Now I want to create a single template with lot of parameters for all reports. My question is how I might use one template to create multiple reports with different column orders in the reports?
For example, client A has the following column order in their report: firstName, LastName, DateBirth, SSN. Client B has the following column order in their report: SSN, DateBirth, LastName, FirstName. Can I use one template to create these multiple reports?
Thanks in advance.

-As I mentioned you can find Template Field object in Insert menu of crystal report Designer 2008.
As soon as you select Template Field object you will be able to see a box beside ur mouse cursor. Drag and drop this box at required place. Now when you place this object in report you will get some thing like this "<TemplateField1>" listed under Formula Fields tree. Double click "<TemplateField1>" formula to add your conditions.
-I have not tested it for large amount of Customers. However as the number of customer increases the number of conditions will get increased.
For the customer that want the same ordering of columns you can put those customers under one condition.
    for Example...
          if(?Client in ['clientA','clientA']) then
                    <TableName>.<Field>......
-What do you mean be 40 columns? Do you mean 40 fields or you want to show 40 columns in report? Please elaborate.
Regards,
Amrita
Edited by: Amrita Singh on May 21, 2009 7:54 AM

How to generate a second csv file with different report columns selected?

Hi. Everybody:
How to generate a second csv file with different report columns selected?
The first csv file is easy (report attributes -> report export -> enable CSV output Yes). However, our users demand 2 csv files with different report columns selected to meet their different needs.
(The users don't want to have one csv file with all report columns included. They just want to get whatever they need directly, no extra columns)
Thank you for any help!
MZ

Hello,
I'm doing it usually. Typically example would be in the report only the column "FIRST_NAME" and "LAST_NAME" displayed whereas
in the csv exported with the UTL_FILE the complete address (street, housenumber, additions, zip, town, state ... ) is written, these things are needed e.g. the form letters.
You do not need another page, just an additional button named e.g. "export_to_csv" on your report page.
The csv export itself is handled from a plsql procedure "stored procedure" ( I like to have business logic outside of apex) which is invoked by pressing the button "export_to_csv". Of course the stored procedure can handle also parameters
An example code would be something like
PROCEDURE srn_brief_mitglieder (
     p_start_mg_nr IN NUMBER,
     p_ende_mg_nr IN NUMBER
AS
export_file          UTL_FILE.FILE_TYPE;
l_line               VARCHAR2(20000);
l_lfd               NUMBER;
l_dateiname          VARCHAR2(100);
l_datum               VARCHAR2(20);
l_hilfe               VARCHAR2(20);
CURSOR c1 IS
SELECT
MG_NR
,TO_CHAR(MG_BEITRITT,'dd.mm.yyyy') AS MG_BEITRITT ,TO_CHAR(MG_AUFNAHME,'dd.mm.yyyy') AS MG_AUFNAHME
,MG_ANREDE ,MG_TITEL ,MG_NACHNAME ,MG_VORNAME
,MG_STRASSE ,MG_HNR ,MG_ZUSATZ ,MG_PLZ ,MG_ORT
FROM MITGLIEDER
WHERE MG_NR >= p_start_mg_nr
AND MG_NR <= p_ende_mg_nr
--WHERE ROWNUM < 10
ORDER BY MG_NR;
BEGIN
SELECT TO_CHAR(SYSDATE, 'yyyy_mm_dd' ) INTO l_datum FROM DUAL;
SELECT TO_CHAR(SYSDATE, 'hh24miss' ) INTO l_hilfe FROM DUAL;
l_datum := l_datum||'_'||l_hilfe;
--DBMS_OUTPUT.PUT_LINE ( l_datum);
l_dateiname := 'SRNBRIEF_MITGLIEDER_'||l_datum||'.CSV';
--DBMS_OUTPUT.PUT_LINE ( l_dateiname);
export_file := UTL_FILE.FOPEN('EXPORTDIR', l_dateiname, 'W');
l_line := '';
--HEADER
l_line := '"NR"|"BEITRITT"|"AUFNAHME"|"ANREDE"|"TITEL"|"NACHNAME"|"VORNAME"';
l_line := l_line||'|"STRASSE"|"HNR"|"ZUSATZ"|"PLZ"|"ORT"';
     UTL_FILE.PUT_LINE(export_file, l_line);
FOR rec IN c1
LOOP
     l_line := '"'||rec.MG_NR||'"';
     l_line := l_line||'|"'||rec.MG_BEITRITT||'"|"' ||rec.MG_AUFNAHME||'"';
     l_line := l_line||'|"'||rec.MG_ANREDE||'"|"'||rec.MG_TITEL||'"|"'||rec.MG_NACHNAME||'"|"'||rec.MG_VORNAME||'"';
     l_line := l_line||'|"'||rec.MG_STRASSE||'"|"'||rec.MG_HNR||'"|"'||rec.MG_ZUSATZ||'"|"'||rec.MG_PLZ||'"|"'||rec.MG_ORT||'"';
--     DBMS_OUTPUT.PUT_LINE (l_line);
-- in datei schreiben
     UTL_FILE.PUT_LINE(export_file, l_line);
END LOOP;
UTL_FILE.FCLOSE(export_file);
END srn_brief_mitglieder;Edited by: wucis on Nov 6, 2011 9:09 AM

How to check-in multiple files with same name having different revision num

Hi
Can anyone please tell me, how to check-in multiple files with the same name with different revision number using RIDC API.
For eg:
First I will check-in a file(TestFile.txt) into a content server with revision number 1 using RIDC API in ADF application. Then after some time, will modify the same file(TestFile.txt) and check-in again. I tried to check-in same file multiple times, however first time its checking-in correctly into server showing revision as 1, while checking-in same file again, its not giving any errror message, and also its not reflecting in server. Only one file(TestFile.txt) is reflecting in server.
How to implement this functinality using RIDC API? Any suggestions would be helpful.
Regards
Raj
Edited by: 887680 on Mar 6, 2013 10:48 AM

Hi Srinath
Thanks for your response. Its not cloning, its like check-in file first, then check-out the file and do some editing and then again upload the same file with different revision number using RIDC. I got the solution now.
Regards
Raj

SSIS 2012 - Importing multiple files with different columns (where columns are stored in a table)

Similar Messages

Maybe you are looking for