Importing multiple csv's (different structures) to destination table

I am using a data flow task with flat file source and ole db destination and looping thru a folder to import multiple csv files into a destination table. What if I wanted to import different csv files, for example, with different structures or layout.
The different csv's would have the same columns for destination table but in different order/name. How do u map each one to the destination table? Multiple data flow tasks or multi flat file sources?

The standard Data Flow Task supports only static columns metadata defined at design time. I would recommend you check the commercial COZYROC
Data Flow Task Plus. It is an extension of the standard Data Flow Task, which supports dynamic columns metadata. You can handle all your input flat files using single
Data Flow Task Plus. No programming skills are required.
SSIS Tasks Components Scripts Services | http://www.cozyroc.com/

Similar Messages

Button to import a csv file to update an existing table

hi ALL,
I'm pretty new with apex, on a page f my application, I want to add a button to import a csv file to update an existing table.
Have you any suggestions?
thanks in advance

Hi,
an easy solutuion would be if ypu could place the CSV file on a location which your database can access.
Access the CSV via an external table (CSV file can be accessed as if it was a readonly table ).
Create a dynamic action (PL/SQL) for the button which reads the external table and adds the data to your target table.
Cheers
Bas

SSIS 2012 - Importing multiple files with different columns (where columns are stored in a table)

Hi everyone,
I have been tasked to upgrade an existing DTS package into an SSIS package.
The old package uses 3 tables.
Table 1) Has a list of file names
Table 2) Has a list of columns per file
Table 3) Is the data destination.
So what is supposed to happen is:
Step 1) the package should load the files into a resultset
Step 2) for each file in the resultset, the package should get the related columns
Step 3) the package should now map the columns to the destination table (variable column numbers here)
Step 4) the import should take place.
I have gotten to step 2 but am not sure how to dynamically map the columns to the destination table.
Any pointers anyone?
Thanks in advance!

Hi Tonyhektic,
What’s the nature of Table 2, does it only store the column names for each file? I don’t know how it is implemented in DTS package, however, in SSIS package, dynamic columns mapping is not supported. To work around this issue, we need to use Script component
to do the columns mapping. You can refer to Sorna’s script above or have a look at the following blogs:
http://munishbansal.wordpress.com/2009/06/09/dynamic-columns-mapping-%E2%80%93-script-component-as-destination-ssis/
http://blog.quasarinc.com/ssis/best-solution-to-load-dynamically-change-csv-file-in-ssis-etl-package/
Regards,
Mike Yin
TechNet Community Support

Importing a csv file into addressbook

So I did a search & found this question has come up quite often, but with the person having a slightly different problem, so here goes.
I am trying to import a csv file into address book, when I go import -> text tile it comes up like it should do and I pick the file, if I select ok at this point, it will quite happily import all of the data no trouble (but in the wrong categories). So I make the changes to what I want ie. Phone other --> Phone (work). Then when I click ok, the button goes from blue to white but nothing happens.
When I open the csv file in textedit I can see all the values separated by commas.
Any Ideas?

I too just had this problem. To clarify all fields work and you can map them. I found that the Outlook CSV file had two problems which were tripping up the AddressBook import function. Note these were both in street address fields so I suspect the poster who said that addresses don't work may have been having that problem.
The CSV load program does not work if you have commas "," in your data. In my case this was very common for some European addresses as well as some company names. And then the second problem was if there was a carriage return or new line character in a field. This later one was a problem with the address fields. I suspect an AddressBook export to CSV might have this same issue. Specifically, when there were multiple lines to the street address, they had been entered as multiple lines in street address one, as opposed to entering them in multiple fields such as "street 1", "street 2", etc.
To solve the problem I opened the CSV file using Numbers and did a global find and replace on: commas, substituting the string --- and on the newline character "¶" (ALT+Rreturn) with *.
Note to reduce some of the remapping I also changed all of the labels with "business" in the header row of the CSV file to "work". This made it a little easier for the AddressBook import function to map fields more correctly.
After these set of changes rather than the alternating blue then white with nothing behavior it cleanly loaded all of my names. I could then search for "---" and "*" in addressbook's search field to identify all addresses which I had to clean up.
I hope that is helpful to someone else. The import function really does work. However, it is not robust enough to deal with certain oddities in one's data.

Import Multiple Excel Into Sql Server

I have 4 Excel spreadsheets all with different formatting and column names that I want to use powershell to import into ONE Sql Server Table that I will want to create with powershell. I have found several instances of importing one workbook, but I
haven't found one where you can import multiple workbooks into the same table. Can someone show sample code for a reference on how to import multiple spreadsheets into the same table with powershell please? I found this link which looks like a
GREAT starting point, but unfortunately it is only showing how to do one table, and I am WAY new to powershell so can't tweak it on my own to set it to import multiple
http://www.sqlserver-dba.com/2013/01/sql-server-export-excel-data-to-sql-server-with-powershell.html

Hi IndigoMontoya,
As a workaroud, how about merge mutiple excel files into one file, then import this excel file to sql table?
To merge excel files, please refer to this script:
$Files = 'd:\merge1.xlsx','d:\merge2.xlsx'
#Launch Excel, and make it do as its told (supress confirmations)
$Excel = New-Object -ComObject Excel.Application
$Excel.Visible = $True
$Excel.DisplayAlerts = $False
#Open up a new workbook
$Dest = $Excel.Workbooks.Add()
#Loop through files, opening each, selecting the Used range, and only grabbing the first 6 columns of it. Then find next available row on the destination worksheet and paste the data
ForEach($File in $Files[0..4]){
$Source = $Excel.Workbooks.Open($File,$true,$true)
If(($Dest.ActiveSheet.UsedRange.Count -eq 1) -and ([String]::IsNullOrEmpty($Dest.ActiveSheet.Range("A1").Value2))){ #If there is only 1 used cell and it is blank select A1
[void]$source.ActiveSheet.Range("A1","F$(($Source.ActiveSheet.UsedRange.Rows|Select -Last 1).Row)").Copy()
[void]$Dest.Activate()
[void]$Dest.ActiveSheet.Range("A1").Select()
}Else{ #If there is data go to the next empty row and select Column A
[void]$source.ActiveSheet.Range("A1","F$(($Source.ActiveSheet.UsedRange.Rows|Select -Last 1).Row)").Copy()
[void]$Dest.Activate()
[void]$Dest.ActiveSheet.Range("A$(($Dest.ActiveSheet.UsedRange.Rows|Select -last 1).row+1)").Select()
[void]$Dest.ActiveSheet.Paste()
$Source.Close()
$Dest.SaveAs("d:\merge3.xlsx",51)
$Dest.close()
$Excel.Quit()
Reference from:
How to use powershell to copy several excel worksheets and make a new one?
I hope this helps.

Error in importing multiple excel sheets to SQL Server

I have a package which imports multiple excel sheets to SQL server using a For each Container. However I am getting the following error message "Excel Source failed validation and returns validation status "VS-NEEDSNEWMETADATA".
Can you please advise me of the steps to potentially resolve this issue ?
Many thanks
Scott

Hi Scott,
Based on your scenario, you need to implement dynamic columns mapping which is not natively supported by managed source/destination adapters. To achieve dynamic columns mapping, we need to make use of Script Component to parse the input columns and dynamically
map them to the destination columns.
References:
http://munishbansal.wordpress.com/2009/06/09/dynamic-columns-mapping-%E2%80%93-script-component-as-destination-ssis/
http://blog.quasarinc.com/ssis/best-solution-to-load-dynamically-change-csv-file-in-ssis-etl-package/
http://stackoverflow.com/questions/13836874/script-task-in-dft-doesnt-get-excecuted
Regards,
Mike Yin
TechNet Community Support

BO xi 3.1 u2013 schedule a template for multiple reports with different column

Hi,
I need to migrate the existing reports from SQL Server Reporting Services to BO, and the column order in the report is very important to the clients since clients have existing code to ingest these reports into their database. Now I want to schedule a single template with different parameters for all reports in BO, or create a single template with lot of parameters for all reports. My question is how I might use one template to create multiple reports with different column orders in the reports?
For example, client A has the following column order in their report: firstName, LastName, DateBirth, SSN. Client B has the following column order in their report: SSN, DateBirth, LastName, FirstName. Can I use one template to create these multiple reports?
Thanks!

Hi,
The only approach I can think of is to create a template report which uses variables
For each column you would need to variable
v_columnAName and v_columnAValue
v_columnAName would have a if statement in it
=if([client]="clientA" or [client]="clientC";NameOf([firstName]);if([client]="clientB";NameOf([SSN]);NameOf([lastName]));
v_columnA would have a if statement in it
=if([client]="clientA" or [client]="clientC";[firstName];if([client]="clientB";[SSN],[lastName]));
This would only work when you had a small set of clients.
This might be more managable if it was done in the universe
Regards
Alan

Import From CSV File statement runs forever, no error, does not finish

Hello,
I am trying to import a CSV file in a JAVA program, with the following statement:
IMPORT FROM CSV FILE '/debug/testdatabase/FILE.csv'
INTO "JOSEPH"."TEST_TABLE"
WITH COLUMN LIST IN FIRST ROW
RECORD DELIMITED BY '\n'
FIELD DELIMITED BY '\t' ERROR LOG '/debug/testdatabase/file.err'
THREADS 10
BATCH 10000
I have two HANA instances on different machines A and B:
Both machines run HANA version 1.00.74.00.389160 (NewDB100_REL), while the OS is
SUSE Linux Enterprise Server 11.1 on machine A and
SUSE Linux Enterprise Server 11.2 on machine B.
The statement above runs fine on machine A and the rows are imported properly from JAVA as well as when executed from HANA Studio SQL console.
If I copy the file to machine B and try the exact same statement with the same file, it does not finish (neither from JAVA nor from HANA Studio SQL console). There is no error either. It cannot be cancelled, only a HANA restart stops the statement. Also the sample file I use has only 2 rows, and memory does not seem to be a problem.
I seem to have a similar problem to the one described here, but the answers there do not help me: http://scn.sap.com/thread/3396582 I specified the record delimiter, and I used a python script to check for any strange characters that are not supposed to be there, but didn't find any.
If I copy the file to my windows PC and use the "File Menu -> Import -> SAP Hana Content -> Data from Local file" function, it imports the file correctly into B, but I need to be able to do it from JAVA.
Machine A administration view:
Machine B administration view:
If you have any idea what might cause this behavior or where I can find more information on this problem please give me a hint.

Hi Joseph,
First from the pics, the revision of your SAP HANA instance is 73 instead of 74. Since I have no identical environment, I cannot test it for you. But can you try the simplest scenario? You can create a table with only one column table try to import a CSV file with only one row.
Best regards,
Wenjun

Script to merge multiple CSV files together with no duplicate records.

I like a Script to merge multiple CSV files together with no duplicate records.
None of the files have any headers and column A got a unique ID. What would be the best way to accomplish that?

OK here is my answer :
2 files in a directory with no headers.
first column is the unique ID, second colomun you put whatever u want
The headers are added when using the import-csv cmdlet
first file contains :
1;a
2;SAMEID-FIRSTFILE
3;c
4;d
5;e
second file contains :
6;a
2;SAMEID-SECONDFILE
7;c
8;d
9;e
the second file contains the line : 2;b wich is the same in the first file
the code :
$i = 0
Foreach($file in (get-childitem d:\yourpath)){
if($i -eq 0){
$ref = import-csv $file.fullname -Header id,value -Delimiter ";"
}else{
$temp = import-csv $file.fullname -Header id,value -Delimiter ";"
foreach($line in $temp){
if(!($ref.id.contains($line.id))){
$objet = new-object Psobject
Add-Member -InputObject $objet -MemberType NoteProperty -Name id -value $line.id
Add-Member -InputObject $objet -MemberType NoteProperty -Name value -value $line.value
$ref += $objet
$i++
$ref
$ref should return:
id
value
1
a
2
SAMEID-FIRSTFILE
3
c
4
d
5
e
6
a
7
c
8
d
9
e
(get-childitem d:\yourpath) -> yourpath containing the 2 csv file

Multiple files in different directory using single FILE adaptor

Hi Guys,
How can we handle multiple files having different format(PDF,TXT,EXCEL) and in different directory with the single FILE adaptor.
I know by using "." we ca get mutilple files, but here tthe files are of different format and in different directory.
Could you please explain as to how we can handle this scenarios?
Thanks
Sahil

Hi,
The short answer is - "No".
The Long Question - "Why do you want to use a SINGLE channel".
If your idea is to reuse the interface then dont worry... Except for the Senderchannel and Sender agreement all your other objects can be reused ...provided your sender apter converts the different files into the same XML format..
For TXT you can use the simple file adpater..
For excel see if you can use save the file as .CSV format...then you can use it directlky by the simple FIle adapter..
For PDF use conversion agent... Here a blog to get u started on that ->
/people/shabarish.vijayakumar/blog/2009/05/17/trouble-writing-out-a-pdf-in-xipi
regards
Arvind R

How to convert an xml to another Xml with a different structure...

hi
i have an xml which should be converted to a standard xml with an entirely different structure(Different elements)....can this be done with xml,xsl alone or do we need to write an java class along with them...if there are any examples..it would be of great help...plz help me out....as this is a work stopper...
given below is the kind i have
<hotelinfo>
<hotelname>RADISSON HOTEL CHENNAI </hotelname>
<address>355 C Gst Road St Thomas Mount Chennai1 600016IN</address>     <price>90.00 - 150.00</price>
<distance>2 (W)</distance>
</hotelinfo>
i need to convert it into
<propertyName>RADISSON HOTEL CHENNAI </propertyName>
<address>
<addressLine>355 C Gst Road St Thomas Mount</addressLine>
<CityName>Chennai</CityName>
<PostalCode>600 001</PostalCode>
<StateName>TamilNadu</StateName>
<CountryName>India</CountryName>
</address>
thanks in advance

Ok here is the code...
package mypack;
import java.io.File;
import javax.xml.parsers.DocumentBuilder;
import javax.xml.parsers.DocumentBuilderFactory;
import javax.xml.transform.Result;
import javax.xml.transform.Source;
import javax.xml.transform.Transformer;
import javax.xml.transform.TransformerConfigurationException;
import javax.xml.transform.TransformerException;
import javax.xml.transform.TransformerFactory;
import javax.xml.transform.dom.DOMSource;
import javax.xml.transform.stream.StreamResult;
import org.w3c.dom.Document;
import org.w3c.dom.Node;
* @author SM23772
* TODO To change the template for this generated type comment go to
* Window - Preferences - Java - Code Style - Code Templates
public class Roopa {
     public static void main(String[] args) {
          File docFile = new File("Roopa.xml");
          Document doc = null;
          Document newDoc = null;
          String hname = null, task = null;
          try {
               DocumentBuilderFactory dbf = DocumentBuilderFactory.newInstance();
               DocumentBuilder db = dbf.newDocumentBuilder();
               doc = db.parse(docFile); //parsing Roopa.xml
               newDoc = db.newDocument(); //Created New document Object for new xml
               hname = doc.getElementsByTagName("hotelname").item(0)
                         .getFirstChild().getNodeValue();
               task = doc.getElementsByTagName("task").item(0).getFirstChild()
                         .getNodeValue();
               Node rootNode = newDoc.createElement("property-info"); //creating root node;
               Node pp = newDoc.createElement("name"); //creating a node called name
               Node taskNode = newDoc.createElement("task"); //creating a node called task
               pp.appendChild(newDoc.createTextNode(hname)); //appending a text node inisde name
               taskNode.appendChild(newDoc.createTextNode(task)); //appending the text node inside task
               rootNode.appendChild(pp); //asppending the nodes inside the root node
               rootNode.appendChild(taskNode);
               newDoc.appendChild(rootNode); //appending the root node to the document
               writeXmlFile(newDoc, "Dest.xml"); //function called to write the document to Dest.xml
          } catch (Exception e) {
               System.out.println(e);
     public static void writeXmlFile(Document doc, String filename) {
          try {
               // Prepare the DOM document for writing
               Source source = new DOMSource(doc);
               // Prepare the output file
               File file = new File(filename);
               Result result = new StreamResult(file);
               // Write the DOM document to the file
               Transformer xformer = TransformerFactory.newInstance()
                         .newTransformer();
               xformer.transform(source, result);
          } catch (TransformerConfigurationException e) {
          } catch (TransformerException e) {
The xml files:
roopa.xml:
<hotel>
<hotelname>RADISSON HOTEL CHENNAI</hotelname>
<more-info><task>give treate to shanu</task></more-info>
</hotel>
Dest.xml
<?xml version="1.0" encoding="UTF-8"?>
<property-info><name>RADISSON HOTEL CHENNAI</name><task>give treate to shanu</task></property-info>Please go through the code.. if u have any more concerns do let me know................
cheers
Shanu

Problem importing a CSV file with forward slashes in a column

I have an Excel csv file of a product database (contains about 6500 products) that contains product codes with forward slahes such as 499/1, 499/3, 499/5.
These are different sizes of a product and as such have different prices etc.
When I import the file into Numbers these numbers appear as 499, 166 1/3, and 99 4/5 respectively.
What seems to be happening is that Numbers is interpreting the forward slash (/) as a divide by command and subsequently performing a calculation on that number on import and hence totally changing the value of the cell so that it is impossible to look up the price related to a product as the product code no longer exists.
Excel can import these files with no problems, why can't Numbers treat each cell as text and leave it alone on import.
Is there any way round this or do I have to revert to using Excel for the import of csv files.
Thanks
Steve

I know I'm a bit(!) late (a year) coming to this party, but there is a simple solution, that worked for me, enclose the field in double quotes, and add a single quote before the number:
Instead of
499/1,"Super Widget 3",12.34
do
"'499/1","Super Widget 3",12.34
In fact the single (unclosed) quote without double-quotes works as well:
'499/1,"Super Widget 3",12.34
I've always found it better to enclose strings with double quotes. This works on loading the file into Mac Numbers and should work with Excel too if that helps. It opens with OpenOffice 4 on the Mac too - if you select "comma" in the "Separated by" checkbox
I can't remember where I picked this info up....
Hope this helps someone, albeit late.
Andy

Importing a CSV file into AD 2012r2

Hi Guys
Just wondering if anyone can help, I am a newbie on scripting and finding it really difficult to understand how to import a csv file into AD
I have created the CSV file with the following headers, see below but I am unable to write a powershell script to import them into AD which work, I have tried multiple help guides but still cannot do it for the life of me.
Any help on this would be really appreciated
objectClass user
cn Jo bloggs
givenName jo
sn blogs
mail [email protected]
sAMAccountname j.blogs
AccountPassword Pa$$w0rd
ChangePasswordAtLogon true
PasswordNeverExpires false
ou OU=staff,DC=test,DC=com

Import-Csv C:\User.csv| %{New-ADUser -SamAccountName $_.SamAccountName `
-UserPrincipalName $_.userprinicpalname `
-Name $_.name `
-Mail = $_.mail `
-DisplayName $_.name `
-GivenName $_.GivenName `
-Path "OU=Test Users,DC=XXX,DC=XXX" `
-PasswordNeverExpires $false `
-ChangePasswordAtLogon $true `
-AccountPassword (ConvertTo-SecureString "test41;" -AsPlainText -force) `
-Enabled $True -PasswordNeverExpires $True -PassThru ` }
help New-ADUser -Examples
You can try the below sample code
Regards Chen V [MCTS SharePoint 2010]

Import Multiple XSD Files

Is it not possible to import multiple xsd files. If a single file is used there is no problem but when the second import statement is added, it will not compile.
It complains that it cannot find one of the element types. Removing the second import it compiles fine.
I also tried it using seperate schema tags for each import.
<types>
<schema xmlns="http://www.w3.org/2001/XMLSchema">
   <import namespace="http://TargetNamespace.com/ftpadapter"
           schemaLocation="delimited_2.txt"/>
   <import namespace="http://xmlns.oracle.com/FTP_Test"
           schemaLocation="FTP_Test.xsd"/>
</schema>
</types>

It is possible to add multiple XSD's in WSDL.
1. Did u add different namespace alias's for these two in <definitions> section ??
2.How you are referring the element which it can't find in the wsdl ?
Try this to identify the issue
"remove the first xsd and use the second one alone. "
If this works then it eliminates that there is an issue with xsd.
Thanks,
Satish
http://soadiscovery.blogspot.com

Difference between the different structures while dealing with User-Exits

While dealing with User-Exits, we come across the different structures/Tables like XVBAP,YVBAP,TVBAP,IVBAP,*VBAP....
What is the difference between them ?

Usually X<tablename> and Y<tablename> are used in change document management to determine records updated (Y is old values and X new ones). (Look at transaction SCDO)
T<tablename is an internal table read from database and I<tablename> is an import table parameter of the same structure. <tablename> is an equivalent of <tablename> declared with statement TABLES and contains usually the "old" value.
Look at exit documentation. start at [User Exits In Sales Document Processing|http://help.sap.com/saphelp_46c/helpdata/en/1c/f62c7dd435d1118b3f0060b03ca329/content.htm]
Regards

Importing multiple csv's (different structures) to destination table

Similar Messages

Maybe you are looking for