ODI reading csv file issue

Hi,
I am in ODI 11.1.1, which is our ELT tool for metedata load for all Planning App.
One issue i am facing is , one data extract coming from an essbase cube using a Calc Script as .csv file.
when i am trying to read that in ODI , it is unable to read. Opening the file and saving it manually . After that step ODI is able to read the file.
in the datastore under the Model , i have defined the file type Delimited as comma and header 2, record separator Ms-Dos.
anyone has come accross this issue ?
Thanks,
KP

Hi,
Most probalby it is due to difference between file creation platform and file read platform.
Say you created a file in Unix and then try to read the file in MS_Dos , ODI will not be able to file , unless you set
record separator as Unix.
Thanks,
Sutirtha

Similar Messages

Question about reading csv file into internal table

Some one (thanks those nice guys!) in this forum have suggested me to use FM KCD_CSV_FILE_TO_INTERN_CONVERT to read csv file into internal table. However, it can be used to read a local file only.
I would like to ask how can I read a CSV file into internal table from files in application server?
I can't simply use SPLIT as there may be comma in the content. e.g.
"abc","aaa,ab",10,"bbc"
My expected output:
abc
aaa,ab
10
bbb
Thanks again for your help.

Hi Gundam,
Try this code. I have made a custom parser to read the details in the record and split them accordingly. I have also tested them with your provided test cases and it work fine.
OPEN DATASET dsn FOR input IN TEXT MODE ENCODING DEFAULT.
DO.
READ DATASET dsn INTO record.
PERFORM parser USING record.
ENDDO.
*DATA str(32) VALUE '"abc",10,"aaa,ab","bbc"'.
*DATA str(32) VALUE '"abc","aaa,ab",10,"bbc"'.
*DATA str(32) VALUE '"a,bc","aaaab",10,"bbc"'.
*DATA str(32) VALUE '"abc","aaa,ab",10,"b,bc"'.
*DATA str(32) VALUE '"abc","aaaab",10,"bbc"'.
FORM parser USING str.
DATA field(12).
DATA field1(12).
DATA field2(12).
DATA field3(12).
DATA field4(12).
DATA cnt TYPE i.
DATA len TYPE i.
DATA temp TYPE i.
DATA start TYPE i.
DATA quote TYPE i.
DATA rec_cnt TYPE i.
len = strlen( str ).
cnt = 0.
temp = 0.
rec_cnt = 0.
DO.
* Start at the beginning
IF start EQ 0.
    "string just ENDED start new one.
    start = 1.
    quote = 0.
    CLEAR field.
ENDIF.
IF str+cnt(1) EQ '"'. "Check for qoutes
    "CHECK IF quotes is already set
    IF quote = 1.
      "Already quotes set
      "Start new field
      start = 0.
      quote = 0.
      CONCATENATE field '"' INTO field.
      IF field IS NOT INITIAL.
        rec_cnt = rec_cnt + 1.
        CONDENSE field.
        IF rec_cnt EQ 1.
          field1 = field.
        ELSEIF rec_cnt EQ 2.
          field2 = field.
        ELSEIF rec_cnt EQ 3.
          field3 = field.
        ELSEIF rec_cnt EQ 4.
          field4 = field.
        ENDIF.
      ENDIF.
*      WRITE field.
    ELSE.
      "This is the start of quotes
      quote = 1.
    ENDIF.
ENDIF.
IF str+cnt(1) EQ ','. "Check end of field
    IF quote EQ 0. "This is not inside quote end of field
      start = 0.
      quote = 0.
      CONDENSE field.
*      WRITE field.
      IF field IS NOT INITIAL.
        rec_cnt = rec_cnt + 1.
        IF rec_cnt EQ 1.
          field1 = field.
        ELSEIF rec_cnt EQ 2.
          field2 = field.
        ELSEIF rec_cnt EQ 3.
          field3 = field.
        ELSEIF rec_cnt EQ 4.
          field4 = field.
        ENDIF.
      ENDIF.
    ENDIF.
ENDIF.
CONCATENATE field str+cnt(1) INTO field.
cnt = cnt + 1.
IF cnt GE len.
    EXIT.
ENDIF.
ENDDO.
WRITE: field1, field2, field3, field4.
ENDFORM.
Regards,
Wenceslaus.

Use servlet read .csv file by ie, but why two more " ?

Hi , friends
following is the key method in my servlet used to read .csv file to ie , but , after I read file to client side , I found that in the both sides of every line " was added, so the csv file can have only one column (many columns was merged into one column), that was not what i want, who guys can find some idea? thanks for your enthusiasm!
void serveRemoteFile(String sFileName, String sContentType, HttpServletRequest req, HttpServletResponse res, StringBuffer sbLog, Runtime rt){
FileInputStream in = null;
ServletOutputStream out = null;
byte bBuf[] = null;
int nLen;
if (isBlank(sContentType) || isBlank(sFileName))
return;
res.setStatus(res.SC_OK);
res.setContentType(sContentType);
res.setHeader("Content-Disposition", "inline;filename= temp.csv");
try {
in = new FileInputStream(sFileName);
out = res.getOutputStream();
bBuf = new byte[1024];
while ( (nLen = in.read(bBuf,0,1024)) != -1) {
out.write(bBuf, 0, nLen);
out.flush();
out.close();
}catch (FileNotFoundException e){
errorPage(req,res,sbLog);
}catch (IOException e){
errorPage(req,res,sbLog);

Excel uses a weird CSV file format. You can find more information about it and a Java library to read data from it here:
http://ostermiller.org/utils/ExcelCSV.html

Drivers to read CSV file

Hi,
In one of my application, I need to read CSV file as a database and read the columns from the CSV file. Can I get any drivers which accepts CSV file as a datasource and I can query on that datasource?
I know this will work in Windows.. But I am working on UNIX and need drivers to be installed on Unix.
Does any one knows how to do it or if they can provide any links, That is of very helpful for me..
Please help me in resolving this problem..

Hi
You need to use a JDBC driver of Type 3 to do your task.
Swaraj

Labview not reading csv file properly.

I'm trying to use the read spreadsheet file vi to read the attached csv.
It does a few things incorrectly:
1) the first cell of the resulting array has ÿþ appended to the front of it, seems like labview is grabbing some special character it shouldn't? (Notepad, notepad++ and excel can all open the file without issue and none show ÿþ or anything like it).
2) It's mistreating the EOL characters as two line feeds, resulting in blank rows between each data row. (Looking at the csv in notepad++ confirms that the lines are ended with a CR and a LF (same as EOL).
3) I'm reading it in as a string, because one of the columns is non-numeric, but then I strip that column out and attempt to convert the remaining string array into a numeric array (this fails). It also fails if I pull out an individual element of the array and attempt to convert it to a number (in both cases using fract/exp string to number), below I have an example of trying to read the 2nd column of the 1st row (index: 0,1) which is formatted as a string: "1.37238168716431" and converting that into a number gives 0 as you can see in the snapshot of the front panel.
Solved!
Go to Solution.
Attachments:
REPORT02.CSV ‏4 KB

I guess the file is in unicode, that doesn't help me very much.
I don't use the "Read from Text File Function" - at least not directly, although it is called in Read from Spreadsheet File. Where is it called, however, right-click doesn't show an option for Convert EOL, instead it's a terminal and it's default is F and as the terminal is disconnected, I'm assuming it's not converting EOL. So it's already "unchecked" in a sense.
No idea how to convert the unicode string to ascii. (no functions show up when i search unicode or ascii in the manager). It displays it just fine in the text box, so I'm not sure why it's able to display it without issue but not read it as a number.
Using Labview 8.6, by the way.

Importing users into WGM from csv file issues/crash

Hi,
i've been importing user information from csv files into WGM via the +server > import+ function .
It worked the first few times but now when i try the import progress bar pops up and promptly disappears without any thing importing.
i've tried restarts, new admin account, reinstalled WGM.
I've also trashed some pref but i don't really know which ones i should be losing.
The servers an OD master.
any help would be appreciated.
as a last resort what do i need to backup/save if i were to format/reinstall osx server? keeping my settings etc.....
thanks
paul

What I did was:
Exported the user list, to create an XML file in the correct format.
Using this format, I created a spreadsheet in Excel (sorry Apple), and in the final column I created a field that concatenated the information I wanted in the ':' deliminated format of the previously export XML.
Then just copy and past via pico into a pure text file and imported that.
You have to be careful with comments in Passenger, using special characters (';!@#$%^ and others can cause the WGM to fail and crash.

Read CSV file into a 1-D array

Hi
I would like to read a csv file into a cluster of 4 elements which would then be read into a 1-D array.
My cluster contains a typedef, a double, a boolean, and another typedef.
Basically it could be seen as:
Bob Runs, 4, T, Bob
Mary sits, 5, F, Mary
Bob Sits, 2, F, Bob
Mary Runs, 9, T, Mary
(keeps growing)
Are there any good examples for what I am trying to put together that I could leverage, or is it better to use a different input file than a csv. I am trying to make my program more flexable and easier to make adjustments even after the executable is created. My line items seem to be growing exponentially and is getting difficult to manage in the LV window.
Thanks
Solved!
Go to Solution.

Unless your CSV file is huge, I'd use "Read from Spreadsheet File" with the delimiter set as "," and the type as string. This will give you a 2D array of strings. You could then separate out each column of the array, convert to the appropriate data type, and use Index & Bundle Cluster Array to build your array of clusters. Something like this (except I'm using a string constant in place of reading from the file).

Read csv file and insert the data to a list in sharePoint

Hi everyone,
i wrote a code that reads from a csv file all his data but i need also to insert all the data to a new list in sharePoint.
how can i do this?
plus, i need to read from the csv file once a day in specific hour. how can i do this? thank you so much!!

Did you look at the example I posted above?
ClientContext in CSOM will allow you to get a handle on the SharePoint objects;
http://msdn.microsoft.com/en-us/library/office/ee539976(v=office.14).aspx
w: http://www.the-north.com/sharepoint | t: @JMcAllisterCH | YouTube: http://www.youtube.com/user/JamieMcAllisterMVP

SQL server 2014 and VS 2013 - Dataflow task, read CSV file and insert data to SQL table

Hello everyone,
I was assigned a work item wherein, I've a dataflow task on For Each Loop container at control flow of SSIS package. This For Each Loop container reads the CSV files from the specified location one by one, and populates a variable with current
file name. Note, the tables where I would like to push the data from each CSV file are also having the same names as CSV file names.
On the dataflow task, I've Flat File component as a source, this component uses the above variable to read the data of a particular file. Now, here my question comes, how can I move the data to destination, SQL table, using the same variable name?
I've tried to setup the OLE DB destination component dynamically but it executes well only for first time. It does not change the mappings as per the columns of the second CSV file. There're around 50 CSV files, each has different set off columns
in it. These files needs to be migrated to SQL tables using the optimum way.
Does anybody know which is the best way to setup the Dataflow task for this requirement?
Also, I cannot use Bulk insert task here as we would like to keep a log of corrupted rows.
Any help would be much appreciated. It's very urgent.
Thanks, <b>Ankit Shah</b> <hr> Inkey Solutions, India. <hr> Microsoft Certified Business Management Solutions Professionals <hr> http://ankit.inkeysolutions.com

The standard Data Flow Task supports only static metadata defined during design time. I would recommend you check the commercial COZYROC
Data Flow Task Plus. It is an extension of the standard Data Flow Task and it supports dynamic metadata at runtime. You can process all your input CSV files using a single Data Flow Task
Plus. No programming skills are required.
SSIS Tasks Components Scripts Services | http://www.cozyroc.com/

Reading csv files efficiently..

Hi..
I need to read,parse and insert required data from csv files of size 1 GB into database efficiently.
Which is the best approach.
Thanks,

I did this search:
http://www.google.com/search?q=site%3Ajakarta.apache.org+csv+parser
and found this:
http://jakarta.apache.org/commons/sandbox/csv/
It took only a few seconds.
Every java project should start with a google search against apache.org.

Problem in reading csv file in servlet

Hi everyone,
I m getting an ClassCastExeption while importing a .csv file to databse...
It works fine in case of .xls...
I m using jxl from Jakarta...and the code is as follows...
Thanks in advance..
FileItemFactory factory1 = new DiskFileItemFactory();
ServletFileUpload upload1 = new ServletFileUpload(factory1);
items1 = upload1.parseRequest(request);
Iterator iter = items1.iterator();
while (iter.hasNext()) {
FileItem item = (FileItem) iter.next();
InputStream uploadedStream = item.getInputStream();
importCSV(uploadedStream);
and in the import method.....
Workbook w;
w = Workbook.getWorkbook((FileInputStream)is);
In the above line i m getting ClassCastException...
Please help..

Thanks gimbal2 ..
w = Workbook.getWorkbook(is); throws the FileNotFoundException
I solved it by tokenizing the inputstream on ','(comma) and then getting the tokens for each column in the database....

Reading csv file how to get the Column name

Hi,
I am trying to read a csv file and then save the data to Oracle.
Class.forName("sun.jdbc.odbc.JdbcOdbcDriver");
Connection c = DriverManager.getConnection("jdbc:odbc:;Driver={Microsoft Text Driver (*.txt; *.csv)};Dbq=.;Extensions=csv,txn");
Statement stmt = c.createStatement();
ResultSet rs = stmt.executeQuery("select * from filename.csv");
while(rs.next())
System.out.println(rs.getString("Num"));
My csv file looks like this:
"CHAM-23","COMPANY NAME","Test","12",20031213,15,16
Number,Environ,Envel,Date,Time
"1","2",3,"4",5
"6","7",8,"9",9
Now is there anyway using the above code I start processing the file from the second row that holds the names of the columns and skip the first row. And also can I get the name of the column using ResultSet something like:
if columnName.equals("Number")
Because I may have a csv file that could have more columns:
"CHAM-24","COMPANY NAME","Test","12",20031213,16,76
Number,Environ,Envel,Date,Time,Total,Count
"1","2","3","4","5",3,9
"6","7","8","9",9",,2
So I want to get the column name and then based on that column I do some other processing.
Once I read the value of each row I want to save the data to an Oracle Table. How do I connect to Oracle from my Application. As the database is on the server. Any help is really appreciated. Thanks

The only thing I could think of (and this is a cluj) would be to attempt to parse the first element of each row as a number. If it fails, you do not have a column name row. You are counting on the fact that you will never have a column name that is a number in the first position.
However, I agree that not always placing the headers in the same location is asking for trouble. If you have control over the file, format it how you want. If someone else has control over the file, find out why they are doing it that way. Maybe there is a "magic" number in the first row telling you where to jump.
Also, I would not use ODBC to simply parse a CSV file. If the file is formatted identically to Microsoft's format (headers in first row, all subsequent rows have same number of columns), then it's fine to take a shortcut and not write your own parser. But if the file is not adhering to that format, don't both using the M$ ODBC driver.
- Saish
"My karma ran over your dogma." - Anon

Reading CSV file, populating text fields, saving each as image to folder

I need to generate a bunch of images by reading a CSV file of records, populating various text fields over an image and then saving each iteration as a new image in a folder. Is Illustrator the best solution to this problem? And if so can someone help me get started?

In my opinion...
While Illustrator is an awesome tool its scripting DOM is a little limited, you may have better luck using InDesign as it has a much broader DOM.
If it is just simple text manipulations and you only require an image (not a document) Photoshop may work well for this as well.
Both the InDesign and Photoshop scripting forums seem to have much more activity as well.
For PhotoShop you can also use the awesome resources @: http://www.ps-scripts.com/bb/
Regards,
Herr Mueller

Reading csv file using file adapter

Hi,
I am working on SOA 11g. I am reading a csv file using a file adapter. Below are the file contents, and the xsd which gets generated by the Jdev.
.csv file:
empid,empname,empsal
100,Ram,20000
101,Shyam,25000
xsd generated by the Jdev:
<?xml version="1.0" encoding="UTF-8" ?>
<xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:nxsd="http://xmlns.oracle.com/pcbpel/nxsd" xmlns:tns="http://TargetNamespace.com/EmpRead" targetNamespace="http://TargetNamespace.com/EmpRead" elementFormDefault="qualified" attributeFormDefault="unqualified"
nxsd:version="NXSD"
nxsd:stream="chars"
nxsd:encoding="ASCII"
nxsd:hasHeader="true"
nxsd:headerLines="1"
nxsd:headerLinesTerminatedBy="${eol}">
<xsd:element name="Root-Element">
<xsd:complexType>
<xsd:sequence>
<xsd:element name="Child-Element" minOccurs="1" maxOccurs="unbounded">
<xsd:complexType>
<xsd:sequence>
<xsd:element name="empid" type="xsd:string" nxsd:style="terminated" nxsd:terminatedBy="," nxsd:quotedBy=""" />
<xsd:element name="empname" minOccurs="1" type="xsd:string" nxsd:style="terminated" nxsd:terminatedBy="," nxsd:quotedBy=""" />
<xsd:element name="empsal" type="xsd:string" nxsd:style="terminated" nxsd:terminatedBy="${eol}" nxsd:quotedBy=""" />
</xsd:sequence>
</xsd:complexType>
</xsd:element>
</xsd:sequence>
</xsd:complexType>
</xsd:element>
</xsd:schema>
For empname i have added minoccurs=1. Now when i remove the empname column, the csv file still gets read from the server, without giving any error.
Now, i created the following xml file, and read it through the file adapter:
<?xml version="1.0" encoding="UTF-8" ?>
<Root-Element xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://TargetNamespace.com/EmpRead xsd/EmpXML.xsd" xmlns="http://TargetNamespace.com/EmpRead">
<Child-Element>
<empid>100</empid>
<empname></empname>
<empsal>20000</empsal>
</Child-Element>
<Child-Element>
<empid>101</empid>
<empname>Shyam</empname>
<empsal>25000</empsal>
</Child-Element>
</Root-Element>
When i removed the value of empname, it throws the proper error for the above xml.
Please tell me why the behaviour of file adapter is different for the csv file and the xml file for the above case.
Thanks

Hi,
I am working on SOA 11g. I am reading a csv file using a file adapter. Below are the file contents, and the xsd which gets generated by the Jdev.
.csv file:
empid,empname,empsal
100,Ram,20000
101,Shyam,25000
xsd generated by the Jdev:
<?xml version="1.0" encoding="UTF-8" ?>
<xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:nxsd="http://xmlns.oracle.com/pcbpel/nxsd" xmlns:tns="http://TargetNamespace.com/EmpRead" targetNamespace="http://TargetNamespace.com/EmpRead" elementFormDefault="qualified" attributeFormDefault="unqualified"
nxsd:version="NXSD"
nxsd:stream="chars"
nxsd:encoding="ASCII"
nxsd:hasHeader="true"
nxsd:headerLines="1"
nxsd:headerLinesTerminatedBy="${eol}">
<xsd:element name="Root-Element">
<xsd:complexType>
<xsd:sequence>
<xsd:element name="Child-Element" minOccurs="1" maxOccurs="unbounded">
<xsd:complexType>
<xsd:sequence>
<xsd:element name="empid" type="xsd:string" nxsd:style="terminated" nxsd:terminatedBy="," nxsd:quotedBy=""" />
<xsd:element name="empname" minOccurs="1" type="xsd:string" nxsd:style="terminated" nxsd:terminatedBy="," nxsd:quotedBy=""" />
<xsd:element name="empsal" type="xsd:string" nxsd:style="terminated" nxsd:terminatedBy="${eol}" nxsd:quotedBy=""" />
</xsd:sequence>
</xsd:complexType>
</xsd:element>
</xsd:sequence>
</xsd:complexType>
</xsd:element>
</xsd:schema>
For empname i have added minoccurs=1. Now when i remove the empname column, the csv file still gets read from the server, without giving any error.
Now, i created the following xml file, and read it through the file adapter:
<?xml version="1.0" encoding="UTF-8" ?>
<Root-Element xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://TargetNamespace.com/EmpRead xsd/EmpXML.xsd" xmlns="http://TargetNamespace.com/EmpRead">
<Child-Element>
<empid>100</empid>
<empname></empname>
<empsal>20000</empsal>
</Child-Element>
<Child-Element>
<empid>101</empid>
<empname>Shyam</empname>
<empsal>25000</empsal>
</Child-Element>
</Root-Element>
When i removed the value of empname, it throws the proper error for the above xml.
Please tell me why the behaviour of file adapter is different for the csv file and the xml file for the above case.
Thanks

Addressbook cant read .csv-files exported from Numbers (Solved)

Hello,
I can hardly believe this myself, but please try. Numbers separates fields by semicolons while exporting my address data to a .csv file, which seems completely feasible to me. When trying to import this into Addressbook, the applications moans that it cant import this file because it is not valid.
Do programmers from separate groups at Apple ever test whether their Apps work together? I just expect them to do so and I cant imagine a case more simple then that. This is really poor. The reason why I put this complaint in the Numbers discussion is, that there is no forum for Adressbook (whereas iCal got his own - why?).
And the reason why I pick up this topic again in a new thread is that the other thread I found related to transferring data from Numbers to Addressbook took a weird twist when they resorted to writing a complicated and elaborate Apple Script for this basic task. The Script provided there messed up with valuable Address data in the first version, so I am not going to try it here.
I just took the search and replace functionality of my text editor to solve it for me. But I still believe that this should work without headaches right out of the box. So Addressbook programmers please hurry up, go, fix it. It is just too distressing. Imagine, I did this in public with Windows guys around. I dont want to experience such again.
Yours, Christian Völker

Hello Yvan,
yes, I agree that Numbers is doing fine and by your explanation I even know why it does the way it does, but Addressbook remains broken.
Regarding your script I did not even bother to read it and I just dont trust it because I cant sue you in case something goes wrong. I prefer to do my own faults and in this case it was an even better and faster solution for me to use my text editor with search and replace.
To make the story complete, after the data was accepted, Addressbook wasnt able to recognize the field labeled "Email" as an Email Address and Street as a Street Name. After telling Addressbook about the meaning of the fields, it consistently crashed while importing. No, this is no good. A Script wont heal these flaws.

ODI reading csv file issue

Similar Messages

Maybe you are looking for