Parse CSV file with some dynamic columns

I have a CSV file that I receive once a week that is in the following format:
"Item","Supplier Item","Description","1","2","3","4","5","6","7","8" ...Linefeed
"","","","Past Due","Day 13-OCT-2014","Buffer 14-OCT-2014","Week 20-OCT-2014","Week 27-OCT-2014", ...LineFeed
"Part1","P1","Big Part","0","0","0","100","50", ...LineFeed
"Part4","P4","Red Part","0","0","0","35","40", ...LineFeed
"Part92","P92","White Part","0","0","0","10","20", ...LineFeed
An explanation of the data - Row 2 is dynamic data signifying the date parts are due. Row 3 begins the part numbers with description and number of parts due on a particular date. So looking at the above data: row 3 column7 shows that PartNo1 has 100 parts
due on the Week of OCT 20 2014 and 50 due on the Week of OCT 27, 2014.
How can I parse this csv to show the data like this:
Item, Supplier Item, Description, Past Due, Due Date Amount Due
Part1 P1 Big Part 0 20 OCT 2014 100
Part1 P1 Big Part 0 27 OCT 2014 50
Part4 P4 Red Part 0 20 OCT 2014 35
Part4 P4 Red Part 0 27 OCT 2014 40
Is there a way to manipulate the format to rearrange the data like I need or what is the best method to resolve this? Moreover how do I go about doing this?

Hello,
If the files have the same structure you can create an Integration Service Package.
see this article
http://www.mssqltips.com/sqlservertip/2923/configure-the-flat-file-source-in-sql-server-integration-services-2012-to-read-csv-files/
Javier Villegas |
@javier_vill | http://sql-javier-villegas.blogspot.com/
Please click "Propose As Answer" if a post solves your problem or "Vote As Helpful" if a post has been useful to you

Similar Messages

How to generate a second csv file with different report columns selected?

Hi. Everybody:
How to generate a second csv file with different report columns selected?
The first csv file is easy (report attributes -> report export -> enable CSV output Yes). However, our users demand 2 csv files with different report columns selected to meet their different needs.
(The users don't want to have one csv file with all report columns included. They just want to get whatever they need directly, no extra columns)
Thank you for any help!
MZ

Hello,
I'm doing it usually. Typically example would be in the report only the column "FIRST_NAME" and "LAST_NAME" displayed whereas
in the csv exported with the UTL_FILE the complete address (street, housenumber, additions, zip, town, state ... ) is written, these things are needed e.g. the form letters.
You do not need another page, just an additional button named e.g. "export_to_csv" on your report page.
The csv export itself is handled from a plsql procedure "stored procedure" ( I like to have business logic outside of apex) which is invoked by pressing the button "export_to_csv". Of course the stored procedure can handle also parameters
An example code would be something like
PROCEDURE srn_brief_mitglieder (
     p_start_mg_nr IN NUMBER,
     p_ende_mg_nr IN NUMBER
AS
export_file          UTL_FILE.FILE_TYPE;
l_line               VARCHAR2(20000);
l_lfd               NUMBER;
l_dateiname          VARCHAR2(100);
l_datum               VARCHAR2(20);
l_hilfe               VARCHAR2(20);
CURSOR c1 IS
SELECT
MG_NR
,TO_CHAR(MG_BEITRITT,'dd.mm.yyyy') AS MG_BEITRITT ,TO_CHAR(MG_AUFNAHME,'dd.mm.yyyy') AS MG_AUFNAHME
,MG_ANREDE ,MG_TITEL ,MG_NACHNAME ,MG_VORNAME
,MG_STRASSE ,MG_HNR ,MG_ZUSATZ ,MG_PLZ ,MG_ORT
FROM MITGLIEDER
WHERE MG_NR >= p_start_mg_nr
AND MG_NR <= p_ende_mg_nr
--WHERE ROWNUM < 10
ORDER BY MG_NR;
BEGIN
SELECT TO_CHAR(SYSDATE, 'yyyy_mm_dd' ) INTO l_datum FROM DUAL;
SELECT TO_CHAR(SYSDATE, 'hh24miss' ) INTO l_hilfe FROM DUAL;
l_datum := l_datum||'_'||l_hilfe;
--DBMS_OUTPUT.PUT_LINE ( l_datum);
l_dateiname := 'SRNBRIEF_MITGLIEDER_'||l_datum||'.CSV';
--DBMS_OUTPUT.PUT_LINE ( l_dateiname);
export_file := UTL_FILE.FOPEN('EXPORTDIR', l_dateiname, 'W');
l_line := '';
--HEADER
l_line := '"NR"|"BEITRITT"|"AUFNAHME"|"ANREDE"|"TITEL"|"NACHNAME"|"VORNAME"';
l_line := l_line||'|"STRASSE"|"HNR"|"ZUSATZ"|"PLZ"|"ORT"';
     UTL_FILE.PUT_LINE(export_file, l_line);
FOR rec IN c1
LOOP
     l_line := '"'||rec.MG_NR||'"';
     l_line := l_line||'|"'||rec.MG_BEITRITT||'"|"' ||rec.MG_AUFNAHME||'"';
     l_line := l_line||'|"'||rec.MG_ANREDE||'"|"'||rec.MG_TITEL||'"|"'||rec.MG_NACHNAME||'"|"'||rec.MG_VORNAME||'"';
     l_line := l_line||'|"'||rec.MG_STRASSE||'"|"'||rec.MG_HNR||'"|"'||rec.MG_ZUSATZ||'"|"'||rec.MG_PLZ||'"|"'||rec.MG_ORT||'"';
--     DBMS_OUTPUT.PUT_LINE (l_line);
-- in datei schreiben
     UTL_FILE.PUT_LINE(export_file, l_line);
END LOOP;
UTL_FILE.FCLOSE(export_file);
END srn_brief_mitglieder;Edited by: wucis on Nov 6, 2011 9:09 AM

Download int table into csv file with each column in separate column in csv

Hi All,
I want to download the data in internal table to CSV file. but each column in the table should come as separate column in csv format.
CALL FUNCTION 'GUI_DOWNLOAD'
    EXPORTING
      FILENAME                = GD_FILE
      FILETYPE                = 'ASC'
      WRITE_FIELD_SEPARATOR   = 'X'
    tables
      DATA_TAB                = I_LINES_NEW
    EXCEPTIONS
      FILE_OPEN_ERROR         = 1
      FILE_READ_ERROR         = 2
      NO_BATCH                = 3
      GUI_REFUSE_FILETRANSFER = 4
      INVALID_TYPE            = 5
      NO_AUTHORITY            = 6
      UNKNOWN_ERROR           = 7
      BAD_DATA_FORMAT         = 8
      HEADER_NOT_ALLOWED      = 9
      SEPARATOR_NOT_ALLOWED   = 10
      HEADER_TOO_LONG         = 11
      UNKNOWN_DP_ERROR        = 12
      ACCESS_DENIED           = 13
      DP_OUT_OF_MEMORY        = 14
      DISK_FULL               = 15
      DP_TIMEOUT              = 16
      OTHERS                  = 17.
IF SY-SUBRC NE 0.
    WRITE: 'Error ', SY-SUBRC, 'returned from GUI_DOWNLOAD SAP OUTBOUND'.
    SKIP.
ENDIF.
with the above values passd , I am getting csv file but all the columns in one column separated by some square symbol.
How to separate them into different columns.
Thanks in advance
rgds,
Madhuri

Below example might help you understand on dowloading CSV file:
TYPE-POOLS: truxs.
DATA: i_t001 TYPE STANDARD TABLE OF t001,
      i_data TYPE truxs_t_text_data.
SELECT * FROM t001 INTO TABLE i_t001 UP TO 20 ROWS.
CALL FUNCTION 'SAP_CONVERT_TO_TEX_FORMAT'
EXPORTING
    i_field_seperator          = ','
*   I_LINE_HEADER              =
*   I_FILENAME                 =
*   I_APPL_KEEP                = ' '
TABLES
    i_tab_sap_data             = i_t001
CHANGING
   i_tab_converted_data       = i_data
EXCEPTIONS
   conversion_failed          = 1
   OTHERS                     = 2.
IF sy-subrc <> 0.
MESSAGE ID sy-msgid TYPE sy-msgty NUMBER sy-msgno
          WITH sy-msgv1 sy-msgv2 sy-msgv3 sy-msgv4.
ENDIF.
DATA: file TYPE string VALUE 'C:\testing.csv'.
CALL METHOD cl_gui_frontend_services=>gui_download
EXPORTING
    filename                = file
CHANGING
    data_tab                = i_data[]
EXCEPTIONS
    file_write_error        = 1
    no_batch                = 2
    gui_refuse_filetransfer = 3
    invalid_type            = 4
    no_authority            = 5
    unknown_error           = 6
    header_not_allowed      = 7
    separator_not_allowed   = 8
    filesize_not_allowed    = 9
    header_too_long         = 10
    dp_error_create         = 11
    dp_error_send           = 12
    dp_error_write          = 13
    unknown_dp_error        = 14
    access_denied           = 15
    dp_out_of_memory        = 16
    disk_full               = 17
    dp_timeout              = 18
    file_not_found          = 19
    dataprovider_exception = 20
    control_flush_error     = 21
    not_supported_by_gui    = 22
    error_no_gui            = 23
    OTHERS                  = 24.
IF sy-subrc <> 0.
MESSAGE ID sy-msgid TYPE sy-msgty NUMBER sy-msgno
             WITH sy-msgv1 sy-msgv2 sy-msgv3 sy-msgv4.
ENDIF.
Regards
Eswar

Unable to Load CSV file with comma inside the column(Sql Server 2008)

Hi,
I am not able load an CSV file with Comma inside a column.
Here is sample File:(First row contain the column names)
_id,qp,c
"1","[ ""0"", ""0"", ""0"" ]","helloworld"
"1","[ ""0"", ""0"", ""0"" ]","helloworld"
When i specify the Text Qualifier as "(Double quotes) it work in SQL Server 2012, where as fail in the SQL Server 2008, complaining with error:
TITLE: Microsoft Visual Studio
The preview sample contains embedded text qualifiers ("). The flat file parser does not support embedding text qualifiers in data. Parsing columns that contain data with text qualifiers will fail at run time.
BUTTONS:
OK
I need to do this in sql server 2008 R2 with Service pack 2, my build version is 10.50.1600.1.
Can someone let me know it is possible in do the same way it is handle in SQL Server 2012?
Or
It got resolved in any successive Cumulative update after 10.50.1600.1?
Regards Harsh

Hello,
If you have CSV with double quotes inside double quotes and with SSIS 2008, I suggest:
in your data flow you use a script transformation component as Source, then define the ouput columns id signed int, Gp unicode string and C unicode string. e.g. Ouput 0 output colmuns
Id - four-byte signed
gp - unicode string
cc - unicode string
Do not use a flat file connection, but use a user variable in which you store the name of the flat file (this could be inside a for each file loop).
The user variable is supplied as a a readonly variable argument to the script component in your dataflow.
In the script component inMain.CreateNewOutputRows use a System.IO.Streamreader with the user variable name to read the file and use the outputbuffer addrow method to add each line to the output of the script component.
Between the ReadLine instraction and the addrow instruction you have to add your code to extract the 3 column values.
public override void CreateNewOutputRows()
Add rows by calling the AddRow method on the member variable named "<Output Name>Buffer".
For example, call MyOutputBuffer.AddRow() if your output was named "MyOutput".
string line;
System.IO.StreamReader file = new System.IO.StreamReader( Variables.CsvFilename);
while ((line = file.ReadLine()) != null)
System.Windows.Forms.MessageBox.Show(line);
if (line.StartsWith("_id,qp,c") != true) //skip header line
Output0Buffer.AddRow();
string[] mydata = Yourlineconversionher(line);
Output0Buffer.Id = Convert.ToInt32(mydata[0]);
Output0Buffer.gp = mydata[1];
Output0Buffer.cc = mydata[2];
file.Close();
Jan D'Hondt - SQL server BI development

Is there a way to open CSV files with more than 255 columns?

I have a CSV file with more than 255 columns of data. It's a fairly standard export of social media data that shows volume of posts by day for the past year, from which I can analyze the data and publish customized charts. Very easy in Excel but I'm hitting the Numbers limit of 255 columns per table. Is there a way to work around the limitation? Perhaps splitting the CSV in two? The data shows up in the CSV file when I open via TextEdit, so it's there. Just can't access it in Numbers. And it's not very usable/useful for me in TextEdit.
Regards,
Tim

You might be better off with Excel. Even if you could find a way to easily split the CSV file into two tables, it would be two tables when you want only one. You said you want to make charts from this data. While a series on a chart can be constructed from data in two different tables, to do so takes a few extra steps for each series on the chart.
For a test to see if you want to proceed, make two small tables with data spanning the tables and make a chart from that data. Make the chart the normal way using the data in the first table then repeat the following steps for each series
Select the series in the chart
Go to Format sidebar
Click in the "Value" box
Add a comma then select the data for this series from the second chart
Press Return
If there is an easier way to do this, maybe someone else will chime in with that info.

I also have a .csv file with the name of a jpeg in one column and a text description of each jpeg in a second column. Is there a way to automatically insert one jpeg (photo) and its corresponding text, each pair on one page, into a Indesign document?

I also have a .csv file with the name of a jpeg in one column and a text description of each jpeg in a second column. Is there a way to automatically insert one jpeg (photo) and its corresponding text, each pair on one page, into a Indesign document?

I would also recommend to write the description into the meta data. This would allow to place a text frame above the image and it is possible to add meta information and file name automatically together with the image, when you place it or even in a prepared template.
Meta data information can be written easily in Bridge in the Meta File Workspace.

How to import data from CSV file with columns separated by semicolon?

I migrate database from MS SQL 2008 to ORACLE 11g
I export data to CSV file from MS SQL
then I try to import it to Oracle
several tables goes fine using Import data option in the SQL Developer
Standard CSV files with data separated by comma were imported.
chars, date (with format string), and integer data are imported via import wizard without problems
the problems were when I try to import table with noninteger numbers with modal part separated by comma not by dot
comma is the standard separator for columns in CSV file
so I must change the standard separator to semicolon
then the import wizard have problem to correct recognize the columns data because it use only standard CSV comma separator :-/
In SQL Developer 1.5.3 Tools -> Preferences -> Migration -> Data Move Options
I change "End of Column Delimiter" to ; but it doens't work
Is this possible to change the standard column separator for import data wizzard in SQL Developer 1.5.3?
Or maybe someone know how to import data in SQL Developer 1.5.3 from CSV when CSV colunn separator is set to semicolon?

A new preference has been added to customize the import delimiter in main code line. This should be available as part of future release.

Reading a csv file with a large number of columns

Hello
I have been attempting to read data from large csv files with 38 columns by reading a line using readline and scanning the linebuffer using scan.
The file size can be up to 100 MB.
Scan does not seem support the large number of fields.
Any suggestions on reading the 38 comma separated fields. There is one header line in the file.
Thanks
Solved!
Go to Solution.

see if strtok() is useful http://www.elook.org/programming/c/strtok.html

Parser - CSV files to Oracle database

Hello all,
I wrote a csv parser that parse csv files to micorsoft access database. Now I need to parse the dataset to oracle database. This is part of my exporter class
class MdbExporter : IExporter
/// <summary>
/// Exportiert das DataSet in eine Mdb-Datei.
/// </summary>
/// <param name="ds">Zu exportierendes DataSet.</param>
public void Write(DataSet ds, string[] names)
string conStr = "Provider=Microsoft.Jet.OLEDB.4.0;" +
"Data Source=" + names[0] + ";";
Console.WriteLine("Exporting to database {0} ...", names[0]);
Any ideas how can I do that?
Thanks for any replay in advance.

I wrote a parser that goes through several folders containing csv files, reads them and creates a dataset that contains 15 tables ( which I need for an intranet application after that). I succeed to write a MDB exporter that export this dataset to microsoft access database. Now I need an exporter that will export my dataset to ORACLE database. Below is the full code for my MDB exporter:
public void Write(DataSet ds, string[] names)
string conStr = "Provider=Microsoft.Jet.OLEDB.4.0;" +
"Data Source=" + names[0] + ";";
Console.WriteLine("Exporting to database {0} ...", names[0]);
DbConnection connection = new OleDbConnection(conStr);
try
connection.Open();
catch (DbException e)
ConsoleEx.WriteException(true, e, "Unable to open database: {0}", names[0]);
throw;
DbCommand command = connection.CreateCommand();
foreach (DataTable table in ds.Tables)
Console.WriteLine("\tDeleting table: {0}", table.TableName);
// delete old tables
command.CommandText = string.Format("drop table {0}", table.TableName);
TryExecute(command, false);
// create new
Console.WriteLine("\tCreating new table: {0}", table.TableName);
string[] columnStrings = new string[table.Columns.Count];
for (int i = 0; i < table.Columns.Count; i++)
columnStrings[i] = "`" + table.Columns.ColumnName + "`" + " varchar";
command.CommandText = string.Format("create table {0} ({1})",
table.TableName, string.Join(", \n", columnStrings));
TryExecute(command, true);
// add rows
for (int row = 0; row < table.Rows.Count; row++)
for (int col = 0; col < table.Columns.Count; col++)
columnStrings[col] = "'" + Convert.ToString(table.Rows[row].ItemArray[col]) + "'";
command.CommandText = string.Format("insert into {0} values ({1})",
table.TableName, string.Join(", \n", columnStrings));
TryExecute(command, true);
connection.Close();
I need similar exporter to Oracle database. Starting with SQL LOADER from the beginning its really not a good in my opinion since I am almost done with this approach.
Any help would be appreciated.
Regards,
Sven

CSV file template - values in columns

Hi experts,
I'm with a requirement to upload transactional data from a CSV file. And it's working, but with values in lines like below:
CostCenter        P_ACCT             Time                  Amount
1101450               341010101     2012.DEZ          56000
1101450               341010102     2012.DEZ          4000
1101450               341010103     2012.DEZ          13000.99
Just in terms of to make the user's life easy, considering that this way the file will have thounsands of lines, I would like to know if is possible to upload data from a CSV file with the key figure in columns. Something like:
CostCenter        P_ACCT             JAN          FEB         MAR        MAY        JUN          JUL          AUG ........
1101450               341010101     56000     56000     56000     56000     56000     56000     56000........
1101450               341010102     4000     56000     56000     56000     56000     56000     56000........
1101450               341010103     13000.99     56000     56000     56000     56000     56000     56000........
If I can't manage this in BPC, I'll have to create a macro to put the values in lines.
Regards!
Lucas

You can use a transformation file to get this in a couple of ways I think.
See the help file topic on *MVAL (NW 7.5 version)
http://help.sap.com/saphelp_bpc75_nw/helpdata/en/b8/a76a1ca9ac4ca698259a8ff397bb61/frameset.htm
Also you could try this (from the help file)
*<Dimension> =<value1>,<value2> has the similar effect of an *MVAL command in the mapping section. This type of header occurs only in the beginning of the data file, not in the middle.
*CATEGORY=ACTUAL
*TIME=1999.JAN, 1999.FEB, 1999.MAR,1999.APR
*ENTITY, ACCOUNT, PRODUCT
*AMOUNT UK, SALES, SEDAN, 100, 200, 300, 400
Good luck!

How to reject external table rows with some blank columns

How to reject external table rows with some blank columns
I have an external table and I would like to reject rows when a number of fields are empty. Here are the details.
CREATE TABLE EXTTAB (
ID NUMBER(10),
TSTAMP DATE,
C1 NUMBER(5,0),
C2 DATE,
C3 FLOAT(126)
ORGANIZATION EXTERNAL (
TYPE ORACLE_LOADER
DEFAULT DIRECTORY EXT_DAT_DIR
ACCESS PARAMETERS (
RECORDS DELIMITED BY NEWLINE
LOAD WHEN (NOT (c1 = BLANKS AND c2 = BLANKS AND c3 = BLANKS))
LOGFILE EXT_LOG_DIR:'exttab.log'
BADFILE EXT_BAD_DIR:'exttab.bad'
DISCARDFILE EXT_BAD_DIR:'exttab.dsc'
FIELDS TERMINATED BY "|"
LRTRIM
MISSING FIELD VALUES ARE NULL
REJECT ROWS WITH ALL NULL
FIELDS (
ID,
TSTAMP DATE 'YYYYMMDDHH24MISS',
C1,
C2 DATE 'YYYYMMDDHH24MISS',
C3
) LOCATION ('dummy.dat')
REJECT LIMIT UNLIMITED
So, as you can see from the LOAD WHEN clause, I'd like to reject rows when C1, C2 and C3 are empty.
The above statement works fine and creates the table. However when I am trying to load data using it, the following error is produced:
ORA-29913: error in executing ODCIEXTTABLEOPEN callout
ORA-29400: data cartridge error
KUP-00554: error encountered while parsing access parameters
KUP-01005: syntax error: found "not": expecting one of: "double-quoted-string, identifier, (, number, single-quoted-string"
KUP-01007: at line 1 column 41
ORA-06512: at "SYS.ORACLE_LOADER", line 14
ORA-06512: at line 1
It seems that external tables driver does not understand the "NOT (...)" condition. Could anyone suggest how I can achieve what I want in a different way?
Thank you.
Denis

Another method would be to simply remove the "LOAD WHEN condition" and create a view on the external table which filters the data.
CREATE EXTTAB_VIEW AS
SELECT * FROM EXTTAB
WHERE not (c1 is null and c2 is null and c3 is null);

How to create a csv file with NCS attributes?

Hi
i installed Cisco Prime NCS and trying to perform bulk update of device credentials with csv file.
How to create a csv file with all required attributes?
This is part of NCS online help talking about this topic:
Bulk Update Devices—To update the device credentials in a bulk, select Bulk Update Devices from the Select a command drop-down list. The Bulk Update Devices page appears.You can choose a CSV file.
Note The CSV file contains a list of devices to be updated, one device per line. Each line is a comma separated list of device attributes. The first line describes the attributes included. The IP address attribute is mandatory.
Bellow is test csv file i created but does not work:
10.64.160.31,v2c,2,10,snmpcomm,ssh,zeus,password,password,enablepwd,enablepwd,60
10.64.160.31,v2c,2,10,snmpcomm,ssh,zeus,password,password,enablepwd,enablepwd,60
The error i am getting while importing this file:
Missing mandatory field [ip_address] on header line:10.64.160.31,v2c,2,10,snmpcomm,ssh,zeus,password,password,enablepwd,enablepwd,60
Assistance appreciated.

It looks like the IP address field is incorrectly set.,
It should be as follows
{Device IP},{Device Subnet Mask}, etc etc
so a practical example of the aboove could be (i dont know if this is completely correct after the IP address / Subnet Mask)
10.64.160.31,255.255.255.0,v2c,2,10,snmpcomm,ssh,zeus,password,password,enablepwd,enablepwd,60
below is a link to the documentation
http://www.cisco.com/en/US/docs/wireless/ncs/1.0/configuration/guide/ctrlcfg.html#wp1840245
HTH
Darren

I have made .indd file with some form fields in it, now haw to read these fields and manipulate its content through script in indesign

I have made .indd file with some form fields in it, now haw to read these fields and manipulate its content through script in indesign

It's probably best to ask in the InDesign Scripting forum:
InDesign Scripting

How to upload .csv file with tab as delimiter.

HI,
I want to upload a .csv file with tab as delimiter to unix path in background.
I know there is function module 'GUI_UPLOAD', but in my case data is available in an internal table .
upload the *.CSV :
OPEN DATASET lv_filename
         FOR OUTPUT IN TEXT MODE ENCODING DEFAULT.
    IF sy-subrc = 0.
*     Write file records into file on application server
      LOOP AT gt_datatab INTO gs_datatab.
        TRANSFER gs_datatab TO lv_filename.
      ENDLOOP.
CLOSE DATASET lv_filename.

Bhanu,
Define a local variable of type CL_ABAP_CHAR_UTILITIES=>HORIZONTAL_TAB.
LV_TAB TYPE CL_ABAP_CHAR_UTILITIES=>HORIZONTAL_TAB
LOOP AT GT_DATATAB INTO GS_DATATAB.
CONCATENATE GS_DATATAB-FIELD1
                         GS_DATATAB-FIELD2
                         GS_DATATAB-FIELDN
                         INTO LV_STRING SEPARATED BY LV_TAB.
TRANSFER LV_STRING TO lv_filename.
ENDLOOP.
Thanks,
Vikram.M

Write the results script of results log pane to XLS or CSV file with VBA.

Hi,
How can I write the results script of results log pane to XLS or CSV file with VBA code or something? I tried so hard but i can't.
Thanks

MoGas,
This is actually not a trivial process. You need to use the results object and code it to write to your file (it is described in the help files).
e-Tester automatically saves the results log as a text file so you may just want to stick with that for simplicity.

Parse CSV file with some dynamic columns

Similar Messages

Maybe you are looking for