CSV dataset columnNames

Hi all,
I'm loading in a CSV file into a dataset, and setting the
columnNames accordingly.
csv: 1,2,3,4,5 names: A,B,C,D,E
However, in some entries in the csv file, the column doesnt
have any content, eg. 1,2,,4,5. In these cases, I'd expect the
empty column to still have a name, but no value. However, this
doesnt appear to be the case, the columns are shuffled along so the
empty ones become undefined at the end, ie.
1,2,3,4,5 = a=1, b=2, c=3, d=4, e=5
1,2,,4,5 = a=1, b=2, c=4, d=5, undefined
I'd expect it to be a=1, b=2, c=undefined, d=4, e=5 ie. c
would have no value. I am currently trying to filterdata on column
D when it contains a certain value, but this is failing all over
the place as it only seems to work when all the other fields are
populated, which in my csv file is only on about 5 lines out of
1800!

I am having this exact problem. Did you ever find a solution?

Similar Messages

Csv dataset not showing repeats in slide show

I created a csv dataset slide show. The repeat areas showed at first but is no longer showing. I have checked the code for misplaced commas, non-allowed special characters, etc. I am baffled as to where the repeat went to.
The page can be seen here: http://kathleenhughesart.com/flowers.html
Thanks for any help, Judy.

Add a carriage return to the last line in the csv file.
Gramps

How do I TRULY remove the CRLF (#) from the last field read in CSV dataset?

Good day, everyone!
PLEASE NOTE: I spent most of yesterday searching SDN and reading all kinds of threads on this topic. Yes, I know, there ARE other threads out there regarding this topic, but I spent all of yesterday afternoon trying every solution posted and nothing worked. So, to my knowledge, nobody has yet to post a definite solution to this problem.
I am reading a comma-delimited CSV file from our Application Server. It was originally in Microsoft Excel but saved as a CSV file. I open the file as follows:
OPEN DATASET p_fname FOR INPUT IN TEXT MODE ENCODING DEFAULT.
Here is my loop to read the entire file into an internal table, splitting it into individual fields:
READ DATASET p_fname INTO wa_unsplit.
WHILE sy-subrc EQ 0.
    ADD 1 TO w_unsplit_tot.
    SPLIT wa_unsplit AT w_comma INTO:
          wa_split-massn
          wa_split-massg
          wa_split-curr
          wa_split-persg
          wa_split-pernr
          wa_split-persid
          wa_split-persk
          wa_split-stat2
          wa_split-fisc_year
          wa_split-funds_center
          wa_split-plans
          wa_split-orgeh
          wa_split-abkrs
          wa_split-werks
          wa_split-sem_posit
          wa_split-ansal
          wa_split-bsgrd
          wa_split-adm_adj_amt
          wa_split-hourly_rate.
    APPEND wa_split TO it_split.
    CLEAR: wa_unsplit,
           wa_split.
    READ DATASET p_fname INTO wa_unsplit.
ENDWHILE.
The problem is that the last field, wa_split-hourly_rate (defined as character length 17) gets a '#' appended to the end of it. This happens with each record, and it appears to be the CR/LF character (the value of it in hex is the same as cl_abap_char_utilities=>cr_lf).
I've tried everything previously recommended to strip this character from my hourly_rate field. I've tried another STRIP command. I've tried REPLACE and TRANSLATE and a whole slew of things. Despite all the threads that exist on SDN about this, I've yet to find something that truly works.
Certainly I can't be the first person reading a file from the Application Server and having this issue.
Please do NOT post links to solutions that DO NOT WORK! Full points will be awarded to anyone who can solve this challenge.
Thanks everyone!

Success! Peluka and Naimesh, I had to combine your two recommendations, and I finally got it to work!
Thanks for the clarification, Rich. I ended up calling my "aux" field "junk" for lack of a better word, but in the end you confirmed what I thought it was.
For anyone else out there who has struggled finding a successful answer to this challenge, here's what worked for me:
CONSTANTS: c_comma(1) TYPE c VALUE ',',
           c_crlf(1)   TYPE c VALUE cl_abap_char_utilities=>cr_lf.
DATA: w_junk          TYPE string.
READ DATASET p_fname INTO wa_unsplit.
WHILE sy-subrc EQ 0.
    ADD 1 TO w_unsplit_tot.
    SPLIT wa_unsplit AT c_comma INTO:
          wa_split-massn
          wa_split-massg
          wa_split-curr
          wa_split-persg
          wa_split-pernr
          wa_split-persid
          wa_split-persk
          wa_split-stat2
          wa_split-fisc_year
          wa_split-funds_center
          wa_split-plans
          wa_split-orgeh
          wa_split-abkrs
          wa_split-werks
          wa_split-sem_posit
          wa_split-ansal
          wa_split-bsgrd
          wa_split-adm_adj_amt
          wa_split-hourly_rate.
And here is the line that removes the CR/LF character from the end of the Hourly Rate field!
    SPLIT wa_split-hourly_rate AT c_crlf INTO:
    wa_split-hourly_rate w_junk.
    APPEND wa_split TO it_split.
    CLEAR: wa_unsplit,
           wa_split.
    READ DATASET p_fname INTO wa_unsplit.
ENDWHILE.
Thanks so much, everyone! Once again, SDN folks save the day. Points awarded!

Is it possible to get ssrs report foldername, reportname, its dataset, columnname and underlinetables column name?

I wrote a query which give report foldername, reportname, username and role but i need to include datasetname, columnname in dataset and colum name in underline tables.
Select distinct a.FolderName, a.ReportName, U.UserName as [AD Group/ USer], Rol.RoleName as ROLE FROM dbo.Users U
inner Join
dbo.PolicyUserRole p ON U.UserID = p.UserID
inner join
dbo.Policies POLICY ON POLICY.PolicyID = p.PolicyID INNER JOIN
dbo.Roles ROL ON ROL.RoleID = P.RoleID INNER JOIN
(select C.PolicyID,C.Name as FolderName ,D.Name as ReportName from dbo.Catalog c
inner Join
dbo.Catalog D
on c.ItemID=d.ParentID ) a
on a.PolicyID=p.PolicyID
where FolderName<>' '

see
http://sornanara.blogspot.in/2011/05/ssrs-reportserver-database-tables.html
Please Mark This As Answer if it solved your issue
Please Mark This As Helpful if it helps to solve your issue
Visakh
My MSDN Page
My Personal Blog
My Facebook Page

Need help in HTML(CSV) Dataset pagination!

Dear sir!
Is it possible to create pagination in dataset (HTML or CSV) like MySQL recordset pagination. It'll be great idea show 10 or 20 rows instead of 500 or 1000. And the second question, is it possible to get specific data (rows) from dataset like query in MySQL.
Thank you!

Hi mutia
If I understand you correctly, the code below is what you're looking for. It is just a crude example with a start screen with a help button, when you click on the help button, a new frame is opened and a string containing HTML tags is displayed. You can compile and test it like this:
javac myFrame.java
java myFrame
myFrame.java
=======================
import java.awt.*;
import java.awt.event.*;
import javax.swing.*;
public class myFrame extends JFrame implements ActionListener {
   public static void main(String args[]) {
      myFrame frame = new myFrame();
      frame.pack();
      frame.setVisible(true);
   public myFrame() {
      setTitle("Main Frame");
      JPanel pan=new JPanel();
      pan.setLayout(new BorderLayout());
      pan.setPreferredSize(new Dimension (200, 100));
      setContentPane(pan);
      JButton helpButton = new JButton("H E L P");
      pan.add(helpButton);
      helpButton.addActionListener(this);
      addWindowListener(new WindowAdapter() {
         public void windowClosing(WindowEvent e) {
            System.exit(0);
   public void actionPerformed(ActionEvent e) {
      JFrame F=new JFrame();
      F.setTitle("Help Screen");
      JPanel pan=new JPanel();
      pan.setLayout(new BorderLayout());
      pan.setPreferredSize(new Dimension (200, 100));
      F.setContentPane(pan);
* lab can be read in from a file but I'll leave that part up to you
      JLabel lab=new JLabel("<html><font size=+1 color=blue>This is a test help<br>message</html>");
      pan.add(lab);
      F.pack();
      F.setVisible(true);
}V.V.

Upload file in to SAP for 'ENTER' key delimited

Hi SAP gurus,
I have one issue while uploading the textfile into sap the file is contains 16bit char i.e. chinese char and req is as below
there are many records in the file and i need to identify the next record by identifying ENTER key pressed.Insted of TAB key here its a ENTER key
so please tell me how to achieve this i am using GUI_UPLOAD to upload text file.
I want to identify where the enter key is pressed in the text file by reading ascii value of ENTER KEY will help me to find the starting of second record .
The file format is as follows
mmmm~~??????
1~20110101~20111231
//???: FIRST RECORD
0~00130698114013434412011042604823700000913.030.172.22TestingCustomer310104120000000???????????????130601000000000??????532611-3357211??~~~
0~CECLOR SUSP125EA13.030.172.2213.030~1510 FROM HERE SECOND RECORD its came after pressing enter
1~0013069811401234512011042604823700001013.030.172.22Testing Customer310104120000000???????????????130601000000000??????532611-3357211??~~~
0~CE125 6 XEA13.030.172.2213.030~1510
Thanks
Nirmal

Are you sure that it's a ENTER key and not CR or CRLF (hex 0D0A)?
If it is CRLF (carriage return-line feed, you can check by looking at the Hex value of the string), you can split as described in this thread:
How do I TRULY remove the CRLF (#) from the last field read in CSV dataset?
Or see the class: cl_abap_char_utilities=>NEWLINE or FORM_FEED for other possibilities.
for instance:
do.
split (your input line) at cl_abap_char_utilities=>newline into line.
if sy-subrc ne 0.
exit.
endif.
append line to internal table.
enddo.
The easiest way to process this would be to have the file come to you as lines of data with horizontal tab delimiters, without those multiple tildes, since that's something else you will have to deal with.
Edited by: DaveL on Aug 1, 2011 7:58 PM

Serving large files for download using JSF

Hello Community,
my JSF Controler Bean has a method which is called using method binding on either a commandButton or commandLink. It's purpose is to stream CSV datasets from a database to the browser, which then opens an application like ms excel, oocalc or gnumeric. Everything works fine if there is only a small number of datasets. But if there are several thousand datasets I get an OutOfMemoryError. So it seems that the data is somehow cached/buffered by the JSF Servlet. Is there a known workaround to my problem?
public void exportFile() {
        FacesContext context = FacesContext.getCurrentInstance();
        HttpServletResponse response =
(HttpServletResponse)context.getExternalContext().getResponse();
        response.setContentType("text/csv");
        response.setHeader("Content-disposition",
"attachment; filename=data.csv");
        try {
                 * This method iterates a java.sql.ResultSet and writes the data
                 * to the ServletOutputStream: "Write and Forget"
                writeData(response.getOutputStream());   // <<< If there are many datasets an OutOfMemoryError is produced
                response.getOutputStream().flush();
                response.getOutputStream().close();
                context.responseComplete();
        } catch (IOException e) {
                e.printStackTrace();
}Thanks, Alexander

Chrisse-
Ann is one of the most knowledgeable Photoshop folks here and I respect her immensely. However she comes from a long film background and is brand new to quality digital camera capture. Ann is
incorrect when she says
>I think that it is probably a toss-up between which would be worse: an interpolated rez-up or a 170 ppi print. Basically, Chrisse needs a better camera if she wants to make prints of this size let alone 11" x 14" ones.
Ann speaks from very extensive film-scan experience, but the reality is that "uprezzing" digital camera image capture is a whole different thing than uprezzing scanned film images. Those of us who do have substantial digicam experience have found a surprising ability to successfully uprez digicam image captures.
Certainly,
well-shot pix from your G9 will normally uprez to print 8x10s just fine and probably 11x14s as well.
Do not hesitate to experiment with uprezzing digicam captures - - including very large amounts of uprez like 2x or more. Test - test - test because each image and how presented/used is different.
I have found however that "well-shot," especially as regards exposure and focus, is important to allow good uprez. Also be especially careful with post-process edits because uprezzing can exacerbate editing distortion.
You need not worry about whether Ann is correct or I am - - just do it and judge the results. Do to typical viewing distances personally I use 300-360 ppi for small prints, 240-300 ppi for 8x10 and 240 ppi for 11x14; 180 ppi for large posters. But like others have said, if the ppi comes reasonably close at your chosen image size do not force a recalculation to reach precisely 240 ppi or whatever, just leave the resample box unchecked.

File transfer Open dataset CSV file Problem

Hi Experts,
I have an issue in transferring Korean characters to a .CSV file using open dataset.
data : c_file(200) TYPE c value '
INTERFACES\In\test8.CSV'.
I have tried
open dataset c_file for output LEGACY TEXT MODE CODE PAGE '4103'.
open dataset c_file for output    in TEXT MODE ENCODING NON-UNICODE.
open dataset c_file for output    in TEXT MODE ENCODING Default.
Nothing is working.
But to download to the presentation server the below code is working. How can the same be achieved for uploading the file to application server.
CALL METHOD cl_gui_frontend_services=>gui_download
      EXPORTING
        filename                = 'D:/test123.xls'
        filetype                = 'ASC'
        write_field_separator   = 'X'
        dat_mode                = 'X'
        codepage                = '4103'
        write_bom               = 'X'
      CHANGING
        data_tab                = t_tab
      EXCEPTIONS
        file_write_error        = 1
        no_batch                = 2
        gui_refuse_filetransfer = 3
        invalid_type            = 4
        no_authority            = 5
        unknown_error           = 6
        header_not_allowed      = 7
        separator_not_allowed   = 8
        filesize_not_allowed    = 9
        header_too_long         = 10
        dp_error_create         = 11
        dp_error_send           = 12
        dp_error_write          = 13
        unknown_dp_error        = 14
        access_denied           = 15
        dp_out_of_memory        = 16
        disk_full               = 17
        dp_timeout              = 18
        file_not_found          = 19
        dataprovider_exception = 20
        control_flush_error     = 21
        not_supported_by_gui    = 22
        error_no_gui            = 23
        OTHERS                  = 24.

Hi,
I would recommend to use OPEN DATASET ... ENCODING UTF-8 ...
If your excel version is unable to open this format, you can convert from 4110 to 4103 with report RSCP_CONVERT_FILE.
Please also have a look at
File upload: Special character
Best regards,
Nils Buerckel

Saving .csv into internal table - using dataset (',' comes between data)

Hi experts,
I need to save .csv from application server to internal table.
i am using the below code.
gt_raw and gwa_raw are dxrawdata format.
OPEN DATASET gv_pfile FOR INPUT IN TEXT MODE ENCODING DEFAULT.
*--- Display error messages if any.
IF sy-subrc NE 0.
    WRITE:/ 'FILE UPLOAD FAILED - ERROR NO. : ', sy-subrc.
    EXIT.
ELSE.
    DO.
      READ DATASET gv_pfile INTO gwa_raw.
      IF sy-subrc NE 0.
        EXIT.
      ELSE.
        APPEND gwa_raw TO gt_raw.
        CLEAR gwa_raw.
      ENDIF.
    ENDDO.
*--Close the Application server file (Mandatory).
    CLOSE DATASET gv_pfile.
ENDIF.
DELETE DATASET gv_pfile.
LOOP AT gt_raw into gwa_raw.
    IF SY-TABIX > 1.
      SPLIT gwa_raw at ',' into gwa_cust-cust_code
                                gwa_cust-cust_name
                                gwa_cust-grp_name
      APPEND gwa_cust TO gt_cust.
      CLEAR: gwa_cust, gwa_raw.
    ENDIF.
ENDLOOP.
My program works fine.
But when the gwa_cust-grp_name contains the value for eg. -> panasonic co., ltd.
it takes till panasonic co., only
and leaves ltd. as i am using SPLIT command.
is there any other way to do this.
plz help me to solve this issue.
thanks.

Hi,
I notice you have marked the message as answered, but I just wanted to let you know there is a solution. The trick is to parse into an internal table and then to find and reassemble fields that were split because they contgain a comma. The ABAP program below is a commented example.
Rgds,
Mark
REPORT zcsv_parse.
DATA:
tokens       TYPE i.
TYPES: BEGIN OF ty_result,
company      TYPE char20,
compnr       TYPE i,
city         TYPE char30,
country      TYPE char30,
END OF ty_result.
DATA:
gt_rawtab    TYPE TABLE OF string,
gw_rawtab    LIKE LINE OF gt_rawtab,
gt_result    TYPE TABLE OF ty_result,
gw_result    LIKE LINE OF gt_result,
gt_parse     TYPE TABLE OF string,
gw_parse     LIKE LINE OF gt_parse.
DEFINE %csvline.
gw_rawtab = &1.
append gw_rawtab to gt_rawtab.
END-OF-DEFINITION.
START-OF-SELECTION.
* Create CSV lines, some with a comma inside a token
%csvline '"CompanyOne NV",500,"Antwerp","Belgium"'.
%csvline '"CompanyTwo,Inc",600,"New York,NY","USA"'.
%csvline '"CompanyThree,Ltd",700,"Sydney,NSW","Australia"'.
* Parse the raw CSV
LOOP AT gt_rawtab INTO gw_rawtab.
    REFRESH gt_parse.
    SPLIT gw_rawtab AT ',' INTO TABLE gt_parse.
    DESCRIBE TABLE gt_parse LINES tokens.
*   If extra commas: token count higher than field count
    IF tokens > 4.
      PERFORM reassemble.
    ENDIF.
*   At this point each entry in GT_PARSE contains exactly
*   one result field => build the result table
    LOOP AT gt_parse INTO gw_parse.
*     Strip quotes from text fields
      REPLACE ALL OCCURRENCES OF '"' IN gw_parse WITH ''.
      CASE sy-tabix.
        WHEN 1. gw_result-company = gw_parse.
        WHEN 2. gw_result-compnr = gw_parse.
        WHEN 3. gw_result-city = gw_parse.
        WHEN 4. gw_result-country = gw_parse.
      ENDCASE.
    ENDLOOP.
    APPEND gw_result TO gt_result.
ENDLOOP.
* Show the formatted result
LOOP AT gt_result INTO gw_result.
    WRITE: / gw_result-company, gw_result-compnr,
             gw_result-city, gw_result-country.
ENDLOOP.
*&      Form reassemble
*       Merges tokens that were split because they contain a comma
FORM reassemble.
DATA: lastpos    TYPE i,
        lastchar   TYPE c,
        currtoken LIKE sy-tabix,
        nexttoken LIKE sy-tabix,
        gw_next    LIKE gw_parse.
LOOP AT gt_parse INTO gw_parse.
    lastpos = STRLEN( gw_parse ) - 1.
    lastchar = gw_parse+lastpos(1).
*   Token starts with quote but does not end with one =>
*   must merge with the next token
    IF gw_parse+0(1) = '"' AND lastchar <> '"'.
      currtoken = sy-tabix.
      nexttoken = sy-tabix + 1.
      READ TABLE gt_parse INTO gw_next INDEX nexttoken.
      CONCATENATE gw_parse gw_next INTO gw_parse SEPARATED BY ','.
      MODIFY gt_parse FROM gw_parse INDEX currtoken.
      DELETE gt_parse INDEX nexttoken.
    ENDIF.
ENDLOOP.
ENDFORM.                    "reassemble

Issues when Downloading Large Datasets to Excel and CSV

Hi,
Hoping someone could lend a hand on the issues described below.
I have a prompted dahsboard that, dependent upon prompts selected, can return detail datasets. THe intent of this dashboard is to AVOID giving end users Answers Access, but still providing the ability to pull large amounts of detail data in an ad-hoc fashion. When large datasets are returned, end users will download the data to thier local machines and use excel for further analysis. I have tried two options:
1) Download to CSV
2) Download data to Excel
For my test, I am uses the dashboard prompts to return 1 years (2009) worth of order data for North America, down to the day level of granularity. Yes alot of detail data...but this is what many "dataheads" at my organization are requesting...(despite best efforts to evangelize the power of OBIEE to do the aggregation for them...). I expext this report to return somewhere around 200k rows...
Here are the results:
1) Download to CSV
Filesize: 78MB
Opening the downloaded file is failrly quick...
126k rows are present in the CSV file...but the dataset abruptly ends in Q3(August) 2009. The following error appears at the end of the incomplete dataset:
<div><script language="javascript" src="res/b_mozilla/browserdom.js"></script><script language="javascript" src="res/b_mozilla/common.js"></script><div class="ErrorMessage">Odbc driver returned an error (SQLFetchScroll).</div><div style="margin-top:2pt" onclick="SAWMoreInfo(event); return false;"><img class="ErrorExpanderImg" border="0" src="res/sk_oracle10/common/errorplus.gif" align="absmiddle">  Error Details<div style="margin-left:15px;display:none" compresssrc="res/sk_oracle10/common/errorminus.gif">
<div class="ErrorCodes">Error Codes: <span dir="ltr">OPR4ONWY:U9IM8TAC</span></div>
<div style="margin-top:4pt"><div class="ErrorSubInfo">State: HY000. Code: 10058. [NQODBC] [SQL_STATE: HY000] [nQSError: 10058] A general error has occurred.
[nQSError: 46073] Operation 'stat()' on file '/opt/apps/oracle/obiee/OracleBIData/tmp/nQS_31951_2986_15442940.TMP' failed with error: (75) ,Çyô@BÀŽB@B¨Ž¡pÇôäìž¬ü5HB. (HY000)</div></div></div></div></div>
2) Download to Excel
Filesize: 46MB
Opening the Excel file is extremely painful...over 20 minutes to open the file...making excel unusable during the opening process...defeinately not acceptable for end users.
When opened the file contains only 65k rows...when there should be over 200k...
Can you please help me understand the limitations of detail data output (downloading) from OBIEE...or provide workarounds for the circumstances above?
Thanks so much in advance.
Adam
Edited by: AdamM on Feb 9, 2010 9:01 PM
Edited by: AdamM on Feb 9, 2010 9:02 PM

@chandrasekhar:
Thanks for your response. I'll try with the export button but also willing to know how to create button on toolbar.And by clicking on that button a popup box will come having two radio buttons asking to download the report either in .xls or in .csv format. I am looking for the subroutines for that.
Thanks.
Message was edited by:
        cinthia nazneen

Trying to import from different csv files into multiple JTables

This might seem like something that isn't commonly done, but I'll see if anyone has done this before anyways.
I currently have a JTable (tab #1) and it successfully reads in the data from a CSV file to populate the cells. Now, I am looking into making 3 more JTables. I can tab between the four different tables, but I can't seem to figure out how to get the data to go to the correct JTable. Tab #1 = table, Tab #2 = table2, Tab #3 = table3, Tab #4 = table4.
Thanks.
        try
            FileInputStream fileInput = new FileInputStream ("upcoming.csv");
            BufferedReader InputCSV = new BufferedReader (new InputStreamReader (fileInput));
            line = InputCSV.readLine(); //start reading into the records.
                //Create data for the table.
                //the row variable = starting row
            int row = 0;
            while(line != null)
                tmp = line.split(",");
                for (int col = 0; col < columnNames.length; ++col)
                    data[row][col] = tmp[col]; //store cells' data to the 2-dimensional array.
                row++;
                line = InputCSV.readLine();
            }//end of WHILE-statement.
        } //END of TRY-statement.
        catch(Exception e)
            System.out.println(e);
        }

I have been successful in my quest to get 4 different datasets into 4 different JTables. The previous posters suggestion to use the Table model led me to this page:
http://java.sun.com/j2se/1.5.0/docs/api/javax/swing/table/TableModel.html
        TableModel myData = new MyTableModel();
        JTable table = new JTable(myData);I had some very similar code in my program. I use the DefaultTableModel and the SortFilterModel before having 4 different JTables.
        DefaultTableModel model = new DefaultTableModel(data, columnNames);
        DefaultTableModel model2 = new DefaultTableModel(data2, columnNames);
        DefaultTableModel model3 = new DefaultTableModel(data3, columnNames);
        DefaultTableModel model4 = new DefaultTableModel(data4, columnNames);I use the code in the first post 4 times, and only change the filename and the data#.
I also have some code that counts the number of lines that will be needed.
Thanks.

Comparing SQL Data Results with CSV file contents

I have the following scenario that I need to resolve and I'm unsure of how to approach it. Let me explain what I am needing and what I have currently done.
I've created an application that automatically marks assessments that delegates complete by comparing SQL Data to CSV file data. I'm using C# to build the objects required that will load the data from SQL into a dataset which is then compared to the
associated CSV file that contains the required results to mark against.
Currently everything is working as expected but I've noticed that if there is a difference in the number of rows returned into the SQL-based dataset, then my application doesn't mark the items at all.
Here is an example:
ScenarioCSV contains 4 rows with 8 columns of information, however, let's say that the delegate was only able to insert 2 rows of data into the dataset. When this happens it marks everything wrong because row 1 in both CSV and dataset files were correct,
however, row 2 in the dataset holds the results found in row 4 in the CSV file and because of this it is comparing it against row 2 in the CSV file.
How can I check whether a row, regardless of its order, can be marked as it does exist but not in the same order, so I don't want the delegate to lose marks just because the row data in the dataset is not perfectly in the same order of the row data
in the CSV file???
I'm at a loss and any assistance will be of huge help to me. I have implemented a ORDER BY clause in the dataset and ensured that the same order is set in the CSV file. This has helped me for scenarios where there are the right number of rows in the dataset,
but as soon as there is 1 row that is missing in the dataset, then the marking just doesn't allow for any marks for both rows even if the data is correct.
I hope I've made sense!! If not, let me know and I will provide a better description and perhaps examples of the dataset data and the csv data that is being compared.
Thanks in advance....

I would read the CSV into a datatable using oledb. Below is code I wrote a few weeks ago to do this.
Then you can compare two datatables by a common primary key (like ID number)
Below is the webpage to compare two datatables
http://stackoverflow.com/questions/10984453/compare-two-datatables-for-differences-in-c
You can find lots of examples by perform following google search
"c# linq compare two dattatable"
//Creates a CSVReader Class
public class CSVReader
public DataSet ReadCSVFile(string fullPath, bool headerRow)
string path = fullPath.Substring(0, fullPath.LastIndexOf("\\") + 1);
string filename = fullPath.Substring(fullPath.LastIndexOf("\\") + 1);
DataSet ds = new DataSet();
try
if (File.Exists(fullPath))
string ConStr = string.Format("Provider=Microsoft.Jet.OLEDB.4.0;Data Source={0}" + ";Extended Properties=\"Text;HDR={1};FMT=Delimited\\\"", path, headerRow ? "Yes" : "No");
string SQL = string.Format("SELECT * FROM {0}", filename);
OleDbDataAdapter adapter = new OleDbDataAdapter(SQL, ConStr);
adapter.Fill(ds, "TextFile");
ds.Tables[0].TableName = "Table1";
foreach (DataColumn col in ds.Tables["Table1"].Columns)
col.ColumnName = col.ColumnName.Replace(" ", "_");
catch (Exception ex)
MessageBox.Show(ex.Message);
return ds;
jdweng

SQL Query results to CSV as two worksheets

I'm trying to take two SQL queries and get the results sent to a CSV file on two worksheets. Looking online I have not found a solid example of using the Excel ComObject to create a CSV then add a new worksheet to the CSV file. An added bonus
would be using AutoFit on the columns so everything is easily visible.
Code found online got me the following script which does work, however it takes 12 minutes to pipe the SQL queries to Excel. Switching to a CSV and the script executes in 5 seconds.
This is another nice to have, I was also looking the best way to look at the results (only 1 column) and depending on the length of the data, insert what Excel would call a Cell thereby shifting cells RIGHT but so far have found no clear examples of how
to accomplish that. My guess would be modifying my SQL queries but I've posted a question on StackOverFlow and someone suggested modifying the PowerShell Table created from the SQL dataset.Tables
Code:
$docs = "C:\Scripts\Output.xlsx"
If (Test-Path $docs){Remove-Item $docs}
Function Run-Query {
param([string[]]$queries,[string[]]$sheetnames)
## - Create an Excel Application instance:
$xlsObj = New-Object -ComObject Excel.Application
$xlsObj.DisplayAlerts = $false
$xlsWb = $xlsobj.Workbooks.Add(1)
## - Create new Workbook and Sheet (Visible = 1 / 0 not visible)
$xlsObj.Visible = 0
$time = 2
for ($i = 0; $i -lt $queries.Count; $i++){
$percentage = $i / $time
$remaining = New-TimeSpan -Seconds ($time - $i)
$message = "{0:p0} complete" -f $percentage, $remaining
Write-Progress -Activity "Creating Daily Reboot Spreadsheet" -status $message -PercentComplete ($percentage * 100)
$query = $queries[$i]
$sheetname = $sheetnames[$i]
$xlsSh = $xlsWb.Worksheets.Add([System.Reflection.Missing]::Value, $xlsWb.Worksheets.Item($xlsWb.Worksheets.Count))
$xlsSh.Name = $sheetname
### SQL query results sent to Excel
$SQLServer = 'ServerName'
$Database = 'DataBase'
## - Connect to SQL Server using non-SMO class 'System.Data':
$SqlConnection = New-Object System.Data.SqlClient.SqlConnection
$SqlConnection.ConnectionString = "Server = $SQLServer; Database = $Database; Integrated Security = True"
$SqlCmd = New-Object System.Data.SqlClient.SqlCommand
$SqlCmd.CommandText = $query
$SqlCmd.Connection = $SqlConnection
## - Extract and build the SQL data object '$Table2':
$SqlAdapter = New-Object System.Data.SqlClient.SqlDataAdapter
$SqlAdapter.SelectCommand = $SqlCmd
$DataSet = New-Object System.Data.DataSet
$SqlAdapter.Fill($DataSet)
$SqlConnection.Close()
$Table1 = $DataSet.Tables["Table"]
## - Build the Excel column heading:
[Array] $getColumnNames = $Table1.Columns | SELECT ColumnName
## - Build column header:
[Int] $RowHeader = 1
foreach ($ColH in $getColumnNames)
$xlsSh.Cells.item(1, $RowHeader).font.bold = $true
$xlsSh.Cells.item(1, $RowHeader) = $ColH.ColumnName
$RowHeader++
## - Adding the data start in row 2 column 1:
[Int] $rowData = 2
[Int] $colData = 1
foreach ($rec in $Table1.Rows)
foreach ($Coln in $getColumnNames)
## - Next line convert cell to be text only:
$xlsSh.Cells.NumberFormat = "@"
## - Populating columns:
$xlsSh.Cells.Item($rowData, $colData) = $rec.$($Coln.ColumnName).ToString()
$ColData++
$rowData++; $ColData = 1
## - Adjusting columns in the Excel sheet:
$xlsRng = $xlsSH.usedRange
[void] $xlsRng.EntireColumn.AutoFit()
}#End For loop.
#Delete unwanted Sheet1.
$xlsWb.Sheets.Item('Sheet1').Delete()
#Set Monday to Active Sheet upon opening Workbook.
$xlsWb.Sheets.Item('Cert').Activate()
## ---------- Saving file and Terminating Excel Application ---------- ##
$xlsFile = "C:\Scripts\Output.xlsx"
[void] $xlsObj.ActiveWorkbook.SaveAs($xlsFile)
$xlsObj.Quit()
## - End of Script - ##
start-sleep 2
While ([System.Runtime.Interopservices.Marshal]::ReleaseComObject($xlsRng)) {'cleanup xlsRng'}
While ([System.Runtime.Interopservices.Marshal]::ReleaseComObject($xlsSh)) {'cleanup xlsSh'}
While ([System.Runtime.Interopservices.Marshal]::ReleaseComObject($xlsWb)) {'cleanup xlsWb'}
While ([System.Runtime.Interopservices.Marshal]::ReleaseComObject($xlsObj)) {'cleanup xlsObj'}
[gc]::collect() | Out-Null
[gc]::WaitForPendingFinalizers() | Out-Null
}#End Function
$queries = @()
$queries += "Query1"
$queries += "Query2"
$sheetnames = @('Cert','Prod')
Run-Query -queries $queries -sheetnames $sheetnames

Here's what I ended up with that accomplishes my goal. I learned it's not possible to create a CSV with two worksheets since Excel will allow this but the CSV cannot be saved with the second worksheet. Instead, I create two CSV files then merge
them into one Excel workbook, one CSV per worksheet. In my case, this happens in 5 seconds. There is one thing which must be mentioned, I've seen this script fail the first time it is run but will successfully run the second time.
Also note, after much trial and error, this code correctly cleans up the Excel ComObject!! -Thanks go to JRV.
$docs = "D:\Scripts\MonthlyReboots.xlsx"
IF (Test-Path $docs){Remove-Item $docs}
$csv1 = "D:\Scripts\Cert.csv"
IF (Test-Path $csv1){Remove-Item $csv1}
$csv2 = "D:\Scripts\Prod.csv"
IF (Test-Path $csv2){Remove-Item $csv2}
Function Run-Query {
param([string[]]$queries,[string[]]$sheetnames,[string[]]$filenames)
Begin{
$SQLServer = 'ServerName'
$Database = 'DataBase'
$SqlConnection = New-Object System.Data.SqlClient.SqlConnection
$SqlConnection.ConnectionString = "Server = $SQLServer; Database = $Database; Integrated Security = True"
$Excel = New-Object -ComObject Excel.Application
$Excel.Visible = 0
$dest = $Excel.Workbooks.Add(1)
}#End Begin
Process{
For($i = 0; $i -lt $queries.Count; $i++){
$SqlCmd = New-Object System.Data.SqlClient.SqlCommand
$SqlCmd.CommandText = $queries[$i]
$SqlCmd.Connection = $SqlConnection
$SqlAdapter = New-Object System.Data.SqlClient.SqlDataAdapter
$SqlAdapter.SelectCommand = $SqlCmd
$DataSet = New-Object System.Data.DataSet
$SqlAdapter.Fill($DataSet)
$DataSet.Tables[0] | Export-Csv -NoTypeInformation -Path "D:\Scripts\$($sheetnames[$i]).csv" -Force
}#end for loop.
}#End Process
End{
$SqlConnection.Close()
#Excel magic test!
For($i = 0; $i -lt $queries.Count; $i++){
$loopy = (Resolve-Path -Path $filenames[$i]).ProviderPath
$Book = $Excel.Workbooks.Open($loopy)
$next = $Excel.workbooks.Open($loopy)
$next.ActiveSheet.Move($dest.ActiveSheet)
$xlsRng = $dest.ActiveSheet.UsedRange
$xlsRng.EntireColumn.AutoFit() | Out-Null
$dest.sheets.item('Sheet1').Delete()
$xlsFile = "D:\Scripts\MonthlyReboots.xlsx"
[void] $Excel.ActiveWorkbook.SaveAs($xlsFile)
$Excel.Quit()
While ([System.Runtime.Interopservices.Marshal]::ReleaseComObject($xlsRng)) {'cleanup xlsRng'}
While ([System.Runtime.Interopservices.Marshal]::ReleaseComObject($next)) {'cleanup xlsSh'}
While ([System.Runtime.Interopservices.Marshal]::ReleaseComObject($Book)) {'cleanup xlsWb'}
While ([System.Runtime.Interopservices.Marshal]::ReleaseComObject($Excel)) {'cleanup xlsObj'}
[gc]::collect() | Out-Null
[gc]::WaitForPendingFinalizers() | Out-Null
}#End end block.
}#End function run-query.
$queries = @()
$queries += @'
Select * from table1
$queries += @'
Select * from table2
$sheetnames = @("Cert","Prod")
$filenames = @("D:\Scripts\Prod.csv","D:\Scripts\Cert.csv")
Run-Query -queries $queries -sheetnames $sheetnames -filenames $filenames
Start-Sleep -Milliseconds 50
Invoke-Item D:\Scripts\MonthlyReboots.xlsx

Parser - CSV files to Oracle database

Hello all,
I wrote a csv parser that parse csv files to micorsoft access database. Now I need to parse the dataset to oracle database. This is part of my exporter class
class MdbExporter : IExporter
/// <summary>
/// Exportiert das DataSet in eine Mdb-Datei.
/// </summary>
/// <param name="ds">Zu exportierendes DataSet.</param>
public void Write(DataSet ds, string[] names)
string conStr = "Provider=Microsoft.Jet.OLEDB.4.0;" +
"Data Source=" + names[0] + ";";
Console.WriteLine("Exporting to database {0} ...", names[0]);
Any ideas how can I do that?
Thanks for any replay in advance.

I wrote a parser that goes through several folders containing csv files, reads them and creates a dataset that contains 15 tables ( which I need for an intranet application after that). I succeed to write a MDB exporter that export this dataset to microsoft access database. Now I need an exporter that will export my dataset to ORACLE database. Below is the full code for my MDB exporter:
public void Write(DataSet ds, string[] names)
string conStr = "Provider=Microsoft.Jet.OLEDB.4.0;" +
"Data Source=" + names[0] + ";";
Console.WriteLine("Exporting to database {0} ...", names[0]);
DbConnection connection = new OleDbConnection(conStr);
try
connection.Open();
catch (DbException e)
ConsoleEx.WriteException(true, e, "Unable to open database: {0}", names[0]);
throw;
DbCommand command = connection.CreateCommand();
foreach (DataTable table in ds.Tables)
Console.WriteLine("\tDeleting table: {0}", table.TableName);
// delete old tables
command.CommandText = string.Format("drop table {0}", table.TableName);
TryExecute(command, false);
// create new
Console.WriteLine("\tCreating new table: {0}", table.TableName);
string[] columnStrings = new string[table.Columns.Count];
for (int i = 0; i < table.Columns.Count; i++)
columnStrings[i] = "`" + table.Columns.ColumnName + "`" + " varchar";
command.CommandText = string.Format("create table {0} ({1})",
table.TableName, string.Join(", \n", columnStrings));
TryExecute(command, true);
// add rows
for (int row = 0; row < table.Rows.Count; row++)
for (int col = 0; col < table.Columns.Count; col++)
columnStrings[col] = "'" + Convert.ToString(table.Rows[row].ItemArray[col]) + "'";
command.CommandText = string.Format("insert into {0} values ({1})",
table.TableName, string.Join(", \n", columnStrings));
TryExecute(command, true);
connection.Close();
I need similar exporter to Oracle database. Starting with SQL LOADER from the beginning its really not a good in my opinion since I am almost done with this approach.
Any help would be appreciated.
Regards,
Sven

10g exporting to CSV using client_text_io is not working correctly.

I have an odd issue which i could do with some help with. I run an function that exports to CSV based on an pre defined record group.
This has been working fine for many months with various customers. Recently a new customer used it and they have 28k rows in his record group and the export is actually not exporting correctly.
The record group has a record count of 28331.
The CSV produced has only 3756 in my CSV file. These are the last 3756 records in the Record group so its as if its overwriting the data as it goes yet all smaller datasets work
FUNCTION fun_export_csv (vgraphid NUMBER, p_filename VARCHAR2)RETURN BOOLEAN IS
out_file                     client_text_io.file_type;
     i                                             NUMBER;
lv_line              VARCHAR2(5000);
BEGIN
rg:=populate_group('RG11_EXP');
synchronize;
lv_line:= ('"GIN","Gin Date","PO Num","PO Required Date","Mat Num","Mat Description","Supplier Part No","On Time Delivery(Yes-1, No-0)"');
client_text_io.put(out_file, lv_line);
client_text_io.new_line(out_file,1);
For i in 1..get_group_row_count('RG11_EXP') Loop--this count is 28331
lv_line:= ('"'||get_group_number_cell('RG11_EXP.col1', i )                ||'"'|| ',' ||'"'||
     get_group_date_cell('RG11_EXP.grn_date', i )                ||'"'|| ',' ||'"'||
     get_group_number_cell('RG11_EXP.po', i )      ||'"'|| ',' ||'"'||
     get_group_date_cell('RG11_EXP.daterqd', i ) ||'"'|| ',' ||'"'||
     get_group_char_cell('RG11_EXP.item_no', i ) ||'"'|| ',' ||'"'||
     get_group_char_cell('RG11_EXP.desc', i )                    ||'"'|| ',' ||'"'||
     get_group_char_cell('RG11_EXP.part_no', i )               ||'"'|| ',' ||'"'||
     get_group_number_cell('RG11_EXP.ontime', i )                    ||'"');
client_text_io.put(out_file, lv_line);
client_text_io.new_line(out_file,1);
END LOOP;
client_text_io.FCLOSE(out_file);
RETURN TRUE;

Hello,
Try to insert a "synchronize" instruction from time to time:
i pls_integer := 1;
Loop
If mod(i, 500) = 0 Then
     synchronize;
End if ;
i := i + 1 ;
End loop;
...<p>But keep in mind the the CLIENT_TEXT_IO generate a lot of network traffic, so it is better and faster to generate the file on the A.S., then after transfer it to the client machine.</p>
Francois

CSV dataset columnNames

Similar Messages

Maybe you are looking for