Reading and Writing large Excel file using JExcel API

hi,
I am using JExcelAPI for reading and writing excel file. My problem is when I read file with 10000 records and 95 columns (file size about 14MB), I got out of memory error and application is crashed. Can anyone tell me is there any way that I can read large file using JExcelAPI throug streams or in any other way. Jakarta POI is also showing this behaviour.
Thanks and advance

Sorry when out of memory error is occurred no stack trace is printed as application is crashed. But I will quote some lines taken from JProfiler where this problem is occurred:
          reader = new FileInputStream(new File(filePath));
          workbook = Workbook.getWorkbook(reader);
          *sheeet = workbook.getSheet(0);* // here out of memory error is occured
           JProfiler tree:
jxl.Workbook.getWorkBook
      jxl.read.biff.File
             jxl.read.biff.CompoundFile.getStream
                   jxl.read.biff.CompoundFile.getBigBlockStream Thanks

Similar Messages

Unable to display double values in Excel sheet using JExcel API

Hi
I am writing code to generate report in the form of Excel Sheet using JExcel API.
Everything is going fine but whenever I want to put some double values in a cell it is only showing 2 decimal places. My problem is "I want to show upto five decimal places".
Any kind of reply might help me lot.
Thank U.

If you enable the submit zero option, it still happens? This is a new feature on the display tabl
#NumericZero Enhancements
To display a numeric zero in place of an error message, you can enter #NumericZero in any of the three Replacement text fields. When you use the #NumericZero option:
· Excel formatting for the cell is retained.
· All calculations with dependency on the cell will compute correctly and will take the value of this cell as zero.
· This numeric zero is for display only. When you submit, the zero value is NOT submitted back to the data source.
You cannot set display strings for cells that contain an invalid member or dimension name (metadata error). Metadata errors produce standard descriptive error messages.
Errors are prioritized in the following order from highest to lowest. The error message for a higher-priority error takes precedence over that for a lower-priority error.
1. (Highest) Metadata errors
2. #No access
3. #Invalid/Meaningless
4. #No data\Missing

Writing into Excel file using PL/SQL and formatting the excel file

Hi,
I am writing into a excel file using PL/SQL and I want to make the first line bold on the excel. Also let me know if there are any other formatting options when writing into excel.
Regards,
-Anand

I am writing into a excel file using PL/SQL
Re: CSV into Oracle and Oracle into CSV
check that thread or search in this forum...

Read very very large excel file into 2d array (strings or varient)

Hi all,
Long time user, first time poster on these boards.
Looking at the copius amounts of great info related to reading Excel data from .xls files into labview, i've found that every one i've found from various people use the ActiveX method (WorkSheet.Range) which two strings are passed, namely excel's LetterNumber format to specify start and end.
However, this function does not work when trying to query huge amounts of information. The error returned is "Type Mismatch, -2147352571" I have a very large excel sheet i need to read data from and then close the excel file (original file remains unchanged). However this file is gigantic (don't ask me, I didn't make it and I can't convince them to use something more appropriate) with over 165 columns at 1000 rows of data.I can read a large number of columns, but only a handful of rows, or vice versa.
Aside from creating a loop to open and close the excel file over and
over reading pieces of it at a time, is there a better way to read more
data using ActiveX? Attached is code uploaded by others (with very minor modification) as an example.
Thanks,
Attachments:
Excel Get Data Specified Field (1-46col).vi ‏23 KB

Hi Maddox731,
I've only had a very quick glance through your thread, and I must admit I haven't really thought it through properly yet. Sounds like you've come up with your own solution anyway. That said I thought I'd take a bit of a scatter gun approach and attach some stuff for you regradless. Please forgive my bluntness.
You'll find my ActiveX Excel worksheet reader, which may or may not contain the problem you've come across. I've never tried it with the data size you are dealing with, so no promises. I've also attached my ADO/SQL approach to the problem. This was something I moved onto when I realised the limitations of AX. One thing I have noticed is that ADO/SQL is much faster than AX, so there may be some gains for you there with large data sets if you can implement it.
I should add that I'm a novice to all this and my efforts are down to bits I've gleamed from MSDN and others' LV examples. I hope it's of some use, if only to spark discussion. Your ctiticism is more than welcome, good or bad.
Regards.
Attachments:
Database Table Reading Stuff.zip ‏119 KB

Very slow painting while reading and writing doubles into file

for 15MB length file i = 7662080
for 50MB length file i = 12414368
Part of Code for writing into file follows like this:
try{
fos = new FileOutputStream("Angel.txt");
File f = new File("Angel.txt");
     if(f.length() >=4)
     f.delete();
fos = new FileOutputStream("Angel.txt");
dos = new DataOutputStream(new BufferedOutputStream(fos,1000000));
int x=0;
double y_last, y_new;
for(int j=0 ;j<i ;j++)
if(some condition)
y_new = ....;
try{
//previously in vectors
y_last = y_new;
vect.add(new Line2D.Double(x, y_last, x, y_new)_;
dos.writeDouble(y_new);
     }catch(Exception e){System.out.println(e);}
dos.close();
fos.close();
x++;
}catch(Exception excp){System.out.println(excp);}
part of code for reading from file follows like this:
public void paint(Graphics g)
try{
     double y1, y2 =0;
     Line2D.Double doub;
     raf = new RandomAccessFile("Angel.txt","r");
     dis = new DataInputStream(new BufferedInputStream(new FileInputStream(raf.getFD(),1000000)));
     raf.seek((rect.x*8));
     for (int i = 0/any value; (i < value as per choice); i++)
          g2.setStroke(new BasicStroke(0)); //2
          y1 = y2;
     y2 =dis.readDouble();
          doub=new Line2D.Double(i,y1,i,y2);
          g2.draw(doub);
dis.close();
     raf.close();
}catch(Exception excp){System.out.println(excp);}
I tried using Object Streams but NotSerializable Exception is thrown as Line2D.Double objects
are not serialized.
Any idea to make reading and writing into file specially from MB files faster is appreciated.

Why are you reading in the file in the paint method ?
Create your data once before painting.
I think you should explain what is your goal and what behavior you want.
Denis

Reading and writing from a file

I am trying to read and write to a file so that I may store an array of string and a double in my application, but I have come upon a couple of problems; first when I put my file path into the path constant it will say that the file is not there ever though it is (so I have to force the issue and when the application asks for a file I have to browse and add the files), secondly when I run the program and it reads from the file it does not display the strings or doubles immediately instead I have to hit the submit button and than they finally come up, thirdly when I reset my values in the .txt files and put in my string and double and hit submit the array jumps one spot and starts at the index of 1 (array[1]) if anyone can help I would appreciate it a lot.
here is my code:
Harold Timmis
[email protected]
Orlando,Fl
*Kudos always welcome
Solved!
Go to Solution.
Attachments:
testingfileIO.vi ‏30 KB
double.txt ‏1 KB
string.txt ‏1 KB

Harold Timmis wrote:
I am trying to read and write to a file so that I may store an array of string and a double in my application, but I have come upon a couple of problems; first when I put my file path into the path constant it will say that the file is not there ever though it is (so I have to force the issue and when the application asks for a file I have to browse and add the files),
I don't see that behavior at all about forcing the issue. Why are you using Not a Path constants? Why not turn those constants into controls you put on your front panel?
secondly when I run the program and it reads from the file it does not display the strings or doubles immediately instead I have to hit the submit button and than they finally come up,
Put your array indicators before your event structure rather than after. Think dataflow. The code pauses at the event structure waiting for an even to fire. Only when it does (such as hitting the submit button) does the data get written to the array indicators.
thirdly when I reset my values in the .txt files and put in my string and double and hit submit the array jumps one spot and starts at the index of 1 (array[1]) if anyone can help I would appreciate it a lot.
I don't see this behavior either. How come your double.txt file has names in it, and your string.txt file has doubles in it?

File adapter reading and writing large files

Hi we are getting error when trying to process large files using file adapters. files of size 80 to 100 MB. we need to read the inbound files and write them to another folder in another server. the error we are getting is out of memory. gracias

Hi,
Use the asynchronous process or a checkpoint(); to see your instance before it time-out.
--Khaleel

Problem reading and ploting large lvm files

Hi everyone,
I have some large .lvm files that I need to process offline; however, the files are quite large and I am regulalry getting "out of memory" messages (7 channels sampled at 4k for 15 minutes or so, maybe longer)... I have managed to reduce the rate this message occurs by first converting the lvm files to tdms, then potting the tdms data, but I still get the "out of memory" error message on somewhat regularly.. .I also down sample the data back to 2k, but that dont help a great deal.
Any suggestions on how I can handle this data? I have read a number of online resources related to managing large data sets (e.g. http://www.ni.com/white-paper/3625/en/) but I am not sure how to implement these suggestions.
Basically, I want to view the content of the ensure file... Then use queues to extract data subsets as selected into another while loop that will handle the analysis/processing (producer/consumer)... I do this regularly for smaller files; so the issue is mainly how to managing the large files... Decimating the data for the intial whole data plot may not work as I have spikes 10ms in width in some channels that I need to see in the main plot.
Any help would be appreciated.
Many thanks,
Jack

jcannon, I did some quick math and I dont think you should be reaching the memory limit of LabVIEW. However, it is possible that you are running out of contiguous memory on your computer while the program is running. See this for a quick breif about contiguous memory.
If I were you I would try to reduce the number of times LabVIEW coppies information in memory. Use show buffer allocations to find out where in your code you are making coppies of memory.
best of luck!

Problem writing to excel file using report generation toolkit

hello everyone, i have this report generation toolkit... and i want to output DAQmx Analog I/P data on to an excel sheet. the DAQmx is programmed to collect
data at 3samples/sec. however, when i see the excel file that Report Generation Toolkit generates, the time stamp is updated every second instead of every 0.33sec.
can anyone please help me? i am using the MS Office Report Express VI.
Now on LabVIEW 10.0 on Win7

@All, I got rid of the express VI, decided to work on the custom low level VIs instead. however, i have a new problem now...
I have a case statement wherein, the user selects if he wants to start generating a report. once the program enters tat loop, the program speed reduces!
can anyone please tell me why is it happening? i ahve attached the vi... also another question.. in this VI, i am capturing the unwanted data into the graph as I am indexin the graph input. how can i make a logic
that the graph captures the data only when I am switching the CREATE REPORT button (which is in the while loop). is there a way that I can append the data to the graph without creating a new graph every iteration? please let me know
thanks
Now on LabVIEW 10.0 on Win7
Attachments:
Untitled 7.vi ‏75 KB
Untitled 7.JPG ‏99 KB

Reading and writing non-English files

Hi there..
I wonder if anyone can tell me how to read and write non-English language files like French or Arabic files...
Im also interested in knowing how can i convert from Unicode to ASCII and vice versa..
thanx.. Mourad

hi there ..
thanx for ur cooperation .. but actually i have tried the following code and it didnt work; so, im looking for more help :)
esp. if there are any notes about the files themselves for example;
by the way i used Notepad files in unicode format..
my code is :
import java.io.*;
public class MainClass {
public static void main(String args[]){
String inputfile = "arabic.txt";
String outfile = "outfile.txt";
try{
RandomAccessFile raf = new RandomAccessFile(inputfile, "r");
InputStreamReader isr = new InputStreamReader(
new FileInputStream(inputfile), "Cp1256");
OutputStreamWriter osw = new OutputStreamWriter(
new FileOutputStream(outfile), "Cp1256");
for(int i=0; i<raf.length(); i++)
osw.write(isr.read());
osw.close();
}catch(UnsupportedEncodingException uee){
System.out.println("UEException: "+ uee.getMessage());
}catch(FileNotFoundException fnfe){
System.out.println("FNFException: "+ fnfe.getMessage());
}catch(IOException ioe){
System.out.println("IOException: "+ ioe.getMessage());
thanx again..Mourad

Read and write into csv file using procedure

I would like to read a particular column value from csv file(which will have many columns) using procedure and for that value I have to retrieve some more values from database table and have to append into same csv file.
can someone help me?
Thanks
Edited by: 1002478 on Apr 25, 2013 5:52 AM
Edited by: 1002478 on Apr 25, 2013 5:55 AM

1002478 wrote:
thanks for your reply.
Using UTL_FILE i'm able to read csv file. I'm stuck up in appending some more columns to same csvYou'll struggle to append data to an existing CSV.
You'd be better to read the data from one CSV, and create another CSV using that data, and the extra data you want, and then when finished, delete the original file and rename the new file to the old filename.
You should be able to do that using External Tables to read the source file (easier than using UTL_FILE) and UTL_FILE to write the new file.

Read an Excel file using POIFSFileSystem API

hello,
I'm working on J2EE web assignment. Following is my requirement
1) User select local excel file and hit submit
2) API takes that file read and create a new Excel file.
I am wondering if I can somehow get inputsteam on form post and then pass it to POI api
POI api syntax
POIFSFileSystem fs = new POIFSFileSystem(java.io.InputStream);
thanks,
Amit

there are plenty of tutorials and docs online
this site is for technical help, general questions should be submitted to Google

How to read data from the excel file using java code.

Hi to all,
I am using below code to getting the data from the excel file but I can't get the corresponding data from the specific file. can anyone give me the correct code to do that... I will waiting for your usefull reply......
advance thanks....
import java.io.*;
import java.sql.*;
    public class sample{
             public static void main(String[] args){
                  Connection connection = null;
                      try{
                           Class.forName("sun.jdbc.odbc.JdbcOdbcDriver");
                           Connection con = DriverManager.getConnection( "jdbc:odbc:Mydsn","","" );
                           Statement st = con.createStatement();
                           ResultSet rs = st.executeQuery( "Select * from [Sheet1$]" );
                           System.out.println("sample:"+rs);
                              ResultSetMetaData rsmd = rs.getMetaData();
                              System.out.println("samplersd:"+rsmd);
                           int numberOfColumns = rsmd.getColumnCount();
                              System.out.println("numberOfColumns:"+numberOfColumns);
                               while (rs.next()) {
                                        System.out.println("sample1:"+rs);
                                        for (int i = 1; i <= numberOfColumns; i++) {
                                             if (i > 1) System.out.print(", ");
                                             String columnValue = rs.getString(i);
                                             System.out.print(columnValue);
                                        System.out.println("");
                                   st.close();
                                   con.close();
                                  } catch(Exception ex) {
                                       System.err.print("Exception: ");
                                       System.err.println(ex.getMessage());
                    }

1: What is the name of the excel sheet?
2: What is printed in this program? null ? anything?
error?Excel file name is "sample.xls" I set excel file connectivity in my JDBC driver(DSN). Here in my program I am not giving that excel file name. I am giving only that excel sheet name. that is followed
ResultSet rs = st.executeQuery( "Select * from [Sheet1$]" );The output of this program is given bellow.
sample:sun.jdbc.odbc.JdbcOdbcResultSet@1b67f74
samplersd:sun.jdbc.odbc.JdbcOdbcResultSetMetaData@530daa
numberOfColumns:2

Saving excel file using HSSF api

I am using HSSF api to download an excel file. But the end users are not able to save the file. Some of them do not get the popup file download window so that they can choose to open or save. Is there a setting in the browser which shows up the window? The users may have different versions of excel on their pcs.
How to show the excel application menu 'file' so that users can save this file?
Any help would be appreciated.

Prakash,
          I have followed your weblog and set the content dispostion.
HttpServletResponse res = request.getServletResponse(true);
               try {
                    res.setContentType("application/vnd.ms-excel");
                    res.setHeader("Content-disposition", "inline; filename=report.xls");
                    wb.write(res.getOutputStream());
               } catch (Exception e) {
                    e.printStackTrace();
When I test this on my pc, I get file download window and I can choose 'open' or'save'. I also have tested this on some of my collegues PCs successfully. But the some of the end users are not getting this window.
when the same is file is downloaded by running an asp page the enire MSExcel application with standard tool bar and menus(file edit etc..) is showing up.
Why is it not poosible to get the same result using HSSF api?

How do I create/update an Excel file using the API?

Hi,
I've successfully created a "Web Intelligence" document using the API (POST /raylight/v1/documents). However, I need to do the same for an Excel file type and have been unable to do so. Attached are the different steps needed to accomplish this with the UI. I need to be able to do the same thing through the API.
I searched through the docs but haven't found anything regarding this particular aspect.
Does anyone have a clue?
Thanks

Thanks for the link. I'm using .Net and thought the process would be fairly similar. It is indeed, however, I'm stuck with the following error when committing on the infoStore object:
File Repository Server error :
ErrorCode: -2147210678
Here's the code I'm using:
var folderID = infoStore.Query("Select TOP 1 SI_ID From CI_INFOOBJECTS Where SI_KIND='Folder' And SI_NAME='myfolder'")[1].ID;
var infoObjects = infoStore.NewInfoObjectCollection();
var infoObject = infoObjects.Add(CrystalDecisions.Enterprise.InfoStore.CeKind.EXCEL);
infoObject.Title = "Book4.xlsx";
infoObject.Files.Add("C:\\Users\\myuser\\Documents\\Book4.xlsx");
infoObject.ParentID = folderID;
infoStore.Commit(infoObjects);
I know the problem is related to "infoObject.Files.Add" because when I omit this statement, it works but the file has no content.
Any idea why??? It's driving me crazy

Reading and Writing large Excel file using JExcel API

Similar Messages

Maybe you are looking for