Need a fast ascii file reader in Java

Hi,
My application needs to read around 50000 small ascii (1-25lines) files. Currently I use the BufferedReader API to read ascii files.
Can you suggest something faster than the below.
Thanks
private ArrayList<String> readDataFile(String fileName) {
String lineData;
List<String> fileContents = new ArrayList<String>();
BufferedReader moFile=null;
try {
moFile = new BufferedReader (new FileReader (new File ( fileName)));
while ((lineData = moFile.readLine()) != null)
fileContents.add(lineData);
} catch(IOException e) {
myLogger.error("Exception caught while reading data file: " + fileName + e.toString());
}finally {
try {
moFile.close();
}catch(Exception e) {
//Ignore
return fileContents;
}

816417 wrote:
Kayman: I will be using all the contents of the file.Yes, but how? You could speed up the processing by reading all the files into a database once, unless these files are for example created every day and you need to always read the new ones.
I don't have a prototype to provide the absolute time, a jprobe on my application shows that it takes 70% of the time.Yes, but your whole application takes 100% of the time. Do you have reason to believe that 70% is too much of time? What does the other 30% do?
I want to reduce the time spent in this method.You want or you need to?

Similar Messages

Having alot of trouble in file reading in java

i dont know. but i feel really bad that Java is a very good OOP language and i suck big time (not very bad but got frustrated) in reading and writing files. some of the time i have accomplish my task with the help of forums. anyways first i would like to ask 2 questions and then i will state my problem. say for instance if i have a file with name amount and dollars. how can i read it?
i know i can use string regex split method in order to split the line but how can i create a method to do some changes in the text file like i have a name, bank account # and amount to deposit in the text file. my question is how can i declare my method to read (Name) first column and price second column and amount to deposit the third colum?
Secondly if i have a text file and i would like to replace all <<A>> with the name and <<G>> with the age. if i wanted to do without reading a file no problem create a string and then use the replaceAll method ("<<A>>", "Jaan") but for some reason i can't do it when i read a file. yes i am searching on google to get more understanding in reading file. but anyone care to explain wil be great. God bless you my Son.
import java.io.*;
import java.util.*;
class ReadData {
     private Scanner s;
     private static final String EMPTY_STRING = "";
     private String line1;
     private static String lineTerminator = System.getProperty("line.separator");
     public static void main(String[] args) throws IOException {
          ReadData read = new ReadData();
          read.start();
     public ReadData() {
          s = new Scanner(System.in);
     public void start() throws FileNotFoundException, IOException {
          System.out.println("Enter the file name");
          try {
          String name = s.next();
          openFile(name);
          catch (FileNotFoundException e) {
               e.printStackTrace();
          catch (IOException e) {
               e.printStackTrace();
     public String openFile(String filename) throws FileNotFoundException, IOException {
          String line;
          String line2;
          StringBuffer document = new StringBuffer(EMPTY_STRING);
          File inFile = new File(filename);
          FileReader fileReader = new FileReader(inFile);
          BufferedReader bufReader = new BufferedReader(fileReader);
          while(true) {
               line = bufReader.readLine();
               if(line==null) break;
               document.append(line + lineTerminator);
              System.out.print(document.toString());
          return document.toString();
}

i dont know. but i feel really bad that Java is a
very good OOP language and i suck big time (not very
bad but got frustrated) in reading and writing files.
some of the time i have accomplish my task with the
help of forums. anyways first i would like to ask 2
questions and then i will state my problem. say for
instance if i have a file with name amount and
dollars. how can i read it?Just like you'd read any other file. How you interpret what you've read depends on the specific format of the file, which you've provided no information about.
i know i can use string regex split method in order
to split the line but how can i create a method to do
some changes in the text file like i have a name,
bank account # and amount to deposit in the text
file. my question is how can i declare my method to
read (Name) first column and price second column and
amount to deposit the third colum? What do you mean? You want to read the entire first column, from all lines, then the entire second column, etc.? Unless you know the length of each line ahead of time so you can use RandomAccesFile, you can't do that. If that's not what you mean, then please clarify.
Secondly if i have a text file and i would like to
replace all <<A>> with the name and <<G>> with the
age. if i wanted to do without reading a file no
problem create a string and then use the replaceAll
method ("<<A>>", "Jaan") but for some reason i can't
do it when i read a file.
Create a tmp file.
For each line {
Read a line.
Do the replacement on the string you read.
Write the new line out to the temp file
rename the temp file to the original
anyone care to explain wil be great. God bless you my
Son.I'm not your son. Keep your god away from me.

Binary File reading in Java

can i read binary format file with java? if so, can you please suggest which class i should be looking into?
Thanks

FileInputStream

I need a fast reliable card reader for HD Video transfer

Hey guys, I've just purchased a Canon 5D Mark II and I use 16GB and 32GB 60MB/s SanDisk flash cards to capture my HD video footage. I must have purchased the worst card reader on the market, because it takes over an hour to transfer the footage and now it crashes my computer everytime. What I was wondering is if anyone out that has any suggestions about a trust worthy card reader that moves a bit faster then the speed of dirt and wont crash my computer.
Thank you in advance,
Crayton

Oh and I have another question. As I have been searching, I see alot of these card readers say they handle "large cards" such as 16GB, however, I use 32GB 60/mbs. It was this card that was destroyed while I was transferring the video. So I was wondering if maybe that is an important factor for me to consider and perhaps I have to find an even beefier card reader? Don't get me wrong, I hope not, but maybe it matters. What is the largest card size you use?
Thanks,
Crayton

Is File handling in Java Swing possible?

Hi I need to know whether file handling in Java Swing is possible or not?

cents wrote:
Thanks for ur answer.....Actually I just wanted to know whether "Reading from a file" and "Writing to a file" in Java Swing is possible or not?
Edited by: cents on Mar 16, 2010 9:35 AMWhat did google tell you? Did you see anything interesting in the API?

Insert time stamp in ASCII file

Hi all!
I am trying to log data values from several channels (so far 4) into one ASCII file including a time stamp in the first column.
The goal is one file that contains one time stamp per row (in the first column) and one column per channel. It may looks like this:
TIME Ch1 Ch2 Ch3 Ch4
t1        a1     b1     c1    d1
t2        a2     b2     c2    d2
I want to save one row (time stamp and 4 channels) each minute as above into the same file. Nice would be to creat a new file each day.
Problems:
1. Using the "Save to ASCII/LVM" step: How can I get SignalExpress to insert a time stamp in the first column of each row?
    I have no problem inserting the 4 signals that generate me one column per signal. And using "Overwrite once, then append to file" I get one file for all signals.
    Any idea how to insert a time stamp this way? Or how to creat a new file every 24 hours?
2. Another way I tried is to use the "Recording Option". But how can I get all the four signals saved in one file?
    After the DAQmx acquisition step I process the 4 channels individually (one scaling step and one threshold step per channel - because of different factors per channel).
    Activating the Recording Option (which would generate a new file every 24 hours, but not starting at 12pm each day...) I get 4 single TDMS-files (and their corresponding tdms_index and meta.txt files),
    one file per signal I am recording. I assume this is because they are 4 individual signals I'm recording and not a group of signals. Can I group individual signals together?
Thanks for your help
SSC
PS: I work with SignalExpress 2.5 and have a NI USB-6210 connected.

In the recording options tab you can set a stop condition to be 24 hours of data. And then select the "Restart Log Automatically" option to start logging for another 24 hours. You can also configure the logging to auto create an ASCII file once logging stops. With this configuration an ASCII file will be created every 24 hours. To enable ACSII file generation, select Tools>>Options.. And then set "Automatically export log to ASCII file to true".
I believe that you will get one ASCII file for each TDMS file. If you are logging 4 channels from the same device you should be able to log the entire group which would allow you to get only 1 TDMS / ASCII file.

JPEG file reading wierd bug using ImageIO.read and buffer strategy

OK I have a slightly (!) bizzare problem. I have a program in Java 1.5.0 /5.0 which is fundamentally using a full screen exclusive mode window to display images loaded from files using ImageIO.read(file). I am getting the graphics context of the buffer strategy used on the FSEM window, and then drawing onto that the image that I have loaded using graphics.drawImage(ImageObj, 0,0, null).
Now ordinarily this works fine, but I have (after much head scratching) discovered that certain specific jpeg images cause me problems. If I load one of these jpeg images, then it loads in with ImageIO.read fine and I get no errors, but the program locks up on the drawImage(ImageObj, 0,0,null) command in the rendering engine. It doesn't crash, but the CPU usage goes to 100% and stays there, and the method never returns. The image is never displayed on screen obviously, as the program never gets to call bufferStrategy.show() for these images. I have to forcibly terminate the VM to get out of the program.
Does anyone know what is going on here? Is there some extraneous meta data in the dodgy jpeg files that is doing this? I don't know what 'extras' you can have stored on a jpeg that would cause this behaviour, and I don't know what restrictions there are in the jpeg file reader in Java.
Its not a major bother to me (now I figured out whats causing it!), as I can just resave a duff jpeg in my image editing app and it will work fine. I just don't know whether I should submit it as a bug or not, and also I thought it would be useful for others to know of this strange behaviour - if I had known about it I would have saved myself about 8 hours!
Thanks
Alex

Oh I forgot to add - I am resizing the images to the screen size using
Image.getScaledInstance(deviceBounds.width,
deviceBounds.height,
Image.SCALE_AREA_AVERAGING);
and then drawing the scaled image onto the graphics context.

Generic - FTP 1.0 Protocol is not creating ASCII file

Hi,
We have a requirement to create ASCII file using Generic - FTP 1.0 Protocol for outbound transactions, but it is creating Binary file now.
Appreciate your immediate response in this regard.
Thanks
Sathish

We are sending EDI data via VAN to our business partners. We have setup the FTP channel to write files to VAN FTP system. Files are getting created but in binary mode and VAN is unable to process them.
Transfer type is set to ASCII only. If i see the logs, i could see that type is ascii, is_binary is false but file.sender.type=BINARY. I have removed user id and server info fom these logs
portProperties.TransportProperties():password=file.sender.ccc=false;type=ascii;channel_mask=None;folder=;marker=false;ccc=false;file.sender.marker=false;host=;keep_connections=false;file.sender.password=file.sender.user=;file.sender.van=false;filename=abc_2010Nov23_14_37_23_088_308_182.dat;is_binary=false;file.sender.type=BINARY;preserve_filename=false;useProxy=false;van=false;polling_interval=5;user=;
Pls let me know how we can transfer ascii files.

Please help me in reading the ascii file in encoded format

hi ,
iam trying to read the ascii file and i need to encode the text file, can u suggest me what arre the different methods are availble for me to encode the text file, please suggest me which is the effiecient method tto use?

This question has been answered before, please search the forums in future.
You could do something like this, it probably not the most efficient though.
If you don't need to do it in code then the native2ascii does this conversion see: http://java.sun.com/j2se/1.4.2/docs/tooldocs/windows/native2ascii.html
        public static void convert(String vsFile) throws IOException
          File f = new File(vsFile);
          FileReader fr = new FileReader(f);
          char[] buf;
          int bytesRead = 0;
          int startAt = 0;
          String content = "";
          do
               buf = new char[512];
               bytesRead = fr.read(buf, 0, 512);
               startAt += bytesRead;
               String tmp = new String(buf);
               content += tmp;
          while (bytesRead == 512);
          FileOutputStream fout = new FileOutputStream(f);
          OutputStreamWriter out = new OutputStreamWriter(fout, "UTF-8");
          out.write(content.trim());
          out.flush();
          out.close();
     }

How to read a web page as an ascii file

I need to open an url and read the contents, as it
were a simple local ascii file. This give me just
garbage, as [B@fd2e84b (that change every time,
seems to be a memory address, not the contents..).
<pre>
import java.io.*;
import java.net.*;
try {
URL j_urlobj = new URL("http://www.google.com";);
URLConnection j_urlcon = j_urlobj.openConnection();
BufferedInputStream j_bis =
(BufferedInputStream)j_urlcon.getInputStream();
byte[] j_data = new byte[4096];
int j_size;
j_size = j_bis.read(j_data, 0, 4096);
while (j_size != -1) {
out.print(j_data);
j_size = j_bis.read(j_data, 0, 4096);
j_bis.close();
} catch (Exception e) { out.print(e.toString()); }
</pre>
What is wrong?? Thank you!
Stefano

I try your code and it's work right with a little change....
The problem is that you read bytes, so if you want something readable try to convert to a String.
String s = new String(j_data)
out.println(s);
Here, the complete code:
try {
URL j_urlobj = new URL("http://www.google.com");
URLConnection j_urlcon = j_urlobj.openConnection();
BufferedInputStream j_bis =
new BufferedInputStream(j_urlcon.getInputStream());
byte[] j_data = new byte[4096];
int j_size;
j_size = j_bis.read(j_data, 0, 4096);
while (j_size != -1) {
out.print(new String(j_data));
j_size = j_bis.read(j_data, 0, 4096);
j_bis.close();
} catch (Exception e) { out.print(e.toString()); }

FAST file reading

Hello,
i need to read large text files, but method i wrote appears to be very slow and cannot be used for larger files.
i have something like this:
BufferedReader br = new BufferedReader(new FileReader(path));
String line = "";
StringBuffer sb = new StringBuffer();
while( line = br.readline()   != null )
sb.append(line);
br.close();I am sure there is faster way to read. Thank you for help.

Here's the old JTextAreaPrintStream for those extra long lines...
import java.io.OutputStream;
import java.io.PrintStream;
import javax.swing.*;
public class JTextAreaPrintStream extends PrintStream {
    public JTextAreaPrintStream(JTextArea textArea){
        super(new JTextAreaOutputStream(textArea));
    private static class JTextAreaOutputStream extends OutputStream {
        private JTextArea textArea;
        private JTextAreaOutputStream(JTextArea textArea) {
            this.textArea = textArea;
        public void write(int i){
            textArea.append(new String(new char[] {(char) i}));
        public void write(byte[] b){
            textArea.append(new String(b));
        public void write(byte[] b, int offset, int len){
            textArea.append(new String(b, offset, len));
    public static void main(String[] args) {
        doDemo();
    static void doDemo() {
        JFrame f = new JFrame("JTextAreaPrintStreamDemo");
        f.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
        JTextArea display = new JTextArea();
        display.setEditable(false);
        JTextAreaPrintStream newOut = new JTextAreaPrintStream(display);
        System.setOut(newOut);
        f.add(new JScrollPane(display));
        f.setSize(300, 300);
        f.setVisible(true);
        System.out.println("How's this?");
        for (int i = 0; i < 5; System.out.println(i++ + 1));
}~

Fast file reader

I have big image file to read. I need a native fast code to read it
any help?

Hi,
This is a very fast method. I use since 4 year.
public static void copy(java.io.InputStream in, java.io.OutputStream out, int ibuffer) throws java.io.IOException
          byte[] buffer = new byte[ibuffer];
          int byteleidos;                                                                 // Bytes que hemos leido
          while(( byteleidos = in.read(buffer,0,buffer.length) ) != -1){
               out.write(buffer,0,byteleidos);
               out.flush();
     }

Read an ascii file line by line

I am new with CVI, and I need to read an ascii file with multiple lines in the following format:
at_test_2_to_13=0.861000
at_testabcd_9_to_21=0.712000
at_abdgx_5_to_8=0.892000
here is the code for writing in the ascii file:
sprintf(ctResult, "%s=%f\n", TestName1,result1);
WriteFile (fileHdl,ctResult ,StringLength(ctResult) );
At a later time, I need to read the text file, find the test name (for example "at_testabcd_9_to_21") and record the value after the "=" sign (in this case 0.712000). The number of lines in the text file is not fixed.
I am writing the data to the file with no problems, but I am stack in reading the data.
I would appreciate any help on how to write this code in CVI.
Thank you in advance
Solved!
Go to Solution.

As Wolfgang already proposed, you can use the ini-file library.
With this library, you can write the results in a structured manner and access an individual measurement by its name, without the need of reading every line above.
I also attached a sample project for what you asked.
Check the ini-file library help and samples for the other option.
S. Eren BALCI
www.aselsan.com.tr
Attachments:
AsciiRead.zip ‏2 KB

Reading Ascii file and display in Text

I have a file containes ascii code, can i read in in a form of text using java, or i have to convert it to text? Any idea?

The ascii file was converted from binary file, intend to use Java to view it as a text and save it. The sample ascii codes are listed below. kindly have a look.
"[email protected].", 24, 25401, -481703714, 0.000, -0.000
".D..Q..!..", 46, -15828, -1606219592, 802866557195523194880.000, 0.000
"b..Y....(?", -30, 16800, 423994384, -0.000, 0.000
"]*.....F..", 120, 10035, -1588323828, -0.000, 0.000
"R...9...ac", -92, -22320, 91620164, -364193873920.000, -0.000
"..8d..9.(.", -80, -24735, -1019651914, 0.000, -237768.381
Pls comment on it ...
Thanks,
Best Rgds,
_calv

What is the easiest way to read fixed width ascii files?

There does not seem to be an easy way to read fixed with ascii files in labview. CSV files and other delimited files are easy, but I cannot think of a simple way to read data from a fixed width ascii file.

jgkempen wrote:
There does not seem to be an easy way to read fixed with ascii files in labview.
Reading a file does not depend on its contents. Unless the file is gigantic, you can just read it as one big string and parse it later, especiall if you want the entire contents of the file.
If you only need a subset, you can calculate the offsets and sizes as multiples of your record size. That's the beauty of fixed size records.
It is typically not efficient to read a file in a loop, small portions at a time, so I would not recommend that except in some very specific situations.
Could you be a bit more specific on what exactly you need to do? Is your problem with reading or with parsing the fixed sized records? Could you attach a sample file and some description on what you need to read from it.
Message Edited by altenbach on 09-05-2007 07:50 PM
LabVIEW Champion . Do more with less code and in less time .

Need a fast ascii file reader in Java

Similar Messages

Maybe you are looking for