Reading large binary files into an array for parsing

I have a large binary log file, consisting of binary data separted by header flags scattered nonuniformly thorughout the data. The file size is about 50M Byte. When I read the file into an array, I get the Labview Memory full error. The design of this is to read the file in and then parse it fro the flags to determine where to separate the data blocks in the byte stream.
There are a few examples that I have read on this site but none seem to give a straight answer for such a simple matter. Does anyone have an example of how I should approach this?

I agree with Gerd. If you are working with binaries, why not use U8 instead of doubles.
If the file is indeed 50MB, then the array should be expecting 52428800 elements, not 50000000. So if you read the file in a loop and populate an element at a time, you could run out of memory fast because any additional element insertion above 50000000 may require additional memory allocation of the size above 50000000 (potentially for each iteration). This is just speculation since I don't see the portion of your code that populates the array.
Question: Why do you need an array? What do you do with the data after you read it? I agree with Altenbach, 50MB is not that big, so working with a file of such a size should not be a problem.

Similar Messages

Read large binary file

How would you read a large binary file into memory please? I had a thought about creating a Byte Array on the fly, however you cannot createa byte array with a long so what happens when you reach the maximum value an integer can store?

a) You can map the file, instead of reading it physically.
b) Let's suppose that you are running Sun JVM in Windows 2000/XP/2003 and you have 4-GB of RAM in your machine (memory is so cheap nowadays...)
- Windows can not use the full 4GB (it reserves 2GB for itself, and if you buy a special server version, it reserves only 1GB for itself.)
- Java can't use more than 1.6 GB in Windows due to some arcane reasons. (Someone posted the Bug Database explanation in this forum.)
So you have an upper limit of 1.6 GB.
1.6GB is a value smaller than the size of an int, so you could have an array as big (using the class java.nio.ByteBuffer and the like). Try and see.

Read very very large excel file into 2d array (strings or varient)

Hi all,
Long time user, first time poster on these boards.
Looking at the copius amounts of great info related to reading Excel data from .xls files into labview, i've found that every one i've found from various people use the ActiveX method (WorkSheet.Range) which two strings are passed, namely excel's LetterNumber format to specify start and end.
However, this function does not work when trying to query huge amounts of information. The error returned is "Type Mismatch, -2147352571" I have a very large excel sheet i need to read data from and then close the excel file (original file remains unchanged). However this file is gigantic (don't ask me, I didn't make it and I can't convince them to use something more appropriate) with over 165 columns at 1000 rows of data.I can read a large number of columns, but only a handful of rows, or vice versa.
Aside from creating a loop to open and close the excel file over and
over reading pieces of it at a time, is there a better way to read more
data using ActiveX? Attached is code uploaded by others (with very minor modification) as an example.
Thanks,
Attachments:
Excel Get Data Specified Field (1-46col).vi ‏23 KB

Hi Maddox731,
I've only had a very quick glance through your thread, and I must admit I haven't really thought it through properly yet. Sounds like you've come up with your own solution anyway. That said I thought I'd take a bit of a scatter gun approach and attach some stuff for you regradless. Please forgive my bluntness.
You'll find my ActiveX Excel worksheet reader, which may or may not contain the problem you've come across. I've never tried it with the data size you are dealing with, so no promises. I've also attached my ADO/SQL approach to the problem. This was something I moved onto when I realised the limitations of AX. One thing I have noticed is that ADO/SQL is much faster than AX, so there may be some gains for you there with large data sets if you can implement it.
I should add that I'm a novice to all this and my efforts are down to bits I've gleamed from MSDN and others' LV examples. I hope it's of some use, if only to spark discussion. Your ctiticism is more than welcome, good or bad.
Regards.
Attachments:
Database Table Reading Stuff.zip ‏119 KB

How do I read directly from file into byte array

I am reading an image from a file into a BuffertedImage then writing it out again into an array of bytes which I store and use later on in the program. Currently Im doing this in two stages is there a way to do it it one go to speed things up.
try
            //Read File Contents into a Buffered Image
            /** BUG 4705399: There was a problem with some jpegs taking ages to load turns out to be
             * (at least partially) a problem with non-standard colour models, which is why we set the
             * destination colour model. The side effect should be standard colour model in subsequent reading.
            BufferedImage bi = null;
            ImageReader ir = null;
            ImageInputStream stream = ImageIO.createImageInputStream(new File(path));
            final Iterator i = ImageIO.getImageReaders(stream);
            if (i.hasNext())
                ir = (ImageReader) i.next();
                ir.setInput(stream);
                ImageReadParam param = ir.getDefaultReadParam();
                ImageTypeSpecifier typeToUse = null;
                for (Iterator i2 = ir.getImageTypes(0); i2.hasNext();)
                    ImageTypeSpecifier type = (ImageTypeSpecifier) i2.next();
                    if (type.getColorModel().getColorSpace().isCS_sRGB())
                        typeToUse = type;
                if (typeToUse != null)
                    param.setDestinationType(typeToUse);
                bi = ir.read(0, param);
                //ir.dispose(); seem to reference this in write
                //stream.close();
            //Write Buffered Image to Byte ArrayOutput Stream
            if (bi != null)
                //Convert to byte array
                final ByteArrayOutputStream output = new ByteArrayOutputStream();
                //Try and find corresponding writer for reader but if not possible
                //we use JPG (which is always installed) instead.
                final ImageWriter iw = ImageIO.getImageWriter(ir);
                if (iw != null)
                    if (ImageIO.write(bi, ir.getFormatName(), new DataOutputStream(output)) == false)
                        MainWindow.logger.warning("Unable to Write Image");
                else
                    if (ImageIO.write(bi, "JPG", new DataOutputStream(output)) == false)
                        MainWindow.logger.warning("Warning Unable to Write Image as JPEG");
                //Add to image list
                final byte[] imageData = output.toByteArray();
                Images.addImage(imageData);


If you don't need to manipulate the image in any way I would suggest you just read the image file directly into a byte array (without ImageReader) and then create the BufferedImage from that byte array.

Reading Large Matlab files into LabVIEW

Hi All,
I am using the 'Open Data Storage [MATLAB] block to read data from a mat file into MATLAB. The data is very large (in the order of millions). When I try to run my vi which is relatively simple to read in and manipulate the data, my LabVIEW said it does not have enough memory. Is there any way to around this problem without physically increasing computer memory.
Thanks in advance for any help and assistance.
Chris

Hi Chris,
Did the 'not enough memory' happen on 'Open Data Storage' or 'Read Data' ?
If it happened on the 'Read Data', you might want to try specifying 'index/count' inputs of the 'Read Data' to read channel values chunk by chunk.
Best Regards,
Mavis

Reading files into an array

so im new to this and having problems.
we have to read a .txt file into an array so when a user types a word in a text box it will search the file and reutrn found if the word is there and not found it the word is not there. but it is not reading the text file here is the code so far
import java.awt.*;
import java.awt.event.*;
import javax.swing.*;
import java.io.*;
public class Checker
// Declares Values in the array
     static String [] anArray = new String[1000];
     // declares variables needed in code later
     int i;
     String s;
     static int count = 0;
     static String b;
     static JTextArea j = new JTextArea();
     public void readFile() throws IOException{
     File inFile = new File("words.txt");
     FileReader fileReader = new FileReader(inFile);
BufferedReader bufReader = new BufferedReader(fileReader);
while (true){
b = bufReader.readLine();
anArray[count] = b;
count++;
if(b==null) throw new EOFException("Hello, end of file reached");
j.append(b);
j.append("\n");
     // public static void main(String args[]){
     // File outFile = new File("words.txt");
     //FileOutputStream outFileStream = new FileOutputStream(outFile);
     // PrintWriter outStream = new PrintWriter(outFileStream);
     // outStream.println("help");
// File inFile = new File("words.txt");
// FileReader fileReader = new FileReader(inFile);
// BufferedReader bufReader = new BufferedReader(fileReader);
/* while (true){
b = bufReader.readLine();
anArray[count] = b;
count++;
if(b==null) throw new EOFException("Hello, end of file reached");
j.append(b);
j.append("\n");
public String checkword(String Word ) {
// Tries the code, runs through until an exception is found.
try {
for ( i = 0; i < anArray.length; i++)
if (Word.equals(anArray))
// If exception is found, throws a new exception
if (Word.equals(anArray[i])) throw new Exception(); // end if
} // end if
} // end for
} // end try
catch (Exception ae ) {
// if the word is found return a message found and its position in array.
return "Found, at position "+i;
}// end catch
// if its not in the array return a not found message
return "Not found";
     } // end
} // end
and the gui class :
import java.awt.*;
import java.awt.event.*;
import javax.swing.*;
import java.io.*;
public class CountDown extends JFrame implements ActionListener
// Defines the GUI components
private JTextField Word = new JTextField(9);
private JButton Guess = new JButton("Enter String");
     private JLabel Message = new JLabel("not found");
     // Creates a new instance of the Checker class.
Checker ch = new Checker();
public CountDown() // constructor
setTitle("Count Down");
setSize(300,200);
setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
setVisible(true);
Container c = getContentPane();
c.setBackground(Color.white);
c.setLayout(new GridLayout(3,1));
// Add the components
c.add(Word);
c.add(Guess);
c.add(Message);
c.add(ch.j);
// Add action Listener for Button to work
Guess.addActionListener(this);
pack();
setContentPane(c);
public void actionPerformed(ActionEvent e) {
// When the button is pressed do this code
if(e.getSource() == Guess)
// Looks in the checker class to compare the word user enters, to the words in the array.
// Returns a message if the word exists with its position in the list or
// Returns a message if the word does not exisit.
Message.setText( ch.checkword(Word.getText()));
public class CountDownA
public void main(String[] args)
CountDown Game = new CountDown();
any help is appriciated asap

Yuck! I echo jverd's sentiment. Please only post the specific pieces of code that you are having trouble with, and use the forum 'code' tags.
But to answer your question, I don't see where you're calling the readFile() method from. If you're not calling readFile, that might explain why nothing is being read.
As for the design of the readFile method, another 'yuck'. heh Don't use exceptions to break out of a loop. If you MUST break out of a loop, use break. But a loop like this doesn't need break. Try something like this:while( (b = bufReader.readLine()) != null)
     anArray[count++] = b;
     j.append(b+"\n");
}Also, in stead of creating an array of some arbitrary fixed length, I recommend using a collection class such as ArrayList.
In stead of this:static String [] anArray = new String[1000];
anArray[count++] = b;it would be better to use something like this:static ArrayList<String> arrayList = new ArrayList<String>();
arrayList.add(b);

How to read binary file into a 2D float array??

Hi All,
I really need help to get this one as I am stuck and can't seem to find any way out of it. I am given a '.dat' file that has float values in it. I want to read this file in java and put it in a 2D float array. The file has basically a matrix of float values. What I want to do is to read this binary file and put all its data into 2D float array (float [] []) so that I can use it in my program. Is there a way to read file like this? I did find a similar matlab code (below) but cant seem to find anything in java and i really want to do this in java only.. I will appreciate ur help in this one.
thanks very much
Nitya
fid = fopen('datafile.dat');
A = fread(fid,[50 50],'float32');
fclose(fid);

I shud have shown the two ways that i Already tried. here they are..
first one using DataInputStream and then trying to readFloat()
DataInputStream dis = ....
Float f = dis.readFloat();This code gives code gives me some random values like this.. (i had a loop)
5.8758974E-14
-0.41055492
1.5724557E-30
1.06822824E14
-1.91934371E15
3.43829601E13
Other way i tried was this.. which seems right but here i have to convert byte to float and i thnk that code is giving some different results (slightly different float values) not sure why....may be my indexing of the array is wrong to make it a matrix.. or something else...
is.read(bytes, offset....);
int cnt = 0;
int j = 0;
for (int start = 0; start < offset; start = start + 4) {
if(j<50){
       myarray[cnt][j] = this.arr2float(bytes, start);
         System.out.println(cnt + "-" + j + " = " + myarray[cnt][j]);
       j++;
}else{
if(cnt < 50){
       cnt++;
       j = 0;
}else{
       break;
public float arr2float (byte[] arr, int start) {
          int i = 0;
          int len = 4;
          int cnt = 0;
          byte[] tmp = new byte[len];
          for (i = start; i < (start + len); i++) {
               tmp[cnt] = arr;
               cnt++;
          int accum = 0;
          i = 0;
          for ( int shiftBy = 0; shiftBy < 32; shiftBy += 8 ) {
               accum |= ( (long)( tmp[i] & 0xff ) ) << shiftBy;
               i++;
          return Float.intBitsToFloat(accum);
Not sure if i am missing some other way to do this...

Large Binary file to array

Hi,
I'm relatively new to Labview and am having trouble with large arrays. I'm using Win32 IO functions to read a 8MB file into Labview and this products a large U8 array. I want to produce a binary array containing the second bit of each byte in this array. For example:
File:00000000 00000010 00000000 00000010 00000010
Binary array: 01011
At the moment I'm using a For Loop and clearly its very inefficient. Is there any way to do the above in a relatively short period of time?
I used a subvi containing a For Loop and enabled Reentrant execution but this isnt very fast either.
Any help would be greatly appreciated,
John
Attachments:
image_bin.png ‏9 KB

Hi John,
You can use boolean operators on integers. No need to convert to boolean array. To extract a bit, just AND with a constant with the bit in question set. Also, you don't need the loop.
See attached example.
The quotient & remainder is one way to scale the resulting array to 1. Without this you'd get an array of 0's and 2's.
Hope this helps,
Daniel
EDIT: Sorry for repeating 10degree, I was distracted while writing the message
Message Edited by dan_u on 07-07-2009 03:52 PM

How to read sampled data info from a "*.au"file into an array?

I need to manipulate the sound file directly using DSP, the first step is to read sampled data from "*.au" file into an array. How can I do that? Thanks a lot!!

There is a file I/O tutorial in the Java Tutorial (google for Java Tutorial). You can read an .au file as bytes like any binary file.
Or if you are using the Java Sound API, google for java sound api or java audio tutorial etc etc.

Reading from file into an array

Hello, new to Java and we need to modify a mortgage calculator to read interest rate from a file into an array and not have them hard coded in the program. I have read many post on how to perform this but am lost on where to put the new code and format. Here is my code and I hope I posted this right.
import javax.swing.*;                                              // Imports the Main Swing Package
import javax.swing.event.*;
import javax.swing.text.*;                                         // Used for Text Box Caret Position
import java.awt.*;                                                // Imports the main AWT Package
import java.awt.event.*;                                         // Event handling class are defined here
import java.text.NumberFormat;
import java.text.*;                                              // Imports the Main Text Package
import java.util.*;                                              // Imports the Main Utility Package
public class mortgageCalculator1 extends JFrame implements ActionListener             // Creates class mortgageCalculator
    JLabel AmountLabel = new JLabel("   Enter Mortgage Amount:$ ");                   // Declares Mortgage Amount Label
    JTextField mortgageAmount = new JTextField(10);                                     // Declares Mortgage Amount Text Field
    JButton IntandTerm1B = new JButton("7 years at 5.35%");                    // Declares 1st Mortgage Term and Interest Rate
    JButton IntandTerm2B = new JButton("15 years at 5.50%");                    // Declares 2nd Mortgage Term and Interest Rate
    JButton IntandTerm3B = new JButton("30 years at 5.75%");                    // Declares 3rd Mortgage Term and Interest Rate
    JLabel PaymentLabel = new JLabel("   Monthly Payment: ");                     // Declares Monthly Payment Label
    JTextField monthlyPayment = new JTextField(10);                          // Declares Monthly Payment Text Field
    JButton exitButton = new JButton("Exit");                              // Declares Exit Button
    JButton newcalcButton = new JButton("New Calculation");                     // Declares New Calculation Button
    JTextArea mortgageTable = new JTextArea(35,65);                         // Declares Mortgage Table Area
    JScrollPane scroll = new JScrollPane(mortgageTable);                         // Declares ScrollPane and puts the Mortgage Table inside
    public mortgageCalculator1()                                        // Creates Method
         super("MORTGAGE CALCULATOR");                              // Title of Frame
         JMenuBar mb = new JMenuBar();                                   // Cretes Menu Bar
            JMenu fileMenu = new JMenu("File");                                 // Creates File Menu
         fileMenu.setMnemonic('F');                                      // Enables alt + f to Access File Menu
         JMenuItem exitItem = new JMenuItem("Exit");                     // Creates Exit in File Menu
         fileMenu.add(exitItem);                                   // Adds Exit to File Menu
             exitItem.addActionListener(new ActionListener()                     // Adds Action Listener to the Exit Item
                 public void actionPerformed(ActionEvent e)                     // Tests to Verify if File->Exit is Pressed
                     System.exit(0);                               // Exits the Programs when File->Exit is Pressed
            mb.add(fileMenu);                                      // Adds the File Menu
            setJMenuBar(mb);
     setSize(600, 400);                                        // Sets Size of Frame
        setLocation(200,200);                                         // Sets the Location of the Window
     setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);                                  // Command on how to close frame
     JPanel pane = new JPanel();                                   // Declares the JPanel
     pane.setLayout(new BoxLayout(pane, BoxLayout.Y_AXIS));                          // Sets Panel Layout to BoxLayout
     Container grid = getContentPane();                               // Declares a Container called grid
     grid.setLayout(new GridLayout(4,3,5,5));                          // Sets grid Layout to GridLayout
     pane.add(grid);                                             // Adds the grid to the Panel
     pane.add(scroll);                                        // Addes the scrollPane to the Panel
        grid.setBackground(Color.yellow);                              // Set grid color to Yellow
     setCursor(new Cursor(Cursor.HAND_CURSOR));                         // Makes the cursor look like a hand
     mortgageAmount.setBackground(Color.black);                         // Sets mortgageAmount JPanel JTextField Background Color
        mortgageAmount.setForeground(Color.white);                         // Sets mortgageAmount JPanel JTextField Foreground Color
     mortgageAmount.setCaretColor(Color.white);                         // Sets mortgageAmount JPanel JTextField Caret Color
     mortgageAmount.setFont(new Font("Lucida Sans Typewriter", Font.PLAIN, 18));     // Sets mortgageAmount JPanel JTextField Font
     monthlyPayment.setBackground(Color.black);                         // Sets monthlyPayment JPanel JTextField Background Color
     monthlyPayment.setForeground(Color.white);                         // Sets monthlyPayment JPanel JTextField Foreground Color
        monthlyPayment.setFont(new Font("Lucida Sans Typewriter", Font.PLAIN, 18));     // Sets monthlyPayment JPanel JTextField Font
     mortgageTable.setBackground(Color.yellow);                         // Sets mortgageTable JTextArea Background Color
     mortgageTable.setForeground(Color.black);                         // Sets mortgageTable JTextArea Foreground Color
     mortgageTable.setFont(new Font("Arial", Font.PLAIN, 18));               // Sets JTextArea Font
     grid.add(AmountLabel);                                        // Adds the Mortgage Amount Label
     grid.add(mortgageAmount);                                   // Adds the Mortgage Amount Text Field
     grid.add(IntandTerm1B);                                        // Adds 1st Loan and Rate Button
     grid.add(PaymentLabel);                                    // Adds the Payment Label
     grid.add(monthlyPayment);                                    // Adds the Monthly Payment Text Field
     monthlyPayment.setEditable(false);                              // Disables editing in this Text Field
        grid.add(IntandTerm2B);                                        // Adds 2nd Loan and Rate Button
        grid.add(exitButton);
     grid.add(newcalcButton);                                    // Adds the New Calc Button
        grid.add(IntandTerm3B);                                        // Adds the Exit Button
     setContentPane(pane);                                         // Enables the Content Pane
     setVisible(true);                                         // Sets JPanel to be Visable
     exitButton.addActionListener(this);                               // Adds Action Listener to the Exit Button
     newcalcButton.addActionListener(this);                            // Adds Action Listener to the New Calc Button
        IntandTerm1B.addActionListener(this);                              // Adds Action Listener to the 1st loan Button
     IntandTerm2B.addActionListener(this);                              // Adds Action Listener to the 2nd loan Button
     IntandTerm3B.addActionListener(this);                               // Adds Action Listener to the 3rd loan Button
     mortgageAmount.addActionListener(this);                              // Adds Action Listener to the Mortgage Amount Text Field
     monthlyPayment.addActionListener(this);                              // Adds Action Listener to the Monthly payment Text Field
    public void actionPerformed(ActionEvent e)                               // Tests to Verify Which Button is Pressed
        Object command = e.getSource();                                                 // Enables command to get data
        if (command == exitButton) //sets exitButton                         // Activates the Exit Button
            System.exit(0); // Exits from exit button                         // Exits from exit button
        int loanTerm = 0;                                             // Declares loanTerm
        if (command == IntandTerm1B)                                   // Activates the 1st Loan Button
            loanTerm = 0;                                        // Sets 1st value of Array
        if (command == IntandTerm2B)                                   // Activates the 2nd Loan Button
            loanTerm = 1;                                        // Sets 2nd value of Array
        if (command == IntandTerm3B)                                   // Activates the 3rd Loan Button
         loanTerm = 2;                                        // Sets 3rd value of Array
            double mortgage = 0;                                   // Declares and Initializes mortgage
         double rate = 0;                                        // Declares and Initializes rate
         double [][] loans = {{7, 5.35}, {15, 5.50}, {30, 5.75},};                 // Array Data for Calculation
                try
                    mortgage = Double.parseDouble(mortgageAmount.getText());            // Gets user input from mortgageAmount Text Field
                  catch (NumberFormatException nfe)                          // Checks for correct number fformatting of user input
                   JOptionPane.showMessageDialog (this, "Error! Invalid input!");      // Outputs error if number is wrong format or nothing is entered
                   return;
         double interestRate = loans [loanTerm][1];                         // Sets interestRate amount
         double intRate = (interestRate / 100) / 12;                         // Calculates Interst Rate
            double loanTermMonths = loans [loanTerm] [0];                    // Calculates Loan Term in Months
            int months = (int)loanTermMonths * 12;                          // Converts Loan Term to Months
            double interestRateMonthly = (intRate / 12);                                // monthly interst rate
         double payment = mortgage * intRate / (1 - (Math.pow(1/(1 + intRate), months)));    // Calculation for Monthly payment
            double remainingLoanBalance = mortgage;                              // Sets Reamaining Balance
            double monthlyPaymentInterest = 0;                                       // holds current interest payment
            double monthlyPaymentPrincipal = 0;                                    // holds current principal payment
            NumberFormat myCurrencyFormatter = NumberFormat.getCurrencyInstance(Locale.US);     // Number formatter to format output in table
            monthlyPayment.setText(myCurrencyFormatter.format(payment));
            mortgageTable.setText("Month\tPrincipal\tInterest\tEnding Balance\n" +                // Formats morgageTable Header
                                  "---------\t----------\t------------\t---------------------\n");
                for (;months > 0 ; months -- )
                 monthlyPaymentInterest = (remainingLoanBalance * intRate);               // Calculation for Monthly Payment Toward Interest
                 //Calculate H = R x I
                 monthlyPaymentPrincipal = (payment - monthlyPaymentInterest);          // Calculation for Monthly Payment Toward Principal
                 //Calculate C = P - H
              remainingLoanBalance = (remainingLoanBalance - monthlyPaymentPrincipal);     // Calculation for Reamining loan Balance
              // Calculate R = R - C
              // H = monthlyPaymentInterest
              // R = remainingLoanBalance
              // P = payment
              // C = monthlyPaymentPrincipal
              // I = interestRateMonthly
              mortgageTable.setCaret (new DefaultCaret());                    // Sets Scroll position to the top left corner
              mortgageTable.append(String.valueOf(months) + "\t" +               // Pulls in data and formats MortgageTable
              myCurrencyFormatter.format(monthlyPaymentPrincipal) + "\t" +
                 myCurrencyFormatter.format(monthlyPaymentInterest) + "\t" +
                 myCurrencyFormatter.format(remainingLoanBalance) + "\n");
                       if(command == newcalcButton)                               // Activates the new calculation Button
                     mortgageAmount.setText(null);                         //clears mortgage amount fields
                  monthlyPayment.setText(null);                         //clears monthly payment fields
                        mortgageTable.setText(null);                              //clears mortgage table
public static void main(String[] args)                               //This is the signature of the entry point of all the desktop apps
     new mortgageCalculator1();
}

OK, making a little progress but am still very confused.
What I have is a file (int&term.dat) with three lines;
5.75, 30
5.5, 15
5.35 ,7
I have three JButtom that I what to read a seperate line and place the term in a term TextField and a rate in a Rate TextField
I have added the following code and all it does now is output a black space to the screen; I am working with one Button and Just the rate for now to try and get it to work. I have been looking at the forums, reading the internet and several books to try and figure this out. I think I may be getting closer.
public static void read()
    String line;
    StringTokenizer tokenizer;
    String rate;
    String term;
    try
        FileReader fr = new FileReader ("int&term.dat");
        BufferedReader inFile = new BufferedReader (fr);
        line = inFile.readLine();
        while (line != null)
        tokenizer = new StringTokenizer(line);
        rate = tokenizer.nextToken();
        line = inFile.readLine();
         inFile.close();
         System.out.println(new String());
         catch (FileNotFoundException exception)
               System.out.println ("The file was not found.");
          catch (IOException exception)
                System.out.println (exception);
}

Reading one line from a text file into an array

i want to read one line from a text file into an array, and then the next line into a different array. both arays are type string...i have this:
public static void readAndProcessData(FileInputStream stream){
     InputStreamReader iStrReader = new InputStreamReader (stream);
     BufferedReader reader = new BufferedReader (iStrReader);
     String line = "";
     try{
     int i = 0;
          while (line != null){
               names[i] = reader.readLine();
               score[i] = reader.readLine();
               line = reader.readLine();
               i++;
          }catch (IOException e){
          System.out.println("Error in file access");
this section calls it:
try{
     FileInputStream stream = new FileInputStream("ISU.txt");
          HighScore.readAndProcessData(stream);
          stream.close();
          names = HighScore.getNames();
          scores = HighScore.getScores();
     }catch(IOException e){
          System.out.println("Error in accessing file." + e.toString());
it gives me an array index out of bounds error

oh wait I see it when I looked at the original quote.
They array you made called names or the other one is prob too small for the amount of names that you have in the file. Hence as I increases it eventually goes out of bounds of the array so you should probably resize the array if that happens.

What's the best way for reading this binary file?

I've written a program that acquires data from a DAQmx card and writes it on a binary file (attached file and picture). The data that I'm acquiring comes from 8 channels, at 2.5MS/s for, at least, 5 seconds. What's the best way of reading this binary file, knowing that:
-I'll need it also on graphics (only after acquiring)
-I also need to see these values and use them later in Matlab.
I've tried the "Array to Spreadsheet String", but LabView goes out of memory (even if I don't use all of the 8 channels, but only 1).
LabView 8.6
Solved!
Go to Solution.
Attachments:
AcquireWrite02.vi ‏15 KB
myvi.jpg ‏55 KB

But my real problem, at least now, is how can I divide the information to get not only one graphic but eight?
I can read the file, but I get this (with only two channels):
So what I tried was, using a for loop, saving 250 elements in different arrays and then writing it to the .txt file. But it doesn't come right... I used 250 because that's what I got from the graphic: at each 250 points it plots the other channel.
Am I missing something here? How should I treat the information coming from the binary file, if not the way I'm doing?
(attached are the .vi files I'm using to save in the .txt format)
(EDITED. I just saw that I was dividing my graph's data in 4 just before plotting it... so It isn't 250 but 1000 elements for each channel... Still, the problem has not been solved)
Message Edited by Danigno on 11-17-2008 08:47 AM
Attachments:
mygraph.jpg ‏280 KB
Read Binary File and Save as txt - 2 channels - with SetFilePosition.vi ‏14 KB
Read Binary File and Save as txt - with SetFilePosition_b_save2files_with_array.vi ‏14 KB

Large binary file reading

Hi,
I'm currently using a java.io.BufferedInputStream to read a large binary file.
I recently discovered there is a chunk of data that shows up near the end of each file. (these files are binary and are XX to XXX mb in size)
Loading it all to memory first would kill my performance so I'd like to be able to come up with an alternate method.
Does Java or the class above offer a way I can
1) get the length of the file
2) seek to a point say 2000 bytes from the end so I can start reading the binary data?
Ideally I'd like to do a backwards read as that would be quickest. Is there a way to change the operation so that a 'read' would be reading backwards (from end to beginning)?
For me speed is the #1 thing i have to worry about. So to be able to seek forward several hundred thousand bytes at a time would help tremendously.

how does the 'skip' method work? Probably by using OS specific calls to read to a point in the file.
maybe I could
'skip' length - 20k from the start or something like
that.
Yep.

How can i read local excel file into internal table in webdynpro for abap a

Could someone tell me how How can i read local excel file into an internal table in webdynpro for abap application.
thank u for your reply

Deep,
File manuplations...............................
1. At the presentation level:
->GUI_UPLOAD
->GUI_DOWNLOAD
->CL_GUI_FRONTEND
2. At the application server level:
->OPEN DATASET : open a file in the application server for reading or writing.
->READ DATASET : used to read from a file on the application server that has been opened for reading
-> TRANSFER DATASET : writing data to a file.
-> CLOSE DATASET : closes the file
-> DELETE DATASET : delete file
If file is on the local PC,use the function module GUI_UPLOAD to upload it into an internal table by passing the given parameters......
call function 'GUI_UPLOAD'
exporting
filename = p_file
filetype = 'ASC'
has_field_separator = '#'
tables
data_tab = t_data
p_file : excel file path.
t_data : internal table
<b>reward points if useful.</b>
regards,
Vinod Samuel.

Any info on CRC, checksum, or other file integity VIs for large binary files?

Working on send rather large binary files (U16 stream to file) via internet. Would like to check for file integity via CRC or comparable checksum. Would appreciate any comments/suggestions

Hi Brian,
You said;
"Would appreciate any comments/suggestions".
You did not mention what transport mechanism you plan on using.
As I understand ALL of the standard mechanism use CRC of some form to ensure the validity of the packet BEFORE it is ever passed up the OSI 7-Layer model.
TCP/IP based protocols will see to it that all of the segments of a transfer are completed and in order.
UDP on the other hand is a broadcast type protocol and does not ensure any packets are recieved.
So,
At the very worst you should be able to handle your "sanity checks" by simply using a sequence value that is included in your out-going message. The reciever should just have to check if the current seq value is equal to the previous +1.
I co-developed an app that ut
ilized this technique to transfer status messages from a RT platform to a Windows machine. The status messages in this app where concidered FYI, so the sequence counter served as a way of determining if anything was missed.
I am insterested in others thoughts on this subject.
Ben
Ben Rayner
I am currently active on.. MainStream Preppers
Rayner's Ridge is under construction

Reading large binary files into an array for parsing

Similar Messages

Maybe you are looking for