File i/o qs with html

Question

hey everybody,
these java forums are a valuable resource so i decided to ask a question about a problem i have been having with my code.
i am trying to read in a file (it's a txt file but with all html code in it)...simple reading is not a problem, but i am trying to filter out the tags and HTML special characters (& ; ,etc)...
eventually i will want to store the "nice" words in an array or something.
here is what i have so far but i cannot seem to figure out what to change? the delimeter setting in the string buffer..or? i've tried a lot of stuff and stripped it back down to how it was at first.
import java.util.*;
import java.io.*;
public class LexicalAnalysis
     public static void main(String args[])
          new LexicalAnalysis();
     StringBuffer dest = new StringBuffer(" ");  // Temporary storage
     private String tag;
     public LexicalAnalysis()
        try
            BufferedReader br = new BufferedReader(new FileReader("cs214sp02.txt"));    // Open source doc   
            String line = null;
            StringBuffer source = new StringBuffer();
            while((line = br.readLine()) != null)   // Read document to StringBuffer
                source.append(line);
            filter(source.toString());
        catch(IOException ioe)
            ioe.printStackTrace();
        private void filter(String src_)
        StringTokenizer st = new StringTokenizer(src_, " <");  // need to decide deilmeters.          
         int tokens = st.countTokens();  // Number of tokens in document          
        for(int i=0; i<tokens; i++) // go through all tokens
            String token = st.nextToken();                                
               System.out.print(token+" "+"\n");
        save(dest.toString());  // Save to disk  */ 
    private void save(String dest_)
        try
            String fileName = "C:/Windows/Desktop/TEST.txt";
            PrintWriter pw = new PrintWriter(new FileWriter(fileName));
            pw.println(dest_);   
            pw.flush();
            System.out.println("\n\n\nend doc!");       
        catch(IOException ioe)
            ioe.printStackTrace();

Answer 1

hey everybody,
i switched up the code - i now read line by line from the file.. i also used a tokenizer in a different fashion.. it seems to work nicely except one problem which i hope someone can help me with!
when i print out the tokens to the screen.. for some reason, the last word in the html file that it reads is printed multiple times! i have no idea why!
maybe ill make this a new post on the forum as well for new people to take a look too :)
import java.util.*;
import java.io.*;
public class LexicalAnalysis
     public static void main(String args[])
          new LexicalAnalysis();
     StringBuffer dest = new StringBuffer(" ");  // Temporary storage
     private String tag;
     public LexicalAnalysis()
        try
            BufferedReader br = new BufferedReader(new FileReader("cs430fa02.txt"));    // Open source doc   
            String line = null;
            while((line = br.readLine()) != null)   // Read document to StringBuffer
                filter(line);
        catch(IOException ioe)
            ioe.printStackTrace();
     private void filter(String src_)
          StringTokenizer st = new StringTokenizer(src_, " <>;&#.", true); //StringTokenizer(src_, " "); // need to decide deilmeters.       
          int tokens = st.countTokens();        // Number of tokens in document       
          String token="";     
          for(int i=0; i<tokens; i++)
               if(st.hasMoreTokens())
                    token = st.nextToken();
               if(token.equals("<"))
                    while(!token.equals(">"))
                         token = st.nextToken();
               if(token.equals("&"))
                    while(!token.equals(";"))
                         token=st.nextToken();
               if(!token.equals(">") && (!token.equals(";")))// && (!token.equals("."))))
                    System.out.print(token.trim()+" ");
               //save(dest.toString());        // Save to disk  */   
    private void save(String dest_)
        try
            String fileName = "C:/Windows/Desktop/TEST.txt";
            PrintWriter pw = new PrintWriter(new FileWriter(fileName));
            pw.println(dest_);   
            pw.flush();
            System.out.println("\n\n\nend doc!");       
        catch(IOException ioe)
            ioe.printStackTrace();