Problems with string tokenizer

Question

okay heres my problem, i have a textarea displaying a program listing and i have to extract variables i.e public int something;
i have tried many different approaches but still the output is not what i was looking for. Can someone have a look at my code and check for mistakes that i cant find
public void createDataDictionary() {
          if(fullText.equals("")) return;//do nothing if no text exists
          String checkText = fileContents.getText();//store text area contents into string
          String dataType = "";
          String variable = "";
          String accessModifier = "";
          StringTokenizer str = new StringTokenizer(checkText," \n", true);
          fileCheckBox.setText("");//clear file check box
          while(str.hasMoreTokens()) {//loop while there are more tokens to tokenize
               try{                    
                    checkText = str.nextToken();
                    //check for comments
                    if((checkText.startsWith("//")) || (checkText.equals("//")) ||
                       (checkText.startsWith("/**")) || (checkText.equals("/**")) ||
                       (checkText.startsWith("*")) || (checkText.equals("*"))) {
                         isComment = true;                    
                    if((checkText.equals("\n"))) isComment = false;
                    if(!isComment)
                         //check for access modifiers
                         if((checkText.equals("public")) || (checkText.equals("private")) ||
                            (checkText.equals("protected"))) {
                                      accessModifier = checkText;
                                 }else {
                                      accessModifier = "";
                         //check for data types             
                         if((checkText.equals("boolean")) || (checkText.equals("char")) ||
                                 (checkText.startsWith("String")) || (checkText.equals("int"))) {
                                   dataType = checkText;
                                   variable = str.nextToken();//get variable expression
                                   System.out.println(accessModifier + " " + dataType + " " + variable);
               }catch(NoSuchElementException nsee) {//not enough tokens to complete operation
                    JOptionPane.showMessageDialog(null, "End of file");
                    return;//break from method
     }here is sample text
private String seven = "help";
char five[];
// String here
//int found
public boolean okay
and here is the output
String
char
String
boolean
//note the space before each output
hope someone can help
thanx in advance

Answer 1

1. Why do you check to see if the token starts with
//, /*, etc, etc. Later you check if the token
equals private, public, protected. It couldn't
equal if it started with a comment.if the token starts with a comment i dont want to read the rest of the line, so i set the isComment flag to true
2. I strongly suggest that you do it line by line.
Perhaps a string tokenizer that only tokenizes by
lines, then within that loop, another that tokenizes
by whitespace. i take it you mean putting the text into a bufferedreader and then using readLine()?? Bit new to string tokenization as you can possibly see
i managed to get the small test text to work more or less as i wanted but when ever i load a large code listing, the results are erratic to say the least
heres a section of this code
private int textNum = 0;/**Integer used to hold the text position within any given line*/
     private int lineNum = 0;/**Integer used to hold the line number within any given file*/
     static boolean application = false;/**Used to track if applet is ran from a browser or JAR file*/
     static boolean fileOpened = false;/**Used to track if file has already been opened*/
     static boolean isComment = false;
     private char lCurve = '(';
     private char rCurve = ')';
     private char lCurly = '{';
     private char rCurly = '}';
     private char lSquare = '[';
     private char rSquare = ']';
     String fullText = "";and heres the output
public int textNum //should be private!!!!
int lineNum //missing private
boolean application //missing static
boolean fileOpened //missing static
boolean isComment //missing static
//all below missing private     
char lCurve
char rCurve
char lCurly
char rCurly
char lSquare
char rSquare
String fullText //not there at all