Peeking or pushing back into a StringTokenizer?

Question

Hello all, I am working on creating a Lexer for C using Java, and am having a hard time with the StringTokenizer, which I have to use. To recognize tokens such as "==" and "<=" I know I need either a peek method or a pushback method, however I have no idea how to construct these with a StringTokenizer, without consuming the token. My code is something like this:
class Lexer:
private StringTokenizer st;
method nextToken():
String nextVal;
if (token.equals("="))
     while (st.hasMoreTokens())
          nextVal = st.nextToken();
          if (!nextVal.equals("="))
                    pushBack(nextVal); //PUSH TOKEN BACK INTO THE STRING TOKENIZER
                    return new Token(Token.EQUALS);     
               else {
                    return new Token(Token.REPBY);
The code should work as follows.... if the token read in is a "=" it should check for the token after that. As long as it is not another "=" then return the Equals token: a single "=". If it reaches the Else statement, we know the next token is another "=", so we return the ReplacedBy token: "==". The code works for every "==" but does not for the single "=". This is because it is consumed by the "st.nextToken()" call, so the token after the single "=" is never read. ("=true" would output as just an "=", the "true" is consumed and lost) Anyways, I know the token must be pushed back into the StringTokenizer stream when there is only a single "=", but I am unsure about what the "pushBack" method should be. I was thinking something like this (but of course it doesnt work):
public void pushBack(String token)
          st = new StringTokenizer(token + st.nextToken());
Or, we could use a peek method instead of a pushBack method.... using some sort of temp variable as a buffer.... but again I am unsure how. Any help?

Answer 1

Yeah I thought about using a Stack... something like this:
while (st.hasMoreTokens())
               stack.push(st.nextToken());
               if (!stack.peek().equals("="))
                    stack.pop();
                    return new Token(Token.EQUALS);     
               else {
                    return new Token(Token.REPBY);
}The above makes sense to me, but I'm getting a NullPointerException when I attempt to push the nextToken from the StringTokenizer onto the stack. Any ideas?