Command line style regular expression string parser

Question

Hi people,
I am currently working on a program where I need to parse a file (or any input stream) line by line. I then need to parse every line for arguments. Each line is formatted similar to how arguments are passed to the command line. The regular expression needs to split every line by any encountered whitespace, but needs to be able to retain any whitespace within double quotes (i.e. "some spaced text here"). Arguments can be numbers, booleans and (quoted) strings. Quoted strings must also be able to have escaped quotes in it (as below). The quotes for the quoted string (the outer ones, obviously not the escaped ones) do not necessarily have to be retained.
An example input line:
arg1 arg2 "arg 3" "arg 4" 987 arg6 "arg \"arg \"arg 7"
Desired example output:
arg1
arg2
arg 3
arg 4
987
arg6
arg "arg "arg 7
After the input line has been split up the program will handle any parsing (i.e. numbers, booleans, etc.). The program currently uses a simple for loop to iterate over all characters in the line and splits it up appropriately by checking every character. However, if this can be done automatically by using a regular expression passed to String.split() (or with some use of the regex package), it would remove quite a bit of redunant code and make the program that much more maintainable.
I do not have much experience with regular expressions since I have never really had the need to use them, but if they can work in this case it would be great.
Thanks in advance for any help.

Answer 1

Almost any parsing problem can be solved if you throw a big enough and ugly enough regex at it, or so I'm told.
I think what you are doing is also amenable to java.io.StreamTokenizer:
import java.io.*;
import static java.io.StreamTokenizer.*;
public class StreamTokenizerExample {
    public static void main(String[] args) throws IOException {
        StringReader input = new StringReader("arg1 arg2 \"arg 3\" \"arg 4\" 987 arg6 \"arg \\\"arg\\\" arg 7\"\nnextline");
        StreamTokenizer in = new StreamTokenizer(input);
        in.eolIsSignificant(true);
        for(int ttype; (ttype = in.nextToken()) != TT_EOF; ) {
            switch (ttype) {
                case TT_WORD:
                    System.out.println("String[" + in.sval + "]");
                    break;
                case TT_NUMBER:
                    System.out.println("number[" + in.nval + "]");
                    break;
                case TT_EOL:
                    System.out.println("[EOL]");
                    break;
                case '"':
                    System.out.println("quoted[" + in.sval + "]");
                    break;
                default:
                    System.out.println("unexpected " + ttype);
{code}