OR ('|') in regular expressions (e.g. split a String into lines)

Question

Which match gets used when you use OR ('|') to specify multiple possible matches in a regex, and there are multiple matches among the supplied patterns? The first one (in the order written) which matches? Or the one which matches the most characters?
To make this concrete, suppose that you want to split a String into lines, where the line delimiters are the same as the [line terminators used by Java regex|http://java.sun.com/javase/6/docs/api/java/util/regex/Pattern.html#lt] :
     A newline (line feed) character ('\n'),
     A carriage-return character followed immediately by a newline character ("\r\n"),
     A standalone carriage-return character ('\r'),
     A next-line character ('\u0085'),
     A line-separator character ('\u2028'), or
     A paragraph-separator character ('\u2029)
This problem has [been considered before|http://forums.sun.com/thread.jspa?forumID=4&threadID=464846] .
If we ignore the idiotic microsoft two char \r\n sequence, then no problem; the Java code would be:
String[] lines = s.split("[\\n\\r\\u0085\\u2028\\u2029]");How do we add support for \r\n? If we try
String[] lines = s.split("[\\n\\r\\u0085\\u2028\\u2029]|\\r\\n");which pattern of the compound (OR) regex gets used if both match? The
[\\n\\r\\u0085\\u2028\\u2029]or the
\\r\\n?
For instance, if the above code is called when
s = "a\r\nb";and if the first pattern
[\\n\\r\\u0085\\u2028\\u2029]is used for the match when the \r is encountered, then the tokens will be
"a", "", "b"
because there is an empty String between the \r and following \n. On the other hand, if the rule is use the pattern which matches the most characters, then the
\\r\\n
pattern will match that entire \r\n and the tokens will be
"a", "b"
which is what you want.
On my particular box, using jdk 1.6.0_17, if I run this code
String s = "a\r\nb";
String[] lines = s.split("[\\n\\r\\u0085\\u2028\\u2029]|\\r\\n");
System.out.print(lines.length + " lines: ");
for (String line : lines) System.out.print(" \"" + line + "\"");
System.out.println();
if (true) return;the answer that I get is
3 lines:  "a" "" "b"So it seems like the first listed pattern is used, if it matches.
Therefore, to get the desired behavior, it seems like I should use
"\\r\\n|[\\n\\r\\u0085\\u2028\\u2029]"instead as the pattern, since that will ensure that the 2 char sequence is first tried for matches. Indeed, if change the above code to use this pattern, it generates the desired output
2 lines:  "a" "b"But what has me worried is that I cannot find any documentation concerning this "first pattern of an OR" rule. This means that maybe the Java regex engine could change in the future, which is worrisome.
The only bulletproof way that I know of to do line splitting is the complicated regex
"(?:(?<=\\r)\\n)" + "|" + "(?:\\r(?!\\n))" + "|" + "(?:\\r\\n)" + "|" + "\\u0085" + "|" + "\\u2028" + "|" + "\\u2029"Here, I use negative lookbehind and lookahead in the first two patterns to guarantee that they never match on the end or start of a \r\n, but only on isolated \n and \r chars. Thus, no matter which order the patterns above are applied by the regex engine, it will work correctly. I also used non-capturing groups
(?:X)
to avoid memory wastage (since I am only interested in grouping, and not capturing).
Is the above complicated regex the only reliable way to do line splitting?

Answer 1

bbatman wrote:
Which match gets used when you use OR ('|') to specify multiple possible matches in a regex, and there are multiple matches among the supplied patterns? The first one (in the order written) which matches? Or the one which matches the most characters?
The longest match wins, normally. Except for alternation (or) as can be read from the innocent sentence
The Pattern engine performs traditional NFA-based matching with ordered alternation as occurs in Perl 5.
in the javadocs. More information can be found in Friedl's book, the relevant page of which google books shows at
[http://books.google.de/books?id=GX3w_18-JegC&pg=PA175&lpg=PA175&dq=regular+expression+%22ordered+alternation%22&source=bl&ots=PHqgNmlnM-&sig=OcDjANZKl0VpJY0igVxkQ3LXplg&hl=de&ei=Dcg7S43NIcSi_AbX-83EDQ&sa=X&oi=book_result&ct=result&resnum=1&ved=0CA0Q6AEwAA#v=onepage&q=&f=false|http://books.google.de/books?id=GX3w_18-JegC&pg=PA175&lpg=PA175&dq=regular+expression+%22ordered+alternation%22&source=bl&ots=PHqgNmlnM-&sig=OcDjANZKl0VpJY0igVxkQ3LXplg&hl=de&ei=Dcg7S43NIcSi_AbX-83EDQ&sa=X&oi=book_result&ct=result&resnum=1&ved=0CA0Q6AEwAA#v=onepage&q=&f=false]
If this link does not survive, search google for
regular expression "ordered alternation"
My first hit went right into Friedl's book.
Harald.