Regex: how can Matcher.matches return true, but Matcher.find return false?

Question

Consider the class below:
import java.util.regex.Matcher;
import java.util.regex.Pattern;
public class RegexBugDemo {
     private static final Pattern numberPattern;
     static {
               // The BigDecimal grammar below was adapted from the BigDecimal(String) constructor.
               // See also p. 46 of http://www.javaregex.com/RegexRecipesV1.pdf for a regex that matches Java floating point literals; uses similar techniques as below.
          String Sign = "[+-]";
          String Sign_opt = "(?:" + Sign + ")" + "?";     // Note: the "(?:" causes this to be a non-capturing group
          String Digits = "\\p{Digit}+";
          String IntegerPart = Digits;
          String FractionPart = Digits;
          String FractionPart_opt = "(?:" + FractionPart + ")" + "?";
          String Significand = "(?:" + IntegerPart + "\\." + FractionPart_opt + ")|(?:" + "\\." + FractionPart + ")|(?:" + IntegerPart + ")";
          String ExponentIndicator = "[eE]";
          String SignedInteger = Sign_opt + Digits;
          String Exponent = ExponentIndicator + SignedInteger;
          String Exponent_opt = "(?:" +Exponent + ")" + "?";
          numberPattern = Pattern.compile(Sign_opt + Significand + Exponent_opt);
//     private static final Pattern numberPattern = Pattern.compile("\\p{Digit}+");
     public static void main(String[] args) throws Exception {
          String s = "0";
//          String s = "01";
          Matcher m1 = numberPattern.matcher(s);
          System.out.println("m1.matches() = " + m1.matches());
          Matcher m2 = numberPattern.matcher(s);
          if (m2.find()) {
               int i0 = m2.start();
               int i1 = m2.end();
               System.out.println("m2 found this substring: \"" + s.substring(i0, i1) + "\"");
          else {
               System.out.println("m2 NOT find");
          System.exit(0);
}Look at the main method: it constructs Matchers from numberPattern for the String "0" (a single zero). It then reports whether or not Matcher.matches works as well as Matcher.find works. When I ran this code on my box just now, I get:
m1.matches() = true
m2 NOT findHow the heck can matches work and find NOT work? matches has to match the entire input sequence, whereas find can back off if need be! I am really pulling my hair out over this one--is it a bug with the JDK regex engine? Did not seem to turn up anything in the bug database...
There are at least 2 things that you can do to get Matcher.find to work.
First, you can change s to more than 1 digit, for example, using the (originaly commented out) line
          String s = "01";yields
m1.matches() = true
m2 found this substring: "01"Second, I found that this simpler regex for numberPattern
     private static final Pattern numberPattern = Pattern.compile("\\p{Digit}+");yields
m1.matches() = true
m2 found this substring: "0"So, the problem seems to be triggered by a short source String and a complicated regex. But I need the complicated regex for my actual application, and cannot see why it is a problem.
Here is a version of main which has a lot more diagnostic printouts:
     public static void main(String[] args) throws Exception {
          String s = "0";
          Matcher m1 = numberPattern.matcher(s);
          System.out.println("m1.regionStart() = " + m1.regionStart());
          System.out.println("m1.regionEnd() = " + m1.regionEnd());
          System.out.println("m1.matches() = " + m1.matches());
          System.out.println("m1.hitEnd() = " + m1.hitEnd());
          m1.reset();
          System.out.println("m1.regionStart() = " + m1.regionStart());
          System.out.println("m1.regionEnd() = " + m1.regionEnd());
          System.out.println("m1.lookingAt() = " + m1.lookingAt());
          System.out.println("m1.hitEnd() = " + m1.hitEnd());
          Matcher m2 = numberPattern.matcher(s);
          System.out.println("m2.regionStart() = " + m2.regionStart());
          System.out.println("m2.regionEnd() = " + m2.regionEnd());
          if (m2.find()) {
               int i0 = m2.start();
               int i1 = m2.end();
               System.out.println("m2 found this substring: \"" + s.substring(i0, i1) + "\"");
          else {
               System.out.println("m2 NOT find");
               System.out.println("m2.hitEnd() = " + m2.hitEnd());
          System.out.println("m2.regionStart() = " + m2.regionStart());
          System.out.println("m2.regionEnd() = " + m2.regionEnd());
          System.out.println("m1 == m2: " + (m1 == m2));
          System.out.println("m1.equals(m2): " + m1.equals(m2));
          System.exit(0);
     }Unfortunately, the output gave me no insights into what is wrong.
I looked at the source code of Matcher. find ends up calling
boolean search(int from)and it executes with NOANCHOR. In contrast, matches ends up calling
boolean match(int from, int anchor)and executes almost the exact same code but with ENDANCHOR. Unfortunately, this too makes sense to me, and gives me no insight into solving my problem.

Answer 1

bbatman wrote:
I -think- that my originally posted regex is correct, albeit possibly a bit verbose, No, there's a (small) mistake. The optional sign is always part of the first OR-ed part (A) and the exponent is always part of the last part (C). Let me explain.
This is your regex:
(?:[+-])?(?:\p{Digit}+\.(?:\p{Digit}+)?)|(?:\.\p{Digit}+)|(?:\p{Digit}+)(?:[eE](?:[+-])?\p{Digit}+)?which can be read as:
(?:[+-])?(?:\p{Digit}+\.(?:\p{Digit}+)?)        # A
|                                               # or
(?:\.\p{Digit}+)                                # B
|                                               # or
(?:\p{Digit}+)(?:[eE](?:[+-])?\p{Digit}+)?      # COnly one of A, B or C is matched of course. So B can never have a exponent or sign (and A cannot have an exponent and C cannot have a sign).
What you probably meant is this:
(?:[+-])?                                   # sign       
    (?:\p{Digit}+\.(?:\p{Digit}+)?)         #   A
    |                                       #   or
    (?:\.\p{Digit}+)                        #   B
    |                                       #   or
    (?:\p{Digit}+)                          #   C
(?:[eE](?:[+-])?\p{Digit}+)?                # exponent
and that this must be a sun regex engine bug, but would love to be educated otherwise. Yes, it looks like a bug to me too.
A simplified version of this behavior (in case you want to file a bug report) would look like this:
When `test` is a single digit, m.find() returns _false_ but matches() returns true.
When `test` is two or more digits, both return true, as expected.
public class Test {
    public static void main(String[] args) {
        String test = "0";
        String regex = "(?:[+-])?(?:\\p{Digit}+\\.(?:\\p{Digit}+)?)|(?:\\.\\p{Digit}+)|(?:\\p{Digit}+)(?:[eE](?:[+-])?\\p{Digit}+)?";
        java.util.regex.Matcher m = java.util.regex.Pattern.compile(regex).matcher(test);
        System.out.println("matches() -> "+test.matches(regex));
        if(m.find()) {
            System.out.println("find()    -> true");
        } else {
            System.out.println("find()    -> false");
}