Thread pool rejecting threads when I don't think it should, ideas?

Question

Hi,
I have a server application in which I only want a specific number of simultaneous requests. If the server gets more then this number it is suppose to close the connection (sends an HTTP 503 error to the client). To do this I used a fix thread pool. When I start the server and submit the max number of requests I get the expected behavior. However if I resubmit the request (within a small period of time, e.g. 1-15 seconds after the first one) I get very odd behavior in that some of the requests are rejected. For example if I set the max to 100 the first set of requests will work fine (100 requests, 100 responses). I then submit again and a small number will be rejected (I've seen it range from 1 to 15 rejected)....
I made a small app which kind of duplicates this behavior (see below). Basically when I see is that the first time submitting requests works fine but the second time I get a rejected one. As best as I can tell none should be rejected....
Here is the code, I welcome your thoughts or if you see something I am doing wrong here...
<pre>
import java.util.concurrent.*;
import java.util.concurrent.atomic.AtomicInteger;
public class ThreadPoolTest {
     static AtomicInteger count = new AtomicInteger();
     public static class threaded implements Runnable {
          @Override
          public void run() {
               System.out.println("In thread: " + Thread.currentThread().getId());
               try {
                    Thread.sleep(500);
               } catch (InterruptedException e) {
                    System.out.println("Thread: " + Thread.currentThread().getId()
                              + " interuptted");
               System.out.println("Exiting run: " + Thread.currentThread().getId());
     private static int maxThreads = 3;
     private ThreadPoolExecutor pool;
     public ThreadPoolTest() {
          super();
          pool = new java.util.concurrent.ThreadPoolExecutor(
                    1, maxThreads - 1, 60L, TimeUnit.SECONDS,
                    new ArrayBlockingQueue<Runnable>(1));
     public static void main(String[] args) throws InterruptedException {
          ThreadPoolTest object = new ThreadPoolTest();
          object.doThreads();
          Thread.sleep(3000);
          object.doThreads();
          object.pool.shutdown();
          try {
               object.pool.awaitTermination(60, TimeUnit.SECONDS);
          } catch (InterruptedException e) {
               // TODO Auto-generated catch block
               e.printStackTrace();
     private void doThreads() {
          int submitted = 0, rejected = 0;
          int counter = count.getAndIncrement();
          for (int x = 0; x < maxThreads ; x++) {
               try {
                    System.out.println("Run #: " + counter + " submitting " + x);
                    pool.execute(new threaded());
                    submitted++;
               catch (RejectedExecutionException re) {
                    System.err.println("\tRun #: " + counter + ", submission " + x
                              + " was rejected");
                    System.err.println("\tQueue active: " + pool.getActiveCount());
                    System.err.println("\tQueue size: " + pool.getPoolSize());
                    rejected++;
          System.out.println("\n\n\tRun #: " + counter);
          System.out.println("\tSubmitted: " + (submitted + rejected));
          System.out.println("\tAccepted: " + submitted);
          System.out.println("\tRejected: " + rejected + "\n\n");
</pre>

Answer 1

First thank you for taking the time to reply, I do appreciate it.
jtahlborn - The code provided here is a contrived example trying to emulate the bigger app as best as I could. The actual program doesn't have any sleeps, the sleep in the secondary thread is to simulate the program doing some work & replying to a request. The sleep in the primary thread is to simulate a small delay between 'requests' to the pool. I can make this 1 second and up to (at least) 5 seconds with the same results. Additionally I can take out the sleep in the secondary thread and still see the a rejection.
EJP - Yes I am aware of the TCP/IP queue, however; I don't see that as relevant to my question. The idea is not to prevent the connection but to respond to the client saying we can't process the request (send an "HTTP 503" error). So basically if we have, say, 100 threads running then the 101st, connection will get a 503 error and the connection will be closed.
Also my test platform - Windows 7 64bit running Java 1.6.0_24-b07 (32bit) on an Intel core i7.
It occurred to me that I did not show the output of the test program. As the output shows below, the first set of requests are all processed properly. The second set of requests is not. The pool should have 2 threads and 1 slot in the queue, so by the time the second "request" is made at least 2 of the requests from the first call should be done processing, so I could possibly understand run 1, submit #2 failing but not submit 1.
<pre>
Run #: 0 submitting 0
Run #: 0 submitting 1
Run #: 0 submitting 2
In thread: 8
In thread: 9
Exiting run: 8
Exiting run: 9
     Run #: 0
     Submitted: 3
     Accepted: 3
     Rejected: 0
In thread: 8
Exiting run: 8
Run #: 1 submitting 0
In thread: 9
Run #: 1 submitting 1
     Run #: 1, submission 1 was rejected
     Queue active: 1
     Queue size: 2
Run #: 1 submitting 2
     Run #: 1
     Submitted: 3
     Accepted: 2
     Rejected: 1
In thread: 8
Exiting run: 9
Exiting run: 8
</pre>