Risk, if any, of closing ServletOutputStream using a user thread?

Question

Hello All,
What is the risk of writing/flushing/closing ServletOutputStream using a user thread other than the HTTP thread associated with the request?
I want to do that to implement client request timeouts. Currently in our application, for every request received we are creating a new thread to make the HTTP thread timeout after a configurable amount of time is elapsed. Though creating a new thread is doing the job just fine but that approach is creating another problem that the servers are not scaling to the load we receive. We frequently get "unable to create threads due to unavailable memory" errors. At any point in time, any of our servers receive ~200 requests, which means that server has to create/handle 400+ threads. If the servers are slow due to external systems, then a lot more threads will be accumulated in the server. When the thread count reaches 500+ it fails to accept any further requests. So as a possible alternative to current approach, I am thinking of running few threads (may be a couple of threads) in the background and timeout the requests by closing the associated ServletOutputStream with TimeoutException.
Note: I know that as another possible solution, I can go for few more servers in the cluster but before that I want to check if the above approach is a possible solution because it is less complex and saves money on the hardware.
Thanks,
Srinivas

Answer 1

Hello Kaj,
Sorry for the cross-post. Actually, I couldn't decide if I should post it as a concurrency, Java Servlet or as a design question.
Maxideon,
Sure, it can be done with single thread also. I said a couple of threads because a single thread may not get sufficient CPU time to handle all timeouts. But otherwise yes, we can try it with single thread also.
ejp,
We do not own the client application. We are a web services application and there are many clients who consume our services.
The standard solution to handle timeouts in the server is by creating an additional thread, which is what we have followed. Now, that design is posing a new problem to us.
No. A custom TimeoutException.
I want to use 2 or 3 user threads to timeout the request if needed. That way I do not have to create 1 thread for each request. In the current approach server creates/handles 400 threads for 200 requests. With the alternative solution I can make it to create/handle 202 or 203 threads for 200 requests.
dubwai,
Here I am answering to your reply to my same question in the other forum.
The current design is working just fine for timeouts. But sometimes it is failing to scale to the load we receive. In the current design we have created 1 thread for each request to timeout HTTP thread, which means that for 200 requests we will have 400 threads (200 HTTP threads and 200 user threads) in the server. For some reason, if the load increases to 250 requests or if external systems are slow then more threads accumulate in the server and it fails to accept incoming requests when thread count reaches ~500. That is the problem we are having.
In the alternative design I proposed that we will create just a couple (may be 2 or 3) of threads running in the background, instead of 1 thread for each request. These background threads monitor HTTP threads and timeout the request if needed by writing and closing ServlerOutputStream. This way server will have to create/handle only ~200 HTTP threads plus a couple of user threads. For this approach to work I have to close ServletOutputStream using background threads instead of HTTP thread and want to know what is the rick of doing that?
To All,
Here is a rough implementation of user thread. The HTTP thread registers itself in threadMap variable with its starttime and in threadResponse variable with its associated HttpServletResponse. This user thread uses these values to find request timeout (10 seconds) and close response if needed.
public class TimeoutMonitor implements Runnable {
    public static Map<Thread, Long> threadMap = new HashMap<Thread, Long>();
    public static Map<Thread, HttpServletResponse> threadResponse =
        new HashMap<Thread, HttpServletResponse>();
     @Override
     public void run() {
         while(true) {
          try {
              wait(1000);
              Set<Thread> keys = threadMap.keySet();
              Iterator<Thread> threads = keys.iterator();
              while(threads.hasNext()) {
                  Thread thread = threads.next();
                  Long startTime = threadMap.get(thread);
               if((System.currentTimeMillis() - startTime) > 10000) {
                   HttpServletResponse response =
                                threadResponse.get(thread);
                   try {
                    PrintWriter writer = response.getWriter();
                    writer.write("Timeout Exception");
                    writer.flush();
                    writer.close();
                    threadMap.remove(thread);
                    threadResponse.remove(thread);
                   } catch (IOException e) {
                    e.printStackTrace();
          } catch (InterruptedException e1) {
              // TODO Auto-generated catch block
              e1.printStackTrace();
    }I have test run this program in tomcat and seem to be working fine. Please note that here response/writer are closed by user thread and not HTTP thread. My questions is, do you see any problem in doing that?
Thanks,
Srinivas