Estimating number of concurrent requests

Question

Hello,
I want to estimate the number of concurrent requests that my server application may need to handle.
Here are 3 approaches I tried to compute the probability of concurrent requests. Each of them gives different results, and obviously some of the results are palin wrong.
It's been a while since I've learnt probabilities, I used to be comfortable with them back then, but this time's over it seems.
Can you help me point out the flaw in the reasoning of each approach (that's for my culture), and, better yet, suggest how to correctly estimate the reasonable number of concurrent requests the server will have to handle (that's for my current problem)?
Preliminary assumptions:
Here are the figures, to fix ideas, and illustrate examples given below of the approach I took.
The application will have 1000 users, each of them issuing 10 requests per day.
Each request take approx 1 minute to be processed.
Users are supposed evenly distributed over worlwide timezones, and the requests are supposed to be evenly distributed over the time of a day (which is not true for a single given user, but is true if enough users are evenly distributed over timezones).
Users are supposed to work independantly of each other, so the probabilities of their actions are independant.
*1) Approach 1: brute linear approach:*
Each user uses 10 requests * 60s/rq = 600s of server time per day.
1000 users therefore use 600,000 seconds of server time; as there are 86400s per day, in average the server is serving *600K/86,K = approx 7* request at any instant of the day.
*2) Approach 2: estimate probability of N concurrent requests*
Each user uses 600s of server time per day.
So the probability to be serving one given user at any given instant of one day seems to be *600/86,400 = approx 0.007*
The probability to be serving N given users at any given instant is therefore 0.007 pow N.
As we don't care about which specific N users are served, the probability to be serving whatever N users is therefore: *(0.007 pow N)xC(1000,N)* where C(1000,N) is the binomial coefficient for subsets of N elements among 1000.
For N=7 it gives a probability of approx 151%
For N=10 it gives a mighty 68%
You have to wait above N=20 to fall under a 1% probability
*3) Approach 3: estimate probability of "not N" concurrent requests*
Each user uses 600s of server time per day.
So the probability to not serve one given user at any given instant of one day seems to be *85,800/86,400 = approx 0.993*
The probability to not serve N given users at any given instant is therefore *0.993 pow N* .
This probability is interesting for big values of N (the probability to serve m users = probability to not serve N=1000-m users), which for N=993 gives 0.09%
But if I multiply by the binomial coefficient, the value is much above 1 for almost all values of N (interestingly, N=999 gives 95%, and N=1000 (0 user served) is 0.09%, but these results are probably artifacts of the floating point computations).
What mixes me up are the following questions:
Q1) Is approach 1 reasonable enough?
It seems to be modelling a degenerate form of approach 2, where the requests would not be randomly spread over the day, but optimally organized in queues.
I cannot force this usage pattern.
Q2) How would you approach this estimation? Do you have a better approach?
Q3) Obviously the formula in approach 2 is wrong at some point as for certain values of N it is above 1.
It is expected that the probability for these values of N is 1, as the linear formula prooves that it is not possible to fit all 40000 requests into one day under a concurrency level of 7. But >1 is mathematically incorrect, so what is the flaw in the reasoning, that leads to this formula?
Q4) What worries me in approach 2 is that I actually use approach 1 to compute the basic probability of 0.007, then use this result in the supposedly more general formula. I'm not sure whether the computing of the 0.007 qualifies as a deduction or as an assumption?
Q5) What's wrong with approach 3 (probability to not serve N clients)? Why are the results inconsistent with approach 2?
Q6) What worries me too is that I based all three approaches on instants of duration zero ("the probability that at any instant..."). Should I instead try to compute "the probability that over a period of 1 second, one user issues a request"? How does this modify the reasoning and the formulaes?
Thank you in advance for your help.
J.

Answer 1

When it becomes difficult to get closed form solutions then resort to simulation. The following is a basic event simulation of 1 days of your problem as I understand it but using a uniform random active period. It will be trivial to change the active distribution to match your exact requirement. You will need to add JFreeChart to the classpath to run it.
One advantage of this form of simulation is that one can easily add logic such as that to throttle the response and then examine how this affects users.
import java.awt.Dimension;
import java.awt.geom.Point2D;
import java.util.ArrayList;
import java.util.List;
import java.util.PriorityQueue;
import java.util.Queue;
import java.util.Random;
import javax.swing.JFrame;
import org.jfree.chart.ChartFactory;
import org.jfree.chart.ChartPanel;
import org.jfree.chart.JFreeChart;
import org.jfree.chart.plot.PlotOrientation;
import org.jfree.data.DomainOrder;
import org.jfree.data.xy.AbstractXYDataset;
public class Sabre20090403
    private static final double MEAN_TIME_BETWEEN_ACTIVE = 24D * 60 * 60 / 10; // seconds
    private static final double MEAN_PROCESSING_TIME = 60; // seconds
    static class Context
        public String toString()
            return "Number active = " + numberActive;
        public void appendPoint(double x, double y)
            Point2D.Double p = new Point2D.Double(x, y);
            results.add(p);
        public Point2D.Double[] getPoints()
            return results.toArray(new Point2D.Double[0]);
        private final List<Point2D.Double> results = new ArrayList<Point2D.Double>();
        private int numberActive = 0;
    abstract static class Task implements Comparable<Task>
        abstract void run();
        public double getEventTime()
            return time_;
        public int compareTo(Task other)
            if (time_ < other.time_)
                return -1;
            else if (time_ > other.time_)
                return +1;
            else
                return 0;
        protected double time_ = 0; // in seconds
    static class TerminalTask extends Task
        public TerminalTask(double simulationDuration)
            time_ = simulationDuration;
        public void run()
            System.out.println("Simulation finished");
    static class PrintTask extends Task
        public PrintTask(Context context, double printInterval)
            context_ = context;
            printInterval_ = printInterval;
        public void run()
            context_.appendPoint(time_, context_.numberActive);
            System.out.println("Time " + time_ + "\t" + context_);
            time_ += printInterval_;
        private final double printInterval_;
        private final Context context_;
    static class User extends Task
        public User(Context context)
            context_ = context;
            run();
        public int getID()
            return ID;
        public void run()
            active = !active;
            if (active)
                context_.numberActive++;
                // Uniformly distributed from 0.5 mpt to 1.5 mpt
                double p = random.nextDouble();
                time_ += MEAN_PROCESSING_TIME * (0.5 + p);
            } else
                double p = random.nextDouble();
                // Exponential distribution with mean of MEAN_TIME_BETWEEN_ACTIVE
                double delta = -MEAN_TIME_BETWEEN_ACTIVE * Math.log(p);
                time_ += delta;
                if (context_.numberActive > 0)
                    context_.numberActive--;
        private boolean active = true;
        private final int ID = nextID++;
        private static int nextID = 0;
        private final Context context_;
        private static Random random = new Random();
    static class Problem implements Runnable
        private Queue<Task> eventQueue = new PriorityQueue<Task>(20000);
        public void run()
            final Context context = new Context();
            for (int i = 0; i < 1000; i++)
                eventQueue.offer(new User(context));
            eventQueue.offer(new PrintTask(context, 60 * 10L)); // 10 minutes print interval
            eventQueue.offer(new TerminalTask(24L * 60 * 60)); // 1 day
            while (true)
                final Task task = eventQueue.poll();
                task.run();
                if (task instanceof TerminalTask)
                    break;
                else
                    eventQueue.offer(task);
            final AbstractXYDataset xyDataset = new AbstractXYDataset()
                private final Point2D.Double[] points = context.getPoints();
                public double getXValue(int series, int item)
                    return points[item].x / 60.0;
                public double getYValue(int series, int item)
                    return points[item].y;
                public DomainOrder getDomainOrder()
                    return DomainOrder.ASCENDING;
                public Comparable getSeriesKey(int index)
                    return "P" + index;
                public int getSeriesCount()
                    return 1;
                public Number getX(int series, int item)
                    return new Double(getXValue(series, item));
                public Number getY(int series, int item)
                    return new Double(getYValue(series, item));
                public int getItemCount(int series)
                    return points.length;
            final JFrame frame = new JFrame("");
            final JFreeChart jfreechart = ChartFactory.createXYLineChart("Simulation", "Time (minutes)", "Number active", xyDataset, PlotOrientation.VERTICAL, true, true, false);
            final ChartPanel panel = new ChartPanel(jfreechart);
            panel.setPreferredSize(new Dimension(1024, 768));
            frame.setContentPane(panel);
            frame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
            frame.pack();
            frame.setLocationRelativeTo(null);
            frame.setVisible(true);
        public static void main(String[] args)
            new Problem().run();
}Edited by: sabre150 on Apr 3, 2009 7:34 PM