How to calculate the 99th percentile of a number stream ...

Question

Environment:
Oracle 11.2.0.3 EE on Solaris 10.5
I have a stream of numbers (say 1000), I need to calculate the 99th percentile of the distribution of that stream such that when a new number 'n' is introduced I can tell if the new number is above the 99th percentile of the distribution of my stream.
I don't have a good feel for the nature of the distribution if that's important.
I am NOT, I repeat NOT a statistician! :-)
I have read the docs on the different functions available, ntile, percent_rank, percentile_cont, percentile_disc, etc. I have also read many articles referenced via Google.
The examples do not do exactly what I'm attempting and I was unable to get the result I need by trial and error (mostly!).
Any suggestions are most welcome!!!
If you need additional information I'll try to supply what I know.
-gary

Answer 1

Here are the CREATE TABLE and INSERT statements for some sample data:
create table sales(custno number, tran_dt date, amount number)Here are the INSERTs to load:
INSERT INTO SALES(custno, tran_dt, amount) VALUES (5, TO_DATE('23-MAR-12','DD-MON-RRRR'),3065);
INSERT INTO SALES(custno, tran_dt, amount) VALUES (5, TO_DATE('29-FEB-12','DD-MON-RRRR'),8435);
INSERT INTO SALES(custno, tran_dt, amount) VALUES (5, TO_DATE('29-FEB-12','DD-MON-RRRR'),9712);
INSERT INTO SALES(custno, tran_dt, amount) VALUES (5, TO_DATE('20-MAR-12','DD-MON-RRRR'),1869);
INSERT INTO SALES(custno, tran_dt, amount) VALUES (5, TO_DATE('03-AUG-12','DD-MON-RRRR'),647);
INSERT INTO SALES(custno, tran_dt, amount) VALUES (5, TO_DATE('03-MAR-12','DD-MON-RRRR'),7434);
INSERT INTO SALES(custno, tran_dt, amount) VALUES (5, TO_DATE('23-AUG-12','DD-MON-RRRR'),225);
INSERT INTO SALES(custno, tran_dt, amount) VALUES (5, TO_DATE('01-SEP-12','DD-MON-RRRR'),28);
INSERT INTO SALES(custno, tran_dt, amount) VALUES (5, TO_DATE('10-APR-12','DD-MON-RRRR'),9393);
INSERT INTO SALES(custno, tran_dt, amount) VALUES (26, TO_DATE('19-MAY-12','DD-MON-RRRR'),1010);
INSERT INTO SALES(custno, tran_dt, amount) VALUES (26, TO_DATE('29-FEB-12','DD-MON-RRRR'),2625);
INSERT INTO SALES(custno, tran_dt, amount) VALUES (26, TO_DATE('18-MAR-12','DD-MON-RRRR'),2511);
INSERT INTO SALES(custno, tran_dt, amount) VALUES (26, TO_DATE('15-AUG-12','DD-MON-RRRR'),1156);
INSERT INTO SALES(custno, tran_dt, amount) VALUES (26, TO_DATE('17-AUG-12','DD-MON-RRRR'),5106);
INSERT INTO SALES(custno, tran_dt, amount) VALUES (26, TO_DATE('19-AUG-12','DD-MON-RRRR'),714);
INSERT INTO SALES(custno, tran_dt, amount) VALUES (26, TO_DATE('07-APR-12','DD-MON-RRRR'),6105);
INSERT INTO SALES(custno, tran_dt, amount) VALUES (26, TO_DATE('25-MAY-12','DD-MON-RRRR'),1592);
INSERT INTO SALES(custno, tran_dt, amount) VALUES (26, TO_DATE('14-MAY-12','DD-MON-RRRR'),1880);
INSERT INTO SALES(custno, tran_dt, amount) VALUES (26, TO_DATE('29-JUL-12','DD-MON-RRRR'),8497);
INSERT INTO SALES(custno, tran_dt, amount) VALUES (26, TO_DATE('13-MAY-12','DD-MON-RRRR'),1658);
INSERT INTO SALES(custno, tran_dt, amount) VALUES (26, TO_DATE('06-APR-12','DD-MON-RRRR'),3071);
INSERT INTO SALES(custno, tran_dt, amount) VALUES (26, TO_DATE('04-SEP-12','DD-MON-RRRR'),8277);
INSERT INTO SALES(custno, tran_dt, amount) VALUES (26, TO_DATE('20-MAR-12','DD-MON-RRRR'),3929);
INSERT INTO SALES(custno, tran_dt, amount) VALUES (26, TO_DATE('04-MAR-12','DD-MON-RRRR'),9239);
INSERT INTO SALES(custno, tran_dt, amount) VALUES (26, TO_DATE('24-APR-12','DD-MON-RRRR'),4390);
INSERT INTO SALES(custno, tran_dt, amount) VALUES (26, TO_DATE('12-MAY-12','DD-MON-RRRR'),8362);
INSERT INTO SALES(custno, tran_dt, amount) VALUES (26, TO_DATE('06-APR-12','DD-MON-RRRR'),4157);
INSERT INTO SALES(custno, tran_dt, amount) VALUES (26, TO_DATE('01-SEP-12','DD-MON-RRRR'),9260);
INSERT INTO SALES(custno, tran_dt, amount) VALUES (26, TO_DATE('31-MAR-12','DD-MON-RRRR'),8017);
INSERT INTO SALES(custno, tran_dt, amount) VALUES (26, TO_DATE('20-MAY-12','DD-MON-RRRR'),2420);
INSERT INTO SALES(custno, tran_dt, amount) VALUES (26, TO_DATE('21-JUL-12','DD-MON-RRRR'),5272);
INSERT INTO SALES(custno, tran_dt, amount) VALUES (26, TO_DATE('23-AUG-12','DD-MON-RRRR'),313);
INSERT INTO SALES(custno, tran_dt, amount) VALUES (26, TO_DATE('24-MAY-12','DD-MON-RRRR'),4747);
INSERT INTO SALES(custno, tran_dt, amount) VALUES (26, TO_DATE('25-APR-12','DD-MON-RRRR'),272);
INSERT INTO SALES(custno, tran_dt, amount) VALUES (26, TO_DATE('02-MAY-12','DD-MON-RRRR'),4329);
INSERT INTO SALES(custno, tran_dt, amount) VALUES (26, TO_DATE('28-JUL-12','DD-MON-RRRR'),3149);
INSERT INTO SALES(custno, tran_dt, amount) VALUES (26, TO_DATE('18-MAR-12','DD-MON-RRRR'),1740);
INSERT INTO SALES(custno, tran_dt, amount) VALUES (26, TO_DATE('07-MAR-12','DD-MON-RRRR'),6868);
INSERT INTO SALES(custno, tran_dt, amount) VALUES (26, TO_DATE('02-JUN-12','DD-MON-RRRR'),5661);
INSERT INTO SALES(custno, tran_dt, amount) VALUES (26, TO_DATE('08-MAY-12','DD-MON-RRRR'),6136);
INSERT INTO SALES(custno, tran_dt, amount) VALUES (26, TO_DATE('23-AUG-12','DD-MON-RRRR'),512);
INSERT INTO SALES(custno, tran_dt, amount) VALUES (26, TO_DATE('16-AUG-12','DD-MON-RRRR'),8784);
INSERT INTO SALES(custno, tran_dt, amount) VALUES (26, TO_DATE('31-AUG-12','DD-MON-RRRR'),7300);
INSERT INTO SALES(custno, tran_dt, amount) VALUES (26, TO_DATE('12-MAY-12','DD-MON-RRRR'),9303);
INSERT INTO SALES(custno, tran_dt, amount) VALUES (26, TO_DATE('29-MAR-12','DD-MON-RRRR'),2626);
INSERT INTO SALES(custno, tran_dt, amount) VALUES (26, TO_DATE('14-JUL-12','DD-MON-RRRR'),6365);
INSERT INTO SALES(custno, tran_dt, amount) VALUES (26, TO_DATE('19-JUN-12','DD-MON-RRRR'),7880);
INSERT INTO SALES(custno, tran_dt, amount) VALUES (26, TO_DATE('20-JUN-12','DD-MON-RRRR'),6096);
INSERT INTO SALES(custno, tran_dt, amount) VALUES (26, TO_DATE('05-AUG-12','DD-MON-RRRR'),1980);
INSERT INTO SALES(custno, tran_dt, amount) VALUES (26, TO_DATE('27-MAY-12','DD-MON-RRRR'),3004);I can now calculate the average sales over the past 6 months (180 day) and 1 month (30 day) periods. I don't need the 6 month average but I do need the individual sales to get the distribution and hence the standard deviations and hopefully the 99th percentile number. I know it's around 3 standard deviations but the spec I'm working with requires using the 99th percentile. Not my choice! :-)
select custno,
         avg(case when tran_dt between sysdate -210 and sysdate -31 then amount else 0 end) sales6mo,
         avg(case when tran_dt >= sysdate -30 then amount else 0 end) sales1mo,
         stddev(amount) sd,
         stddev(amount)*3 sd3
from sales
group by custno;I want to see if there are any customers' (I know I only generated 2) 1 month sales that are more than the 99th percentile of the 6 month distribution.
If it's easier to use a lower percentile I can use the examples to plug in the 99.
Here is the result of the above query:
    CUSTNO   SALES6MO   SALES1MO         SD        SD3
         5 4506.11111 28.1111111 4142.61146 12427.8344
        26 3563.81579 1090.05263  2952.1488 8856.44641So, from this data I would expect my 99th percentile to be close to the SD3 column. Correct?
Thanks very much for the assistance!!
-gary
Edited by: garywicke on Sep 5, 2012 9:54 AM