Query to Report on Parallel Jobs Running

Question

Morning!
I would like to get a query that reports on my parallel jobs.
For each minute that a procedure is running I would like to know what stages are running.
I log the whole procedure in a table called run_details and the start and end of each stage in a table called incident.
I'm running Oracle 9i
Here is some sample data based on 2 threads Expected output at the bottom
SQL>CREATE TABLE run_details
  2  (run_details_key  NUMBER(10)
  3  ,start_time       DATE
  4  ,end_time         DATE
  5  ,description      VARCHAR2(50)
  6  );
SQL>CREATE TABLE incident
  2  (run_details_key NUMBER(10)
  3  ,stage           VARCHAR2(20)
  4  ,severity        VARCHAR2(20)
  5  ,time_stamp      DATE
  6  );
SQL>INSERT INTO run_details
  2  VALUES (1
  3         ,TO_DATE('08/10/2007 08:00','DD/MM/YYYY HH24:MI')
  4         ,TO_DATE('08/10/2007 08:10','DD/MM/YYYY HH24:MI')
  5         ,'Test'
  6         );
SQL>INSERT INTO incident
  2  VALUES (1
  3         ,'Stage1'
  4         ,'START'
  5         ,TO_DATE('08/10/2007 08:00','DD/MM/YYYY HH24:MI')
  6         );
SQL>INSERT INTO incident
  2  VALUES (1
  3         ,'Stage1'
  4         ,'END'
  5         ,TO_DATE('08/10/2007 08:08:53','DD/MM/YYYY HH24:MI:SS')
  6         );
SQL>INSERT INTO incident
  2  VALUES (1
  3         ,'Stage2'
  4         ,'START'
  5         ,TO_DATE('08/10/2007 08:00','DD/MM/YYYY HH24:MI')
  6         );
SQL>INSERT INTO incident
  2  VALUES (1
  3         ,'Stage2'
  4         ,'END'
  5         ,TO_DATE('08/10/2007 08:04:23','DD/MM/YYYY HH24:MI:SS')
  6         );
SQL>INSERT INTO incident
  2  VALUES (1
  3         ,'Stage3'
  4         ,'START'
  5         ,TO_DATE('08/10/2007 08:04:24','DD/MM/YYYY HH24:MI:SS')
  6         );
SQL>INSERT INTO incident
  2  VALUES (1
  3         ,'Stage3'
  4         ,'END'
  5         ,TO_DATE('08/10/2007 08:10','DD/MM/YYYY HH24:MI')
  6         );
SQL>select * from incident;
RUN_DETAILS_KEY STAGE      SEVERITY   TIME_STAMP
              1 Stage1     START      08/10/2007 08:00:00
              1 Stage1     END        08/10/2007 08:08:53
              1 Stage2     START      08/10/2007 08:00:00
              1 Stage2     END        08/10/2007 08:04:23
              1 Stage3     START      08/10/2007 08:04:24
              1 Stage3     END        08/10/2007 08:10:00  So stages 1 and 2 run in parallel from 08:00, then at 08:04:23 stage 2 stops and a second later stage 3 starts.
set some variables
SQL>define start_time = null
SQL>col start_time new_value start_time
SQL>define end_time = null
SQL>col end_time new_value end_time
SQL>
SQL>SELECT start_time-(1/(24*60)) start_time
  2        ,end_time
  3  FROM   run_details
  4  WHERE  run_details_key =  1;
START_TIME          END_TIME
08/10/2007 07:59:00 08/10/2007 08:10:00Get every minute that the process is running for:
SQL>WITH t AS (SELECT TRUNC(TO_DATE('&start_time','dd/mm/yyyy hh24:mi:ss'),'MI') + rownum/24/60 tm
  2             FROM   dual
  3             CONNECT BY ROWNUM <= (TO_DATE('&end_time','dd/mm/yyyy hh24:mi:ss')
  4                                   -TO_DATE('&start_time','dd/mm/yyyy hh24:mi:ss')
  5                                  )*24*60
  6            )
  7  SELECT tm
  8  FROM t;
old   1: WITH t AS (SELECT TRUNC(TO_DATE('&start_time','dd/mm/yyyy hh24:mi:ss'),'MI') + rownum/24/60 tm
new   1: WITH t AS (SELECT TRUNC(TO_DATE('08/10/2007 07:59:00','dd/mm/yyyy hh24:mi:ss'),'MI') + rownum/24/60 tm
old   3:            CONNECT BY ROWNUM <= (TO_DATE('&end_time','dd/mm/yyyy hh24:mi:ss')
new   3:            CONNECT BY ROWNUM <= (TO_DATE('08/10/2007 08:10:00','dd/mm/yyyy hh24:mi:ss')
old   4:                                -TO_DATE('&start_time','dd/mm/yyyy hh24:mi:ss')
new   4:                                -TO_DATE('08/10/2007 07:59:00','dd/mm/yyyy hh24:mi:ss')
TM
08/10/2007 08:00:00
08/10/2007 08:01:00
08/10/2007 08:02:00
08/10/2007 08:03:00
08/10/2007 08:04:00
08/10/2007 08:05:00
08/10/2007 08:06:00
08/10/2007 08:07:00
08/10/2007 08:08:00
08/10/2007 08:09:00
08/10/2007 08:10:00
11 rows selected.Get stage, start & end times and duration
SQL>SELECT ai1.stage
  2        ,ai1.time_stamp start_time
  3        ,ai2.time_stamp end_time
  4        ,SUBSTR(numtodsinterval(ai2.time_stamp-ai1.time_stamp, 'DAY'), 12, 8) duration
  5  FROM   dw2.incident ai1
  6  JOIN   dw2.incident ai2
  7         ON ai1.run_details_key = ai2.run_details_key
  8         AND ai1.stage = ai2.stage
  9  WHERE ai1.severity = 'START'
10  AND ai2.severity = 'END'
11  AND ai1.run_details_key  = 1
12  ORDER BY ai1.time_stamp
13  /
STAGE      START_TIME          END_TIME            DURATION
Stage1     08/10/2007 08:00:00 08/10/2007 08:08:53 00:08:52
Stage2     08/10/2007 08:00:00 08/10/2007 08:04:23 00:04:22
Stage3     08/10/2007 08:04:24 08/10/2007 08:10:00 00:05:36Then combine both (or do something else) to get this:
TM                  THREAD_1 THREAD_2
08/10/2007 08:00:00 Stage1   Stage2
08/10/2007 08:01:00 Stage1   Stage2
08/10/2007 08:02:00 Stage1   Stage2
08/10/2007 08:03:00 Stage1   Stage2
08/10/2007 08:04:00 Stage1   Stage2
08/10/2007 08:05:00 Stage1   Stage3
08/10/2007 08:06:00 Stage1   Stage3
08/10/2007 08:07:00 Stage1   Stage3
08/10/2007 08:08:00 Stage1   Stage3
08/10/2007 08:09:00          Stage3
08/10/2007 08:10:00          Stage3Ideally I'd like this to work for n-threads, as I want this to run on different environments that have different numbers of CPUs.
Thank you for your time.

Answer 1

> Ideally I'd like this to work for n-threads, as I want this to run on
different environments that have different numbers of CPUs.
The number of CPUs are not always a good indication of the processing load that a platform can take - especially when the processing load involves a lot of I/O.
You can have 99% CPU idle time with a 1000 active processes... as that idle time is in fact CPU time spend waiting on I/O completion. Courtesy of a severely strained I/O channel that is the bottleneck.
Another factor is memory (resources). You for example have 4 CPUs with 8GB physical memory.. where a single process (typically a Java VM for a complex process) grabs a huge amount of memory. Assuming that 4 threads/CPU or 1 threads/CPU can be a severe overestimate due to the amount of memory needed. Getting this wrong leads in turn to excessive virtual memory paging and reduces the platform's performance drastically.
CPU alone is a very poor choice when deciding on the platform's capacity to run parallel processes.