Update query performance - Challenging one

Question

Hi All,
Below is the procedure I have created to update the five columns of one table. I have to update almost 9673533 records.
With the below procedure for updating 500 records it took around 1.5 hrs.
What would be the best strategy to enhance the performance of the procedure.. Please throw your ideas to make it in less time.
create or replace procedure test is
  cursor cur_web_prd is
    select rp_id
      from dw_retl_prds t;
  lc_short_brief       varchar2(300);
  lc_long_brief        varchar2(2000);
  ld_date_first_online date;
  ld_date_sent         date;
  lc_rp_id             varchar2(20);
  lc_copos_ssl2        varchar2(40);
  lc_qoslistid         varchar2(40);
  ln_web_prd_cnt       number;
  ln_copos_cnt         number;
  ln_qolist_cnt        number;     
begin
  for i in cur_web_prd loop
      select count(*)
      into ln_web_prd_cnt
      from dw_retl_prds t, web_prds@nrstp s
     where t.rp_id = s.rp_rp_id
       and t.rp_id = i.rp_id;
  if ln_web_prd_cnt = 1 then
    select t.rp_id,
           s.short_brief,
           s.long_brief,
           s.date_first_online,
           s.date_sent
      into lc_rp_id,
           lc_short_brief,
           lc_long_brief,
           ld_date_first_online,
           ld_date_sent
      from dw_retl_prds t, web_prds@nrstp s
     where t.rp_id = s.rp_rp_id
       and t.rp_id = i.rp_id;
      select count(*)
      into ln_copos_cnt
      from dw_retl_prd_cat_val_us@dwenq t, dw_retl_prd_cat_vals@dwenq s
     where t.rp_rp_id = i.rp_id
       and trunc(sysdate) between t.dw_eftv_from and
           nvl(t.dw_eftv_to, trunc(sysdate))
       and s.pc_pc_id = 'COPOS_SSL2'
       and trunc(sysdate) between s.dw_eftv_from and
           nvl(s.dw_eftv_to, trunc(sysdate))
       and t.pc_pc_id = s.pc_pc_id
       and t.pcv_pcv_id = s.pcv_id;
       if ln_copos_cnt = 1 then
    select s.descr
      into lc_copos_ssl2
      from dw_retl_prd_cat_val_us@dwenq t, dw_retl_prd_cat_vals@dwenq s
     where t.rp_rp_id = i.rp_id
       and trunc(sysdate) between t.dw_eftv_from and
           nvl(t.dw_eftv_to, trunc(sysdate))
       and s.pc_pc_id = 'COPOS_SSL2'
       and trunc(sysdate) between s.dw_eftv_from and
           nvl(s.dw_eftv_to, trunc(sysdate))
       and t.pc_pc_id = s.pc_pc_id
       and t.pcv_pcv_id = s.pcv_id;
       else
           lc_copos_ssl2 := null;
       end if;
        select count(*)
      into ln_qolist_cnt
      from dw_retl_prd_cat_val_us@dwenq t, dw_retl_prd_cat_vals@dwenq s
     where t.rp_rp_id = i.rp_id
       and trunc(sysdate) between t.dw_eftv_from and
           nvl(t.dw_eftv_to, trunc(sysdate))
       and s.pc_pc_id = 'QOSLISTID'
       and trunc(sysdate) between s.dw_eftv_from and
           nvl(s.dw_eftv_to, trunc(sysdate))
       and t.pc_pc_id = s.pc_pc_id
       and t.pcv_pcv_id = s.pcv_id;
       if ln_qolist_cnt = 1 then
    select s.descr
      into lc_qoslistid
      from dw_retl_prd_cat_val_us@dwenq t, dw_retl_prd_cat_vals@dwenq s
     where t.rp_rp_id = i.rp_id
       and trunc(sysdate) between t.dw_eftv_from and
           nvl(t.dw_eftv_to, trunc(sysdate))
       and s.pc_pc_id = 'QOSLISTID'
       and trunc(sysdate) between s.dw_eftv_from and
           nvl(s.dw_eftv_to, trunc(sysdate))
       and t.pc_pc_id = s.pc_pc_id
       and t.pcv_pcv_id = s.pcv_id;
       else
           lc_qoslistid := null;
       end if;
    update dw_retl_prds t
       set t.web_short_brief   = lc_short_brief,
           t.web_long_brief    = lc_long_brief,
           t.date_first_online = ld_date_first_online,
           t.date_sent         = ld_date_sent,
           t.copos_ssl#2       = lc_copos_ssl2,
           t.qoslistid         = lc_qoslistid
     where t.rp_id = i.rp_id;
   end if;
   ln_web_prd_cnt            := 0;   
   lc_short_brief            := null;          
   lc_long_brief             := null;
   ld_date_first_online      := null;
   ld_date_sent              := null;
   lc_copos_ssl2             := null;
   lc_qoslistid              := null;
  end loop;
  commit;
  end;I have taken count(*) to check and avoid the too_many_rows and no_data_found exceptions. As there is no Unique key on the table Web_prds@nrstp.
For dw_retl_prds the unique key is rp_id.

Answer 1

Hello
First things first, you need to trace it to find out where it is spending it's time. Find out which statements are taking the longest and target them.
That said however, you are using the slowest possible method for processing this data by using a cursor for loop. You are then doing separate lookups over a database link - and you are doing the queries twice! You could reduce the time of the lookups by 50% just by combining the queries...for example
CREATE OR REPLACE PROCEDURE test
IS
   CURSOR cur_web_prd
   IS
      SELECT rp_id
        FROM dw_retl_prds t;
   lc_short_brief         VARCHAR2 (300);
   lc_long_brief          VARCHAR2 (2000);
   ld_date_first_online   DATE;
   ld_date_sent           DATE;
   lc_rp_id               VARCHAR2 (20);
   lc_copos_ssl2          VARCHAR2 (40);
   lc_qoslistid           VARCHAR2 (40);
   ln_web_prd_cnt         NUMBER;
   ln_copos_cnt           NUMBER;
   ln_qolist_cnt          NUMBER;
BEGIN
   FOR i IN cur_web_prd
   LOOP
      BEGIN
               SELECT t.rp_id,
                s.short_brief,
                s.long_brief,
                s.date_first_online,
                s.date_sent
           INTO lc_rp_id,
                lc_short_brief,
                lc_long_brief,
                ld_date_first_online,
                ld_date_sent
           FROM dw_retl_prds t, web_prds@nrstp s
          WHERE t.rp_id = s.rp_rp_id AND t.rp_id = i.rp_id;
          BEGIN
              SELECT s.descr
                  INTO lc_copos_ssl2
                  FROM dw_retl_prd_cat_val_us@dwenq t,
                       dw_retl_prd_cat_vals@dwenq s
                 WHERE t.rp_rp_id = i.rp_id
                       AND TRUNC (SYSDATE) BETWEEN t.dw_eftv_from
                                               AND NVL (t.dw_eftv_to,
                                                        TRUNC (SYSDATE))
                       AND s.pc_pc_id = 'COPOS_SSL2'
                       AND TRUNC (SYSDATE) BETWEEN s.dw_eftv_from
                                               AND NVL (s.dw_eftv_to,
                                                        TRUNC (SYSDATE))
                       AND t.pc_pc_id = s.pc_pc_id
                       AND t.pcv_pcv_id = s.pcv_id;  
          EXCEPTION
                WHEN NO_DATA_FOUND THEN
                     lc_copos_ssl2 := NULL;
          END;
          BEGIN
              SELECT s.descr
                  INTO lc_qoslistid
                  FROM dw_retl_prd_cat_val_us@dwenq t,
                       dw_retl_prd_cat_vals@dwenq s
                 WHERE t.rp_rp_id = i.rp_id
                       AND TRUNC (SYSDATE) BETWEEN t.dw_eftv_from
                                               AND NVL (t.dw_eftv_to,
                                                        TRUNC (SYSDATE))
                       AND s.pc_pc_id = 'QOSLISTID'
                       AND TRUNC (SYSDATE) BETWEEN s.dw_eftv_from
                                               AND NVL (s.dw_eftv_to,
                                                        TRUNC (SYSDATE))
                       AND t.pc_pc_id = s.pc_pc_id
                       AND t.pcv_pcv_id = s.pcv_id;
            EXCEPTION
                WHEN NO_DATA_FOUND THEN
                    lc_qoslistid := NULL;
            END;
            UPDATE dw_retl_prds t
            SET t.web_short_brief = lc_short_brief,
                t.web_long_brief = lc_long_brief,
                t.date_first_online = ld_date_first_online,
                t.date_sent = ld_date_sent,
                t.copos_ssl#2 = lc_copos_ssl2,
                t.qoslistid = lc_qoslistid
          WHERE t.rp_id = i.rp_id;
      EXCEPTION
            WHEN
                NO_DATA_FOUND THEN
                    NULL;
      END;
      ln_web_prd_cnt := 0;
      lc_short_brief := NULL;
      lc_long_brief := NULL;
      ld_date_first_online := NULL;
      ld_date_sent := NULL;
      lc_copos_ssl2 := NULL;
      lc_qoslistid := NULL;
   END LOOP;
   COMMIT;
END;But this is still not an efficient implementation - you should try to put all of this into 1 or more Update statements without the loop. Something like...
            UPDATE dw_retl_prds t
            SET (   t.web_short_brief,
                    t.web_long_brief,
                    t.date_first_online,
                    t.date_sent
                ) =
                (   SELECT
                        s.short_brief,
                        s.long_brief,
                        s.date_first_online,
                        s.date_sent
                    FROM
                        web_prds@nrstp s
                    WHERE
                        t.rp_id = s.rp_rp_id
            WHERE
                EXISTS( SELECT
                            NULL
                        FROM
                            web_prds@nrstp s
                        WHERE
                            t.rp_id = s.rp_rp_id
                        );That way you would have 1, 2 or 3 update statements each of which will run in a fraction of the time it takes for your process to run. You may need to do more work after this, such as materializing the results locally from the remote database rather than joining over the database link, but that's a bridge you can cross when you come to it.
HTH
David