Hirarchial query

Question

Hi
I use Oracle 9.2.0.1 Database
I try to bulid a temporary table which will be loaded by data from a procedure. The data should be gathered from two other tables:
1 - hirarchial table of items
2 - financial data that exist in the a different table that uses the lowest items in the hirarchial table (mentioned in item 1)
Table 1 structure name=fin_structure:
structure_id number(5)
catalog_id number(5)
master_item_id varchar2(15)
detailed_item_id varchar2(15)
Table 2 structure name=fin_data:
structure_id number(5)
catalog_id number(5)
org_unit_id varchar2(15)
item_id varchar2(15)
sum_1 number
sum_2 number
sum_3 number
sum_4 number
Eample of data:
fin_structure
4 3 0 1
4 3 1 35
4 3 35 35000
4 3 35 35010
4 3 0 3
4 3 3 50
4 3 50 50000
4 3 3 51
4 3 51 51000
4 3 51 51200
fin_data
4 3 15010 35000 -3000000 -15000 0 0
4 3 15010 50000 -56886.25 -25000 0 1
4 3 15010 51000 61600 32000 0 0
4 3 15010 51200 10800 0 1 1
4 3 20010 35000 -2000000 -10000 0 0
4 3 20010 50000 1 1 0 1
4 3 20010 51000 1 1 0 0
4 3 20010 51200 1 1 1 1
I want for each org_unit_it to aggregate all item_id according to the hirarchial structure. Once I've asked about this issue, and I've got a solution. It worked fine, but with a lot of data it takes lots of time, (and I had to duplicate several of the query for each sum column).
Here is a link to previous reply:
{thread:id=2351632}
The output shold look like that:
Structure:
org_unit_id
master_item_id
detailed_item_id ( in leaf levels = item_id in fin_data)
sum_1
sum_2
sum_3
sum_4
Data:
15010 35 35000 -3000000 -15000 0 0
15010 35 -3000000 -15000 0 0
15010 1 -3000000 -15000 0 0
15010 50 50000 -56886.25 -25000 0 1
15010 50 -56886.25 -25000 0 1
15010 51 51000 61600 32000 0 0
15010 51 51200 10800 0 1 1
15010 51 72400 32000 1 0
15010 3 15513.75 7000 1 1
15010 -2984486.25 8000 1 1
20010 35 35000 -2000000 -10000 0 0
20010 35 -2000000 -10000 0 0
20010 1 -2000000 -10000 0 0
20010 50 50000 1 1 0 1
20010 50 1 1 0 1
20010 51 51000 1 1 0 0
20010 51 51200 1 1 1 1
20010 51 2 2 1 1
20010 3 3 3 1 2
20010 -1999997 -9997 1 2
Here is the code I've used (its a part from a package, so there are functions and procedures that i call and not in this post. These are not the problems. sum1 ... sum4 has different names in the procedure, and there are some more of them. these are examples):
cursor c1 is
      select max(level) --, max(rownum)
        from fin_structure
       where structure_id = inStructureId
       start with master_item_id = 'TOTAL'
      connect by prior detailed_item_id = master_item_id;
    cursor c2 is
      select distinct organisation_unit_id,
                      get_org_unit_desc(organisation_unit_id, catalog_id) org_unit_desc
        from fin_data
       where structure_id = inStructureId
         and organisation_unit_id between inFromOrgUnitId and inToOrgUnitId
       order by organisation_unit_id;
    cursor c3 is
      with t_main as
       (select bis.structure_id,
               bis.catalog_id,
               bis.master_item_id,
               bis.detailed_item_id,
               sys_connect_by_path(bis.detailed_item_id, ',') || ',' path,
               bbv.org_unit_desc,
               bbv.item_desc,
               bbv.sum_planned,
               bbv.sum_add,
               bbv.sum_freeze,
               bbv.total_budget_nf,
               bbv.total_budget_wf,
               bbv.sum_reserved,
               bbv.sum_actual,
               bbv.balance_nf,
               bbv.balance_wf
          from fin_structure bis,
               (select *
                  from fin_data
                 where structure_id = inStructureId
                   and organisation_unit_id = vDetailedOrgUnitId
                   and item_id between inFromItemId and inToItemId
                   and budget_year = inYear) bbv
         where bbv.structure_id(+) = bis.structure_id
           and bbv.catalog_id(+) = bis.catalog_id
           and bbv.item_id(+) = bis.detailed_item_id
         start with bis.master_item_id = 'TOTAL'
        connect by bis.master_item_id = prior bis.detailed_item_id),
      t_planned as
       (select path,
               length(path || chr(1)) -
               length(translate(path || chr(1), chr(0) || ',', chr(0))) - 1 lvl,
               master_item_id,
               get_item_desc(master_item_id, catalog_id) master_item_desc,
               detailed_item_id,
               get_item_desc(detailed_item_id, catalog_id) detailed_item_desc,
               (select sum(sum_planned)
                  from t_main b
                 where b.structure_id = a.structure_id
                   and b.catalog_id = a.catalog_id
                   and b.path like a.path || '%'
                   and (length(path || chr(1)) -
                       length(translate(path || chr(1),
                                         chr(0) || ',',
                                         chr(0))) - 1) <= inToLevel) sum_planned
          from t_main a),
      t_add as
       (select master_item_id,
               detailed_item_id,
               (select sum(sum_add)
                  from t_main b
                 where b.structure_id = a.structure_id
                   and b.catalog_id = a.catalog_id
                   and b.path like a.path || '%') sum_add
          from t_main a),
      t_freeze as
       (select master_item_id,
               detailed_item_id,
               (select sum(sum_freeze)
                  from t_main b
                 where b.structure_id = a.structure_id
                   and b.catalog_id = a.catalog_id
                   and b.path like a.path || '%') sum_freeze
          from t_main a),
      t_total_budget_nf as
       (select master_item_id,
               detailed_item_id,
               (select sum(total_budget_nf)
                  from t_main b
                 where b.structure_id = a.structure_id
                   and b.catalog_id = a.catalog_id
                   and b.path like a.path || '%') total_budget_nf
          from t_main a),
      t_total_budget_wf as
       (select master_item_id,
               detailed_item_id,
               (select sum(total_budget_wf)
                  from t_main b
                 where b.structure_id = a.structure_id
                   and b.catalog_id = a.catalog_id
                   and b.path like a.path || '%') total_budget_wf
          from t_main a),
      t_reserved as
       (select master_item_id,
               detailed_item_id,
               (select sum(sum_reserved)
                  from t_main b
                 where b.structure_id = a.structure_id
                   and b.catalog_id = a.catalog_id
                   and b.path like a.path || '%') sum_reserved
          from t_main a),
      t_actual as
       (select master_item_id,
               detailed_item_id,
               (select sum(sum_actual)
                  from t_main b
                 where b.structure_id = a.structure_id
                   and b.catalog_id = a.catalog_id
                   and b.path like a.path || '%') sum_actual
          from t_main a),
      t_balance_nf as
       (select master_item_id,
               detailed_item_id,
               (select sum(balance_nf)
                  from t_main b
                 where b.structure_id = a.structure_id
                   and b.catalog_id = a.catalog_id
                   and b.path like a.path || '%') balance_nf
          from t_main a),
      t_balance_wf as
       (select master_item_id,
               detailed_item_id,
               (select sum(balance_wf)
                  from t_main b
                 where b.structure_id = a.structure_id
                   and b.catalog_id = a.catalog_id
                   and b.path like a.path || '%') balance_wf
          from t_main a)
      select path,
             lvl,
             tp.master_item_id,
             tp.master_item_desc,
             lpad(' ', lvl) || tp.detailed_item_id tree_detailed_item_id,
             tp.detailed_item_id,
             tp.detailed_item_desc,
             sum_planned,
             sum_add,
             sum_freeze,
             total_budget_nf,
             total_budget_wf,
             sum_reserved,
             sum_actual,
             balance_nf,
             balance_wf
        from t_planned         tp,
             t_add             tad,
             t_freeze          tf,
             t_total_budget_nf ttbnf,
             t_total_budget_wf ttbwf,
             t_reserved        tr,
             t_actual          tac,
             t_balance_nf      tbnf,
             t_balance_wf      tbwf
       where (sum_planned is not null or sum_add is not null or
             sum_freeze is not null or total_budget_nf is not null or
             total_budget_wf is not null or sum_reserved is not null or
             sum_actual is not null or balance_nf is not null or
             balance_wf is not null)
         and tp.master_item_id = tad.master_item_id
         and tp.master_item_id = tf.master_item_id
         and tp.master_item_id = ttbnf.master_item_id
         and tp.master_item_id = ttbwf.master_item_id
         and tp.master_item_id = tad.master_item_id
         and tp.master_item_id = tr.master_item_id
         and tp.master_item_id = tac.master_item_id
         and tp.master_item_id = tbnf.master_item_id
         and tp.master_item_id = tbwf.master_item_id
         and tp.detailed_item_id = tad.detailed_item_id
         and tp.detailed_item_id = tf.detailed_item_id
         and tp.detailed_item_id = ttbnf.detailed_item_id
         and tp.detailed_item_id = ttbwf.detailed_item_id
         and tp.detailed_item_id = tr.detailed_item_id
         and tp.detailed_item_id = tac.detailed_item_id
         and tp.detailed_item_id = tbnf.detailed_item_id
         and tp.detailed_item_id = tbwf.detailed_item_id
      union all
      select null,
             null,
             null,
             null,
             null,
             null,
             null,
             sum(sum_planned),
             sum(sum_add),
             sum(sum_freeze),
             sum(total_budget_nf),
             sum(total_budget_wf),
             sum(sum_reserved),
             sum(sum_actual),
             sum(balance_nf),
             sum(balance_wf)
        from t_main
       order by lvl, master_item_id, detailed_item_id;
            begin
              select structure_sort
                into vC4_RowNum
                from fin_structure
               where structure_id = inStructureId
                 and detailed_item_id = vDetailedItemId
                 and master_item_id = vMasterItemId;
            exception
              when no_data_found then
                vC4_RowNum := '';
            end;
            if nTreeLevel < nMaxLevel then
              vMasterOrgUnitId := 'B';
            else
              vMasterOrgUnitId := '';
            end if;
            vMasterOrgUnitDesc := vC4_RowNum;
            insert_budget_tree_gtmp(inStructureId,
                                    nTreeLevel,
                                    vPath,
                                    inYear,
                                    vMasterOrgUnitId,
                                    vMasterOrgUnitDesc,
                                    vDetailedOrgUnitId,
                                    vDetailedOrgUnitDesc,
                                    vMasterItemId,
                                    vMasterItemDesc,
                                    vTreeDetailedItemId,
                                    vDetailedItemId,
                                    vDetailedItemDesc,
                                    nSumPlanned,
                                    nSumAdd,
                                    nSumFreeze,
                                    nTotalBudgetNF,
                                    nTotalBudgetWF,
                                    nSumReserved,
                                    nSumActual,
                                    nBalanceNF,
                                    nBalanceWF);
          end if;
        end loop;
      end loop;Thanks for help
Edited by: user400097 on Apr 13, 2012 9:42 AM

Answer 1

Ok, that's a lot for us to trawl through and not something we can put on our own test database and replicate.
What you're essentially saying is that your process runs slowly when you have a lot of data.
Looking at your code, it's not clear exactly what you're doing because you've obviously left out some of the code.
What is clear though is that you appear to be doing nested cursor loops...
        end loop;
      end loop;and inserting individual records inside those loops.
This is known as row-by-row processing, or more affectionatly known as slow-by-slow processing because it's one of the slowest ways of processing data.
Essentually what you are trying to achieve is the insertion of a lot of records based on some queries. The most performant way of doing that would be to try and combine everything into a single SQL statement that juts does an INSERT .... SELECT .... and leaves out the PL/SQL loops completely. The reason... every time you use a pl/sql cursor loop with SQL statements inside it, you are causing a context switch between the PL/SQL engine and the SQL engine. It takes time to do that switch; to pass across the information from one engine to the other. With a single SQL statement on the other hand it would just be a single context switch, so the actual processing would be performed by the SQL engine at it's optimum speed.