PLSQL design question

Question

Hi All,
I have a database with about 500 tables, which can be divided into 100 sets. Each set has 5 tables. There are certain parent-child relationship within the set. All of the 100 sets have the identical logic. However the non-key fields are different. Any suggestion on how to design my PLSQL procedure and function?
Thanks,
Mike

Answer 1

user550903 wrote:
All the tables are in the same schema. I know it is a bad design from schema point of view. However, it is there already, and I have no right to modify it. All I want to do is to foger out a good solution for PLSQL. Actually, I am able to use dynamic SQL to handle different operations on different tables based on Oracle system table. But, I don't know how I can pass the augment for different row type, and how can I call different function bases on the table name. If you are formilar to C++, all I want is similar to template and pointer to a function.
Thanks in advance.Well, the moment you get into dynamic SQL you start having to write more complex code and sometimes dynamic PL/SQL, which is a nightmare to debug and maintain and doesn't allow for validation at compile time, but can lead to bugs becoming apparent at run time, and sometimes only with certain data or situations. It's buggy and doesn't scale well.
Because you want it "generic" you are saying that you don't want to hard code the individual table structures, and when you come to do that you have to get into the bowels of SQL and use something like the DBMS_SQL package to build your queries and bind parameters and then process the results based on column position rather than column name, looking at the definition of each column to determine the column type and then handling each type on-the-fly.
Such code can have it's uses, such as one I wrote for my own purposes to quickly export data from queries as CSV files. The following code gives an example of that (it's limited to just specifically handling numbers, varchar2's and dates, but could be expanded upon)...
As sys user:
CREATE OR REPLACE DIRECTORY TEST_DIR AS '\tmp\myfiles'
GRANT READ, WRITE ON DIRECTORY TEST_DIR TO myuser
/As myuser:
CREATE OR REPLACE PROCEDURE run_query(p_sql IN VARCHAR2
                                     ,p_dir IN VARCHAR2
                                     ,p_header_file IN VARCHAR2
                                     ,p_data_file IN VARCHAR2 := NULL) IS
  v_finaltxt  VARCHAR2(4000);
  v_v_val     VARCHAR2(4000);
  v_n_val     NUMBER;
  v_d_val     DATE;
  v_ret       NUMBER;
  c           NUMBER;
  d           NUMBER;
  col_cnt     INTEGER;
  f           BOOLEAN;
  rec_tab     DBMS_SQL.DESC_TAB;
  col_num     NUMBER;
  v_fh        UTL_FILE.FILE_TYPE;
  v_samefile  BOOLEAN := (NVL(p_data_file,p_header_file) = p_header_file);
BEGIN
  c := DBMS_SQL.OPEN_CURSOR;
  DBMS_SQL.PARSE(c, p_sql, DBMS_SQL.NATIVE);
  d := DBMS_SQL.EXECUTE(c);
  DBMS_SQL.DESCRIBE_COLUMNS(c, col_cnt, rec_tab);
  FOR j in 1..col_cnt
  LOOP
    CASE rec_tab(j).col_type
      WHEN 1 THEN DBMS_SQL.DEFINE_COLUMN(c,j,v_v_val,2000);
      WHEN 2 THEN DBMS_SQL.DEFINE_COLUMN(c,j,v_n_val);
      WHEN 12 THEN DBMS_SQL.DEFINE_COLUMN(c,j,v_d_val);
    ELSE
      DBMS_SQL.DEFINE_COLUMN(c,j,v_v_val,2000);
    END CASE;
  END LOOP;
  -- This part outputs the HEADER
  v_fh := UTL_FILE.FOPEN(upper(p_dir),p_header_file,'w',32767);
  FOR j in 1..col_cnt
  LOOP
    v_finaltxt := ltrim(v_finaltxt||','||lower(rec_tab(j).col_name),',');
  END LOOP;
  --  DBMS_OUTPUT.PUT_LINE(v_finaltxt);
  UTL_FILE.PUT_LINE(v_fh, v_finaltxt);
  IF NOT v_samefile THEN
    UTL_FILE.FCLOSE(v_fh);
  END IF;
  -- This part outputs the DATA
  IF NOT v_samefile THEN
    v_fh := UTL_FILE.FOPEN(upper(p_dir),p_data_file,'w',32767);
  END IF;
  LOOP
    v_ret := DBMS_SQL.FETCH_ROWS(c);
    EXIT WHEN v_ret = 0;
    v_finaltxt := NULL;
    FOR j in 1..col_cnt
    LOOP
      CASE rec_tab(j).col_type
        WHEN 1 THEN DBMS_SQL.COLUMN_VALUE(c,j,v_v_val);
                    v_finaltxt := ltrim(v_finaltxt||',"'||v_v_val||'"',',');
        WHEN 2 THEN DBMS_SQL.COLUMN_VALUE(c,j,v_n_val);
                    v_finaltxt := ltrim(v_finaltxt||','||v_n_val,',');
        WHEN 12 THEN DBMS_SQL.COLUMN_VALUE(c,j,v_d_val);
                    v_finaltxt := ltrim(v_finaltxt||','||to_char(v_d_val,'DD/MM/YYYY HH24:MI:SS'),',');
      ELSE
        v_finaltxt := ltrim(v_finaltxt||',"'||v_v_val||'"',',');
      END CASE;
    END LOOP;
  --  DBMS_OUTPUT.PUT_LINE(v_finaltxt);
    UTL_FILE.PUT_LINE(v_fh, v_finaltxt);
  END LOOP;
  UTL_FILE.FCLOSE(v_fh);
  DBMS_SQL.CLOSE_CURSOR(c);
END;This allows for the header row and the data to be written to seperate files if required.
e.g.
SQL> exec run_query('select * from emp','TEST_DIR','output.txt');
PL/SQL procedure successfully completed.Output.txt file contains:
empno,ename,job,mgr,hiredate,sal,comm,deptno
7369,"SMITH","CLERK",7902,17/12/1980 00:00:00,800,,20
7499,"ALLEN","SALESMAN",7698,20/02/1981 00:00:00,1600,300,30
7521,"WARD","SALESMAN",7698,22/02/1981 00:00:00,1250,500,30
7566,"JONES","MANAGER",7839,02/04/1981 00:00:00,2975,,20
7654,"MARTIN","SALESMAN",7698,28/09/1981 00:00:00,1250,1400,30
7698,"BLAKE","MANAGER",7839,01/05/1981 00:00:00,2850,,30
7782,"CLARK","MANAGER",7839,09/06/1981 00:00:00,2450,,10
7788,"SCOTT","ANALYST",7566,19/04/1987 00:00:00,3000,,20
7839,"KING","PRESIDENT",,17/11/1981 00:00:00,5000,,10
7844,"TURNER","SALESMAN",7698,08/09/1981 00:00:00,1500,0,30
7876,"ADAMS","CLERK",7788,23/05/1987 00:00:00,1100,,20
7900,"JAMES","CLERK",7698,03/12/1981 00:00:00,950,,30
7902,"FORD","ANALYST",7566,03/12/1981 00:00:00,3000,,20
7934,"MILLER","CLERK",7782,23/01/1982 00:00:00,1300,,10The procedure allows for the header and data to go to seperate files if required. Just specifying the "header" filename will put the header and data in the one file.
Adapt to output different datatypes and styles are required.
===========================================================================================================================
FIXED WIDTH FORMAT
===========================================================================================================================
As sys user:
CREATE OR REPLACE DIRECTORY TEST_DIR AS '\tmp\myfiles'
GRANT READ, WRITE ON DIRECTORY TEST_DIR TO myuser
/As myuser:
CREATE OR REPLACE PROCEDURE run_query(p_sql IN VARCHAR2
                                     ,p_dir IN VARCHAR2
                                     ,p_header_file IN VARCHAR2
                                     ,p_data_file IN VARCHAR2 := NULL) IS
  v_finaltxt  VARCHAR2(4000);
  v_v_val     VARCHAR2(4000);
  v_n_val     NUMBER;
  v_d_val     DATE;
  v_ret       NUMBER;
  c           NUMBER;
  d           NUMBER;
  col_cnt     INTEGER;
  f           BOOLEAN;
  rec_tab     DBMS_SQL.DESC_TAB;
  col_num     NUMBER;
  v_fh        UTL_FILE.FILE_TYPE;
  v_samefile  BOOLEAN := (NVL(p_data_file,p_header_file) = p_header_file);
BEGIN
  c := DBMS_SQL.OPEN_CURSOR;
  DBMS_SQL.PARSE(c, p_sql, DBMS_SQL.NATIVE);
  d := DBMS_SQL.EXECUTE(c);
  DBMS_SQL.DESCRIBE_COLUMNS(c, col_cnt, rec_tab);
  FOR j in 1..col_cnt
  LOOP
    CASE rec_tab(j).col_type
      WHEN 1 THEN DBMS_SQL.DEFINE_COLUMN(c,j,v_v_val,2000);
      WHEN 2 THEN DBMS_SQL.DEFINE_COLUMN(c,j,v_n_val);
      WHEN 12 THEN DBMS_SQL.DEFINE_COLUMN(c,j,v_d_val);
    ELSE
      DBMS_SQL.DEFINE_COLUMN(c,j,v_v_val,2000);
    END CASE;
  END LOOP;
  -- This part outputs the HEADER
  v_fh := UTL_FILE.FOPEN(upper(p_dir),p_header_file,'w',32767);
  FOR j in 1..col_cnt
  LOOP
    CASE rec_tab(j).col_type
      WHEN 1 THEN v_finaltxt := v_finaltxt||rpad(lower(rec_tab(j).col_name),rec_tab(j).col_max_len,' ');
      WHEN 2 THEN v_finaltxt := v_finaltxt||rpad(lower(rec_tab(j).col_name),rec_tab(j).col_max_len,' ');
      WHEN 12 THEN v_finaltxt := v_finaltxt||rpad(lower(rec_tab(j).col_name),greatest(19,length(rec_tab(j).col_name)),' ');
    END CASE;
  END LOOP;
  UTL_FILE.PUT_LINE(v_fh, v_finaltxt);
  IF NOT v_samefile THEN
    UTL_FILE.FCLOSE(v_fh);
  END IF;
  -- This part outputs the DATA
  IF NOT v_samefile THEN
    v_fh := UTL_FILE.FOPEN(upper(p_dir),p_data_file,'w',32767);
  END IF;
  LOOP
    v_ret := DBMS_SQL.FETCH_ROWS(c);
    EXIT WHEN v_ret = 0;
    v_finaltxt := NULL;
    FOR j in 1..col_cnt
    LOOP
      CASE rec_tab(j).col_type
        WHEN 1 THEN DBMS_SQL.COLUMN_VALUE(c,j,v_v_val);
                    v_finaltxt := v_finaltxt||rpad(nvl(v_v_val,' '),rec_tab(j).col_max_len,' ');
        WHEN 2 THEN DBMS_SQL.COLUMN_VALUE(c,j,v_n_val);
                    v_finaltxt := v_finaltxt||rpad(nvl(to_char(v_n_val,'fm99999999999999999999999999999999999999'),' '),rec_tab(j).col_max_len,' ');
        WHEN 12 THEN DBMS_SQL.COLUMN_VALUE(c,j,v_d_val);
                    v_finaltxt := v_finaltxt||rpad(nvl(to_char(v_d_val,'DD/MM/YYYY HH24:MI:SS'),' '),greatest(19,length(rec_tab(j).col_name)),' ');
      END CASE;
    END LOOP;
    UTL_FILE.PUT_LINE(v_fh, v_finaltxt);
  END LOOP;
  UTL_FILE.FCLOSE(v_fh);
  DBMS_SQL.CLOSE_CURSOR(c);
END;This allows for the header row and the data to be written to seperate files if required.
e.g.
SQL> exec run_query('select * from emp','TEST_DIR','output.txt');
PL/SQL procedure successfully completed.Output.csv file contains:
empno                 ename     job      mgr                   hiredate           sal                   comm                  deptno               
7369                  SMITH     CLERK    7902                  17/12/1980 00:00:00800                                         20                   
7499                  ALLEN     SALESMAN 7698                  20/02/1981 00:00:001600                  300                   30                   
7521                  WARD      SALESMAN 7698                  22/02/1981 00:00:001250                  500                   30                   
7566                  JONES     MANAGER  7839                  02/04/1981 00:00:002975                                        20                   
7654                  MARTIN    SALESMAN 7698                  28/09/1981 00:00:001250                  1400                  30                   
7698                  BLAKE     MANAGER  7839                  01/05/1981 00:00:002850                                        30                   
7782                  CLARK     MANAGER  7839                  09/06/1981 00:00:002450                                        10                   
7788                  SCOTT     ANALYST  7566                  19/04/1987 00:00:003000                                        20                   
7839                  KING      PRESIDENT                      17/11/1981 00:00:005000                                        10                   
7844                  TURNER    SALESMAN 7698                  08/09/1981 00:00:001500                  0                     30                   
7876                  ADAMS     CLERK    7788                  23/05/1987 00:00:001100                                        20                   
7900                  JAMES     CLERK    7698                  03/12/1981 00:00:00950                                         30                   
7902                  FORD      ANALYST  7566                  03/12/1981 00:00:003000                                        20                   
7934                  MILLER    CLERK    7782                  23/01/1982 00:00:001300                                        10                   
               The procedure allows for the header and data to go to seperate files if required. Just specifying the "header" filename will put the header and data in the one file.
As you can see, the query has to be parsed and executed by the code and then we can use the Describe functionality to get the DBMS_SQL package to tell us what the results look like, but then we have to handle each type of column based on it's internal datatype number (details can be found on the internet or in the documentation).
So, you can write PL/SQL code to query your tables "generically", but if you want to somehow define a structure to pass back to a 3rd party language, you are actually going to have to look at using Ref Cursors, and then the 3rd party application has to do what I was showing in the example code above, which is to describe the results and process the columns based on position etc. A lot of 3rd party software does this automatically with ref cursors.
Simple example of a Ref Cursor:
SQL> CREATE OR REPLACE PACKAGE reftest IS
  2    PROCEDURE test(P_no in number, cur_o OUT sys_refcursor);
  3  end;
  4  /
Package created.
SQL>
SQL> CREATE OR REPLACE PACKAGE body reftest as
  2    PROCEDURE test(P_no in number, cur_o OUT sys_refcursor) as
  3      myexc exception;
  4    BEGIN
  5      if P_no = 1 then
  6        open cur_o for select empno, ename from emp;
  7      elsif p_no =2 then
  8        open cur_o for select deptno, dname from dept;
  9      else
10        RAISE myexc;
11      END IF;
12    exception
13      when myexc then
14        raise_application_error(20991,'input must be 1 or 2');
15    end ;
16  end reftest;
17  /
Package body created.
SQL> var x refcursor;
SQL> exec reftest.test(1,:x);
PL/SQL procedure successfully completed.
SQL> print x;
     EMPNO ENAME
      7369 SMITH
      7499 ALLEN
      7521 WARD
      7566 JONES
      7654 MARTIN
      7698 BLAKE
      7782 CLARK
      7788 SCOTT
      7839 KING
      7844 TURNER
      7876 ADAMS
      7900 JAMES
      7902 FORD
      7934 MILLER
14 rows selected.
SQL> exec reftest.test(2,:x);
PL/SQL procedure successfully completed.
SQL> print x;
    DEPTNO DNAME
        10 ACCOUNTING
        20 RESEARCH
        30 SALES
        40 OPERATIONS
SQL>Here, it may not be that obvious, but the SQL*Plus client tool obtains the ref cursor back from the procedure and then it is actually SQL*Plus that has to describe the cursor and determine how it will display the results based on the datatypes of the returned columns. The PL/SQL code hasn't had to concern itself with passing back any description except what is in-built to the ref cursor itself.
In reality the same thing happens when you just execute an SQL query in SQL*Plus, as SQL*Plus has to do the process of opening a cursor for the SQL statmeent, requesting a parse of the query, getting a description of the results, fetching the data back row by row and then displaying the results based on the description. However, that's all hidden from you.