Bulk & collection concept
Can u please explain me the bulk & collection concept ?
why we use bulk & collection ?
when to use bulk & collection ?
and performance wise which is better ?
thanks in advance
user10685034 wrote:
Can u please explain me the bulk & collection concept ?
why we use bulk & collection ?
when to use bulk & collection ?
and performance wise which is better ? In PL/SQL programming the SQL part is executed by SQL Engine and PL is executed by PL/SQL Engine.
So in general any SQL statement available in PL/SQL is send to the SQL engine by the PL/SQL Engine and processed and returned back. This process of sending and getting it back is called Context Switch.
Consider the following code.
for i in (select empno from emp where deptno = 100)
loop
update emp set sal = sal*100 where empno = i.empno;
end loop; Lets consider that there are 20 employee under deptno 100. So now the UPDATE statement is executed 20 times. That is there is 20 context switch.
Introduction of bulk collect reduces this. Consider the following code.
declare
Type EmpTbl as table of number(10);
lEmpNo EmpTbl;
begin
select empno bulk collect into lEmpNo from emp where deptno = 100;
forall i in 1..lEmpNo.count
update emp set sal = sal*100 where empno = lEmpNo(i);
end; Here in the above code what i am doing is
1. Created a PL/SQL Table called EmpTbl.
2. Created a variable lEmpNo of type EmpTbl.
3. Collected all the empno for deptno = 100 into the lEmpNo.
4. Using FORALL send all the empno in the PL/SQL table to update at a single shot. By this the context switch is reduced extensively.
Similar Messages
-
BULK COLLECT and FORALL with dynamic INSERT.
Hello,
I want to apply BULK COLLECT and FORALL feature for a insert statement in my procedure for performance improvements as it has to insert a huge amount of data.
But the problem is that the insert statement gets generated dynamically and even the table name is found at the run-time ... so i am not able to apply the performance tuning concepts.
See below the code
PROCEDURE STP_MES_INSERT_GLOBAL_TO_MAIN
(P_IN_SRC_TABLE_NAME VARCHAR2 ,
P_IN_TRG_TABLE_NAME VARCHAR2 ,
P_IN_ED_TRIG_ALARM_ID NUMBER ,
P_IN_ED_CATG_ID NUMBER ,
P_IN_IS_PIECEID_ALARM IN CHAR,
P_IN_IS_LAST_RECORD IN CHAR
IS
V_START_DATA_ID NUMBER;
V_STOP_DATA_ID NUMBER;
V_FROM_DATA_ID NUMBER;
V_TO_DATA_ID NUMBER;
V_MAX_REC_IN_LOOP NUMBER := 30000;
V_QRY1 VARCHAR2(32767);
BEGIN
EXECUTE IMMEDIATE 'SELECT MIN(ED_DATA_ID), MAX(ED_DATA_ID) FROM '|| P_IN_SRC_TABLE_NAME INTO V_START_DATA_ID , V_STOP_DATA_ID;
--DBMS_OUTPUT.PUT_LINE('ORIGINAL START ID := '||V_START_DATA_ID ||' ORIGINAL STOP ID := ' || V_STOP_DATA_ID);
V_FROM_DATA_ID := V_START_DATA_ID ;
IF (V_STOP_DATA_ID - V_START_DATA_ID ) > V_MAX_REC_IN_LOOP THEN
V_TO_DATA_ID := V_START_DATA_ID + V_MAX_REC_IN_LOOP;
ELSE
V_TO_DATA_ID := V_STOP_DATA_ID;
END IF;
LOOP
BEGIN
LOOP
V_QRY1 := ' INSERT INTO '||P_IN_TRG_TABLE_NAME||
' SELECT * FROM '||P_IN_SRC_TABLE_NAME ||
' WHERE ED_DATA_ID BETWEEN ' || V_FROM_DATA_ID ||' AND ' || V_TO_DATA_ID;
EXECUTE IMMEDIATE V_QRY1;
commit;
V_FROM_DATA_ID := V_TO_DATA_ID + 1;
IF ( V_STOP_DATA_ID - V_TO_DATA_ID > V_MAX_REC_IN_LOOP ) THEN
V_TO_DATA_ID := V_TO_DATA_ID + V_MAX_REC_IN_LOOP;
ELSE
V_TO_DATA_ID := V_TO_DATA_ID + (V_STOP_DATA_ID - V_TO_DATA_ID);
END IF;
EXCEPTION
WHEN OTHERS THEN.............
....................so on Now you can observer here that P_IN_SRC_TABLE_NAME is the source table name which we get as a parameter at run-time. I have used 2 table in the insert statement P_IN_TRG_TABLE_NAME (in which i have to insert data) and P_IN_SRC_TABLE_NAME(from where i have to insert data)
V_QRY1 := ' INSERT INTO '||P_IN_TRG_TABLE_NAME||
' SELECT * FROM '||P_IN_SRC_TABLE_NAME ||
' WHERE ED_DATA_ID BETWEEN ' || V_FROM_DATA_ID ||' AND ' || V_TO_DATA_ID;
EXECUTE IMMEDIATE V_QRY1;now when i appy the bulk collect and forall feature i am facing the out of scope problem....see the code below ...
BEGIN
EXECUTE IMMEDIATE 'SELECT MIN(ED_DATA_ID), MAX(ED_DATA_ID) FROM '|| P_IN_SRC_TABLE_NAME INTO V_START_DATA_ID , V_STOP_DATA_ID;
--DBMS_OUTPUT.PUT_LINE('ORIGINAL START ID := '||V_START_DATA_ID ||' ORIGINAL STOP ID := ' || V_STOP_DATA_ID);
V_FROM_DATA_ID := V_START_DATA_ID ;
IF (V_STOP_DATA_ID - V_START_DATA_ID ) > V_MAX_REC_IN_LOOP THEN
V_TO_DATA_ID := V_START_DATA_ID + V_MAX_REC_IN_LOOP;
ELSE
V_TO_DATA_ID := V_STOP_DATA_ID;
END IF;
LOOP
DECLARE
TYPE TRG_TABLE_TYPE IS TABLE OF P_IN_SRC_TABLE_NAME%ROWTYPE;
V_TRG_TABLE_TYPE TRG_TABLE_TYPE;
CURSOR TRG_TAB_CUR IS
SELECT * FROM P_IN_SRC_TABLE_NAME
WHERE ED_DATA_ID BETWEEN V_FROM_DATA_ID AND V_TO_DATA_ID;
V_QRY1 varchar2(32767);
BEGIN
OPEN TRG_TAB_CUR;
LOOP
FETCH TRG_TAB_CUR BULK COLLECT INTO V_TRG_TABLE_TYPE LIMIT 30000;
FORALL I IN 1..V_TRG_TABLE_TYPE.COUNT
V_QRY1 := ' INSERT INTO '||P_IN_TRG_TABLE_NAME||' VALUES V_TRG_TABLE_TYPE(I);'
EXECUTE IMMEDIATE V_QRY1;
EXIT WHEN TRG_TAB_CUR%NOTFOUND;
END LOOP;
CLOSE TRG_TAB_CUR;
V_FROM_DATA_ID := V_TO_DATA_ID + 1;
IF ( V_STOP_DATA_ID - V_TO_DATA_ID > V_MAX_REC_IN_LOOP ) THEN
V_TO_DATA_ID := V_TO_DATA_ID + V_MAX_REC_IN_LOOP;
ELSE
V_TO_DATA_ID := V_TO_DATA_ID + (V_STOP_DATA_ID - V_TO_DATA_ID);
END IF;
EXCEPTION
WHEN OTHERS THEN.........so on
But the above code is not helping me , what i am doing wrong ??? how can i tune this dynamically generated statement to use bulk collect for better performace ......
Thanks in Advance !!!!Hello,
a table name cannot be bind as a parameter in SQL, this wont't compile:
EXECUTE IMMEDIATE ' INSERT INTO :1 VALUES ......
USING P_IN_TRG_TABLE_NAME ...but this should work:
EXECUTE IMMEDIATE ' INSERT INTO ' || P_IN_TRG_TABLE_NAME || ' VALUES ......You cannot declare a type that is based on a table which name is in a variable.
PL/SQL is stronly typed language, a type must be known at compile time, a code like this is not allowed:
PROCEDURE xx( src_table_name varchar2 )
DECLARE
TYPE tab IS TABLE OF src_table_name%ROWTYPE;
...This can be done by creating one big dynamic SQL - see example below (tested on Oracle 10 XE - this is a slightly simplified version of your procedure):
CREATE OR REPLACE
PROCEDURE stp1(
p_in_src_table_name VARCHAR2 ,
p_in_trg_table_name VARCHAR2 ,
v_from_data_id NUMBER := 100,
v_to_data_id NUMBER := 100000
IS
BEGIN
EXECUTE IMMEDIATE q'{
DECLARE
TYPE trg_table_type IS TABLE OF }' || p_in_src_table_name || q'{%ROWTYPE;
V_TRG_TABLE_TYPE TRG_TABLE_TYPE;
CURSOR TRG_TAB_CUR IS
SELECT * FROM }' || p_in_src_table_name ||
q'{ WHERE ED_DATA_ID BETWEEN :V_FROM_DATA_ID AND :V_TO_DATA_ID;
BEGIN
OPEN TRG_TAB_CUR;
LOOP
FETCH TRG_TAB_CUR BULK COLLECT INTO V_TRG_TABLE_TYPE LIMIT 30000;
FORALL I IN 1 .. V_TRG_TABLE_TYPE.COUNT
INSERT INTO }' || p_in_trg_table_name || q'{ VALUES V_TRG_TABLE_TYPE( I );
EXIT WHEN TRG_TAB_CUR%NOTFOUND;
END LOOP;
CLOSE TRG_TAB_CUR;
END; }'
USING v_from_data_id, v_to_data_id;
COMMIT;
END;But this probably won't give any performace improvements. Bulk collect and forall can give performance improvements when there is a DML operation inside a loop,
and this one single DML operates on only one record or relatively small number of records, and this DML is repeated many many times in the loop.
I guess that your code is opposite to this - it contains insert statement that operates on many records (one single insert ~ 30000 records),
and you are trying to replace it with bulk collect/forall - INSERT INTO ... SELECT FROM will almost alwayst be faster than bulk collect/forall.
Look at simple test - below is a procedure that uses INSERT ... SELECT :
CREATE OR REPLACE
PROCEDURE stp(
p_in_src_table_name VARCHAR2 ,
p_in_trg_table_name VARCHAR2 ,
v_from_data_id NUMBER := 100,
v_to_data_id NUMBER := 100000
IS
V_QRY1 VARCHAR2(32767);
BEGIN
V_QRY1 := ' INSERT INTO '|| P_IN_TRG_TABLE_NAME ||
' SELECT * FROM '|| P_IN_SRC_TABLE_NAME ||
' WHERE ed_data_id BETWEEN :f AND :t ';
EXECUTE IMMEDIATE V_QRY1
USING V_FROM_DATA_ID, V_TO_DATA_ID;
COMMIT;
END;
/and we can compare both procedures:
SQL> CREATE TABLE test333
2 AS SELECT level ed_data_id ,
3 'XXX ' || LEVEL x,
4 'YYY ' || 2 * LEVEL y
5 FROM dual
6 CONNECT BY LEVEL <= 1000000;
Table created.
SQL> CREATE TABLE test333_dst AS
2 SELECT * FROM test333 WHERE 1 = 0;
Table created.
SQL> set timing on
SQL> ed
Wrote file afiedt.buf
1 BEGIN
2 FOR i IN 1 .. 100 LOOP
3 stp1( 'test333', 'test333_dst', 1000, 31000 );
4 END LOOP;
5* END;
SQL> /
PL/SQL procedure successfully completed.
Elapsed: 00:00:22.12
SQL> ed
Wrote file afiedt.buf
1 BEGIN
2 FOR i IN 1 .. 100 LOOP
3 stp( 'test333', 'test333_dst', 1000, 31000 );
4 END LOOP;
5* END;
SQL> /
PL/SQL procedure successfully completed.
Elapsed: 00:00:14.86without bulk collect ~ 15 sec.
bulk collect version ~ 22 sec. .... 7 sec longer / 15 sec. = about 45% performance decrease. -
Any way to use cursor values inside other cursor by bulk collect?
hi,
Is there any way to use cursor get_tables value insdide loop get column if i am using bulk collect in both cursors?
I tried a lot but i am nt able to do it.kindly help...
create or replace procedure MULTIPLE_CURSORS_PROC is
v_owner varchar2(40);
v_table_name varchar2(40);
v_column_name varchar2(100);
cursor get_tables is
select distinct tbl.owner, tbl.table_name
from all_tables tbl
where tbl.owner = 'SYSTEM';
cursor get_columns is
select distinct col.column_name
from all_tab_columns col
where col.owner = v_owner
and col.table_name = v_table_name;
begin
open get_tables;
loop
fetch get_tables into v_owner, v_table_name;
open get_columns;
loop
fetch get_columns into v_column_name;
end loop;
close get_columns;
end loop;
close get_tables;
end ;hi there
Refer this
CREATE OR REPLACE PROCEDURE MULTIPLE_CURSORS_PROC
IS
TYPE scol IS VARRAY (10000) OF VARCHAR2 (32767);
v_table_name scol;
v_column_name scol;
TYPE curtyp IS REF CURSOR;
get_columns curtyp;
CURSOR get_tables
IS
SELECT DISTINCT tbl.table_name
FROM all_tables tbl
WHERE tbl.owner = 'SYSTEM';
BEGIN
OPEN get_tables;
LOOP
FETCH get_tables BULK COLLECT INTO v_table_name;
FOR indx IN v_table_name.FIRST .. v_table_name.LAST
LOOP
SELECT DISTINCT col.column_name
BULK COLLECT
INTO v_column_name
FROM all_tab_columns col
WHERE col.table_name = v_table_name (indx);
FOR ind IN v_column_name.FIRST .. v_column_name.LAST
LOOP
DBMS_OUTPUT.put_line (v_column_name (ind));
END LOOP;
END LOOP;
EXIT WHEN get_tables%NOTFOUND;
END LOOP;
CLOSE get_tables;
END MULTIPLE_CURSORS_PROC;regards
Hitesh -
Hi all,
I have a performance issue in the below code,where i am trying to insert the data from table_stg into target_tab and in parent_tab tables and then to child tables via cursor with bulk collect .the target_tab and parent_tab are huge tables and have a row wise trigger enabled on it .the trigger is mandatory . This timetaken for this block to execute is 5000 seconds.Now my requirement is to reduce it to 5 to 10 mins.
can someone please guide me here.Its bit urgent .Awaiting for your response.
declare
vmax_Value NUMBER(5);
vcnt number(10);
id_val number(20);
pc_id number(15);
vtable_nm VARCHAR2(100);
vstep_no VARCHAR2(10);
vsql_code VARCHAR2(10);
vsql_errm varchar2(200);
vtarget_starttime timestamp;
limit_in number :=10000;
idx number(10);
cursor stg_cursor is
select
DESCRIPTION,
SORT_CODE,
ACCOUNT_NUMBER,
to_number(to_char(CORRESPONDENCE_DATE,'DD')) crr_day,
to_char(CORRESPONDENCE_DATE,'MONTH') crr_month,
to_number(substr(to_char(CORRESPONDENCE_DATE,'DD-MON-YYYY'),8,4)) crr_year,
PARTY_ID,
GUID,
PAPERLESS_REF_IND,
PRODUCT_TYPE,
PRODUCT_BRAND,
PRODUCT_HELD_ID,
NOTIFICATION_PREF,
UNREAD_CORRES_PERIOD,
EMAIL_ID,
MOBILE_NUMBER,
TITLE,
SURNAME,
POSTCODE,
EVENT_TYPE,
PRIORITY_IND,
SUBJECT,
EXT_PRD_ID_TX,
EXT_PRD_HLD_ID_TX,
EXT_SYS_ID,
EXT_PTY_ID_TX,
ACCOUNT_TYPE_CD,
COM_PFR_TYP_TX,
COM_PFR_OPT_TX,
COM_PFR_RSN_CD
from table_stg;
type rec_type is table of stg_rec_type index by pls_integer;
v_rt_all_cols rec_type;
BEGIN
vstep_no := '0';
vmax_value := 0;
vtarget_starttime := systimestamp;
id_val := 0;
pc_id := 0;
success_flag := 0;
vstep_no := '1';
vtable_nm := 'before cursor';
OPEN stg_cursor;
vstep_no := '2';
vtable_nm := 'After cursor';
LOOP
vstep_no := '3';
vtable_nm := 'before fetch';
--loop
FETCH stg_cursor BULK COLLECT INTO v_rt_all_cols LIMIT limit_in;
vstep_no := '4';
vtable_nm := 'after fetch';
--EXIT WHEN v_rt_all_cols.COUNT = 0;
EXIT WHEN stg_cursor%NOTFOUND;
FOR i IN 1 .. v_rt_all_cols.COUNT
LOOP
dbms_output.put_line(upper(v_rt_all_cols(i).event_type));
if (upper(v_rt_all_cols(i).event_type) = upper('System_enforced')) then
vstep_no := '4.1';
vtable_nm := 'before seq sel';
select PC_SEQ.nextval into pc_id from dual;
vstep_no := '4.2';
vtable_nm := 'before insert corres';
INSERT INTO target1_tab
(ID,
PARTY_ID,
PRODUCT_BRAND,
SORT_CODE,
ACCOUNT_NUMBER,
EXT_PRD_ID_TX,
EXT_PRD_HLD_ID_TX,
EXT_SYS_ID,
EXT_PTY_ID_TX,
ACCOUNT_TYPE_CD,
COM_PFR_TYP_TX,
COM_PFR_OPT_TX,
COM_PFR_RSN_CD,
status)
VALUES
(pc_id,
v_rt_all_cols(i).party_id,
decode(v_rt_all_cols(i).product_brand,'LTB',2,'HLX',1,'HAL',1,'BOS',3,'VER',4,0),
v_rt_all_cols(i).sort_code,
'XXXX'||substr(trim(v_rt_all_cols(i).ACCOUNT_NUMBER),length(trim(v_rt_all_cols(i).ACCOUNT_NUMBER))-3,4),
v_rt_all_cols(i).EXT_PRD_ID_TX,
v_rt_all_cols(i).EXT_PRD_HLD_ID_TX,
v_rt_all_cols(i).EXT_SYS_ID,
v_rt_all_cols(i).EXT_PTY_ID_TX,
v_rt_all_cols(i).ACCOUNT_TYPE_CD,
v_rt_all_cols(i).COM_PFR_TYP_TX,
v_rt_all_cols(i).COM_PFR_OPT_TX,
v_rt_all_cols(i).COM_PFR_RSN_CD,
NULL);
vstep_no := '4.3';
vtable_nm := 'after insert corres';
else
select COM_SEQ.nextval into id_val from dual;
vstep_no := '6';
vtable_nm := 'before insertcomm';
if (upper(v_rt_all_cols(i).event_type) = upper('REMINDER')) then
vstep_no := '6.01';
vtable_nm := 'after if insertcomm';
insert into parent_tab
(ID ,
CTEM_CODE,
CHA_CODE,
CT_CODE,
CONTACT_POINT_ID,
SOURCE,
RECEIVED_DATE,
SEND_DATE,
RETRY_COUNT)
values
(id_val,
lower(v_rt_all_cols(i).event_type),
decode(v_rt_all_cols(i).product_brand,'LTB',2,'HLX',1,'HAL',1,'BOS',3,'VER',4,0),
'Email',
v_rt_all_cols(i).email_id,
'IADAREMINDER',
systimestamp,
systimestamp,
0);
else
vstep_no := '6.02';
vtable_nm := 'after else insertcomm';
insert into parent_tab
(ID ,
CTEM_CODE,
CHA_CODE,
CT_CODE,
CONTACT_POINT_ID,
SOURCE,
RECEIVED_DATE,
SEND_DATE,
RETRY_COUNT)
values
(id_val,
lower(v_rt_all_cols(i).event_type),
decode(v_rt_all_cols(i).product_brand,'LTB',2,'HLX',1,'HAL',1,'BOS',3,'VER',4,0),
'Email',
v_rt_all_cols(i).email_id,
'CORRESPONDENCE',
systimestamp,
systimestamp,
0);
END if;
vstep_no := '6.11';
vtable_nm := 'before chop';
if (v_rt_all_cols(i).ACCOUNT_NUMBER is not null) then
v_rt_all_cols(i).ACCOUNT_NUMBER := 'XXXX'||substr(trim(v_rt_all_cols(i).ACCOUNT_NUMBER),length(trim(v_rt_all_cols(i).ACCOUNT_NUMBER))-3,4);
insert into child_tab
(COM_ID,
KEY,
VALUE)
values
(id_val,
'IB.Correspondence.AccountNumberMasked',
v_rt_all_cols(i).ACCOUNT_NUMBER);
end if;
vstep_no := '6.1';
vtable_nm := 'before stateday';
if (v_rt_all_cols(i).crr_day is not null) then
insert into child_tab
(COM_ID,
KEY,
VALUE)
values
(id_val,
--'IB.Correspondence.Date.Day',
'IB.Crsp.Date.Day',
v_rt_all_cols(i).crr_day);
end if;
vstep_no := '6.2';
vtable_nm := 'before statemth';
if (v_rt_all_cols(i).crr_month is not null) then
insert into child_tab
(COM_ID,
KEY,
VALUE)
values
(id_val,
--'IB.Correspondence.Date.Month',
'IB.Crsp.Date.Month',
v_rt_all_cols(i).crr_month);
end if;
vstep_no := '6.3';
vtable_nm := 'before stateyear';
if (v_rt_all_cols(i).crr_year is not null) then
insert into child_tab
(COM_ID,
KEY,
VALUE)
values
(id_val,
--'IB.Correspondence.Date.Year',
'IB.Crsp.Date.Year',
v_rt_all_cols(i).crr_year);
end if;
vstep_no := '7';
vtable_nm := 'before type';
if (v_rt_all_cols(i).product_type is not null) then
insert into child_tab
(COM_ID,
KEY,
VALUE)
values
(id_val,
'IB.Product.ProductName',
v_rt_all_cols(i).product_type);
end if;
vstep_no := '9';
vtable_nm := 'before title';
if (trim(v_rt_all_cols(i).title) is not null) then
insert into child_tab
(COM_ID,
KEY,
VALUE )
values
(id_val,
'IB.Customer.Title',
trim(v_rt_all_cols(i).title));
end if;
vstep_no := '10';
vtable_nm := 'before surname';
if (v_rt_all_cols(i).surname is not null) then
insert into child_tab
(COM_ID,
KEY,
VALUE)
values
(id_val,
'IB.Customer.LastName',
v_rt_all_cols(i).surname);
end if;
vstep_no := '12';
vtable_nm := 'before postcd';
if (trim(v_rt_all_cols(i).POSTCODE) is not null) then
insert into child_tab
(COM_ID,
KEY,
VALUE)
values
(id_val,
'IB.Customer.Addr.PostCodeMasked',
substr(replace(v_rt_all_cols(i).POSTCODE,' ',''),length(replace(v_rt_all_cols(i).POSTCODE,' ',''))-2,3));
end if;
vstep_no := '13';
vtable_nm := 'before subject';
if (trim(v_rt_all_cols(i).SUBJECT) is not null) then
insert into child_tab
(COM_ID,
KEY,
VALUE)
values
(id_val,
'IB.Correspondence.Subject',
v_rt_all_cols(i).subject);
end if;
vstep_no := '14';
vtable_nm := 'before inactivity';
if (trim(v_rt_all_cols(i).UNREAD_CORRES_PERIOD) is null or
trim(v_rt_all_cols(i).UNREAD_CORRES_PERIOD) = '3' or
trim(v_rt_all_cols(i).UNREAD_CORRES_PERIOD) = '6' or
trim(v_rt_all_cols(i).UNREAD_CORRES_PERIOD) = '9') then
insert into child_tab
(COM_ID,
KEY,
VALUE)
values
(id_val,
'IB.Correspondence.Inactivity',
v_rt_all_cols(i).UNREAD_CORRES_PERIOD);
end if;
vstep_no := '14.1';
vtable_nm := 'after notfound';
end if;
vstep_no := '15';
vtable_nm := 'after notfound';
END LOOP;
end loop;
vstep_no := '16';
vtable_nm := 'before closecur';
CLOSE stg_cursor;
vstep_no := '17';
vtable_nm := 'before commit';
DELETE FROM table_stg;
COMMIT;
vstep_no := '18';
vtable_nm := 'after commit';
EXCEPTION
WHEN OTHERS THEN
ROLLBACK;
success_flag := 1;
vsql_code := SQLCODE;
vsql_errm := SUBSTR(sqlerrm,1,200);
error_logging_pkg.inserterrorlog('samp',vsql_code,vsql_errm, vtable_nm,vstep_no);
RAISE_APPLICATION_ERROR (-20011, 'samp '||vstep_no||' SQLERRM:'||SQLERRM);
end;
ThanksIts bit urgent
NO - it is NOT urgent. Not to us.
If you have an urgent problem you need to hire a consultant.
I have a performance issue in the below code,
Maybe you do and maybe you don't. How are we to really know? You haven't posted ANYTHING indicating that a performance issue exists. Please read the FAQ for how to post a tuning request and the info you need to provide. First and foremost you have to post SOMETHING that actually shows that a performance issue exists. Troubleshooting requires FACTS not just a subjective opinion.
where i am trying to insert the data from table_stg into target_tab and in parent_tab tables and then to child tables via cursor with bulk collect .the target_tab and parent_tab are huge tables and have a row wise trigger enabled on it .the trigger is mandatory . This timetaken for this block to execute is 5000 seconds.Now my requirement is to reduce it to 5 to 10 mins.
Personally I think 5000 seconds (about 1 hr 20 minutes) is very fast for processing 800 trillion rows of data into parent and child tables. Why do you think that is slow?
Your code has several major flaws that need to be corrected before you can even determine what, if anything, needs to be tuned.
This code has the EXIT statement at the beginning of the loop instead of at the end
FETCH stg_cursor BULK COLLECT INTO v_rt_all_cols LIMIT limit_in;
vstep_no := '4';
vtable_nm := 'after fetch';
--EXIT WHEN v_rt_all_cols.COUNT = 0;
EXIT WHEN stg_cursor%NOTFOUND;
The correct place for the %NOTFOUND test when using BULK COLLECT is at the END of the loop; that is, the last statement in the loop.
You can use a COUNT test at the start of the loop but ironically you have commented it out and have now done it wrong. Either move the NOTFOUND test to the end of the loop or remove it and uncomment the COUNT test.
WHEN OTHERS THEN
ROLLBACK;
That basically says you don't even care what problem occurs or whether the problem is for a single record of your 10,000 in the collection. You pretty much just throw away any stack trace and substitute your own message.
Your code also has NO exception handling for any of the individual steps or blocks of code.
The code you posted also begs the question of why you are using NAME=VALUE pairs for child data rows? Why aren't you using a standard relational table for this data?
As others have noted you are using slow-by-slow (row by row processing). Let's assume that PL/SQL, the bulk collect and row-by-row is actually necessary.
Then you should be constructing the parent and child records into collections and then inserting them in BULK using FORALL.
1. Create a collection for the new parent rows
2. Create a collection for the new child rows
3. For each set of LIMIT source row data
a. empty the parent and child collections
b. populate those collections with new parent/child data
c. bulk insert the parent collection into the parent table
d. bulk insert the child collection into the child table
And unless you really want to either load EVERYTHING or abandon everything you should use bulk exception handling so that the clean data gets processed and only the dirty data gets rejected. -
Opening two cursors using open cursor with bulk collect on colections ..
Is it possible to have the implementatiion of using bulk collect with collections using two open cursors ..
first c1
second c2
open c1
loop
open c2
loop
end loop
close c2
end loop;
close c1
what i found is for every outer loop of cursor c1 , cursor c2 is open and closed for every record.
is this willl imporove the performace .?
EXAMPLE:-
NOTE: The relatoin between finc and minc is one to many ..finc is parent and minc is child
function chk_notnull_blank ( colname IN number ) return number is
BEGIN
if ( colname is NOT NULL and colname not in ( -8E14, -7E14, -6E14, -5E14, -4E14, -3E14, -2E14, -1E14, -1E9 )) then
RETURN colname ;
else
RETURN 0;
end if;
END chk_notnull_blank;
procedure Proc_AnnualFmlyTotIncSummary is
CURSOR c_cur_finc IS SELECT FAMID FROM FINC ;
CURSOR c_cur_minc IS SELECT FAMID, MEMBNO , ANFEDTX, ANGOVRTX, ANPRVPNX, ANRRDEDX, ANSLTX, SALARYX, SALARYBX, NONFARMX, NONFRMBX , FARMINCX, FRMINCBX, RRRETIRX, RRRETRBX, SOCRRX, INDRETX, JSSDEDX, SSIX, SSIBX from MINC minc WHERE FAMID IN ( SELECT FAMID FROM FINC finc WHERE minc.FAMID = finc.FAMID );
v_tot_fsalaryx number := 0;
v_tot_fnonfrmx number := 0;
v_tot_ffrmincx number := 0;
v_tot_frretirx number := 0;
v_tot_findretx number := 0;
v_tot_fjssdedx number := 0;
v_tot_fssix number := 0;
v_temp_sum_fsalaryx number := 0;
v_temp_sum_fnonfrmx number := 0;
v_temp_sum_ffrmincx number := 0;
v_temp_sum_frretirx number := 0;
v_temp_sum_findretx number := 0;
v_temp_sum_fjssdedx number := 0;
v_temp_sum_fssix number := 0;
TYPE minc_rec IS RECORD (FAMID MINC.FAMID%TYPE, MEMBNO MINC.MEMBNO%TYPE , ANFEDTX MINC.ANFEDTX%TYPE, ANGOVRTX MINC.ANGOVRTX%TYPE , ANPRVPNX MINC.ANPRVPNX%TYPE , ANRRDEDX MINC.ANRRDEDX%TYPE , ANSLTX MINC.ANSLTX%TYPE, SALARYX MINC.SALARYX%TYPE , SALARYBX MINC.SALARYBX%TYPE , NONFARMX MINC.NONFARMX%TYPE , NONFRMBX MINC.NONFRMBX%TYPE, FARMINCX MINC.FARMINCX%TYPE , FRMINCBX MINC.FRMINCBX%TYPE , RRRETIRX MINC.RRRETIRX%TYPE , RRRETRBX MINC.RRRETRBX%TYPE, SOCRRX MINC.SOCRRX%TYPE , INDRETX MINC.INDRETX%TYPE , JSSDEDX MINC.JSSDEDX%TYPE , SSIX MINC.SSIX%TYPE , SSIBX MINC.SSIBX%TYPE );
v_flag_boolean boolean := false;
v_famid number ;
v_stmt varchar2(3200) ;
v_limit number := 50;
v_temp_FAMTFEDX number := 0 ;
v_temp_FGOVRETX number := 0 ;
v_temp_FPRIVPENX number := 0 ;
v_temp_FRRDEDX number := 0 ;
v_temp_FSLTAXX number := 0 ;
v_temp_FSALARYX number := 0 ;
v_temp_FNONFRMX number := 0 ;
v_temp_FFRMINCX number := 0 ;
v_temp_FRRETIRX number := 0 ;
v_temp_FINDRETX number := 0 ;
v_temp_FJSSDEDX number := 0 ;
v_temp_FSSIX number := 0 ;
BEGIN
OPEN c_cur_finc ;
LOOP
FETCH c_cur_finc BULK COLLECT INTO famid_type_tbl LIMIT v_limit;
EXIT WHEN famid_type_tbl.COUNT = 0;
FOR i in famid_type_tbl.FIRST..famid_type_tbl.LAST
LOOP
OPEN c_cur_minc ;
LOOP
FETCH c_cur_minc BULK COLLECT INTO minc_rec_type_tbl LIMIT v_limit;
EXIT WHEN minc_rec_type_tbl.COUNT = 0;
FOR j IN minc_rec_type_tbl.FIRST..minc_rec_type_tbl.LAST
LOOP
if ( famid_type_tbl(i) = minc_rec_type_tbl(j).FAMID ) THEN
v_temp_FAMTFEDX := v_temp_FAMTFEDX + chk_notnull_blank(minc_rec_type_tbl(j).ANFEDTX );
v_temp_FGOVRETX := v_temp_FGOVRETX + chk_notnull_blank(minc_rec_type_tbl(j).ANGOVRTX);
v_temp_FPRIPENX := v_temp_FPRIPENX + chk_notnull_blank(minc_rec_type_tbl(j).ANPRVPNX);
v_temp_FRRDEDX := v_temp_FRRDEDX + chk_notnull_blank(minc_rec_type_tbl(j).ANRRDEDX);
v_temp_FSLTAXX := v_temp_FSLTAXX + chk_notnull_blank(minc_rec_type_tbl(j).ANSLTX );
v_temp_FSALARYX := v_temp_FSALARYX + chk_notnull_blank(minc_rec_type_tbl(j).SALARYX ) + chk_notnull_blank(minc_rec_type_tbl(j).SALARYBX);
v_temp_FNONFRMX := v_temp_FNONFRMX + chk_notnull_blank(minc_rec_type_tbl(j).NONFARMX) + chk_notnull_blank(minc_rec_type_tbl(j).NONFRMBX);
v_temp_FFRMINCX := v_temp_FFRMINCX + chk_notnull_blank(minc_rec_type_tbl(j).FARMINCX) + chk_notnull_blank(minc_rec_type_tbl(j).FRMINCBX );
v_temp_FRRETIRX := v_temp_FRRETIRX + chk_notnull_blank(minc_rec_type_tbl(j).RRRETIRX) + chk_notnull_blank(minc_rec_type_tbl(j).RRRETRBX ) + chk_notnull_blank(minc_rec_type_tbl(j).SOCRRX);
v_temp_FINDREXT := v_temp_FINDRETX + chk_notnull_blank(minc_rec_type_tbl(j).INDRETX);
v_temp_FJSSDEDX := v_temp_FJSSDEDX + chk_notnull_blank(minc_rec_type_tbl(j).JSSDEDX);
v_temp_FSSIX := v_temp_FSSIX + chk_notnull_blank(minc_rec_type_tbl(j).SSIX ) + chk_notnull_blank(minc_rec_type_tbl(j).SSIBX);
END IF;
END LOOP;
END LOOP ;
CLOSE c_cur_minc;
UPDATE FINC SET FAMTFEDX = v_temp_FAMTFEDX WHERE FAMID = famid_type_tbl(i);
END LOOP;
END LOOP;
CLOSE c_cur_finc;
END;
EXCEPTION
WHEN OTHERS THEN
raise_application_error(-20001,'An error was encountered - '||SQLCODE||' -ERROR- '||SQLERRM);
v_err_code := SQLCODE;
v_err_msg := substr(SQLERRM, 1, 200);
INSERT INTO audit_table (error_number, error_message) VALUES (v_err_code, v_err_msg);
error_logging(p_error_code => substr(sqlerrm,1,9), p_error_message => substr(sqlerrm,12), p_package =>'PKG_FCI_APP',p_procedure => 'Proc_Annual_Deductions_FromPay ' , p_location => v_location);
end Proc_AnnualFmlyTotIncSummary ;
Is the proga efficient and free from compilation errors ..?
thanks/kumar
Edited by: kumar73 on Sep 22, 2010 12:48 PMfunction chk_notnull_blank ( colname IN number ) return number is Maybe this function should have its own forum:
how to use case in this program
Re: how to declare a formal parameter in a function of type record and access ?
Re: how to define a function with table type parameter
Re: creation of db trigger with error ..
Re: How to write a trigger for the below scenario
how to improve the code using advanced methods
yours advice in improving the coding ..
How to use bulk in multiple cursors !!
;-) -
How to view errors if bulk collect has thrown errors
Hi,
I have few questions.
1.How to view error whether bulk collect is successful or not
2.What is identified & unidentified relationships in ERWIN
3.How to see the errors whether the sql loder is successful or not
and how to open the log file.Is there any specific command in UNIX
which tells loader is successful or thrown error
4.When executing the pl/sql procedure from UNIX.how to check for errors.
Please provide the answers for this
ThanksUse SAVE EXCEPTIONS clause in your FORALL loop.
Is this for homework/test? -
Can I use Bulk Collect results as input parameter for another cursor
MUSIC ==> remote MUSIC_DB database, MUSIC table has 60 million rows
PRICE_DATA ==> remote PRICING_DB database, PRICE_DATE table has 1 billion rows
These two table once existed in same database, but size of database exceeded available hardware size and hardware budget, so the PRICE_DATA table was moved to another Oracle database. I need to create a single report that combines data from both of these tables, and a distributed join with DRIVING_SITE hint will not work because the size of both table is too large to push to one DRIVING_SITE location, so I wrote this PLSQL block to process in small blocks.
QUESTION: how can use bulk collect from one cursor and pass that bulk collected information as input to second cursor without specifically listing each cell of the PLSQL bulk collection? See sample pseudo-code below, I am trying to determine more efficient way to code than hard-coding 100 parameter names into 2nd cursor.
NOTE: below is truly pseudo-code, I had to change the names of everything to adhere to NDA, but below works and is fast enough for my purposes, but if I want to change from 100 input parameters to 200, I have to add more hard-coded values. There has got to be a better way.
DECLARE
-- define cursor that retrieves distinct SONG_IDs from MUSIC table in remote music database
CURSOR C_CURRENT_MUSIC
IS
select distinct SONG_ID
from MUSIC@MUSIC_DB
where PRODUCTION_RELEASE=1
/* define a parameterized cursor that accepts 100 SONG_IDs and retrieves
required pricing information
CURSOR C_get_music_price_data
P_SONG_ID_001 NUMBER, P_SONG_ID_002 NUMBER, P_SONG_ID_003 NUMBER, P_SONG_ID_004 NUMBER, P_SONG_ID_005 NUMBER, P_SONG_ID_006 NUMBER, P_SONG_ID_007 NUMBER, P_SONG_ID_008 NUMBER, P_SONG_ID_009 NUMBER, P_SONG_ID_010 NUMBER,
P_SONG_ID_011 NUMBER, P_SONG_ID_012 NUMBER, P_SONG_ID_013 NUMBER, P_SONG_ID_014 NUMBER, P_SONG_ID_015 NUMBER, P_SONG_ID_016 NUMBER, P_SONG_ID_017 NUMBER, P_SONG_ID_018 NUMBER, P_SONG_ID_019 NUMBER, P_SONG_ID_020 NUMBER,
P_SONG_ID_021 NUMBER, P_SONG_ID_022 NUMBER, P_SONG_ID_023 NUMBER, P_SONG_ID_024 NUMBER, P_SONG_ID_025 NUMBER, P_SONG_ID_026 NUMBER, P_SONG_ID_027 NUMBER, P_SONG_ID_028 NUMBER, P_SONG_ID_029 NUMBER, P_SONG_ID_030 NUMBER,
P_SONG_ID_031 NUMBER, P_SONG_ID_032 NUMBER, P_SONG_ID_033 NUMBER, P_SONG_ID_034 NUMBER, P_SONG_ID_035 NUMBER, P_SONG_ID_036 NUMBER, P_SONG_ID_037 NUMBER, P_SONG_ID_038 NUMBER, P_SONG_ID_039 NUMBER, P_SONG_ID_040 NUMBER,
P_SONG_ID_041 NUMBER, P_SONG_ID_042 NUMBER, P_SONG_ID_043 NUMBER, P_SONG_ID_044 NUMBER, P_SONG_ID_045 NUMBER, P_SONG_ID_046 NUMBER, P_SONG_ID_047 NUMBER, P_SONG_ID_048 NUMBER, P_SONG_ID_049 NUMBER, P_SONG_ID_050 NUMBER,
P_SONG_ID_051 NUMBER, P_SONG_ID_052 NUMBER, P_SONG_ID_053 NUMBER, P_SONG_ID_054 NUMBER, P_SONG_ID_055 NUMBER, P_SONG_ID_056 NUMBER, P_SONG_ID_057 NUMBER, P_SONG_ID_058 NUMBER, P_SONG_ID_059 NUMBER, P_SONG_ID_060 NUMBER,
P_SONG_ID_061 NUMBER, P_SONG_ID_062 NUMBER, P_SONG_ID_063 NUMBER, P_SONG_ID_064 NUMBER, P_SONG_ID_065 NUMBER, P_SONG_ID_066 NUMBER, P_SONG_ID_067 NUMBER, P_SONG_ID_068 NUMBER, P_SONG_ID_069 NUMBER, P_SONG_ID_070 NUMBER,
P_SONG_ID_071 NUMBER, P_SONG_ID_072 NUMBER, P_SONG_ID_073 NUMBER, P_SONG_ID_074 NUMBER, P_SONG_ID_075 NUMBER, P_SONG_ID_076 NUMBER, P_SONG_ID_077 NUMBER, P_SONG_ID_078 NUMBER, P_SONG_ID_079 NUMBER, P_SONG_ID_080 NUMBER,
P_SONG_ID_081 NUMBER, P_SONG_ID_082 NUMBER, P_SONG_ID_083 NUMBER, P_SONG_ID_084 NUMBER, P_SONG_ID_085 NUMBER, P_SONG_ID_086 NUMBER, P_SONG_ID_087 NUMBER, P_SONG_ID_088 NUMBER, P_SONG_ID_089 NUMBER, P_SONG_ID_090 NUMBER,
P_SONG_ID_091 NUMBER, P_SONG_ID_092 NUMBER, P_SONG_ID_093 NUMBER, P_SONG_ID_094 NUMBER, P_SONG_ID_095 NUMBER, P_SONG_ID_096 NUMBER, P_SONG_ID_097 NUMBER, P_SONG_ID_098 NUMBER, P_SONG_ID_099 NUMBER, P_SONG_ID_100 NUMBER
IS
select
from PRICE_DATA@PRICING_DB
where COUNTRY = 'USA'
and START_DATE <= sysdate
and END_DATE > sysdate
and vpc.SONG_ID IN
P_SONG_ID_001 ,P_SONG_ID_002 ,P_SONG_ID_003 ,P_SONG_ID_004 ,P_SONG_ID_005 ,P_SONG_ID_006 ,P_SONG_ID_007 ,P_SONG_ID_008 ,P_SONG_ID_009 ,P_SONG_ID_010,
P_SONG_ID_011 ,P_SONG_ID_012 ,P_SONG_ID_013 ,P_SONG_ID_014 ,P_SONG_ID_015 ,P_SONG_ID_016 ,P_SONG_ID_017 ,P_SONG_ID_018 ,P_SONG_ID_019 ,P_SONG_ID_020,
P_SONG_ID_021 ,P_SONG_ID_022 ,P_SONG_ID_023 ,P_SONG_ID_024 ,P_SONG_ID_025 ,P_SONG_ID_026 ,P_SONG_ID_027 ,P_SONG_ID_028 ,P_SONG_ID_029 ,P_SONG_ID_030,
P_SONG_ID_031 ,P_SONG_ID_032 ,P_SONG_ID_033 ,P_SONG_ID_034 ,P_SONG_ID_035 ,P_SONG_ID_036 ,P_SONG_ID_037 ,P_SONG_ID_038 ,P_SONG_ID_039 ,P_SONG_ID_040,
P_SONG_ID_041 ,P_SONG_ID_042 ,P_SONG_ID_043 ,P_SONG_ID_044 ,P_SONG_ID_045 ,P_SONG_ID_046 ,P_SONG_ID_047 ,P_SONG_ID_048 ,P_SONG_ID_049 ,P_SONG_ID_050,
P_SONG_ID_051 ,P_SONG_ID_052 ,P_SONG_ID_053 ,P_SONG_ID_054 ,P_SONG_ID_055 ,P_SONG_ID_056 ,P_SONG_ID_057 ,P_SONG_ID_058 ,P_SONG_ID_059 ,P_SONG_ID_060,
P_SONG_ID_061 ,P_SONG_ID_062 ,P_SONG_ID_063 ,P_SONG_ID_064 ,P_SONG_ID_065 ,P_SONG_ID_066 ,P_SONG_ID_067 ,P_SONG_ID_068 ,P_SONG_ID_069 ,P_SONG_ID_070,
P_SONG_ID_071 ,P_SONG_ID_072 ,P_SONG_ID_073 ,P_SONG_ID_074 ,P_SONG_ID_075 ,P_SONG_ID_076 ,P_SONG_ID_077 ,P_SONG_ID_078 ,P_SONG_ID_079 ,P_SONG_ID_080,
P_SONG_ID_081 ,P_SONG_ID_082 ,P_SONG_ID_083 ,P_SONG_ID_084 ,P_SONG_ID_085 ,P_SONG_ID_086 ,P_SONG_ID_087 ,P_SONG_ID_088 ,P_SONG_ID_089 ,P_SONG_ID_090,
P_SONG_ID_091 ,P_SONG_ID_092 ,P_SONG_ID_093 ,P_SONG_ID_094 ,P_SONG_ID_095 ,P_SONG_ID_096 ,P_SONG_ID_097 ,P_SONG_ID_098 ,P_SONG_ID_099 ,P_SONG_ID_100
group by
vpc.SONG_ID
,vpc.STOREFRONT_ID
TYPE SONG_ID_TYPE IS TABLE OF MUSIC@MUSIC_DB%TYPE INDEX BY BINARY_INTEGER;
V_SONG_ID_ARRAY SONG_ID_TYPE ;
v_commit_counter NUMBER := 0;
BEGIN
/* open cursor you intent to bulk collect from */
OPEN C_CURRENT_MUSIC;
LOOP
/* in batches of 100, bulk collect ADAM_ID mapped TMS_IDENTIFIER into PLSQL table or records */
FETCH C_CURRENT_MUSIC BULK COLLECT INTO V_SONG_ID_ARRAY LIMIT 100;
EXIT WHEN V_SONG_ID_ARRAY.COUNT = 0;
/* to avoid NO DATA FOUND error when pass 100 parameters to OPEN cursor, if the arrary
is not fully populated to 100, pad the array with nulls to fill up to 100 cells. */
IF (V_SONG_ID_ARRAY.COUNT >=1 and V_SONG_ID_ARRAY.COUNT <> 100) THEN
FOR j IN V_SONG_ID_ARRAY.COUNT+1..100 LOOP
V_SONG_ID_ARRAY(j) := null;
END LOOP;
END IF;
/* pass a batch of 100 to cursor that get price information per SONG_ID and STOREFRONT_ID */
FOR j IN C_get_music_price_data
V_SONG_ID_ARRAY(1) ,V_SONG_ID_ARRAY(2) ,V_SONG_ID_ARRAY(3) ,V_SONG_ID_ARRAY(4) ,V_SONG_ID_ARRAY(5) ,V_SONG_ID_ARRAY(6) ,V_SONG_ID_ARRAY(7) ,V_SONG_ID_ARRAY(8) ,V_SONG_ID_ARRAY(9) ,V_SONG_ID_ARRAY(10) ,
V_SONG_ID_ARRAY(11) ,V_SONG_ID_ARRAY(12) ,V_SONG_ID_ARRAY(13) ,V_SONG_ID_ARRAY(14) ,V_SONG_ID_ARRAY(15) ,V_SONG_ID_ARRAY(16) ,V_SONG_ID_ARRAY(17) ,V_SONG_ID_ARRAY(18) ,V_SONG_ID_ARRAY(19) ,V_SONG_ID_ARRAY(20) ,
V_SONG_ID_ARRAY(21) ,V_SONG_ID_ARRAY(22) ,V_SONG_ID_ARRAY(23) ,V_SONG_ID_ARRAY(24) ,V_SONG_ID_ARRAY(25) ,V_SONG_ID_ARRAY(26) ,V_SONG_ID_ARRAY(27) ,V_SONG_ID_ARRAY(28) ,V_SONG_ID_ARRAY(29) ,V_SONG_ID_ARRAY(30) ,
V_SONG_ID_ARRAY(31) ,V_SONG_ID_ARRAY(32) ,V_SONG_ID_ARRAY(33) ,V_SONG_ID_ARRAY(34) ,V_SONG_ID_ARRAY(35) ,V_SONG_ID_ARRAY(36) ,V_SONG_ID_ARRAY(37) ,V_SONG_ID_ARRAY(38) ,V_SONG_ID_ARRAY(39) ,V_SONG_ID_ARRAY(40) ,
V_SONG_ID_ARRAY(41) ,V_SONG_ID_ARRAY(42) ,V_SONG_ID_ARRAY(43) ,V_SONG_ID_ARRAY(44) ,V_SONG_ID_ARRAY(45) ,V_SONG_ID_ARRAY(46) ,V_SONG_ID_ARRAY(47) ,V_SONG_ID_ARRAY(48) ,V_SONG_ID_ARRAY(49) ,V_SONG_ID_ARRAY(50) ,
V_SONG_ID_ARRAY(51) ,V_SONG_ID_ARRAY(52) ,V_SONG_ID_ARRAY(53) ,V_SONG_ID_ARRAY(54) ,V_SONG_ID_ARRAY(55) ,V_SONG_ID_ARRAY(56) ,V_SONG_ID_ARRAY(57) ,V_SONG_ID_ARRAY(58) ,V_SONG_ID_ARRAY(59) ,V_SONG_ID_ARRAY(60) ,
V_SONG_ID_ARRAY(61) ,V_SONG_ID_ARRAY(62) ,V_SONG_ID_ARRAY(63) ,V_SONG_ID_ARRAY(64) ,V_SONG_ID_ARRAY(65) ,V_SONG_ID_ARRAY(66) ,V_SONG_ID_ARRAY(67) ,V_SONG_ID_ARRAY(68) ,V_SONG_ID_ARRAY(69) ,V_SONG_ID_ARRAY(70) ,
V_SONG_ID_ARRAY(71) ,V_SONG_ID_ARRAY(72) ,V_SONG_ID_ARRAY(73) ,V_SONG_ID_ARRAY(74) ,V_SONG_ID_ARRAY(75) ,V_SONG_ID_ARRAY(76) ,V_SONG_ID_ARRAY(77) ,V_SONG_ID_ARRAY(78) ,V_SONG_ID_ARRAY(79) ,V_SONG_ID_ARRAY(80) ,
V_SONG_ID_ARRAY(81) ,V_SONG_ID_ARRAY(82) ,V_SONG_ID_ARRAY(83) ,V_SONG_ID_ARRAY(84) ,V_SONG_ID_ARRAY(85) ,V_SONG_ID_ARRAY(86) ,V_SONG_ID_ARRAY(87) ,V_SONG_ID_ARRAY(88) ,V_SONG_ID_ARRAY(89) ,V_SONG_ID_ARRAY(90) ,
V_SONG_ID_ARRAY(91) ,V_SONG_ID_ARRAY(92) ,V_SONG_ID_ARRAY(93) ,V_SONG_ID_ARRAY(94) ,V_SONG_ID_ARRAY(95) ,V_SONG_ID_ARRAY(96) ,V_SONG_ID_ARRAY(97) ,V_SONG_ID_ARRAY(98) ,V_SONG_ID_ARRAY(99) ,V_SONG_ID_ARRAY(100)
LOOP
/* do stuff with data from Song and Pricing Database coming from the two
separate cursors, then continue processing more rows...
END LOOP;
/* commit after each batch of 100 SONG_IDs is processed */
COMMIT;
EXIT WHEN C_CURRENT_MUSIC%NOTFOUND; -- exit when there are no more rows to fetch from cursor
END LOOP; -- bulk fetching loop
CLOSE C_CURRENT_MUSIC; -- close cursor that was used in bulk collection
/* commit rows */
COMMIT; -- commit any remaining uncommitted data.
END;I've got a problem when using passing VARRAY of numbers as parameter to remote cursor: it takes a super long time to run, sometimes doesn't finish even after an hour as passed.
Continuing with my example in original entry, I replaced the bulk collect into PLSQL table collection with a VARRAY and i bulk collect into the VARRAY, this is fast and I know it works because I can DBMS_OUTPUT.PUT_LINE cells of VARRAY so I know it is getting populated correctly. However, when I pass the VARRAY containing 100 cells populated with SONG_IDs as parameter to cursor, execution time is over an hour and when I am expecting a few seconds.
Below code example strips the problem down to it's raw details, I skip the bulk collect and just manually populate a VARRAY with 100 SONG_ID values, then try to pass to as parameter to a cursor, but the execution time of cursor is unexpectedly long, over 30 minutes, sometime longer, when I am expecting seconds.
IMPORTANT: If I take the same 100 SONG_IDs and place them directly in the cursor query's where IN clause, the SQL runs in under 5 seconds and returns result. Also, if I pass the 100 SONG_IDs as individual cells of a PLSQL table collection, then it also runs fast.
I thought that since the VARRAY is used via select subquery that is it queried locally, but the cursor is remote, and that I had a distribute problem on my hands, so I put in the DRIVING_SITE hint to attempt to force the result of query against VARRAY to go to remote server and rest of query will run there before returning result, but that didn't work either, still got slow response.
Is something wrong with my code, or I am running into a Oracle problem that may require support to resolve?
DECLARE
/* define a parameterized cursor that accepts XXX number of in SONG_IDs and
retrieves required pricing information
CURSOR C_get_music_price_data
p_array_song_ids SYS.ODCInumberList
IS
select /*+DRIVING_SITE(pd) */
count(distinct s.EVE_ID)
from PRICE_DATA@PRICING_DB pd
where pd.COUNTRY = 'USA'
and pd.START_DATE <= sysdate
and pd.END_DATE > sysdate
and pd.SONG_ID IN
select column_value from table(p_array_song_ids)
group by
pd.SONG_ID
,pd.STOREFRONT_ID
V_ARRAY_SONG_IDS SYS.ODCInumberList := SYS.ODCInumberList();
BEGIN
V_ARRAY_SONG_IDS.EXTEND(100);
V_ARRAY_SONG_IDS( 1 ) := 31135 ;
V_ARRAY_SONG_IDS( 2 ) := 31140 ;
V_ARRAY_SONG_IDS( 3 ) := 31142 ;
V_ARRAY_SONG_IDS( 4 ) := 31144 ;
V_ARRAY_SONG_IDS( 5 ) := 31146 ;
V_ARRAY_SONG_IDS( 6 ) := 31148 ;
V_ARRAY_SONG_IDS( 7 ) := 31150 ;
V_ARRAY_SONG_IDS( 8 ) := 31152 ;
V_ARRAY_SONG_IDS( 9 ) := 31154 ;
V_ARRAY_SONG_IDS( 10 ) := 31156 ;
V_ARRAY_SONG_IDS( 11 ) := 31158 ;
V_ARRAY_SONG_IDS( 12 ) := 31160 ;
V_ARRAY_SONG_IDS( 13 ) := 33598 ;
V_ARRAY_SONG_IDS( 14 ) := 33603 ;
V_ARRAY_SONG_IDS( 15 ) := 33605 ;
V_ARRAY_SONG_IDS( 16 ) := 33607 ;
V_ARRAY_SONG_IDS( 17 ) := 33609 ;
V_ARRAY_SONG_IDS( 18 ) := 33611 ;
V_ARRAY_SONG_IDS( 19 ) := 33613 ;
V_ARRAY_SONG_IDS( 20 ) := 33615 ;
V_ARRAY_SONG_IDS( 21 ) := 33617 ;
V_ARRAY_SONG_IDS( 22 ) := 33630 ;
V_ARRAY_SONG_IDS( 23 ) := 33632 ;
V_ARRAY_SONG_IDS( 24 ) := 33636 ;
V_ARRAY_SONG_IDS( 25 ) := 33638 ;
V_ARRAY_SONG_IDS( 26 ) := 33640 ;
V_ARRAY_SONG_IDS( 27 ) := 33642 ;
V_ARRAY_SONG_IDS( 28 ) := 33644 ;
V_ARRAY_SONG_IDS( 29 ) := 33646 ;
V_ARRAY_SONG_IDS( 30 ) := 33648 ;
V_ARRAY_SONG_IDS( 31 ) := 33662 ;
V_ARRAY_SONG_IDS( 32 ) := 33667 ;
V_ARRAY_SONG_IDS( 33 ) := 33669 ;
V_ARRAY_SONG_IDS( 34 ) := 33671 ;
V_ARRAY_SONG_IDS( 35 ) := 33673 ;
V_ARRAY_SONG_IDS( 36 ) := 33675 ;
V_ARRAY_SONG_IDS( 37 ) := 33677 ;
V_ARRAY_SONG_IDS( 38 ) := 33679 ;
V_ARRAY_SONG_IDS( 39 ) := 33681 ;
V_ARRAY_SONG_IDS( 40 ) := 33683 ;
V_ARRAY_SONG_IDS( 41 ) := 33685 ;
V_ARRAY_SONG_IDS( 42 ) := 33700 ;
V_ARRAY_SONG_IDS( 43 ) := 33702 ;
V_ARRAY_SONG_IDS( 44 ) := 33704 ;
V_ARRAY_SONG_IDS( 45 ) := 33706 ;
V_ARRAY_SONG_IDS( 46 ) := 33708 ;
V_ARRAY_SONG_IDS( 47 ) := 33710 ;
V_ARRAY_SONG_IDS( 48 ) := 33712 ;
V_ARRAY_SONG_IDS( 49 ) := 33723 ;
V_ARRAY_SONG_IDS( 50 ) := 33725 ;
V_ARRAY_SONG_IDS( 51 ) := 33727 ;
V_ARRAY_SONG_IDS( 52 ) := 33729 ;
V_ARRAY_SONG_IDS( 53 ) := 33731 ;
V_ARRAY_SONG_IDS( 54 ) := 33733 ;
V_ARRAY_SONG_IDS( 55 ) := 33735 ;
V_ARRAY_SONG_IDS( 56 ) := 33737 ;
V_ARRAY_SONG_IDS( 57 ) := 33749 ;
V_ARRAY_SONG_IDS( 58 ) := 33751 ;
V_ARRAY_SONG_IDS( 59 ) := 33753 ;
V_ARRAY_SONG_IDS( 60 ) := 33755 ;
V_ARRAY_SONG_IDS( 61 ) := 33757 ;
V_ARRAY_SONG_IDS( 62 ) := 33759 ;
V_ARRAY_SONG_IDS( 63 ) := 33761 ;
V_ARRAY_SONG_IDS( 64 ) := 33763 ;
V_ARRAY_SONG_IDS( 65 ) := 33775 ;
V_ARRAY_SONG_IDS( 66 ) := 33777 ;
V_ARRAY_SONG_IDS( 67 ) := 33779 ;
V_ARRAY_SONG_IDS( 68 ) := 33781 ;
V_ARRAY_SONG_IDS( 69 ) := 33783 ;
V_ARRAY_SONG_IDS( 70 ) := 33785 ;
V_ARRAY_SONG_IDS( 71 ) := 33787 ;
V_ARRAY_SONG_IDS( 72 ) := 33789 ;
V_ARRAY_SONG_IDS( 73 ) := 33791 ;
V_ARRAY_SONG_IDS( 74 ) := 33793 ;
V_ARRAY_SONG_IDS( 75 ) := 33807 ;
V_ARRAY_SONG_IDS( 76 ) := 33809 ;
V_ARRAY_SONG_IDS( 77 ) := 33811 ;
V_ARRAY_SONG_IDS( 78 ) := 33813 ;
V_ARRAY_SONG_IDS( 79 ) := 33815 ;
V_ARRAY_SONG_IDS( 80 ) := 33817 ;
V_ARRAY_SONG_IDS( 81 ) := 33819 ;
V_ARRAY_SONG_IDS( 82 ) := 33821 ;
V_ARRAY_SONG_IDS( 83 ) := 33823 ;
V_ARRAY_SONG_IDS( 84 ) := 33825 ;
V_ARRAY_SONG_IDS( 85 ) := 33839 ;
V_ARRAY_SONG_IDS( 86 ) := 33844 ;
V_ARRAY_SONG_IDS( 87 ) := 33846 ;
V_ARRAY_SONG_IDS( 88 ) := 33848 ;
V_ARRAY_SONG_IDS( 89 ) := 33850 ;
V_ARRAY_SONG_IDS( 90 ) := 33852 ;
V_ARRAY_SONG_IDS( 91 ) := 33854 ;
V_ARRAY_SONG_IDS( 92 ) := 33856 ;
V_ARRAY_SONG_IDS( 93 ) := 33858 ;
V_ARRAY_SONG_IDS( 94 ) := 33860 ;
V_ARRAY_SONG_IDS( 95 ) := 33874 ;
V_ARRAY_SONG_IDS( 96 ) := 33879 ;
V_ARRAY_SONG_IDS( 97 ) := 33881 ;
V_ARRAY_SONG_IDS( 98 ) := 33883 ;
V_ARRAY_SONG_IDS( 99 ) := 33885 ;
V_ARRAY_SONG_IDS(100 ) := 33889 ;
/* do stuff with data from Song and Pricing Database coming from the two
separate cursors, then continue processing more rows...
FOR i IN C_get_music_price_data( v_array_song_ids ) LOOP
. (this is the loop where I pass in v_array_song_ids
. populated with only 100 cells and it runs forever)
END LOOP;
END; -
How to improve performance using bulk collects with plsql tables or arrays
Hi All,
my procedure is like this
declare
cursor c1 is select ----------------------
begin
assigning to variables
validations on that variables
--50 validations are here --
insert into a table
end;
we have created indexes on primary keys,
i want to use
DECLARE
CURSOR a_cur IS
SELECT program_id
FROM airplanes;
TYPE myarray IS TABLE OF a_cur%ROWTYPE;
cur_array myarray;
BEGIN
OPEN a_cur;
LOOP
FETCH a_cur BULK COLLECT INTO cur_array LIMIT 100;
***---------can i assign cursor data to the plsql table variables or array***
***validate on the pl sql variable as---***
i
nsert into a table
EXIT WHEN a_cur%NOTFOUND;
END LOOP;
CLOSE a_cur;
END;
Edited by: Veekay on Oct 21, 2011 4:28 AMFastest way often is this:
insert /*+append */
into aTable
select * from airplanes;
commit;The select and insert part can even be done in parallel if needed.
However if the oparation is complex or the dataset is very very very very very large or the programmer is decent but not excellent then the bulk approach should be considered. It is often a pretty stable and linear scaling approach.
The solution depends a little on the database version.
LOOP
FETCH a_cur BULK COLLECT INTO cur_array LIMIT 100;
EXIT WHEN a_cur.count = 0;
forall i in a_cur.first.. a_cur.last
insert into aTable (id)
values (a_cur(i));
END LOOP;
...If you have more then one column then you might need a single collection for each column. Other possibilities depend on the db version.
Also: do not exit using a_cur%NOTFOUND. This is wrong! You might loose records from the end of the data set. -
Use of FOR Cursor and BULK COLLECT INTO
Dear all,
in which case we prefer to use FOR cursor and cursor with BULK COLLECT INTO? The following contains two block that query identically where one is using FOR cursor, the other is using BULK COLLECT INTO . Which one that performs better given in the existing task? How do we measure performance between these two?
I'm using sample HR schema:
declare
l_start number;
BEGIN
l_start:= DBMS_UTILITY.get_time;
dbms_lock.sleep(1);
FOR employee IN (SELECT e.last_name, j.job_title FROM employees e,jobs j
where e.job_id=j.job_id and e.job_id LIKE '%CLERK%' AND e.manager_id > 120 ORDER BY e.last_name)
LOOP
DBMS_OUTPUT.PUT_LINE ('Name = ' || employee.last_name || ', Job = ' || employee.job_title);
END LOOP;
DBMS_OUTPUT.put_line('total time: ' || to_char(DBMS_UTILITY.get_time - l_start) || ' hsecs');
END;
declare
l_start number;
type rec_type is table of varchar2(20);
name_rec rec_type;
job_rec rec_type;
begin
l_start:= DBMS_UTILITY.get_time;
dbms_lock.sleep(1);
SELECT e.last_name, j.job_title bulk collect into name_rec,job_rec FROM employees e,jobs j
where e.job_id=j.job_id and e.job_id LIKE '%CLERK%' AND e.manager_id > 120 ORDER BY e.last_name;
for j in name_rec.first..name_rec.last loop
DBMS_OUTPUT.PUT_LINE ('Name = ' || name_rec(j) || ', Job = ' || job_rec(j));
END LOOP;
DBMS_OUTPUT.put_line('total time: ' || to_char(DBMS_UTILITY.get_time - l_start) || ' hsecs');
end;
/In this code, I put timestamp in each block, but they are useless since they both run virtually instantaneous...
Best regards,
ValIf you want to get 100% benifit of bulk collect then it must be implemented as below
declare
Cursor cur_emp
is
SELECT e.last_name, j.job_title
FROM employees e,jobs j
where e.job_id=j.job_id
and e.job_id LIKE '%CLERK%'
AND e.manager_id > 120
ORDER BY e.last_name;
l_start number;
type rec_type is table of varchar2(20);
name_rec rec_type;
job_rec rec_type;
begin
l_start:= DBMS_UTILITY.get_time;
dbms_lock.sleep(1);
/*SELECT e.last_name, j.job_title bulk collect into name_rec,job_rec FROM employees e,jobs j
where e.job_id=j.job_id and e.job_id LIKE '%CLERK%' AND e.manager_id > 120 ORDER BY e.last_name;
OPEN cur_emp;
LOOP
FETCH cur_emp BULK COLLECT INTO name_rec LIMIT 100;
EXIT WHEN name_rec.COUNT=0;
FOR j in 1..name_rec.COUNT
LOOP
DBMS_OUTPUT.PUT_LINE ('Name = ' || name_rec(j) || ', Job = ' || job_rec(j));
END LOOP;
EXIT WHEN cur_emp%NOTFOUND;
END LOOP;
CLOSE cur_emp;
DBMS_OUTPUT.put_line('total time: ' || to_char(DBMS_UTILITY.get_time - l_start) || ' hsecs');
end;
/ -
I want to share the link and post the:
Tell Congress:
It’s Time to Stop Mass Surveillance
Mass surveillance threatens people’s privacy and security online.
Tell Congress to stop bulk collection of our personal information.
there is no easy way to post this or forward without sending the entire page link and nobody will touch those if something like a preview doesn't show up, just looks like a bunch of unsafe links to most people, you need to have a share button on this if you really want us to share itHeres the link to "Take action" http://mzl.la/1CgTnKg. In the future if you want to copy links, right click the address bar, and the whole link should be selected, then click "Copy" in the menu that comes up.
Have a great day!!
Jon -
Is there a way to BULK COLLECT with FOR UPDATE and not lock ALL the rows?
Currently, we fetch a cursor on a few million rows using BULK COLLECT.
In a FORALL loop, we update the rows.
What is happening now, is that we run this procedure at the same time, and there is another session running a MERGE statement on the same table, and a DEADLOCK is created between them.
I'd like to add to the cursor the FOR UPDATE clause, but from what i've read,
it seems that this will cause ALL the rows in the cursor to become locked.
This is a problem, as the other session is running MERGE statements on the table every few seconds, and I don't want it to fail with ORA-0054 (resource busy).
What I would like to know is if there is a way, that only the rows in the
current bulk will be locked, and all the other rows will be free for updates.
To reproduce this problem:
1. Create test table:
create table TEST_TAB
ID1 VARCHAR2(20),
ID2 VARCHAR2(30),
LAST_MODIFIED DATE
2. Add rows to test table:
insert into TEST_TAB (ID1, ID2, LAST_MODIFIED)
values ('416208000770698', '336015000385349', to_date('15-11-2009 07:14:56', 'dd-mm-yyyy hh24:mi:ss'));
insert into TEST_TAB (ID1, ID2, LAST_MODIFIED)
values ('208104922058401', '336015000385349', to_date('15-11-2009 07:11:15', 'dd-mm-yyyy hh24:mi:ss'));
insert into TEST_TAB (ID1, ID2, LAST_MODIFIED)
values ('208104000385349', '336015000385349', to_date('15-11-2009 07:15:13', 'dd-mm-yyyy hh24:mi:ss'));
3. Create test procedure:
CREATE OR REPLACE PROCEDURE TEST_PROC IS
TYPE id1_typ is table of TEST_TAB.ID1%TYPE;
TYPE id2_typ is table of TEST_TAB.ID2%TYPE;
id1_arr id1_typ;
id2_arr id2_typ;
CURSOR My_Crs IS
SELECT ID1, ID2
FROM TEST_TAB
WHERE ID2 = '336015000385349'
FOR UPDATE;
BEGIN
OPEN My_Crs;
LOOP
FETCH My_Crs bulk collect
INTO id1_arr, id2_arr LIMIT 1;
Forall i in 1 .. id1_arr.COUNT
UPDATE TEST_TAB
SET LAST_MODIFIED = SYSDATE
where ID2 = id2_arr(i)
and ID1 = id1_arr(i);
dbms_lock.sleep(15);
EXIT WHEN My_Crs%NOTFOUND;
END LOOP;
CLOSE My_Crs;
COMMIT;
EXCEPTION
WHEN OTHERS THEN
RAISE_APPLICATION_ERROR(-20000,
'Test Update ' || SQLCODE || ' ' || SQLERRM);
END TEST_PROC;
4. Create another procedure to check if table rows are locked:
create or replace procedure check_record_locked(p_id in TEST_TAB.ID1%type) is
cursor c is
select 'dummy'
from TEST_TAB
WHERE ID2 = '336015000385349'
and ID1 = p_id
for update nowait;
e_resource_busy exception;
pragma exception_init(e_resource_busy, -54);
begin
open c;
close c;
dbms_output.put_line('Record ' || to_char(p_id) || ' is not locked.');
rollback;
exception
when e_resource_busy then
dbms_output.put_line('Record ' || to_char(p_id) || ' is locked.');
end check_record_locked;
5. in one session, run the procedure TEST_PROC.
6. While it's running, in another session, run this block:
begin
check_record_locked('208104922058401');
check_record_locked('416208000770698');
check_record_locked('208104000385349');
end;
7. you will see that all records are identified as locked.
Is there a way that only 1 row will be locked, and the other 2 will be unlocked?
Thanks,
Yoni.I don't have database access on weekends (look at it as a template)
suppose you
create table help_iot
(bucket number,
id1 varchar2(20),
constraint help_iot_pk primary key (bucket,id1)
organization index;not very sure about the create table syntax above.
declare
maximal_bucket number := 10000; -- will update few hundred rows at a time if you must update few million rows
the_sysdate date := sysdate;
begin
truncate table help_iot;
insert into help_iot
select ntile(maximal_bucket) over (order by id1) bucket,id1
from test_tab
where id2 = '336015000385349';
for i in 1 .. maximal_bucket
loop
select id1,id2,last_modified
from test_tab
where id2 = '336015000385349'
and id1 in (select id1
from help_iot
where bucket = i
for update of last_modified;
update test_tab
set last_modified = the_sysdate
where id2 = '336015000385349'
and id1 in (select id1
from help_iot
where bucket = i
commit;
dbms_lock.sleep(15);
end loop;
end;Regards
Etbin
introduced the_sysdate if last_modified must be the same for all updated rows
Edited by: Etbin on 29.11.2009 16:48 -
How can I fill a table of objects from cursor with select * bulk collect???
Hi All, I have a TYPE as OBJECT
create or replace type dept2_o as object (
deptno NUMBER(2),
dname VARCHAR2(14),
loc VARCHAR2(13));
I can fill a table of objects from cursor with out select * bulk collect...., row by row
declare
TYPE dept2_t IS TABLE of dept2_o;
dept_o_tab dept2_t:=dept2_t();
i integer;
begin
i:=0;
dept_o_tab.extend(20);
for rec in (select * from dept) loop
i:=i+1;
dept_o_tab(i):=dept2_o(
deptno => rec.deptno,
dname => rec.dname,
loc =>rec.loc
end loop;
for k IN 1..i loop
dbms_output.put_line(dept_o_tab(k).deptno||' '||dept_o_tab(k).dname||' '||dept_o_tab(k).loc);
end loop;
end;
RESULT
10 ACCOUNTING NEW YORK
20 RESEARCH DALLAS
30 SALES CHICAGO
40 OPERATIONS BOSTON
But I can't fill a table of objects from cursor with select * bulk collect construction ...
declare
TYPE dept2_t IS TABLE of dept2_o;
dept_o_tab dept2_t:=dept2_t();
begin
dept_o_tab.extend(20);
select * bulk collect into dept_o_tab from dept;
end;
RESULT
ORA-06550: line 6, column 39;
PL/SQL: ORA-00947: not enough values ....
How can I fill a table of objects from cursor with select * bulk collect???create or replace type dept_ot as object (
deptno NUMBER(2),
dname VARCHAR2(14),
loc VARCHAR2(13));
create table dept
(deptno number
,dname varchar2(14)
,loc varchar2(13)
insert into dept values (10, 'x', 'xx');
insert into dept values (20, 'y', 'yy');
insert into dept values (30, 'z', 'zz');
select dept_ot (deptno, dname, loc)
from dept
create type dept_nt is table of dept_ot
declare
l_depts dept_nt;
begin
select dept_ot (deptno, dname, loc)
bulk collect
into l_depts
from dept
for i in l_depts.first .. l_depts.last
loop
dbms_output.put_line (l_depts(i).deptno);
dbms_output.put_line (l_depts(i).dname);
dbms_output.put_line (l_depts(i).loc);
end loop;
end;
/ -
Error while doing Bulk Collect to a table type
I'm using a Table type to accumulate resultset from a loop and finally return the records in the table type as a ref cursor to the front end.
But when I'm using Bult collect to insert into the table type object it keeps throwing an error
'PLS-00597: expression 'TAB_CALENDAR_AVAIL_RESULTSET' in the INTO list is of wrong type'. Can someone help me to let me know what could be the reason for this error. I'm not able to proceed further, please help.
Here is the code.
CREATE OR REPLACE PACKAGE hotel
AS
TYPE calendar_cursor IS REF CURSOR;
TYPE type_calendar_avail is RECORD(
HOTEL_ID AVAILABILITY_CALENDAR.hotel_id%TYPE,--varchar2(4), --AVAILABILITY_CALENDAR.hotel_id%TYPE,
AVAIL_DATE AVAILABILITY_CALENDAR.AVAIL_DATE%TYPE ,
TOTAL_COUNT number
TYPE type_calendar_avail_resultset IS TABLE OF type_calendar_avail;
tab_calendar_avail_resultset type_calendar_avail_resultset ; -- declare variable of type type_calendar_avail_resultset
PROCEDURE sp_get_calendar_results (
sallhotelswithavaildate VARCHAR2,
ilengthofstay NUMBER,
sorcowner VARCHAR2,
all_unittypes VARCHAR2, --DBMS_SQL.VARCHAR2S
calendar_resultset OUT calendar_cursor
-- tab_calendar_avail_resultset out type_calendar_avail_resultset
PROCEDURE sp_get_calendar_results (
sallhotelswithavaildate VARCHAR2,
ilengthofstay NUMBER,
-- ivariant NUMBER,
sorcowner VARCHAR2,
all_unittypes VARCHAR2, --DBMS_SQL.VARCHAR2S
calendar_resultset OUT calendar_cursor
AS
sbuf VARCHAR2 (200);
sepr VARCHAR2 (1);
shotelwithdate VARCHAR2 (200);
shotelid VARCHAR2 (10);
savaildate VARCHAR2 (8);
sactualavaildate VARCHAR2 (8);
pos NUMBER;
istart NUMBER;
sstartdate VARCHAR2 (8);
senddate VARCHAR2 (8);
squery VARCHAR2 (32767) := '';
sunittypecond VARCHAR2 (500) := '';
sunitdesccond VARCHAR2 (500) := '';
v_unit_cond a_unit_cond;
tempunitcond VARCHAR2 (50) := '';
BEGIN
istart := 1;
LOOP
tempunitcond := hotel.stringtokenizer (all_unittypes, istart, '|');
IF tempunitcond IS NOT NULL
THEN
v_unit_cond (istart) := tempunitcond;
istart := istart + 1;
END IF;
EXIT WHEN tempunitcond IS NULL;
END LOOP;
sunitdesccond := hotel.get_unit_description_cond (v_unit_cond);
DBMS_OUTPUT.put_line ('unit description : ' || sunitdesccond);
sbuf := sallhotelswithavaildate;
sepr := '|';
istart := 1;
LOOP
shotelwithdate := hotel.stringtokenizer (sbuf, istart, sepr);
EXIT WHEN shotelwithdate IS NULL;
shotelid :=
SUBSTR (shotelwithdate, 1, INSTR (shotelwithdate, ',') - 1);
savaildate :=
SUBSTR (shotelwithdate, INSTR (shotelwithdate, ',') + 1);
squery :=
' SELECT MIN (ad.avail_date) '
|| ' FROM wvo_fonres.fpavail_daily ad'
|| ' WHERE ad.hotel_id = '
|| shotelid
|| ' AND ad.days_left >= '
|| ilengthofstay
|| ' AND ad.avail_date >= '
|| savaildate;
IF UPPER (sorcowner) = 'N'
THEN
squery :=
squery
|| ' AND ad.ORC_TYPE != ''R'' and ad.ORC_TYPE != ''P'' and ad.ORC_TYPE != ''E'' ';
END IF;
squery := squery || ' AND ( ' || sunitdesccond || ') ';
EXECUTE IMMEDIATE squery
INTO sactualavaildate;
DBMS_OUTPUT.put_line ('Actual available Date: ' || sactualavaildate);
hotel.sp_get_startdate_enddate (sactualavaildate,
--ivariant,
sstartdate,
senddate
sunittypecond := hotel.get_unittype_cond (v_unit_cond, sorcowner);
-- execute immediate
squery :=
'select HOTEL_ID, AVAIL_DATE, ' || sunittypecond || ' AS TOTAL_COUNT '
|| ' FROM AVAILABILITY_CALENDAR A '
|| 'WHERE '
|| 'AVAIL_DATE >= '''
|| sstartdate
|| ''' '
|| 'AND '
|| 'AVAIL_DATE <= '''
|| senddate
|| ''' '
||'AND '
|| 'A.HOTEL_ID IN ('
|| shotelid
|| ') '
|| 'AND ('
|| sunittypecond
|| '> 0) '
|| -- where total available count of unit type is greater than 0
' ORDER BY AVAIL_DATE'; --order clause
open calendar_resultset for squery;
fetch calendar_resultset BULK COLLECT INTO tab_calendar_avail_resultset;
istart := istart + 1;
END LOOP;
COMMIT;
EXCEPTION
WHEN NO_DATA_FOUND
THEN
NULL;
WHEN OTHERS
THEN
DBMS_OUTPUT.put_line
(SQLERRM (SQLCODE));
RAISE;
END sp_get_calendar_results;
END hotel;
/1. put tags [co[/b][b]de] and [co[/b][b]de] around your code, so it's readable
B. what does "hotel.get_unittype_cond (v_unit_cond, sorcowner)" actually retun?
and third, try this for the array declaration:
tab_calendar_avail_resultset type_calendar_avail_resultset := type_calendar_avail_resultset ; () ; -
ORA-01722: invalid number error with Bulk collect
Hi ,
I have been using the script to delete old seasonal data from my application DB tables. The stored procedure has been created successfully but when i try to run the proc it has been throwing 'ORA-01722: invalid number' exception at line 'FETCH C1_CUR BULK COLLECT INTO C1_TYPE_VAR LIMIT v_bulklimit;'.
Could you please help me here?
Below is the stored proc:
CREATE OR REPLACE PROCEDURE clean_old_season_data(P_SEASON VARCHAR2) AS
CURSOR C1_CUR IS SELECT ROWID RID,pro.* FROM PROPS pro where pro.ITEMPK IN
(SELECT sve.pk FROM SAVEDVALUEENTRY sve WHERE sve.p_parent IN
(SELECT s.pk FROM SAVEDVALUES s WHERE s.P_MODIFIEDITEM IN
(SELECT a.PK
FROM products a
WHERE a.p_season IN (select s.pk from Seasons s where s.P_code=P_SEASON)
) ) ) and rownum<5;
CURSOR C2_DEL IS SELECT RID FROM PROPS_HISTORY;
TYPE C1_TYPE IS TABLE OF C1_CUR%ROWTYPE;
C1_TYPE_VAR C1_TYPE;
TYPE C2_TYPE IS TABLE OF UROWID;
C2_TYPE_VAR C2_TYPE;
ex_dml_errors EXCEPTION;
PRAGMA EXCEPTION_INIT(ex_dml_errors, -24381);
l_error_count NUMBER;
err_num NUMBER;
err_msg VARCHAR2 (300);
COMMIT_VARIABLE PLS_INTEGER:=0;
v_bulklimit NUMBER:=2;
BEGIN
/*------------------ Data Selection and INSERTION IN HISTORY TABLE ---------------------------------------*/
OPEN C1_CUR;
LOOP
DBMS_OUTPUT.put_line('Cursor opend now in loop');
FETCH C1_CUR BULK COLLECT INTO C1_TYPE_VAR LIMIT v_bulklimit;//ERROR OCCURS HERE
DBMS_OUTPUT.put_line('Cursor count is'|| C1_TYPE_VAR.COUNT);
FORALL I IN 1..C1_TYPE_VAR.COUNT SAVE EXCEPTIONS
INSERT INTO PROPS_HISTORY VALUES C1_TYPE_VAR(I);
COMMIT_VARIABLE := COMMIT_VARIABLE + v_bulklimit;
DBMS_OUTPUT.put_line('Commit variable'|| COMMIT_VARIABLE.COUNT);
IF COMMIT_VARIABLE = v_bulklimit THEN
COMMIT;
COMMIT_VARIABLE := 0;
END IF;
EXIT WHEN C1_CUR%NOTFOUND;
END LOOP;
DBMS_OUTPUT.put_line('Cursor closed now in loop and data inserted in history table');
CLOSE C1_CUR;
/*------------------ Data Selection and DELETION IN Live TABLE ---------------------------------------*/
COMMIT_VARIABLE := 0;
OPEN C2_DEL;
LOOP
FETCH C2_DEL BULK COLLECT INTO C2_TYPE_VAR LIMIT 2;
FORALL I IN 1..C2_TYPE_VAR.COUNT SAVE EXCEPTIONS
DELETE FROM PROPS WHERE ROWID = C2_TYPE_VAR(I);
COMMIT_VARIABLE := COMMIT_VARIABLE + 2;
IF COMMIT_VARIABLE = 2 THEN
COMMIT;
COMMIT_VARIABLE := 0;
END IF;
EXIT WHEN C2_DEL%NOTFOUND;
END LOOP;
CLOSE C2_DEL;
END;Although there are many things which should not have been done in the posted code, I could not find any reason why the Invalid number error should occur at the Fetch clause.
I would suggest you to Insert into Table by providing the Order of Columns i.e. Insert into table (col1, ... colN) values (coll(i).col1...col(i).colN);
I tested below code and it did not give any errors.
drop table test_table;
create table test_Table
rid varchar2(100),
emp_id number(5),
fname varchar2(20),
lname varchar2(50)
set serveroutput on;
declare
cursor c_cur is
select rowid rid, e.*
from employees e
where rownum < 10;
type typ_cur is table of c_cur%rowtype;
typ typ_cur;
l_bulk_limit number := 5;
begin
open c_cur;
loop
fetch c_cur bulk collect into typ limit l_bulk_limit;
dbms_output.put_line('Collection Count :: ' || typ.count);
forall i in 1..typ.count --typ.first..typ.last
insert into test_Table (rid, emp_id, fname, lname) values (typ(i).rid,typ(i).employee_id,typ(i).first_name,typ(i).last_name);
dbms_output.put_line('Processed ' || l_bulk_limit || ' records.');
exit when c_cur%notfound;
end loop;
commit;
end;
select * from test_table;PS:- 1. When you are processing only 4 Records, then why are you breaking them in 2 Loops?
2. Why Commit every time you are processing a DML? Why not maintain an Error Flag and Rollback the Transaction as soon as error is encountered?
3. Use "{code}" (Exclude Double Quotes) to format the code. I am not sure if works.
Regards,
P. -
Hi,
below procedure is throwing error like "invalid sql". i doubt problem is with insert statement inside the body of the procedure, please help me how to load table with individual columns in bulk collect
CREATE OR REPLACE PROCEDURE subscriber_load
IS
TYPE r_subscriber_data IS RECORD
( acct_no LPDADMIN.SUBSCRIBER.acct_no%TYPE,
acct_status_cd LPDADMIN.SUBSCRIBER.acct_status_cd%TYPE,
connect_date LPDADMIN.SUBSCRIBER.connect_date%TYPE,
disconnect_date LPDADMIN.SUBSCRIBER.disconnect_date%TYPE,
bill_salutation_cd LPDADMIN.SUBSCRIBER.bill_salutation_cd%TYPE,
bill_first_name LPDADMIN.SUBSCRIBER.bill_first_name%TYPE,
bill_last_name LPDADMIN.SUBSCRIBER.bill_last_name%TYPE,
bill_addr_1 LPDADMIN.SUBSCRIBER.bill_addr_1%TYPE,
bill_addr_2 LPDADMIN.SUBSCRIBER.bill_addr_2%TYPE,
bill_city LPDADMIN.SUBSCRIBER.bill_city%TYPE,
bill_postal_code LPDADMIN.SUBSCRIBER.bill_postal_code%TYPE,
bill_province_cd LPDADMIN.SUBSCRIBER.bill_province_cd%TYPE,
bill_cycle_day LPDADMIN.SUBSCRIBER.bill_cycle_day%TYPE,
home_phone LPDADMIN.SUBSCRIBER.home_phone%TYPE,
business_phone LPDADMIN.SUBSCRIBER.business_phone%TYPE,
first_name LPDADMIN.SUBSCRIBER.first_name%TYPE,
last_name LPDADMIN.SUBSCRIBER.last_name%TYPE,
home_phone LPDADMIN.SUBSCRIBER.home_phone%TYPE,
delql_status_cd LPDADMIN.SUBSCRIBER.delql_status_cd%TYPE,
service_address_1 LPDADMIN.SUBSCRIBER.service_address_1%TYPE,
service_address_2 LPDADMIN.SUBSCRIBER.service_address_2%TYPE,
service_city LPDADMIN.SUBSCRIBER.service_city%TYPE,
service_province_cd LPDADMIN.SUBSCRIBER.service_province_cd%TYPE
TYPE t_subscriber_data IS TABLE OF r_subscriber_data INDEX BY BINARY_INTEGER;
table_subscriber_data t_subscriber_data;
CURSOR c_subscriber_data IS
SELECT s.acct_no
,lb.lob_status_cd
,s.connect_date
,s.disconnect_date
,s.bill_to_salutation_cd
,substr(s.bill_to_name,instr(bill_to_name,',',1)+1) bill_first_name
,substr(s.bill_to_name,1,instr(bill_to_name,',',1)-1) bill_last_name
,s.bill_to_addr_1
,s.bill_to_addr_2
,s.bill_to_city
,s.bill_to_postal_code
,s.bill_to_province_cd
,s.bill_create_day_of_month
,s.home_phone
,s.business_phone
,substr(s.name,instr(bill_to_name,',',1)+1) first_name
,substr(s.name,1,instr(bill_to_name,',',1)-1) last_name
,s.home_phone
,s.delq_status_cd
,h.addr_1
,h.addr_2
,h.city
,h.province_cd
FROM ccsadmin.subscriber s
inner join CCSADMIN.HOUSE h
on s.HOUSE_KEY = h.HOUSE_NO
left outer join CCSADMIN.LINE_OF_BUSINESS lb
on h.HOUSE_NO = lb.HOUSE_NO
WHERE lb.LOB_TYPE_CD ='C'
AND lb.HOUSE_NO IS NOT NULL;
BEGIN
OPEN c_subscriber_data;
LOOP
FETCH c_subscriber_data BULK COLLECT INTO table_subscriber_data LIMIT 100000;
EXIT WHEN table_subscriber_data.COUNT = 0;
if table_subscriber_data.COUNT>0 then
FOR idx IN table_subscriber_data.first..table_subscriber_data.last loop
INSERT INTO LPDADMIN.SUBSCRIBER(acct_no
,acct_status_cd
,connect_date
,disconnect_date
,bill_salutation_cd
,bill_first_name
,bill_last_name
,bill_addr_1
,bill_addr_2
,bill_city
,bill_postal_code
,bill_province_cd
,bill_cycle_day
,home_phone
,business_phone
,first_name
,last_name
,home_phone
,delql_status_cd
,service_address_1
,service_address_2
,service_city
,service_province_cd
VALUES (table_subscriber_data(idx).acct_no
,table_subscriber_data(idx).acct_status_cd
,table_subscriber_data(idx).connect_date
,table_subscriber_data(idx).disconnect_date
,table_subscriber_data(idx).bill_salutation_cd
,table_subscriber_data(idx).bill_first_name
,table_subscriber_data(idx).bill_last_name
,table_subscriber_data(idx).bill_addr_1
,table_subscriber_data(idx).bill_addr_2
,table_subscriber_data(idx).bill_city
,table_subscriber_data(idx).bill_postal_code
,table_subscriber_data(idx).bill_province_cd
,table_subscriber_data(idx).bill_cycle_day
,table_subscriber_data(idx).home_phone
,table_subscriber_data(idx).business_phone
,table_subscriber_data(idx).first_name
,table_subscriber_data(idx).last_name
,table_subscriber_data(idx).home_phone
,table_subscriber_data(idx).delql_status_cd
,table_subscriber_data(idx).service_address_1
,table_subscriber_data(idx).service_address_2
,table_subscriber_data(idx).service_city
,table_subscriber_data(idx).service_province_cd);
END LOOP;
end if;
END subscriber_load;
/no PL/SQL required
INSERT INTO LPDADMIN.SUBSCRIBER(acct_no
,acct_status_cd
,connect_date
,disconnect_date
,bill_salutation_cd
,bill_first_name
,bill_last_name
,bill_addr_1
,bill_addr_2
,bill_city
,bill_postal_code
,bill_province_cd
,bill_cycle_day
,home_phone
,business_phone
,first_name
,last_name
,home_phone
,delql_status_cd
,service_address_1
,service_address_2
,service_city
,service_province_cd
) SELECT .....
Maybe you are looking for
-
Reading items from a list view in SharePoint 2013
I am trying to read items from a Calendar, and have created a view that contains all the items/events that I would like to retrieve to package in a CSV file, specifically the Title, Description, and the Start and End Dates. The problem with some even
-
Mid 2010 iMac Wifi signal strength issue
I am having issues maintaining a robust Wifi signal on my 21.5 inch 2010 iMac with 10.6.5 installed. my symptoms are very similar to what is described in this post: http://discussions.apple.com/thread.jspa?threadID=2596933&tstart=0 in addition to tha
-
Satellite A300-1N8 - Need 64bit drivers
Hello, fellow "Toshibers". I got a Satellite A300-1N8 two days ago and I've installed Windows Vista Ultimate x64 on it, but guess what - my graphics card and my network adapter (and several other things as well) would not start. It is important to sa
-
Problems with pattern matching Regular expression
String emailRegEx = "(\\w+)@(\\w+\\.)(\\w+)(\\.\\w+)*"; String test = "[email protected]"; if(test.matches(emailRegEx)) { }throws an exception java.util.regex.PatternSyntaxException: Illegal repetition near index 5.
-
PRODUCT NO.: LG660ET#ABZ UTILIZING A USB HP V210W 16GB WHEN THE NOTEBOOK IS ON APPEAR A BLUE SCREEN WITH A CRASH DUMP. IF WE INTRODUCE THE USB KEY WHEN THE NOTEBOOK IS OFF AND THEN WE START THE NOTEBOOK EVERYTHING GOES WELL. THE USB KEY NORMALLY WORK