Opening two cursors using open cursor with bulk collect on colections ..
Is it possible to have the implementatiion of using bulk collect with collections using two open cursors ..
first c1
second c2
open c1
loop
open c2
loop
end loop
close c2
end loop;
close c1
what i found is for every outer loop of cursor c1 , cursor c2 is open and closed for every record.
is this willl imporove the performace .?
EXAMPLE:-
NOTE: The relatoin between finc and minc is one to many ..finc is parent and minc is child
function chk_notnull_blank ( colname IN number ) return number is
BEGIN
if ( colname is NOT NULL and colname not in ( -8E14, -7E14, -6E14, -5E14, -4E14, -3E14, -2E14, -1E14, -1E9 )) then
RETURN colname ;
else
RETURN 0;
end if;
END chk_notnull_blank;
procedure Proc_AnnualFmlyTotIncSummary is
CURSOR c_cur_finc IS SELECT FAMID FROM FINC ;
CURSOR c_cur_minc IS SELECT FAMID, MEMBNO , ANFEDTX, ANGOVRTX, ANPRVPNX, ANRRDEDX, ANSLTX, SALARYX, SALARYBX, NONFARMX, NONFRMBX , FARMINCX, FRMINCBX, RRRETIRX, RRRETRBX, SOCRRX, INDRETX, JSSDEDX, SSIX, SSIBX from MINC minc WHERE FAMID IN ( SELECT FAMID FROM FINC finc WHERE minc.FAMID = finc.FAMID );
v_tot_fsalaryx number := 0;
v_tot_fnonfrmx number := 0;
v_tot_ffrmincx number := 0;
v_tot_frretirx number := 0;
v_tot_findretx number := 0;
v_tot_fjssdedx number := 0;
v_tot_fssix number := 0;
v_temp_sum_fsalaryx number := 0;
v_temp_sum_fnonfrmx number := 0;
v_temp_sum_ffrmincx number := 0;
v_temp_sum_frretirx number := 0;
v_temp_sum_findretx number := 0;
v_temp_sum_fjssdedx number := 0;
v_temp_sum_fssix number := 0;
TYPE minc_rec IS RECORD (FAMID MINC.FAMID%TYPE, MEMBNO MINC.MEMBNO%TYPE , ANFEDTX MINC.ANFEDTX%TYPE, ANGOVRTX MINC.ANGOVRTX%TYPE , ANPRVPNX MINC.ANPRVPNX%TYPE , ANRRDEDX MINC.ANRRDEDX%TYPE , ANSLTX MINC.ANSLTX%TYPE, SALARYX MINC.SALARYX%TYPE , SALARYBX MINC.SALARYBX%TYPE , NONFARMX MINC.NONFARMX%TYPE , NONFRMBX MINC.NONFRMBX%TYPE, FARMINCX MINC.FARMINCX%TYPE , FRMINCBX MINC.FRMINCBX%TYPE , RRRETIRX MINC.RRRETIRX%TYPE , RRRETRBX MINC.RRRETRBX%TYPE, SOCRRX MINC.SOCRRX%TYPE , INDRETX MINC.INDRETX%TYPE , JSSDEDX MINC.JSSDEDX%TYPE , SSIX MINC.SSIX%TYPE , SSIBX MINC.SSIBX%TYPE );
v_flag_boolean boolean := false;
v_famid number ;
v_stmt varchar2(3200) ;
v_limit number := 50;
v_temp_FAMTFEDX number := 0 ;
v_temp_FGOVRETX number := 0 ;
v_temp_FPRIVPENX number := 0 ;
v_temp_FRRDEDX number := 0 ;
v_temp_FSLTAXX number := 0 ;
v_temp_FSALARYX number := 0 ;
v_temp_FNONFRMX number := 0 ;
v_temp_FFRMINCX number := 0 ;
v_temp_FRRETIRX number := 0 ;
v_temp_FINDRETX number := 0 ;
v_temp_FJSSDEDX number := 0 ;
v_temp_FSSIX number := 0 ;
BEGIN
OPEN c_cur_finc ;
LOOP
FETCH c_cur_finc BULK COLLECT INTO famid_type_tbl LIMIT v_limit;
EXIT WHEN famid_type_tbl.COUNT = 0;
FOR i in famid_type_tbl.FIRST..famid_type_tbl.LAST
LOOP
OPEN c_cur_minc ;
LOOP
FETCH c_cur_minc BULK COLLECT INTO minc_rec_type_tbl LIMIT v_limit;
EXIT WHEN minc_rec_type_tbl.COUNT = 0;
FOR j IN minc_rec_type_tbl.FIRST..minc_rec_type_tbl.LAST
LOOP
if ( famid_type_tbl(i) = minc_rec_type_tbl(j).FAMID ) THEN
v_temp_FAMTFEDX := v_temp_FAMTFEDX + chk_notnull_blank(minc_rec_type_tbl(j).ANFEDTX );
v_temp_FGOVRETX := v_temp_FGOVRETX + chk_notnull_blank(minc_rec_type_tbl(j).ANGOVRTX);
v_temp_FPRIPENX := v_temp_FPRIPENX + chk_notnull_blank(minc_rec_type_tbl(j).ANPRVPNX);
v_temp_FRRDEDX := v_temp_FRRDEDX + chk_notnull_blank(minc_rec_type_tbl(j).ANRRDEDX);
v_temp_FSLTAXX := v_temp_FSLTAXX + chk_notnull_blank(minc_rec_type_tbl(j).ANSLTX );
v_temp_FSALARYX := v_temp_FSALARYX + chk_notnull_blank(minc_rec_type_tbl(j).SALARYX ) + chk_notnull_blank(minc_rec_type_tbl(j).SALARYBX);
v_temp_FNONFRMX := v_temp_FNONFRMX + chk_notnull_blank(minc_rec_type_tbl(j).NONFARMX) + chk_notnull_blank(minc_rec_type_tbl(j).NONFRMBX);
v_temp_FFRMINCX := v_temp_FFRMINCX + chk_notnull_blank(minc_rec_type_tbl(j).FARMINCX) + chk_notnull_blank(minc_rec_type_tbl(j).FRMINCBX );
v_temp_FRRETIRX := v_temp_FRRETIRX + chk_notnull_blank(minc_rec_type_tbl(j).RRRETIRX) + chk_notnull_blank(minc_rec_type_tbl(j).RRRETRBX ) + chk_notnull_blank(minc_rec_type_tbl(j).SOCRRX);
v_temp_FINDREXT := v_temp_FINDRETX + chk_notnull_blank(minc_rec_type_tbl(j).INDRETX);
v_temp_FJSSDEDX := v_temp_FJSSDEDX + chk_notnull_blank(minc_rec_type_tbl(j).JSSDEDX);
v_temp_FSSIX := v_temp_FSSIX + chk_notnull_blank(minc_rec_type_tbl(j).SSIX ) + chk_notnull_blank(minc_rec_type_tbl(j).SSIBX);
END IF;
END LOOP;
END LOOP ;
CLOSE c_cur_minc;
UPDATE FINC SET FAMTFEDX = v_temp_FAMTFEDX WHERE FAMID = famid_type_tbl(i);
END LOOP;
END LOOP;
CLOSE c_cur_finc;
END;
EXCEPTION
WHEN OTHERS THEN
raise_application_error(-20001,'An error was encountered - '||SQLCODE||' -ERROR- '||SQLERRM);
v_err_code := SQLCODE;
v_err_msg := substr(SQLERRM, 1, 200);
INSERT INTO audit_table (error_number, error_message) VALUES (v_err_code, v_err_msg);
error_logging(p_error_code => substr(sqlerrm,1,9), p_error_message => substr(sqlerrm,12), p_package =>'PKG_FCI_APP',p_procedure => 'Proc_Annual_Deductions_FromPay ' , p_location => v_location);
end Proc_AnnualFmlyTotIncSummary ;
Is the proga efficient and free from compilation errors ..?
thanks/kumar
Edited by: kumar73 on Sep 22, 2010 12:48 PM
function chk_notnull_blank ( colname IN number ) return number is Maybe this function should have its own forum:
how to use case in this program
Re: how to declare a formal parameter in a function of type record and access ?
Re: how to define a function with table type parameter
Re: creation of db trigger with error ..
Re: How to write a trigger for the below scenario
how to improve the code using advanced methods
yours advice in improving the coding ..
How to use bulk in multiple cursors !!
;-)
Similar Messages
-
Adobe Reader Splash--I have a PDF file that whenever I try to open it suggests using Adobe Reader with a link to download Adobe Reader. I can't get beyond this. I have made Preview the default for all PDF's and yet this persists. I have also deleted Adobe Reader. This particular file is a portfolio, btw.
I have similar issue. Did u find an answer?
-
How can I have a single cursor using elements 9 with Windows 7?
Currently the cursor on my screen is 3 small boxes in place of the single arrow. I am using Photoshop Elements 9 With Windows 7. I have loaded the patch files currently in the Adobe system without any success.
Thank-you that solved the issue.
In a message dated 4/20/2014 8:57:10 P.M. Eastern Daylight Time,
[email protected] writes:
Re: How can I have a single cursor using elements 9 with Windows 7?
created by Barbara B. (http://forums.adobe.com/people/Barbara+B.) in
Photoshop Elements - View the full discussion
(http://forums.adobe.com/message/6315096#6315096) -
How to use table type in bulk collect
Hi experts,
How to use table type in bulk collect see the procedure used( oracle 10g)
and error is
PLS-00597: expression 'REQ_REC' in the INTO list is of wrong type
CREATE OR REPLACE PROCEDURE SAMPLE_SP IS
TYPE TYP_A AS OBJECT
( COLMN1 TABLE1.COLM1%TYPE,
COLMN2 TABLE1.COLM2%TYPE,
COLMN3 TABLE1.COLM3%TYPE
TYPE REC_A IS TABLE OF TYP_A;
REQ_REC A_REC;
CURSOR REQ_CUR IS SELECT COLM1,COLM2,COLM3 FROM TABLE1 WHERE <CONDITION>;
BEGIN
OPEN REQ_REC;
LOOP
EXIT WHEN REQ_REC%NOTFOUND;
FETCH REQ_REC BULK COLLECT INTO REQ_REC LIMIT 1000;
FOR I IN 1..REQ_REC.COUNT
LOOP
<insert statement>
END LOOP;
COMMIT;
END LOOP;
END SAMPLE_SP;
Many thanks,
Kalingaok but that is not an issue..
Hi experts,
How to use table type in bulk collect see the procedure used( oracle 10g)
and error is
PLS-00597: expression 'REQ_REC' in the INTO list is of wrong type
CREATE OR REPLACE PROCEDURE SAMPLE_SP IS
TYPE TYP_A AS OBJECT
( COLMN1 TABLE1.COLM1%TYPE,
COLMN2 TABLE1.COLM2%TYPE,
COLMN3 TABLE1.COLM3%TYPE
TYPE REC_A IS TABLE OF TYP_A;
REQ_REC A_REC;
CURSOR REQ_CUR IS SELECT COLM1,COLM2,COLM3 FROM TABLE1 WHERE <CONDITION>;
BEGIN
OPEN REQ_CUR;
LOOP
EXIT WHEN REQ_REC%NOTFOUND;
FETCH REQ_REC BULK COLLECT INTO REQ_REC LIMIT 1000;
FOR I IN 1..REQ_REC.COUNT
LOOP
<insert statement>
END LOOP;
COMMIT;
END LOOP;
END SAMPLE_SP;
Many thanks,
Kalinga
Message was edited by:
Kalinga -
Cursor with bulk collect ...
Hi experts,
CURSOR ZIP_CUR IS SELECT * FROM ZIP_MASTER WHERE ..(SUBQUERY)
OPEN ZIP_CUR;
LOOP
EXIT WHEN C1%NOTFOUND;
FETCH ZIP_CUR BULK COLLECT INTO ZIP_REC LIMIT 1000;
FOR I IN ZIP_REC.FIRST..ZIP_REC.LAST
LOOP
END LOOP
END LOOP
If there is no data in cusor then the looping after bulk collect is throughing VALUE_ERROR
but why EXIT WHEN C1%NOTFOUND not terminating procedure??
now i am using the same select statement in cursor to find the data and raise user defined exception..
is there any other way to handle in execption??
Many thanks,
Kalinga..You can try something like this -
satyaki>
satyaki>declare
2 CURSOR c_emp
3 IS
4 SELECT *
5 FROM emp_s
6 WHERE deptno in (
7 SELECT deptno
8 from emp_s
9 where job = 'ANALYST'
10 );
11
12 type emp_tt is table of emp_s%rowtype index by pls_integer;
13
14 emp_rec emp_tt;
15 begin
16
17 OPEN c_emp;
18 LOOP
19 EXIT WHEN c_emp%NOTFOUND;
20 FETCH c_emp BULK COLLECT INTO emp_rec LIMIT 1000;
21 FOR I IN 1..emp_rec.COUNT
22 LOOP
23 dbms_output.put_line('Empno: '||emp_rec(I).empno);
24 dbms_output.put_line('Ename: '||emp_rec(I).ename);
25 dbms_output.put_line('Mgr: '||emp_rec(I).mgr);
26 END LOOP;
27 END LOOP;
28 exception
29 when others then
30 dbms_output.put_line(sqlerrm);
31 end;
32 /
Empno: 7902
Ename: FORD
Mgr: 7566
Empno: 7839
Ename: KING
Mgr: 7839
Empno: 7788
Ename: SCOTT
Mgr: 7566
Empno: 7698
Ename: BLAKE
Mgr: 7839
PL/SQL procedure successfully completed.
satyaki>Regards.
Satyaki De. -
Trouble with Bulk Collect to Ref cursor
I'm trying to open a ref cursor to a dynamic query, and the fetch the cursor (BULK COLLECT)to the table type variable.But I keep getting the compilation error as 'PLS-00597: expression 'EMP_RESULTSET' in the INTO list is of wrong type'
But when I use a simple select from a table and Bulk Collect directly to the table type variable it works. But that is not what I want.
Can someone tell me where I have gone wrong in this stored proc I have listed below.
your help will be highly appreciated.
PROCEDURE SP_TEST_EMP_TABLE_TYPE (
p_resultset OUT result_cursor -- ref cursor as out parameter
AS
TYPE TYPE_REC_EMP is RECORD(EMPNO EMPLOYEE.EMPNO%TYPE, JOIN_DATE EMPLOYEE.JOIN_DATE%TYPE, SALARY EMPLOYEE.SALARY%TYPE); -- declare record type
TYPE TYPE_TAB_EMP IS TABLE OF TYPE_REC_EMP; -- declare table type
EMP_RESULTSET TYPE_TAB_EMP; -- declare variable of type type_calendar_avail_resultset
SQUERY VARCHAR2(32000):='';
BEGIN
SQUERY:='SELECT EMPNO,JOIN_DATE,SALARY FROM EMPLOYEE WHERE EMPNO= 1001 AND JOIN_DATE=''20070530'' ';
--select EMPNO,JOIN_DATE,SALARY BULK COLLECT INTO EMP_RESULTSET from EMPLOYEE WHERE EMPNO=1001 AND JOIN_DATE='20070525';
OPEN p_resultset FOR SQUERY;
FETCH p_resultset BULK COLLECT INTO EMP_RESULTSET;
EXCEPTION
WHEN NO_DATA_FOUND
THEN
NULL;
WHEN OTHERS
THEN
DBMS_OUTPUT.put_line (SQLERRM (SQLCODE));
END SP_TEST_EMP_TABLE_TYPE ;> i) I use a ref cursor to return back to the java
front end, so I had to use a ref cursor.
What is a ref cursor? It is not a data set. It is a pointer to a "SQL program". This program is created by the SQL Engine and the CBO that parses the SQL source code and determine an execution plan to get to the requested rows.
When the client fetches from a (ref) cursor, the client is running this program and it find and returns the next row.
There is no such thing as a physical result set for a cursor - no "copy" of the rows found for the source code SQL is made as a result set.
> ii) I also use a dynamic sql, but I was thinking it
wasn't useful for this posting, so tried to write a
simple sql
What is dynamic SQL? SQL where object names (e.g name of the table) is only known at run-time. Or where the filter (e.g. WHERE conditions) can only be determined at run time.
If these are known, the SQL is static SQL.
For both static and dynamic SQL, bind variables are critical. It is the biggest performance mistake (in Oracle) to hardcode values and literals into a SQL.
> ii) I use a Bulk Collect to the table type
collection, since I use a loop, for which I had to
collect the results from each loop and finally send
the resultset thru the ref cursor.
Impossible. Nor does it make any sense. As stated, a ref cursor is a program and not a result set.
What you intend to do is run a SQL against data. Copy this data into private/local PL/SQL memory. Construct another SQL against this data - which means that it needs to be shipped back to the SQL engine as it cannot use/read local PL/SQL memory structures. And the pass that SQL cursor to the client.
What for?
> I had earlier used the logic to for this using a
temporary table, which works perfectly fine, but our
DBA says we should avoid temporary tables since it
makes additional read/write to the disk. This is the
reason I'm using table type collection here.
Your DBA is correct. One should so a single pass through a data set. Which is why simply passing a ref cursor for a SQL statement to the client is the simplest.
It makes no sense copying SQL data into a collection and then copying that back into the SQL engine in order to throw a ref cursor around it.
Basic client-server fundamentals.
And why RTFM the Oracle manuals I've indicated is important. You need to understand the memory and processing architectures in Oracle in order to make an informed and correct decision. -
Yosemite tries to open another photoshop using "open with"
Hi there,
I upgraded to Yosemite a few days ago. When I right click on a jpeg and ask it to "open with" Photoshop it tries to open another Photoshop.
By this I mean that a second Photoshop icon appears in the dock bouncing up and down, even if I already have Photoshop open and I'm working with it. This second Icon never actually opens the
I use Photoshop a lot, and have CS6. I've gone directly from Mountain Lion to Yosemite and never had this problem before.
Any thoughts?
Many thanks.That is a problem with an AVG extension or toolbar.
Start Firefox in <u>[[Safe Mode]]</u> to check if one of the extensions (Aurora/Firefox/Tools > Add-ons > Extensions) is causing the problem (switch to the DEFAULT theme: Firefox/Tools > Add-ons > Appearance/Themes).
* Don't make any changes on the Safe mode start window.
* https://support.mozilla.com/kb/Safe+Mode
* https://support.mozilla.com/kb/Troubleshooting+extensions+and+themes -
Under File menu Open File will not open local files even those downloaded to the desktop by Firefox 5.
If I double click on HTML file it does open
Reinstalled. Same behaviour. Default Browser set correctly.
Permissions set correctly. Odd behaviour.Are you sure it hasn't imported?
Check for the track names (trackname only, not Album or Artist)) and see if they have a blank Artist and Album, because WAV files have no MetaData, so are kind of stupid as far as knowing anything about the Artist, Album & track id.
You can't search by 'Untitled' either as there is literally, no name at all, but iTunes does pick up the actual physical track name in Finder and uses that. As an example, if you renamed your Aerosmith WAV file from say 'Toys In The Attic' to 'Love Song' it would import the track as 'Love Song' as that's all it can go by.
There are a few iTunes AppleScripts that let you import from a file of WAV music, by looking at the enclosing Album in the enclosing Artist as that's how most PC progs create the WAV file structure.
Mike -
Need help with Bulk Collect ForAll Update
Hi - I'm trying to do a Bulk Collect/ForAll Update but am having issues.
My declarations look like this:
CURSOR cur_hhlds_for_update is
SELECT hsh.household_id, hsh.special_handling_type_id
FROM compas.household_special_handling hsh
, scr_id_lookup s
WHERE hsh.household_id = s.id
AND s.scr = v_scr
AND s.run_date = TRUNC (SYSDATE)
AND effective_date IS NULL
AND special_handling_type_id = 1
AND created_by != v_user;
TYPE rec_hhlds_for_update IS RECORD (
household_id HOUSEHOLD_SPECIAL_HANDLING.household_id%type,
spec_handl_type_id HOUSEHOLD_SPECIAL_HANDLING.SPECIAL_HANDLING_TYPE_ID%type
TYPE spec_handling_update_array IS TABLE OF rec_hhlds_for_update;
l_spec_handling_update_array spec_handling_update_array;And then the Bulk Collect/ForAll looks like this:
OPEN cur_hhlds_for_update;
LOOP
FETCH cur_hhlds_for_update BULK COLLECT INTO l_spec_handling_update_array LIMIT 1000;
EXIT WHEN l_spec_handling_update_array.count = 0;
FORALL i IN 1..l_spec_handling_update_array.COUNT
UPDATE compas.household_special_handling
SET effective_date = TRUNC(SYSDATE)
, last_modified_by = v_user
, last_modified_date = SYSDATE
WHERE household_id = l_spec_handling_update_array(i).household_id
AND special_handling_type_id = l_spec_handling_update_array(i).spec_handl_type_id;
l_special_handling_update_cnt := l_special_handling_update_cnt + SQL%ROWCOUNT;
END LOOP;And this is the error I'm receiving:
ORA-06550: line 262, column 31:
PLS-00436: implementation restriction: cannot reference fields of BULK In-BIND table of records
ORA-06550: line 262, column 31:
PLS-00382: expression is of wrong type
ORA-06550: line 263, column 43:
PL/SQL: ORA-22806: not an object or REF
ORA-06550: line 258, column 9:
PL/SQL: SQMy problem is that the table being updated has a composite primary key so I have two conditions in my where clause. This the the first time I'm even attempting the Bulk Collect/ForAll Update and it seems like it would be straight forward if I was only dealing with a single-column primary key. Can anyone please help advise me as to what I'm missing here or how I can accomplish this?
Thanks!
ChristineYou cannot reference a column inside a record when doin a for all. You need to refer as a whole collection . So you will need two collections.
Try like this,
DECLARE
CURSOR cur_hhlds_for_update
IS
SELECT hsh.household_id, hsh.special_handling_type_id
FROM compas.household_special_handling hsh, scr_id_lookup s
WHERE hsh.household_id = s.ID
AND s.scr = v_scr
AND s.run_date = TRUNC (SYSDATE)
AND effective_date IS NULL
AND special_handling_type_id = 1
AND created_by != v_user;
TYPE arr_household_id IS TABLE OF HOUSEHOLD_SPECIAL_HANDLING.household_id%TYPE
INDEX BY BINARY_INTEGER;
TYPE arr_spec_handl_type_id IS TABLE OF HOUSEHOLD_SPECIAL_HANDLING.SPECIAL_HANDLING_TYPE_ID%TYPE
INDEX BY BINARY_INTEGER;
l_household_id_col arr_household_id;
l_spec_handl_type_id_col arr_spec_handl_type_id;
BEGIN
OPEN cur_hhlds_for_update;
LOOP
FETCH cur_hhlds_for_update
BULK COLLECT INTO l_household_id_col, l_spec_handl_type_id_col
LIMIT 1000;
EXIT WHEN cur_hhlds_for_update%NOTFOUND;
FORALL i IN l_household_id_col.FIRST .. l_household_id_col.LAST
UPDATE compas.household_special_handling
SET effective_date = TRUNC (SYSDATE),
last_modified_by = v_user,
last_modified_date = SYSDATE
WHERE household_id = l_household_id_col(i)
AND special_handling_type_id = l_spec_handl_type_id_col(i);
--l_special_handling_update_cnt := l_special_handling_update_cnt + SQL%ROWCOUNT; -- Not sure what this does.
END LOOP;
END;G. -
Problem with BULK COLLECT with million rows - Oracle 9.0.1.4
We have a requirement where are supposed to load 58 millions of rows into a FACT Table in our DATA WAREHOUSE. We initially planned to use Oracle Warehouse Builder but due to performance reasons, decided to write custom code. We wrote a custome procedure which opens a simple cursor and reads all the 58 million rows from the SOURCE Table and in a loop processes the rows and inserts the records into a TARGET Table. The logic works fine but it took 20hrs to complete the load.
We then tried to leverage the BULK COLLECT and FORALL and PARALLEL options and modified our PL/SQL code completely to reflect these. Our code looks very simple.
1. We declared PL/SQL BINARY_INDEXed Tables to store the data in memory.
2. We used BULK COLLECT into FETCH the data.
3. We used FORALL statement while inserting the data.
We did not introduce any of our transformation logic yet.
We tried with the 600,000 records first and it completed in 1 min and 29 sec with no problems. We then doubled the no. of rows to 1.2 million and the program crashed with the following error:
ERROR at line 1:
ORA-04030: out of process memory when trying to allocate 16408 bytes (koh-kghu
call ,pmucalm coll)
ORA-06512: at "VVA.BULKLOAD", line 66
ORA-06512: at line 1
We got the same error even with 1 million rows.
We do have the following configuration:
SGA - 8.2 GB
PGA
- Aggregate Target - 3GB
- Current Allocated - 439444KB (439 MB)
- Maximum allocated - 2695753 KB (2.6 GB)
Temp Table Space - 60.9 GB (Total)
- 20 GB (Available approximately)
I think we do have more than enough memory to process the 1 million rows!!
Also, some times the same program results in the following error:
SQL> exec bulkload
BEGIN bulkload; END;
ERROR at line 1:
ORA-03113: end-of-file on communication channel
We did not even attempt the full load. Also, we are not using the PARALLEL option yet.
Are we hitting any bug here? Or PL/SQL is not capable of mass loads? I would appreciate any thoughts on this?
Thanks,
Haranadh
Following is the code:
set echo off
set timing on
create or replace procedure bulkload as
-- SOURCE --
TYPE src_cpd_dt IS TABLE OF ima_ama_acct.cpd_dt%TYPE;
TYPE src_acqr_ctry_cd IS TABLE OF ima_ama_acct.acqr_ctry_cd%TYPE;
TYPE src_acqr_pcr_ctry_cd IS TABLE OF ima_ama_acct.acqr_pcr_ctry_cd%TYPE;
TYPE src_issr_bin IS TABLE OF ima_ama_acct.issr_bin%TYPE;
TYPE src_mrch_locn_ref_id IS TABLE OF ima_ama_acct.mrch_locn_ref_id%TYPE;
TYPE src_ntwrk_id IS TABLE OF ima_ama_acct.ntwrk_id%TYPE;
TYPE src_stip_advc_cd IS TABLE OF ima_ama_acct.stip_advc_cd%TYPE;
TYPE src_authn_resp_cd IS TABLE OF ima_ama_acct.authn_resp_cd%TYPE;
TYPE src_authn_actvy_cd IS TABLE OF ima_ama_acct.authn_actvy_cd%TYPE;
TYPE src_resp_tm_id IS TABLE OF ima_ama_acct.resp_tm_id%TYPE;
TYPE src_mrch_ref_id IS TABLE OF ima_ama_acct.mrch_ref_id%TYPE;
TYPE src_issr_pcr IS TABLE OF ima_ama_acct.issr_pcr%TYPE;
TYPE src_issr_ctry_cd IS TABLE OF ima_ama_acct.issr_ctry_cd%TYPE;
TYPE src_acct_num IS TABLE OF ima_ama_acct.acct_num%TYPE;
TYPE src_tran_cnt IS TABLE OF ima_ama_acct.tran_cnt%TYPE;
TYPE src_usd_tran_amt IS TABLE OF ima_ama_acct.usd_tran_amt%TYPE;
src_cpd_dt_array src_cpd_dt;
src_acqr_ctry_cd_array src_acqr_ctry_cd;
src_acqr_pcr_ctry_cd_array src_acqr_pcr_ctry_cd;
src_issr_bin_array src_issr_bin;
src_mrch_locn_ref_id_array src_mrch_locn_ref_id;
src_ntwrk_id_array src_ntwrk_id;
src_stip_advc_cd_array src_stip_advc_cd;
src_authn_resp_cd_array src_authn_resp_cd;
src_authn_actvy_cd_array src_authn_actvy_cd;
src_resp_tm_id_array src_resp_tm_id;
src_mrch_ref_id_array src_mrch_ref_id;
src_issr_pcr_array src_issr_pcr;
src_issr_ctry_cd_array src_issr_ctry_cd;
src_acct_num_array src_acct_num;
src_tran_cnt_array src_tran_cnt;
src_usd_tran_amt_array src_usd_tran_amt;
j number := 1;
CURSOR c1 IS
SELECT
cpd_dt,
acqr_ctry_cd ,
acqr_pcr_ctry_cd,
issr_bin,
mrch_locn_ref_id,
ntwrk_id,
stip_advc_cd,
authn_resp_cd,
authn_actvy_cd,
resp_tm_id,
mrch_ref_id,
issr_pcr,
issr_ctry_cd,
acct_num,
tran_cnt,
usd_tran_amt
FROM ima_ama_acct ima_ama_acct
ORDER BY issr_bin;
BEGIN
OPEN c1;
FETCH c1 bulk collect into
src_cpd_dt_array ,
src_acqr_ctry_cd_array ,
src_acqr_pcr_ctry_cd_array,
src_issr_bin_array ,
src_mrch_locn_ref_id_array,
src_ntwrk_id_array ,
src_stip_advc_cd_array ,
src_authn_resp_cd_array ,
src_authn_actvy_cd_array ,
src_resp_tm_id_array ,
src_mrch_ref_id_array ,
src_issr_pcr_array ,
src_issr_ctry_cd_array ,
src_acct_num_array ,
src_tran_cnt_array ,
src_usd_tran_amt_array ;
CLOSE C1;
FORALL j in 1 .. src_cpd_dt_array.count
INSERT INTO ima_dly_acct (
CPD_DT,
ACQR_CTRY_CD,
ACQR_TIER_CD,
ACQR_PCR_CTRY_CD,
ACQR_PCR_TIER_CD,
ISSR_BIN,
OWNR_BUS_ID,
USER_BUS_ID,
MRCH_LOCN_REF_ID,
NTWRK_ID,
STIP_ADVC_CD,
AUTHN_RESP_CD,
AUTHN_ACTVY_CD,
RESP_TM_ID,
PROD_REF_ID,
MRCH_REF_ID,
ISSR_PCR,
ISSR_CTRY_CD,
ACCT_NUM,
TRAN_CNT,
USD_TRAN_AMT)
VALUES (
src_cpd_dt_array(j),
src_acqr_ctry_cd_array(j),
null,
src_acqr_pcr_ctry_cd_array(j),
null,
src_issr_bin_array(j),
null,
null,
src_mrch_locn_ref_id_array(j),
src_ntwrk_id_array(j),
src_stip_advc_cd_array(j),
src_authn_resp_cd_array(j),
src_authn_actvy_cd_array(j),
src_resp_tm_id_array(j),
null,
src_mrch_ref_id_array(j),
src_issr_pcr_array(j),
src_issr_ctry_cd_array(j),
src_acct_num_array(j),
src_tran_cnt_array(j),
src_usd_tran_amt_array(j));
COMMIT;
END bulkload;
SHOW ERRORS
-----------------------------------------------------------------------------do you have a unique key available in the rows you are fetching?
It seems a cursor with 20 million rows that is as wide as all the columnsyou want to work with is a lot of memory for the server to use at once. You may be able to do this with parallel processing (dop over 8) and a lot of memory for the warehouse box (and the box you are extracting data from)...but is this the most efficient (and thereby fastest) way to do it?
What if you used a cursor to select a unique key only, and then during the cursor loop fetch each record, transform it, and insert it into the target?
Its a different way to do a lot at once, but it cuts down on the overall memory overhead for the process.
I know this isnt as elegant as a single insert to do it all at once, but sometimes trimming a process down so it takes less resources at any given moment is much faster than trying to do the whole thing at once.
My solution is probably biased by transaction systems, so I would be interested in what the data warehouse community thinks of this.
For example:
source table my_transactions (tx_seq_id number, tx_fact1 varchar2(10), tx_fact2 varchar2(20), tx_fact3 number, ...)
select a cursor of tx_seq_id only (even at 20 million rows this is not much)
you could then either use a for loop or even bulk collect into a plsql collection or table
then process individually like this:
procedure process_a_tx(p_tx_seq_id in number)
is
rTX my_transactions%rowtype;
begin
select * into rTX from my_transactions where tx_seq_id = p_tx_seq_id;
--modify values as needed
insert into my_target(a, b, c) values (rtx.fact_1, rtx.fact2, rtx.fact3);
commit;
exception
when others
rollback;
--write to a log or raise and exception
end process_a_tx;
procedure collect_tx
is
cursor tx is
select tx_seq_id from my_transactions;
begin
for rTx in cTx loop
process_a_tx(rtx.tx_seq_id);
end loop;
end collect_tx; -
ORA-01722: invalid number error with Bulk collect
Hi ,
I have been using the script to delete old seasonal data from my application DB tables. The stored procedure has been created successfully but when i try to run the proc it has been throwing 'ORA-01722: invalid number' exception at line 'FETCH C1_CUR BULK COLLECT INTO C1_TYPE_VAR LIMIT v_bulklimit;'.
Could you please help me here?
Below is the stored proc:
CREATE OR REPLACE PROCEDURE clean_old_season_data(P_SEASON VARCHAR2) AS
CURSOR C1_CUR IS SELECT ROWID RID,pro.* FROM PROPS pro where pro.ITEMPK IN
(SELECT sve.pk FROM SAVEDVALUEENTRY sve WHERE sve.p_parent IN
(SELECT s.pk FROM SAVEDVALUES s WHERE s.P_MODIFIEDITEM IN
(SELECT a.PK
FROM products a
WHERE a.p_season IN (select s.pk from Seasons s where s.P_code=P_SEASON)
) ) ) and rownum<5;
CURSOR C2_DEL IS SELECT RID FROM PROPS_HISTORY;
TYPE C1_TYPE IS TABLE OF C1_CUR%ROWTYPE;
C1_TYPE_VAR C1_TYPE;
TYPE C2_TYPE IS TABLE OF UROWID;
C2_TYPE_VAR C2_TYPE;
ex_dml_errors EXCEPTION;
PRAGMA EXCEPTION_INIT(ex_dml_errors, -24381);
l_error_count NUMBER;
err_num NUMBER;
err_msg VARCHAR2 (300);
COMMIT_VARIABLE PLS_INTEGER:=0;
v_bulklimit NUMBER:=2;
BEGIN
/*------------------ Data Selection and INSERTION IN HISTORY TABLE ---------------------------------------*/
OPEN C1_CUR;
LOOP
DBMS_OUTPUT.put_line('Cursor opend now in loop');
FETCH C1_CUR BULK COLLECT INTO C1_TYPE_VAR LIMIT v_bulklimit;//ERROR OCCURS HERE
DBMS_OUTPUT.put_line('Cursor count is'|| C1_TYPE_VAR.COUNT);
FORALL I IN 1..C1_TYPE_VAR.COUNT SAVE EXCEPTIONS
INSERT INTO PROPS_HISTORY VALUES C1_TYPE_VAR(I);
COMMIT_VARIABLE := COMMIT_VARIABLE + v_bulklimit;
DBMS_OUTPUT.put_line('Commit variable'|| COMMIT_VARIABLE.COUNT);
IF COMMIT_VARIABLE = v_bulklimit THEN
COMMIT;
COMMIT_VARIABLE := 0;
END IF;
EXIT WHEN C1_CUR%NOTFOUND;
END LOOP;
DBMS_OUTPUT.put_line('Cursor closed now in loop and data inserted in history table');
CLOSE C1_CUR;
/*------------------ Data Selection and DELETION IN Live TABLE ---------------------------------------*/
COMMIT_VARIABLE := 0;
OPEN C2_DEL;
LOOP
FETCH C2_DEL BULK COLLECT INTO C2_TYPE_VAR LIMIT 2;
FORALL I IN 1..C2_TYPE_VAR.COUNT SAVE EXCEPTIONS
DELETE FROM PROPS WHERE ROWID = C2_TYPE_VAR(I);
COMMIT_VARIABLE := COMMIT_VARIABLE + 2;
IF COMMIT_VARIABLE = 2 THEN
COMMIT;
COMMIT_VARIABLE := 0;
END IF;
EXIT WHEN C2_DEL%NOTFOUND;
END LOOP;
CLOSE C2_DEL;
END;Although there are many things which should not have been done in the posted code, I could not find any reason why the Invalid number error should occur at the Fetch clause.
I would suggest you to Insert into Table by providing the Order of Columns i.e. Insert into table (col1, ... colN) values (coll(i).col1...col(i).colN);
I tested below code and it did not give any errors.
drop table test_table;
create table test_Table
rid varchar2(100),
emp_id number(5),
fname varchar2(20),
lname varchar2(50)
set serveroutput on;
declare
cursor c_cur is
select rowid rid, e.*
from employees e
where rownum < 10;
type typ_cur is table of c_cur%rowtype;
typ typ_cur;
l_bulk_limit number := 5;
begin
open c_cur;
loop
fetch c_cur bulk collect into typ limit l_bulk_limit;
dbms_output.put_line('Collection Count :: ' || typ.count);
forall i in 1..typ.count --typ.first..typ.last
insert into test_Table (rid, emp_id, fname, lname) values (typ(i).rid,typ(i).employee_id,typ(i).first_name,typ(i).last_name);
dbms_output.put_line('Processed ' || l_bulk_limit || ' records.');
exit when c_cur%notfound;
end loop;
commit;
end;
select * from test_table;PS:- 1. When you are processing only 4 Records, then why are you breaking them in 2 Loops?
2. Why Commit every time you are processing a DML? Why not maintain an Error Flag and Rollback the Transaction as soon as error is encountered?
3. Use "{code}" (Exclude Double Quotes) to format the code. I am not sure if works.
Regards,
P. -
Problem with Bulk Collect ... FORALL
I've written following code to bulk collect records from a cursor into a collection and insert it into a table using FORALL loop
OPEN x;
LOOP
FETCH x BULK COLLECT INTO v_collection LIMIT 1000;
FORALL i IN 1..v_collection.count
INSERT INTO tablename(column1, column2) VALUES(v_collection(i).val1, v_collection(i).val2);
COMMIT;
EXIT WHEN x%NOTFOUND;
END LOOP;I have verified that query which is executed by cursor returns records but when my procedure is executed the insert statement inside the FORALL loop never executes. Am I missing something over here?
Regards,
FahadYes, the cursor is returning a row.
I've found the solution myself. There was a trigger which was deleting data on commit. Due to this, the records were not inserting. -
Forall with bulk collect .. getting error
it's 10 g.
gettting this error.
drop table t2;
create table t2
( seq_id number,
act number,
is_p varchar2(1),
other varchar2(20)
insert into t2 values(1,2,'N','Test 1');
drop table t3;
create table t3
( seq_id number
-- ,act number
-- ,is_p varchar2(1)
set serveroutput on
declare
type t2_r is table of t2%ROWTYPE;
cursor c1 is select * from t2 ;
t t2_r;
begin
open c1;
fetch c1 BULK collect into t;
forall i in 1..t.count
insert into t3(seq_id) values (t(i).seq_id);
end;
ORA-06550: line 10, column 35:
PLS-00436: implementation restriction: cannot reference fields of BULK In-BIND table of records
ORA-06550: line 10, column 35:
PLS-00382: expression is of wrong type
ORA-06550: line 10, column 35:
PL/SQL: ORA-22806: not an object or REF
ORA-06550: line 10, column 4:
PL/SQL: SQL Statement ignoredThe PLS-00436 is an annoying error message indeed.
On 10g:
rwijk@ORA10GR2> create table t2
2 ( seq_id number,
3 act number,
4 is_p varchar2(1),
5 other varchar2(20)
6 );
Tabel is aangemaakt.
rwijk@ORA10GR2> insert into t2 values(1,2,'N','Test 1');
1 rij is aangemaakt.
rwijk@ORA10GR2> create table t3
2 ( seq_id number
3 -- ,act number
4 -- ,is_p varchar2(1)
5 );
Tabel is aangemaakt.
rwijk@ORA10GR2> declare
2 type t2_r is table of t2%ROWTYPE;
3 cursor c1 is select * from t2 ;
4 t t2_r;
5 begin
6 open c1;
7 fetch c1 BULK collect into t;
8 forall i in 1..t.count
9 insert into t3(seq_id) values (t(i).seq_id);
10 close c1;
11 end;
12 /
insert into t3(seq_id) values (t(i).seq_id);
FOUT in regel 9:
.ORA-06550: Regel 9, kolom 35:
PLS-00436: Implementatierestrictie: kan niet verwijzen naar velden van BULK In-BIND-recordtabel..
ORA-06550: Regel 9, kolom 35:
PLS-00382: Uitdrukking heeft onjuist type..
ORA-06550: Regel 9, kolom 35:
PL/SQL: ORA-22806: Geen object of REF..
ORA-06550: Regel 9, kolom 4:
PL/SQL: SQL Statement ignored.You can code it slightly different using SQL object types like this, to make the code work in 10g:
rwijk@ORA10GR2> create type t2_otype is object
2 ( seq_id number
3 , act number
4 , is_p varchar2(1)
5 , other varchar2(20)
6 );
7 /
Type is aangemaakt.
rwijk@ORA10GR2> create type t2s is table of t2_otype;
2 /
Type is aangemaakt.
rwijk@ORA10GR2> declare
2 cursor c1 is select t2_otype(seq_id,act,is_p,other) from t2 ;
3 t t2s;
4 begin
5 open c1;
6 fetch c1 BULK collect into t;
7 forall i in 1..t.count
8 insert into t3(seq_id) values (treat(t(i) as t2_otype).seq_id);
9 close c1;
10 end;
11 /
PL/SQL-procedure is geslaagd.On 11g, as said, you don't have to modify your code:
rwijk@ORA11G> create table t2
2 ( seq_id number,
3 act number,
4 is_p varchar2(1),
5 other varchar2(20)
6 );
Tabel is aangemaakt.
rwijk@ORA11G> insert into t2 values(1,2,'N','Test 1');
1 rij is aangemaakt.
rwijk@ORA11G> create table t3
2 ( seq_id number
3 -- ,act number
4 -- ,is_p varchar2(1)
5 );
Tabel is aangemaakt.
rwijk@ORA11G> declare
2 type t2_r is table of t2%ROWTYPE;
3 cursor c1 is select * from t2 ;
4 t t2_r;
5 begin
6 open c1;
7 fetch c1 BULK collect into t;
8 forall i in 1..t.count
9 insert into t3(seq_id) values (t(i).seq_id);
10 close c1;
11 end;
12 /
PL/SQL-procedure is geslaagd.Regards,
Rob. -
How can I see error when execute UPDATE with BULK COLLECT
Hi
I have a code
CURSOR c_renov_eleg IS
SELECT t2.ROWID
FROM mytable t2
WHERE BLA BLA BLA
OPEN c_renov_eleg;
LOOP
FETCH c_renov_eleg BULK COLLECT
INTO w_rowid LIMIT 1000;
FORALL i IN 1 .. w_rowid.COUNT
UPDATE mytable
SET id_baixa_elegibilidade = K_SCNOB_BXA_RENOVACAO,
column01 = arr_ciclo(1),
dt_ultima_atualizacao = SYSDATE,
cd_usuario_atualizacao = USER
WHERE ROWID = w_rowid(i);
COMMIT;
EXIT WHEN c_renov_eleg%NOTFOUND;
END LOOP;How can I to sse when and where have some error ? , in Exception return what record ?How can I to sse when and where have some error
?Have a look at
[url=http://download.oracle.com/docs/cd/B28359_01/appd
ev.111/b28370/tuning.htm#i49099]Handling FORALL
Exceptions (%BULK_EXCEPTIONS Attribute).btw. just never ever commit in a for loop ;)
(Tubby is just too fast ;))
Hi , thank all
btw. just never ever commit in a for loopWhen I must to COMMIT using BULK COLLECT ? -
Collection with bulk collect , statement is not executed..
DECLARE
CURSOR cur_upt IS SELECT ts.user_id, ts.lot_id, ts.ml_ac_no, ts.td_prs_dt, ts.unit_cost, ts.cost_basis
FROM tb_xop_sharelot_frac_snap fs, tb_xop_sharelot ts
WHERE fs.lot_id=ts.lot_id AND fs.user_id=ts.user_id;
TYPE tx_tab IS TABLE OF tb_xop_sharelot.user_id%TYPE;
ltab tx_tab;
TYPE tx_tab1 IS TABLE OF tb_xop_sharelot.lot_id%TYPE;
ltab1 tx_tab1;
TYPE tx_tab2 IS TABLE OF tb_xop_sharelot.ml_ac_no%TYPE;
ltab2 tx_tab2;
TYPE tx_tab3 IS TABLE OF tb_xop_sharelot.td_prs_dt%TYPE;
ltab3 tx_tab3;
TYPE tx_tab4 IS TABLE OF tb_xop_sharelot.unit_cost%TYPE;
ltab4 tx_tab4;
TYPE tx_tab5 IS TABLE OF tb_xop_sharelot.cost_basis%TYPE;
ltab5 tx_tab5;
BEGIN
INSERT INTO tb_xop_sharelot_frac_snap (lot_id, jemq_num, lot_qy, activity_type, LOT_SL_CREATE_DT,
LOT_SL_CLOSE_DT, lot_status, frac_recon, hist_flag, create_dt, user_id)
(SELECT lot_id, jemq_num, lot_qy, activity_type, LOT_SL_CREATE_DT,
LOT_SL_CLOSE_DT,lot_status, frac_recon, hist_flag, create_dt, user_id FROM tb_xop_sharelot_fraction);
OPEN Cur_upt;
LOOP
FETCH Cur_upt BULK COLLECT INTO ltab, ltab1, ltab2, ltab3, ltab4, ltab5 LIMIT 5000;
EXIT WHEN cur_upt%NOTFOUND;
END LOOP;
CLOSE cur_upt;
FORALL i IN ltab.FIRST..ltab.LAST
UPDATE tb_xop_sharelot_frac_snap SET ml_ac_no=ltab2(i)/*, td_prs_dt=ltab3(i),
unit_cost=ltab4(i), cost_basis=ltab5(i)*/ WHERE user_id=ltab(i) AND lot_id=ltab1(i);
COMMIT;
EXCEPTION
WHEN OTHERS THEN
DBMS_OUTPUT.PUT_LINE(SQLCODE|| ' ' ||SQLERRM);
END;This is the third question you have posted just putting one short subject and only posting code not formatted.
I suggest you to read SQL and PL/SQL FAQ and avoid posting your question in this way as they will be ignored.
Please:
a) post sample data
b) post your code formatted
i.e.:
{noformat}{noformat}
SELECT ...
{noformat}{noformat}
c) explain what problem you are facing in details (including oracle errors)
d) explain the logic you want to have
e) post your expected output.
Regards.
Al
Maybe you are looking for
-
Why does my new Airport Extreme accept the old password for a guest user?
I have a new N/AC Airport Extreme Time Capsule that a friend can access only by using the password for an older "G" Airport base station from a previous visit. She can't get onto my current network using the correct/current passwrod. Anybody have any
-
How to get the cursor position on a picture?
I'd like to show the cursor position on a picture which is 400 by 400 pixel size and located at (0, 0, 400, 400) on a panel. So suppose that the cursor is at (50, 50), how can the java code know? Thank you.
-
IW32 - restriction or validation to accept only internal order from a plant
HI team!! I´m needing your help Im working with Tcode IW32 when creating a maintenance order and we got through Goto / Settlement rule we can assign any internal order number. there is no restriction or validation to accept only internal order number
-
It is just fine yesterday. And I found it can't open in 20:00PM yesterday. At first, I think it need to be charging. So I charge it immediatly. Now, I still can't open it, need help!
-
When i choose embedded as isolation, the layout is not preserved. In case of URL, the iview is rendered as iframes, but layout is preserved. Is there any way in which i can preserve the layout and avoid iframes?