How to stop killing database using all of pga

Question

I have an application that is already written in java. It calls a package procedure
that has
inputs and outputs
This report kills the database and the dba noticed that it was eating all of the pga and killing the system. I am looking at the code and I see that one of the outputs is a
select ttype(col1,col2,...),tadtype(col1,col2...)
bulk collect into one of the output variables.
Just running a sample I noticed tha the bulk collect is collecting almost 30,000 rows and this ruinning in one session uses almost 250 mb of pga, when 10 users run the report the system dies. I have been told changing the java code will not happen. If there something I can do in the package to send the data back using bulk collect limit xxxx.
I tired using 1000, but the report only see the last 1000 rows. There are other output variables so I am not sure how to not "keep" all of the collection for this until I send it with the rest of the output variables. I am using 10.1.0.5. I am not sure if I can somehow write a function to pipe_row the rows backs to the client wirthout keeping the whole mess.
the psuedo code is something like this
procedure x (
pid IN s.SESSION_ID%TYPE
,piv_sort_on IN VARCHAR2
,piv_bno IN b.NO%TYPE
,piv_batch_type IN VARCHAR2
,piv_vt_cache_flag IN VARCHAR2
,potab_trans OUT TAB_TRANS
,potab_addr OUT TAB_ADDR
,pon_net_count OUT NUMBER
,pov_net_amount OUT VARCHAR2
,pov_currency_symbol OUT VARCHAR2
,pov_flag OUT TFLAG%TYPE
,pov_region_short_name OUT R.NAME%TYPE
,pov_error_msg OUT VARCHAR2
do work ...
huge bullk collect into ,potab_trans ,potab_addr -- about 20,000 - 50,000 rows want to change this
-- send output back and not keep the huge are for
-- these colllections.
more work for rest of output variable info
end
Edited by: smklad on Sep 2, 2008 3:34 PM

Answer 1

It is expected behaviour.
When running PL/SQL, you are using the PL engine - this engine makes calls to the SQL engine whenever it hits SQL in your PL/SQL code. Data has to be transferred between the PL and SQL engines. All this is pretty neat as it allows you to mix the source code of two very different languages, PL and SQL, combine it seamlessly, and treat it as if it is a single language. Makes development and maintenance and design and what not a lot better.
Each time your PL code uses SQL (opening a cursor, fetching from a cursor), that requires a context switch from the PL engine to the SQL engine. Context switching is unavoidable, but as it comes with a performance penalty needs to be reduces as far as possible.
Enter bulk collect. Instead of doing a 100 context switches in a cursor fetch loop to fetch a 100 rows, a single bulk collect can used to collect a 100 rows. 100 context switches versus 1 context switch.
Okay, but now where do that 100 rows go when bulk collecting? It has to be stored by the PL engine. The PL engine uses PGA memory. Thus it needs to allocate PGA to store these 100 rows.
If you bulk collect a 1000 rows, it needs to allocate PGA for a 1000 rows. If a million rows... ouch.
So a bulk collect decreases context switching. However, it increases the demand for PGA. It is therefore a balancing act when it comes to performance - decreasing context switching at the cost of an increase in PGA.
And this is why it is mostly mandatory that you use the LIMIT clause when bulk collecting. Bulk collect a 100 to a 1000 rows at a time. And as you are managing the number of rows that can be bulk collected per context switch, you are managing the amount of PGA to expend.
Last comment - you mention that the driving app is Java. Now I hope that this does not call PL/SQL, with PL/SQL doing a bulk collect, and then passing that collection back to Java. That is, plainly put, very stupid. Why?
The SQL engine has an excellent db buffer cache. Java can bulk fetch from it directly (one can set the array fetch size in an Oracle client driver). It makes absolutely no sense to use the PL engine as an intermediate cache - have the PL engine fetch rows from the SQL engine cache, cache those rows in (expensive) PGA memory, and then pass that to Java. It is a lot slower as it has more moving parts. It cannot scale as this design uses very expensive server memory called PGA as a cache - instead of relying on the SQL db buffer cache that was explicitly designed for this very purpose.