Performance Decreased with group by

Hi,
I am tuning a query in oracle. when I run the query without group by clause, the query runs in 5-6 sec, but after including the group by clause. The same query takes 14mins to run. If anybody has any idea of what is the impact of group by clause on performance of a SQL query, please help me . The table has 1million rows of data.

Definitely group by clause will take more time.. Do you really need group by clause in the query ?
Please check
http://asktom.oracle.com/pls/asktom/f?p=100:11:0::::p11_question_id:561712900346367451
REgards
Rajesh
Edited by: Rajesh on Jun 10, 2010 12:20 PM

Similar Messages

SQL Server 2005 performance decreases with DB size while SQL Server 2012 is fine

Hi,
We have a C# windows service running that polls some files and inserts/updates some fields in database.
The service was tested on a local dev machine with SQL Server 2012 running and performance was quite decent with any number of records. Later on the service was moved to a test stage environment where SQL Server 2005 is installed. At that point database
was still empty and service was running just fine but later on, after some 500k records were written, performance problems came to light. After some more tests we've founds out that, basically, database operation performance in SQL Server 2005 decreases with
a direct correlation with the database size. Here are some testing results:
Run#
1
2
3
4
5
DB size (records)
520k
620k
720k
820k
920k
SQL Server 2005
TotalRunTime
25:25.1
32:25.4
38:27.3
42:50.5
43:51.8
Get1
00:18.3
00:18.9
00:20.1
00:20.1
00:19.3
Get2
01:13.4
01:17.9
01:21.0
01:21.2
01:17.5
Get3
01:19.5
01:24.6
01:28.4
01:29.3
01:24.8
Count1
00:19.9
00:18.7
00:17.9
00:18.7
00:19.1
Count2
00:44.5
00:45.7
00:45.9
00:47.0
00:46.0
Count3
00:21.7
00:21.7
00:21.7
00:22.3
00:22.3
Count4
00:23.6
00:23.9
00:23.9
00:24.9
00:24.5
Process1
03:10.6
03:15.4
03:14.7
03:21.5
03:19.6
Process2
17:08.7
23:35.7
28:53.8
32:58.3
34:46.9
Count5
00:02.3
00:02.3
00:02.3
00:02.3
00:02.1
Count6
00:01.6
00:01.6
00:01.6
00:01.7
00:01.7
Count7
00:01.9
00:01.9
00:01.7
00:02.0
00:02.0
Process3
00:02.0
00:01.8
00:01.8
00:01.8
00:01.8
SQL Server 2012
TotalRunTime
12:51.6
13:38.7
13:20.4
13:38.0
12:38.8
Get1
00:21.6
00:21.7
00:20.7
00:22.7
00:21.4
Get2
01:38.3
01:37.2
01:31.6
01:39.2
01:37.3
Get3
01:41.7
01:42.1
01:35.9
01:44.5
01:41.7
Count1
00:20.3
00:19.9
00:19.9
00:21.5
00:17.3
Count2
01:04.5
01:04.8
01:05.3
01:10.0
01:01.0
Count3
00:24.5
00:24.1
00:23.7
00:26.0
00:21.7
Count4
00:26.3
00:24.6
00:25.1
00:27.5
00:23.7
Process1
03:52.3
03:57.7
03:59.4
04:21.2
03:41.4
Process2
03:05.4
03:06.2
02:53.2
03:10.3
03:06.5
Count5
00:02.8
00:02.7
00:02.6
00:02.8
00:02.7
Count6
00:02.3
00:03.0
00:02.8
00:03.4
00:02.4
Count7
00:02.5
00:02.9
00:02.8
00:03.4
00:02.5
Process3
00:21.7
00:21.0
00:20.4
00:22.8
00:21.5
One more thing is that it's not Process2 table that constantly grows in size but is Process1 table, that gets almost 100k records each run.
After that SQL Server 2005 has also been installed on a dev machine just to test things and we got exactly the same results. Both SQL Server 2005 and 2012 instances are installed using default settings with no changes at all. The same goes for databases
created for the service.
So the question is - why are there such huge differences between performance of SQL Server 2005 and 2012? Maybe there are some settings that are set by default in SQL Server 2012 database that need to be set manually in 2005?
What else can I try to test? The main problem is that production SQL Server will be updated god-knows-when and we can't just wait for that.
Any suggestions/advices are more than welcome.

...One more thing is that it's not Process2 table that constantly grows in size but is
Process1 table, that gets almost 100k records each run....
Hi,
It is not clear to me what is that you are doing, but now we have a better understanding on ONE of your tables an it is obviously you will get worse result as the data become bigger. Actually your table look like a automatic build table by ORM like Entity
Framework, and it's DDL probably do not much your needs. For example if your select query is using a filter on the other column that [setID] then you have no index and the server probably have to scan the entire table in order to find the records that you
need.
Forum is a suitable place to seek advice about a specific system (as I mentioned before we are not familiar with your system), and it is more suitable for general questions. For example the fact that you have no index except the index on the column [setID]
can indicate a problem. Ultimately to optimize the system will need to investigate it more thoroughly (as it is no longer appropriate forum ... but we're not there yet). Another point is that now we can see that you are using [timestamp] column, an this
implies that your are using this column as a filter for selecting the data. If so, then maybe a better DDL will be to use clustered index on this column and if needed a nonclustered index on the [setID] if it is needed at all...
what is obviously is that next is to check if this DDL fit
your specific needs (as i mentioned before).
Next step is to understand what action do you do with this table. (1) what is your query which become slowly in a bigger data set. (2) Are you using ORM (object relational mapping, like Entity Framework
code first), and if so then which one.
[Personal Site] [Blog] [Facebook]

Performance Decrease with ComboBoxes

Memory consumption and bad performance decrease when closing and reopening panels containing ComboBoxes.
Context:
- Running localMode
- Jdev 9.0.3.1 DE
- Business Component Version 9.0.3.10.62
- jdk 1.3.1_02
Panel (within an JInternal frame) with some JUComboBoxes with LOV-Binding to ViewObjects, only with a few entries (2-20)
We observed, that the count of JUIteratorBindings encreases with every new Panel (re)open.
After closing the Internal Frame (including the Panel) the IteratorBindings of all ComboBoxes keep alive and a set of new ones are added to the PanelBinding when opening the (same) Panel once more.
Is it a bug?
Have one to remove the IteratorBindings explicitly?
Is there any configuration possibility, that the IteratorBindings die with the Panel?
Has anyone the same problem?
Roswitha Stepputtis

Yes, we experience similar problems. My suggestion would be to simply hide (setVisible(false)) your internal frames and reuse them when you open them again.
So, instead of dynamically re-creating your JInternalFrame instances, you keep a reference to them once they are created and reuse them each time they should be made visible. I think this will make your application more efficient as a whole.

JDev10g: General performance decrease with JClients like Oracle BC Browser

Hi,
I'm observing a significant performance decrease of JClient applications (homegrown and Oracle Business Components Browser)in JDev10g compared against JDev9i (9033)?
Can anyone confirm my observation?
Just create a default Dept-Emp business components package and start the Business components browser. Scroll to some data (it takes some time...) and resize the window (takes even more time)!
My guess: JDev9i-style data binding for JClients is simulated in JDev10g with ADF-Data-Binding?? My own application is a simple "recompile"-migration of a JDev9i project with JDev10g. Does this apply to Oracle Business Components Browser too?
My own application runs still faster with JDev9i!
What is the problem? Any hints are welcome!
Thanks,
Markus

Excellent Blog. Thank You
Small clarification on Step **6) Oracle Home Directory, ...a) Resize the Root Partition**
Ubuntu 11.10 has Gparted available as a Ubuntu software download, DONT use that while trying the above step, instead download the ISO file from http://sourceforge.net/projects/gparted/files/gparted-live-stable/ gparted-live-0.12.0-5.iso (124.6 MB)
Burn that ISO file on a Blank DVD, reboot the Ubuntu , during startup select Boot from DVD Option if not already selected. this will take to Boot Menu Options of Gparted Live then select the first menu option, and this allows to do further action such as Re-sizing .
and once you have chosen and executed step a) . do NOT run step b) also that is "Setup External Storage"
I hope this minor clarification can avoid some confusion
Regards
Madhusudhan Rao
Edited by: MadhusudhanRao on Mar 24, 2012 11:30 PM

Performance decrease with xorg 1.6 and intel

i have a box with
gigabyte ga-8ig1000mk mainboard
intel 865G graphics
celeron 2.6 cpu
512 M ram
arch linux
x does not start with xorg 1.5/intel or xorg 1.6/intel (video memory related error message)
kde 4 performance with xorg 1.6/intel-legacy is noticably lower than with xorg 1.4/intel
glxgears in its default window size gives 460 fps with xorg 1.4/intel and 300 fps with xorg 1.6/intel-legacy
currently the only way to have xorg 1.4 is
IgnorePkg = xf86-video-intel
IgnorePkg = intel-dri
IgnorePkg = glproto
IgnorePkg = xcb-proto
IgnorePkg = libxcb
IgnorePkg = libx11
IgnorePkg = dri2proto
IgnorePkg = libdrm
IgnorePkg = mesa
IgnorePkg = libgl
IgnorePkg = libxdamage
IgnorePkg = damageproto
IgnorePkg = xcb-util
IgnorePkg = gtk2
IgnorePkg = pango
IgnorePkg = libthai
IgnorePkg = libxinerama
IgnorePkg = libxcursor
IgnorePkg = libxklavier
IgnorePkg = xkeyboard-config
IgnorePkg = libxi
IgnorePkg = libxext
IgnorePkg = libx11
IgnorePkg = libxcb
IgnorePkg = xcb-proto
IgnorePkg = xextproto
IgnorePkg = inputproto
IgnorePkg = cairo
IgnorePkg = libpng
IgnorePkg = pixman
IgnorePkg = xf86-input-keyboard
IgnorePkg = xf86-input-mouse
IgnorePkg = xf86-video-vesa
IgnorePkg = libfontenc
IgnorePkg = freetype2
IgnorePkg = fontsproto
IgnorePkg = libxfont
IgnorePkg = xproto
IgnorePkg = xorg-font-utils
IgnorePkg = xorg-fonts-encodings
IgnorePkg = xorg-fonts-alias
IgnorePkg = expat
IgnorePkg = fontconfig
IgnorePkg = xorg-fonts-100dpi
IgnorePkg = xorg-fonts-75dpi
IgnorePkg = libxau
IgnorePkg = libxdmcp
IgnorePkg = xcb-proto
IgnorePkg = libxcb
IgnorePkg = kbproto
IgnorePkg = inputproto
IgnorePkg = bigreqsproto
IgnorePkg = libx11
IgnorePkg = xextproto
IgnorePkg = libxext
IgnorePkg = libice
IgnorePkg = libsm
IgnorePkg = libxt
IgnorePkg = libxmu
IgnorePkg = libxpm
IgnorePkg = printproto
IgnorePkg = libxp
IgnorePkg = libxaw
IgnorePkg = xorg-res-utils
IgnorePkg = libxkbfile
IgnorePkg = libxkbui
IgnorePkg = xf86miscproto
IgnorePkg = libxxf86misc
IgnorePkg = xf86vidmodeproto
IgnorePkg = libxxf86vm
IgnorePkg = xcursor-themes
IgnorePkg = xorg-xkb-utils
IgnorePkg = xkeyboard-config
IgnorePkg = fontcacheproto
IgnorePkg = libxfontcache
IgnorePkg = libxi
IgnorePkg = liblbxutil
IgnorePkg = renderproto
IgnorePkg = libxrender
IgnorePkg = randrproto
IgnorePkg = libxrandr
IgnorePkg = trapproto
IgnorePkg = libxtrap
IgnorePkg = mcpp
IgnorePkg = xtrans
IgnorePkg = xorg-server-utils
IgnorePkg = xf86dgaproto
IgnorePkg = libxxf86dga
IgnorePkg = recordproto
IgnorePkg = libxtst
IgnorePkg = xineramaproto
IgnorePkg = libxinerama
IgnorePkg = videoproto
IgnorePkg = libxv
IgnorePkg = dmxproto
IgnorePkg = libdmx
IgnorePkg = libxft
IgnorePkg = xorg-utils
IgnorePkg = libdrm
IgnorePkg = xorg-fonts-misc
IgnorePkg = pixman
IgnorePkg = xbitmaps
IgnorePkg = xorg-server
IgnorePkg = xorg-twm
IgnorePkg = xorg-xauth
IgnorePkg = xorg-xinit
IgnorePkg = fixesproto
IgnorePkg = libxfixes
IgnorePkg = libxcursor
IgnorePkg = libpng
IgnorePkg = xorg-apps
IgnorePkg = xterm
i want a better solution
Last edited by ash (2009-04-19 22:48:49)

you do not need nearly as many packages on hold. Haven't logged into my 32-bit arch install for a month now, but I've maintained a xorg 1.4 install with this line just fine:
IgnorePkg = xf86-video-intel xf86-input-evdev xorg-server xorg-apps xorg-xkb-utils xorg-server-utils xorg-utils intel-dri synaptics libgl libdrm xorg-res-utils xf86driproto xf86-input-keyboard xkeyboard-config
That said, I expext to upgrade next time I use it (probably to play around with some fast boot). I would look into troubleshooting with the latest xorg and intel driver as it really should work, if I were you. I presume you have already tried starting x without a xorg.conf as a baseline test with those versions?

Performance issue with grouping components

Hi Guys,
I am building a dashboard in Dashboards 4.1 using Live Office connections. The initial summary view contains multiple charts, labels, customized image components, etc. which have all been grouped into one component. The user needs to able to filter the dashboard based on "Dept Name", "Employee Type" and "Month".
Now, to filter on "Dept Name", there are 5 different check boxes provided for each department inside a pie chart. Based on the selection, all the data in the dashboard will change. The way I am thinking of achieving this is by creating 5 copies of the initial grouped component and then setting dynamic visibility on each based on the check box selection. I will also change the data mapping for each copy of the grouped component.
Similarly, I am thinking about doing the same for the filter for "Employee Type" & "Month"
My question is - Is this a good method to achieve the task ? Will it cause any performance issues ?

copying the same set of components 5 or 7 times will result in a model that is slower to load and mat be slower to use. If possible try to limit the number of components to one set and move the data around the spreadsheet instead. This can be hard in some cases, and depending how you do it could also affect performance.
I have found that the more objects you copy on the canvas the more liable to corruption the file gets as well.
As always, designing a dashboard is a balance between complexity and usability.

Performance decrease with custom cell renderer

Hi,
Finally, I have my cell renderer working fine (it was simpler than I thought). Anyway, I'm suspecting something is not working quite well yet. When I try to reorder my colums (by dragging them) they move like slow motion. If I don't register my custom cell renderer, I can move my columns fine. Next is the code of my custom renderer.
class PriorityRenderer extends DefaultTableCellRenderer {
public Component getTableCellRendererComponent(
JTable table, Object value,     boolean isSelected,
boolean hasFocus, int row, int column) {
    if(value instanceof coloredValue) {
    coloredValue cv = (coloredValue)value;
    if(isSelected) this.setBackground(table.getSelectionBackground());
    else this.setBackground(cv.getbkColor());
    this.setForeground(cv.gettxColor());
    this.setText(value.toString());
    this.setOpaque(true);
    return this;
}All the cells of my JTable are "coloredValue"s. This is a little class with a String, two colors and some getter methods.
Can anyone giveme a hint ?
Thanks!!

OK! Now the performance problem is gone!! (I don't know why, I didn't touch a thing)
Thanks anyway

Performance decrease with flash animation/games

When I try to view embedded flash using firefox or safari, it seems to hog a ton of memory. It starts out at a reasonable ammount (usually ~300MB of Physical plus Virtual) but eventually it gets up to 1.39 GB (out of my 2.00 GB available). The whole system slows considerably and often crashes.
I have noticed that the same flash animations work fine on my windows based PC with only 1 GB of physical memory. I have a 256 MB graphics card along with 2 GB of memory and a fast processor. Is there a way I can pipeline the system more to use the graphics chip rather than the memory or any software that might prevent this issue?
Thank You

Make sure you have the latest versions of Adobe Flash Player Software.
Try running repair permissions from your HD and afterwards run +repair disk+ _from your software install DVD._

IPhone 3G Performance Worse with 4.2.1

Has anyone had their performance decrease with the 4.2.1 update? My phone is responding much more poorly. I can't open maps or any other application that uses location services over the cellular network. Totally bombs. Works ok with wifi, load is still slow but once loaded works ok. Also calendar and camera take even longer to load than they did before. Phone hangs a lot more also and many more calls failing.
Anyone else? I want to chuck my phone out the window. 6 months of fail.

I'm new to this but if you're using a PC (can't speak on Mac) try going to your Control Panel, click on the Add or Remove Programs icon, when the list populates scroll down to the Apple Mobile Device Support icon, click on Add/Remove (on Windows 7 its a right click on the line anywhere) a menu drops down or a window appears depending on your system, and it offers you the choice to Unintall, Change, or Repair. Choose the Repair option and it should update your system. Perhaps that's your problem. There are other options in that list to chose such as a link to click on which takes you to the webpage where help is available. I've discovered that cleaning out my Add/Remove program list to save storage space is good for my computer and also made me realize that we, the computer using public, have real power in that we can thin out that list considerably by just seeing which programs we use frequently or rarely and delete accordingly, scares the heck out of the big boys who design browsers and other useless programs that clog up our system. I just deleted Google Chrome who were so sorry to see me leave and was promptly sent a survey so that they could understand my reasons for leaving. I felt on par with them, for once and let them know that Internet Explorer 9 ripped them off. Good luck.

Performance issues with pipelined table functions

I am testing pipelined table functions to be able to re-use the base_query function. Contrary to my understanding, the with_pipeline procedure runs 6 time slower than the legacy no_pipeline procedure. Am I missing something? The processor function is from [url http://www.oracle-developer.net/display.php?id=429]improving performance with pipelined table functions .
Edit: The underlying query returns 500,000 rows in about 3 minutes. So there are are no performance issues with the query itself.
Many thanks in advance.
CREATE OR REPLACE PACKAGE pipeline_example
IS
 TYPE resultset_typ IS REF CURSOR;
 TYPE row_typ IS RECORD (colC VARCHAR2(200), colD VARCHAR2(200), colE VARCHAR2(200));
 TYPE table_typ IS TABLE OF row_typ;
 FUNCTION base_query (argA IN VARCHAR2, argB IN VARCHAR2)
 RETURN resultset_typ;
 c_default_limit CONSTANT PLS_INTEGER := 100;
 FUNCTION processor (
 p_source_data IN resultset_typ,
 p_limit_size IN PLS_INTEGER DEFAULT c_default_limit)
 RETURN table_typ
 PIPELINED
 PARALLEL_ENABLE(PARTITION p_source_data BY ANY);
 PROCEDURE with_pipeline (argA IN VARCHAR2,
 argB IN VARCHAR2,
 o_resultset OUT resultset_typ);
 PROCEDURE no_pipeline (argA IN VARCHAR2,
 argB IN VARCHAR2,
 o_resultset OUT resultset_typ);
END pipeline_example;
CREATE OR REPLACE PACKAGE BODY pipeline_example
IS
 FUNCTION base_query (argA IN VARCHAR2, argB IN VARCHAR2)
 RETURN resultset_typ
 IS
 o_resultset resultset_typ;
 BEGIN
 OPEN o_resultset FOR
 SELECT colC, colD, colE
 FROM some_table
 WHERE colA = ArgA AND colB = argB;
 RETURN o_resultset;
 END base_query;
 FUNCTION processor (
 p_source_data IN resultset_typ,
 p_limit_size IN PLS_INTEGER DEFAULT c_default_limit)
 RETURN table_typ
 PIPELINED
 PARALLEL_ENABLE(PARTITION p_source_data BY ANY)
 IS
 aa_source_data table_typ;-- := table_typ ();
 BEGIN
 LOOP
 FETCH p_source_data
 BULK COLLECT INTO aa_source_data
 LIMIT p_limit_size;
 EXIT WHEN aa_source_data.COUNT = 0;
 /* Process the batch of (p_limit_size) records... */
 FOR i IN 1 .. aa_source_data.COUNT
 LOOP
 PIPE ROW (aa_source_data (i));
 END LOOP;
 END LOOP;
 CLOSE p_source_data;
 RETURN;
 END processor;
 PROCEDURE with_pipeline (argA IN VARCHAR2,
 argB IN VARCHAR2,
 o_resultset OUT resultset_typ)
 IS
 BEGIN
 OPEN o_resultset FOR
 SELECT /*+ PARALLEL(t, 5) */ colC,
 SUM (CASE WHEN colD > colE AND colE != '0' THEN colD / ColE END)de,
 SUM (CASE WHEN colE > colD AND colD != '0' THEN colE / ColD END)ed,
 SUM (CASE WHEN colD = colE AND colD != '0' THEN '1' END) de_one,
 SUM (CASE WHEN colD = '0' OR colE = '0' THEN '0' END) de_zero
 FROM TABLE (processor (base_query (argA, argB),100)) t
 GROUP BY colC
 ORDER BY colC
 END with_pipeline;
 PROCEDURE no_pipeline (argA IN VARCHAR2,
 argB IN VARCHAR2,
 o_resultset OUT resultset_typ)
 IS
 BEGIN
 OPEN o_resultset FOR
 SELECT colC,
 SUM (CASE WHEN colD > colE AND colE != '0' THEN colD / ColE END)de,
 SUM (CASE WHEN colE > colD AND colD != '0' THEN colE / ColD END)ed,
 SUM (CASE WHEN colD = colE AND colD != '0' THEN 1 END) de_one,
 SUM (CASE WHEN colD = '0' OR colE = '0' THEN '0' END) de_zero
 FROM (SELECT colC, colD, colE
 FROM some_table
 WHERE colA = ArgA AND colB = argB)
 GROUP BY colC
 ORDER BY colC;
 END no_pipeline;
END pipeline_example;
ALTER PACKAGE pipeline_example COMPILE;Edited by: Earthlink on Nov 14, 2010 9:47 AM
Edited by: Earthlink on Nov 14, 2010 11:31 AM
Edited by: Earthlink on Nov 14, 2010 11:32 AM
Edited by: Earthlink on Nov 20, 2010 12:04 PM
Edited by: Earthlink on Nov 20, 2010 12:54 PM

Earthlink wrote:
Contrary to my understanding, the with_pipeline procedure runs 6 time slower than the legacy no_pipeline procedure. Am I missing something? Well, we're missing a lot here.
Like:
- a database version
- how did you test
- what data do you have, how is it distributed, indexed
and so on.
If you want to find out what's going on then use a TRACE with wait events.
All nessecary steps are explained in these threads:
HOW TO: Post a SQL statement tuning request - template posting
http://oracle-randolf.blogspot.com/2009/02/basic-sql-statement-performance.html
Another nice one is RUNSTATS:
http://asktom.oracle.com/pls/asktom/ASKTOM.download_file?p_file=6551378329289980701

Performance issue with two unbanalnced hierarchies in a single report

Hi All
We are facing the performance issue with one of the report which houses two unbalanced hierarchies (having 18 levels) - skipped & ragged. Basically its a part of OBIAPPS financila analytics .
The query is below :
Could anyone let me know how to improve the performane. Any parameter that should be looked at while using unbalanced hierarchies.
WITH SAWITH0
AS ( SELECT SUM (T91707.OTHER_LOC_AMT) AS c1,
MAX (T314768.HIER2_CODE) AS c2,
MAX (T314768.HIER3_CODE) AS c3,
MAX (T314768.HIER4_CODE) AS c4,
MAX (T314768.HIER5_CODE) AS c5,
MAX (T314768.HIER6_CODE) AS c6,
MAX (T314768.HIER7_CODE) AS c7,
MAX (T314768.HIER8_CODE) AS c8,
MAX (T314768.HIER9_CODE) AS c9,
MAX (T314768.HIER10_CODE) AS c10,
MAX (T314768.HIER11_CODE) AS c11,
MAX (T314768.HIER12_CODE) AS c12,
MAX (T314768.HIER13_CODE) AS c13,
MAX (T314768.HIER14_CODE) AS c14,
MAX (T314768.HIER15_CODE) AS c15,
MAX (T314768.HIER16_CODE) AS c16,
MAX (T314768.HIER17_CODE) AS c17,
MAX (T314768.HIER18_CODE) AS c18,
MAX (T314768.HIER19_CODE) AS c19,
MAX (T314768.HIER20_CODE) AS c20,
T314768.HIER1_NAME AS c21,
T314768.HIER1_CODE AS c22,
T314914.HIER1_NAME AS c24,
T314914.HIER10_NAME AS c25,
T314914.HIER11_NAME AS c26,
T314914.HIER12_NAME AS c27,
T314914.HIER13_NAME AS c28,
T314914.HIER14_NAME AS c29,
T314914.HIER15_NAME AS c30,
T314914.HIER16_NAME AS c31,
T314914.HIER17_NAME AS c32,
T314914.HIER18_NAME AS c33,
T314914.HIER19_NAME AS c34,
T314914.HIER2_NAME AS c35,
T314914.HIER20_NAME AS c36,
T314914.HIER3_NAME AS c37,
T314914.HIER4_NAME AS c38,
T314914.HIER5_NAME AS c39,
T314914.HIER6_NAME AS c40,
T314914.HIER7_NAME AS c41,
T314914.HIER8_NAME AS c42,
T314914.HIER9_NAME AS c43,
T314914.HIER20_CODE AS c44,
T314914.HIER1_CODE AS c45,
T314914.HIER10_CODE AS c46,
T314914.HIER11_CODE AS c47,
T314914.HIER12_CODE AS c48,
T314914.HIER13_CODE AS c49,
T314914.HIER14_CODE AS c50,
T314914.HIER15_CODE AS c51,
T314914.HIER16_CODE AS c52,
T314914.HIER17_CODE AS c53,
T314914.HIER18_CODE AS c54,
T314914.HIER19_CODE AS c55,
T314914.HIER2_CODE AS c56,
T314914.HIER3_CODE AS c57,
T314914.HIER4_CODE AS c58,
T314914.HIER5_CODE AS c59,
T314914.HIER6_CODE AS c60,
T314914.HIER7_CODE AS c61,
T314914.HIER8_CODE AS c62,
T314914.HIER9_CODE AS c63
FROM W_HIERARCHY_D T314768 /* Dim_W_HIERARCHY_D_Segment11 */
W_GL_SEGMENT_D T315677 /* Dim_W_GL_SEGMENT_D_Segment11 */
W_HIERARCHY_D T314914 /* Dim_W_HIERARCHY_D_Segment13 */
W_GL_SEGMENT_D T315731 /* Dim_W_GL_SEGMENT_D_Segment13 */
W_GL_ACCOUNT_D T91397 /* Dim_W_GL_ACCOUNT_D */
W_GL_OTHER_F T91707 /* Fact_W_GL_OTHER_F */
WHERE ( T91397.ROW_WID = T91707.GL_ACCOUNT_WID
AND T91397.ACCOUNT_SEG11_CODE = T315677.SEGMENT_VAL_CODE
AND T91397.ACCOUNT_SEG13_CODE = T315731.SEGMENT_VAL_CODE
AND T91397.ACCOUNT_SEG11_ATTRIB = T315677.SEGMENT_LOV_ID
AND T91397.ACCOUNT_SEG13_ATTRIB = T315731.SEGMENT_LOV_ID
AND T314768.HIER_CODE = T315677.SEGMENT_LOV_ID
AND T314768.HIER_NAME = T315677.SEGMENT_LOV_NAME
AND T314768.HIERARCHY_ID = T315677.SEGMENT_VAL_CODE
AND T314914.HIER_CODE = T315731.SEGMENT_LOV_ID
AND T314914.HIER_NAME = T315731.SEGMENT_LOV_NAME
AND T314914.HIERARCHY_ID = T315731.SEGMENT_VAL_CODE
AND T315677.SEGMENT_LOV_NAME =
'Responsibility_Centre_Functional'
AND T315677.SEGMENT_LOV_ID = 1000163
AND T315731.SEGMENT_LOV_NAME = 'Account_Master'
AND T315731.SEGMENT_LOV_ID = 1000165
AND ( T314914.HIER11_CODE IN ('S526002012')
OR T314914.HIER12_CODE IN ('S000001022')
OR T314914.HIER11_CODE IS NULL)
AND (T314914.HIER12_CODE IN ('S000001022')
OR T314914.HIER12_CODE IS NULL)
AND ( T314914.HIER8_CODE IN ('S000005160')
OR T314914.HIER9_CODE IN ('S000000187')
OR T314914.HIER10_CODE IN ('S526003000')
OR T314914.HIER11_CODE IN ('S526002012')
OR T314914.HIER12_CODE IN ('S000001022')
OR T314914.HIER8_CODE IS NULL)
AND ( T314914.HIER9_CODE IN ('S000000187')
OR T314914.HIER10_CODE IN ('S526003000')
OR T314914.HIER11_CODE IN ('S526002012')
OR T314914.HIER12_CODE IN ('S000001022')
OR T314914.HIER9_CODE IS NULL)
AND ( T314914.HIER10_CODE IN ('S526003000')
OR T314914.HIER11_CODE IN ('S526002012')
OR T314914.HIER12_CODE IN ('S000001022')
OR T314914.HIER10_CODE IS NULL)
AND ( T314914.HIER1_CODE IN ('ALL_LI')
OR T314914.HIER2_CODE IN ('S000000001')
OR T314914.HIER3_CODE IN ('S000005150')
OR T314914.HIER4_CODE IN ('S000005151')
OR T314914.HIER5_CODE IN ('S000005153')
OR T314914.HIER6_CODE IN ('S000005154')
OR T314914.HIER7_CODE IN ('S000005062')
OR T314914.HIER8_CODE IN ('S000005160')
OR T314914.HIER9_CODE IN ('S000000187')
OR T314914.HIER10_CODE IN ('S526003000')
OR T314914.HIER11_CODE IN ('S526002012')
OR T314914.HIER12_CODE IN ('S000001022'))
AND ( T314914.HIER2_CODE IN ('S000000001')
OR T314914.HIER3_CODE IN ('S000005150')
OR T314914.HIER4_CODE IN ('S000005151')
OR T314914.HIER5_CODE IN ('S000005153')
OR T314914.HIER6_CODE IN ('S000005154')
OR T314914.HIER7_CODE IN ('S000005062')
OR T314914.HIER8_CODE IN ('S000005160')
OR T314914.HIER9_CODE IN ('S000000187')
OR T314914.HIER10_CODE IN ('S526003000')
OR T314914.HIER11_CODE IN ('S526002012')
OR T314914.HIER12_CODE IN ('S000001022')
OR T314914.HIER2_CODE IS NULL)
AND ( T314914.HIER3_CODE IN ('S000005150')
OR T314914.HIER4_CODE IN ('S000005151')
OR T314914.HIER5_CODE IN ('S000005153')
OR T314914.HIER6_CODE IN ('S000005154')
OR T314914.HIER7_CODE IN ('S000005062')
OR T314914.HIER8_CODE IN ('S000005160')
OR T314914.HIER9_CODE IN ('S000000187')
OR T314914.HIER10_CODE IN ('S526003000')
OR T314914.HIER11_CODE IN ('S526002012')
OR T314914.HIER12_CODE IN ('S000001022')
OR T314914.HIER3_CODE IS NULL)
AND ( T314914.HIER4_CODE IN ('S000005151')
OR T314914.HIER5_CODE IN ('S000005153')
OR T314914.HIER6_CODE IN ('S000005154')
OR T314914.HIER7_CODE IN ('S000005062')
OR T314914.HIER8_CODE IN ('S000005160')
OR T314914.HIER9_CODE IN ('S000000187')
OR T314914.HIER10_CODE IN ('S526003000')
OR T314914.HIER11_CODE IN ('S526002012')
OR T314914.HIER12_CODE IN ('S000001022')
OR T314914.HIER4_CODE IS NULL)
AND ( T314914.HIER5_CODE IN ('S000005153')
OR T314914.HIER6_CODE IN ('S000005154')
OR T314914.HIER7_CODE IN ('S000005062')
OR T314914.HIER8_CODE IN ('S000005160')
OR T314914.HIER9_CODE IN ('S000000187')
OR T314914.HIER10_CODE IN ('S526003000')
OR T314914.HIER11_CODE IN ('S526002012')
OR T314914.HIER12_CODE IN ('S000001022')
OR T314914.HIER5_CODE IS NULL)
AND ( T314914.HIER6_CODE IN ('S000005154')
OR T314914.HIER7_CODE IN ('S000005062')
OR T314914.HIER8_CODE IN ('S000005160')
OR T314914.HIER9_CODE IN ('S000000187')
OR T314914.HIER10_CODE IN ('S526003000')
OR T314914.HIER11_CODE IN ('S526002012')
OR T314914.HIER12_CODE IN ('S000001022')
OR T314914.HIER6_CODE IS NULL)
AND ( T314914.HIER7_CODE IN ('S000005062')
OR T314914.HIER8_CODE IN ('S000005160')
OR T314914.HIER9_CODE IN ('S000000187')
OR T314914.HIER10_CODE IN ('S526003000')
OR T314914.HIER11_CODE IN ('S526002012')
OR T314914.HIER12_CODE IN ('S000001022')
OR T314914.HIER7_CODE IS NULL)
AND T314768.HIER1_CODE IS NOT NULL
AND T314914.HIER20_CODE IS NOT NULL
AND T314914.HIER13_CODE IS NULL
AND T314914.HIER14_CODE IS NULL
AND T314914.HIER15_CODE IS NULL
AND T314914.HIER16_CODE IS NULL
AND T314914.HIER17_CODE IS NULL
AND T314914.HIER18_CODE IS NULL
AND T314914.HIER19_CODE IS NULL)
GROUP BY T314768.HIER1_CODE,
T314768.HIER1_NAME,
T314914.HIER1_CODE,
T314914.HIER1_NAME,
T314914.HIER2_CODE,
T314914.HIER2_NAME,
T314914.HIER3_CODE,
T314914.HIER3_NAME,
T314914.HIER4_CODE,
T314914.HIER4_NAME,
T314914.HIER5_CODE,
T314914.HIER5_NAME,
T314914.HIER6_CODE,
T314914.HIER6_NAME,
T314914.HIER7_CODE,
T314914.HIER7_NAME,
T314914.HIER8_CODE,
T314914.HIER8_NAME,
T314914.HIER9_CODE,
T314914.HIER9_NAME,
T314914.HIER10_CODE,
T314914.HIER10_NAME,
T314914.HIER11_CODE,
T314914.HIER11_NAME,
T314914.HIER12_CODE,
T314914.HIER12_NAME,
T314914.HIER13_CODE,
T314914.HIER13_NAME,
T314914.HIER14_CODE,
T314914.HIER14_NAME,
T314914.HIER15_CODE,
T314914.HIER15_NAME,
T314914.HIER16_CODE,
T314914.HIER16_NAME,
T314914.HIER17_CODE,
T314914.HIER17_NAME,
T314914.HIER18_CODE,
T314914.HIER18_NAME,
T314914.HIER19_CODE,
T314914.HIER19_NAME,
T314914.HIER20_CODE,
T314914.HIER20_NAME),
SAWITH1
AS (SELECT SUM (D1.c1) OVER () AS c1,
MAX (D1.c2) OVER (PARTITION BY D1.c22) AS c2,
MAX (D1.c3) OVER (PARTITION BY D1.c22) AS c3,
MAX (D1.c4) OVER (PARTITION BY D1.c22) AS c4,
MAX (D1.c5) OVER (PARTITION BY D1.c22) AS c5,
MAX (D1.c6) OVER (PARTITION BY D1.c22) AS c6,
MAX (D1.c7) OVER (PARTITION BY D1.c22) AS c7,
MAX (D1.c8) OVER (PARTITION BY D1.c22) AS c8,
MAX (D1.c9) OVER (PARTITION BY D1.c22) AS c9,
MAX (D1.c10) OVER (PARTITION BY D1.c22) AS c10,
MAX (D1.c11) OVER (PARTITION BY D1.c22) AS c11,
MAX (D1.c12) OVER (PARTITION BY D1.c22) AS c12,
MAX (D1.c13) OVER (PARTITION BY D1.c22) AS c13,
MAX (D1.c14) OVER (PARTITION BY D1.c22) AS c14,
MAX (D1.c15) OVER (PARTITION BY D1.c22) AS c15,
MAX (D1.c16) OVER (PARTITION BY D1.c22) AS c16,
MAX (D1.c17) OVER (PARTITION BY D1.c22) AS c17,
MAX (D1.c18) OVER (PARTITION BY D1.c22) AS c18,
MAX (D1.c19) OVER (PARTITION BY D1.c22) AS c19,
MAX (D1.c20) OVER (PARTITION BY D1.c22) AS c20,
D1.c21 AS c21,
D1.c22 AS c22,
SUM (
D1.c1)
OVER (
PARTITION BY D1.c46,
D1.c47,
D1.c48,
D1.c49,
D1.c50,
D1.c51,
D1.c52,
D1.c53,
D1.c54,
D1.c55,
D1.c45,
D1.c44,
D1.c56,
D1.c57,
D1.c58,
D1.c59,
D1.c60,
D1.c61,
D1.c62,
D1.c63,
D1.c22)
AS c23,
D1.c24 AS c24,
D1.c25 AS c25,
D1.c26 AS c26,
D1.c27 AS c27,
D1.c28 AS c28,
D1.c29 AS c29,
D1.c30 AS c30,
D1.c31 AS c31,
D1.c32 AS c32,
D1.c33 AS c33,
D1.c34 AS c34,
D1.c35 AS c35,
D1.c36 AS c36,
D1.c37 AS c37,
D1.c38 AS c38,
D1.c39 AS c39,
D1.c40 AS c40,
D1.c41 AS c41,
D1.c42 AS c42,
D1.c43 AS c43,
D1.c44 AS c44,
D1.c45 AS c45,
D1.c46 AS c46,
D1.c47 AS c47,
D1.c48 AS c48,
D1.c49 AS c49,
D1.c50 AS c50,
D1.c51 AS c51,
D1.c52 AS c52,
D1.c53 AS c53,
D1.c54 AS c54,
D1.c55 AS c55,
D1.c56 AS c56,
D1.c57 AS c57,
D1.c58 AS c58,
D1.c59 AS c59,
D1.c60 AS c60,
D1.c61 AS c61,
D1.c62 AS c62,
D1.c63 AS c63
FROM SAWITH0 D1)
SELECT DISTINCT
38 AS c1,
D1.c24 AS c2,
D1.c25 AS c3,
D1.c26 AS c4,
D1.c27 AS c5,
D1.c28 AS c6,
D1.c29 AS c7,
D1.c30 AS c8,
D1.c31 AS c9,
D1.c32 AS c10,
D1.c33 AS c11,
D1.c34 AS c12,
D1.c35 AS c13,
D1.c36 AS c14,
D1.c37 AS c15,
D1.c38 AS c16,
D1.c39 AS c17,
D1.c40 AS c18,
D1.c41 AS c19,
D1.c42 AS c20,
D1.c43 AS c21,
D1.c21 AS c22,
NULL AS c23,
NULL AS c24,
NULL AS c25,
NULL AS c26,
NULL AS c27,
NULL AS c28,
NULL AS c29,
NULL AS c30,
NULL AS c31,
NULL AS c32,
NULL AS c33,
NULL AS c34,
NULL AS c35,
NULL AS c36,
NULL AS c37,
NULL AS c38,
NULL AS c39,
NULL AS c40,
NULL AS c41,
D1.c44 AS c42,
D1.c45 AS c43,
D1.c46 AS c44,
D1.c47 AS c45,
D1.c48 AS c46,
D1.c49 AS c47,
D1.c50 AS c48,
D1.c51 AS c49,
D1.c52 AS c50,
D1.c53 AS c51,
D1.c54 AS c52,
D1.c55 AS c53,
D1.c56 AS c54,
D1.c57 AS c55,
D1.c58 AS c56,
D1.c59 AS c57,
D1.c60 AS c58,
D1.c61 AS c59,
D1.c62 AS c60,
D1.c63 AS c61,
NULL AS c62,
D1.c22 AS c63,
NULL AS c64,
NULL AS c65,
NULL AS c66,
NULL AS c67,
NULL AS c68,
NULL AS c69,
NULL AS c70,
NULL AS c71,
NULL AS c72,
NULL AS c73,
NULL AS c74,
NULL AS c75,
NULL AS c76,
NULL AS c77,
NULL AS c78,
NULL AS c79,
NULL AS c80,
NULL AS c81,
D1.c23 AS c82,
CASE WHEN 1 = 1 THEN 1 ELSE 0 END AS c83,
CASE
WHEN D1.c2 IS NULL
AND D1.c3 IS NULL
AND D1.c4 IS NULL
AND D1.c5 IS NULL
AND D1.c6 IS NULL
AND D1.c7 IS NULL
AND D1.c8 IS NULL
AND D1.c9 IS NULL
AND D1.c10 IS NULL
AND D1.c11 IS NULL
AND D1.c12 IS NULL
AND D1.c13 IS NULL
AND D1.c14 IS NULL
AND D1.c15 IS NULL
AND D1.c16 IS NULL
AND D1.c17 IS NULL
AND D1.c18 IS NULL
AND D1.c19 IS NULL
AND D1.c20 IS NULL
THEN
1
ELSE
0
END
AS c84
FROM SAWITH1 D1
WHERE ( D1.c44 IS NOT NULL
AND D1.c50 IS NULL
AND D1.c49 IS NULL
AND D1.c22 IS NOT NULL
AND D1.c51 IS NULL
AND D1.c52 IS NULL
AND D1.c53 IS NULL
AND D1.c54 IS NULL
AND D1.c55 IS NULL)
/* Formatted on 12/17/2012 7:49:44 PM (QP5 v5.139.911.3011) */
WITH OBICOMMON0
AS (SELECT T156337.ROW_WID AS c2,
T156337.MCAL_PERIOD_WID AS c3,
ROW_NUMBER ()
OVER (PARTITION BY T156337.MCAL_PERIOD_WID
ORDER BY T156337.MCAL_PERIOD_WID DESC)
AS c4,
T156337.MCAL_PERIOD_NAME AS c5,
T156337.MCAL_PER_NAME_YEAR AS c6
FROM W_MCAL_DAY_D T156337 /* Dim_W_MCAL_DAY_D_Fiscal_Day */
WHERE (T156337.MCAL_CAL_NAME = 'Accounting')),
SAWITH0
AS (SELECT CASE
WHEN CASE D1.c4 WHEN 1 THEN D1.c2 ELSE NULL END
IS NOT NULL
THEN
RANK ()
OVER (
ORDER BY
CASE D1.c4 WHEN 1 THEN D1.c2 ELSE NULL END ASC NULLS LAST)
END
AS c1,
D1.c2 AS c2,
D1.c3 AS c3
FROM OBICOMMON0 D1),
SAWITH1
AS (SELECT DISTINCT
MIN (D1.c1) OVER (PARTITION BY D1.c3) AS c1, D1.c2 AS c2
FROM SAWITH0 D1),
SAWITH2
AS (SELECT CASE
WHEN CASE D1.c4 WHEN 1 THEN D1.c2 ELSE NULL END
IS NOT NULL
THEN
RANK ()
OVER (
ORDER BY
CASE D1.c4 WHEN 1 THEN D1.c2 ELSE NULL END ASC NULLS LAST)
END
AS c1,
D1.c3 AS c2,
D1.c5 AS c3,
D1.c6 AS c4
FROM OBICOMMON0 D1),
SAWITH3 AS (SELECT DISTINCT MIN (D1.c1) OVER (PARTITION BY D1.c2) AS c1,
D1.c2 AS c2,
D1.c3 AS c3,
D1.c4 AS c4
FROM SAWITH2 D1),
SAWITH4
AS ( SELECT SUM (T91707.TD_OTHER_REP_AMT) AS c1,
T314914.HIER1_NAME AS c2,
D2.c3 AS c3,
T314914.HIER1_CODE AS c4,
D2.c2 AS c5
FROM W_HIERARCHY_D T314914 /* Dim_W_HIERARCHY_D_Segment13 */
W_GL_SEGMENT_D T315731 /* Dim_W_GL_SEGMENT_D_Segment13 */
W_GL_ACCOUNT_D T91397 /* Dim_W_GL_ACCOUNT_D */
W_GL_OTHER_F T91707 /* Fact_W_GL_OTHER_F */
SAWITH1 D4,
SAWITH3 D2
WHERE ( T314914.HIER_CODE = T315731.SEGMENT_LOV_ID
AND T314914.HIER_NAME = T315731.SEGMENT_LOV_NAME
AND T91397.ROW_WID = T91707.GL_ACCOUNT_WID
AND T91707.ACCT_PERIOD_END_DT_WID = D4.c2
AND T314914.HIERARCHY_ID = T315731.SEGMENT_VAL_CODE
AND T91397.ACCOUNT_SEG13_CODE = T315731.SEGMENT_VAL_CODE
AND T91397.ACCOUNT_SEG13_ATTRIB = T315731.SEGMENT_LOV_ID
AND T315731.SEGMENT_LOV_NAME =
'Account_Retail_Distribution'
AND T315731.SEGMENT_LOV_ID = 1000165
AND D2.c1 = D4.c1
AND (D2.c4 IN ('2011', '2012')))
GROUP BY T314914.HIER1_CODE,
T314914.HIER1_NAME,
D2.c2,
D2.c3)
SELECT D1.c1 AS c1,
D1.c2 AS c2,
D1.c3 AS c3,
D1.c4 AS c4,
D1.c5 AS c5,
D1.c6 AS c6
FROM ( SELECT DISTINCT 0 AS c1,
D1.c2 AS c2,
D1.c3 AS c3,
D1.c4 AS c4,
D1.c1 AS c5,
D1.c5 AS c6
FROM SAWITH4 D1
ORDER BY c2 NULLS FIRST, c4 NULLS FIRST, c3) D1
WHERE ROWNUM <= 65001

Hello Gurus, Experts
Any help/tips here ...

Performance problems with XMLTABLE and XMLQUERY involving relational data

Hello-
Is anyone out there using XMLTABLE or XMLQUERY with more than a toy set of data? I am running into serious performance problems tyring to do basic things such as:
* Combine records in 10 relational tables into a single table of XMLTYPE records using XMLTABLE. This hangs indefinitely for any more than 800 records. Oracle has confirmed that this is a problem and is working on a fix.
* Combine a single XMLTYPE record with several relational code tables into a single XMLTYPE record using XMLQUERY and ora:view() to insert code descriptions after each code. Performance is 10 seconds for 10 records (terrible) passing a batch of records , or 160 seconds for one record (unacceptable!). How can it take 10 times longer to process 1/10th the number of records? Ironically, the query plan says it will do a full table scan of records for the batch, but an index access for the one record passed to the XMLQUERY.
I am rapidly losing faith in XML DB, and desparately need some hints on how to work around these performance problems, or at least some assurance that others have been able to get this thing to perform.

<Note>Long post, sorry.</Note>
First, thanks for the responses above. I'm impressed with the quality of thought put into them. (Do the forum rules allow me to offer rewards? :) One suggestion in particular made a big performance improvement, and I’m encouraged to hear of good performance in pure XML situations. Unfortunately, I think there is a real performance challenge in two use cases that are pertinent to the XML+relational subject of this post and probably increasingly common as XML DB usage increases:
• Converting legacy tabular data into XML records; and
• Performing code table lookups for coded values in XML records.
There are three things I want to accomplish with this post:
• Clarify what we are trying to accomplish, which might expose completely different approaches than I have tried
• Let you know what I tried so far and the rationale for my approach to help expose flaws in my thinking and share what I have learned
• Highlight remaining performance issues in hopes that we can solve them
What we are trying to accomplish:
• Receive a monthly feed of 10,000 XML records (batched together in text files), each containing information about an employee, including elements that repeat for every year of service. We may need to process an annual feed of 1,000,000 XML records in the future.
• Receive a one-time feed of 500,000 employee records stored in about 10 relational tables, with a maximum join depth of 2 or 3. This is inherently a relational-to-XML process. One record/second is minimally acceptable, but 10 records/sec would be better.
• Consolidate a few records (from different providers) for each employee into a single record. Given the data volume, we need to achieve a minimum rate of 10 records per second. This may be an XML-only process, or XML+relational if code lookups are done during consolidation.
• Allow the records to be viewed and edited, with codes resolved into user-friendly descriptions. Since a user is sitting there, code lookups done when a record is viewed (vs. during consolidation) should not take more than 3 seconds total. We have about 20 code tables averaging a few hundred rows each, though one has 450,000 rows.
As requested earlier, I have included code at the end of this post for example tables and queries that accurately (but simply) replicate our real system.
Why we did and why:
• Stored the source XML records as CLOBS: We did this to preserve the records exactly as they were certified and sent from providers. In addition, we always access the entire XML record as a whole (e.g., when viewing a record or consolidating employee records), so this storage model seemed like a good fit. We can copy them into another format if necessary.
• Stored the consolidated XML employee records as “binary XML”. We did this because we almost always access a single, entire record as a whole (for view/edit), but might want to create some summary statistics at some point. Binary XML seemed the best fit.
• Used ora:view() for both tabular source records and lookup tables. We are not aware of any alternatives at this time. If it made sense, most code tables could be pre-converted into XML documents, but this seemed risky from a performance standpoint because the lookups use both code and date range constraints (the meaning of codes changes over time).
• Stored records as XMLTYPE columns in a table with other key columns on the table, plus an XMLTYPE metadata column. We thought this would facilitate pulling a single record (or a few records for a given employee) quickly. We knew this might be unnecessary given XML indexes and virtual columns, but were not experienced with those and wanted the comfort of traditional keys. We did not used XMLTYPE tables or the XML Repository for documents.
• Used XMLTABLE to consolidate XML records by looping over each distinct employee ID in the source batch. We also tried XMLQUERY and it seems to perform about the same. We can achieve 10 to 20 records/second if we do not do any code lookups during consolidation, just meeting our performance requirement, but still much slower than expected.
• Used PL/SQL with XMLFOREST to convert tabular source records to XML by looping over distinct employee IDs. We tried this outside PL/SQL both with XMLFOREST and XMLTABLE+ora:view(), but it hangs in both cases for more than 800 records (a known/open issue). We were able to get it to work by using an explicit cursor to loop over distinct employee IDs (rather than processing all records at once within the query). The performance is one record/second, which is minimally acceptable and interferes with other database activity.
• Used XMLQUERY plus ora:view() plus XPATH constraints to perform code lookups. When passing a single employee record, the response time ranges from 1 sec to 160 sec depending on the length of the record (i.e., number of years of service). We achieved a 5-fold speedup using an XMLINDEX (thank you Marco!!). The result may be minimally acceptable, but I’m baffled why the index would be needed when processing a single XML record. Other things we tried: joining code tables in the FOR...WHERE clauses, joining code tables using LET with XPATH constraints and LET with WHERE clause constraints, and looking up codes individually via JDBC from the application code at presentation time. All those approaches were slower. Note: the difference I mentioned above in equality/inequality constraint performance was due to data record variations not query plan variations.
What issues remain?
We have a minimally acceptable solution from a performance standpoint with one very awkward PL/SQL workaround. The performance of a mixed XML+relational data query is still marginal IMHO, until we properly utilize available optimizations, fix known problems, and perhaps get some new query optimizations. On the last point, I think the query plan for tabular lookups of codes in XML records is falling short right now. I’m reminded of data warehousing in the days before hash joins and star join optimization. I would be happy to be wrong, and just as happy for viable workarounds if I am right!
Here are the details on our code lookup challenge. Additional suggestions would be greatly appreciated. I’ll try to post more detail on the legacy table conversion challenge later.
-- The main record table:
create table RECORDS (
SSN varchar2(20),
XMLREC sys.xmltype
xmltype column XMLREC store as binary xml;
create index records_ssn on records(ssn);
-- A dozen code tables represented by one like this:
create table CODES (
CODE varchar2(4),
DESCRIPTION varchar2(500)
create index codes_code on codes(code);
-- Some XML records with coded values (the real records are much more complex of course):
-- I think this took about a minute or two
DECLARE
ssn varchar2(20);
xmlrec xmltype;
i integer;
BEGIN
xmlrec := xmltype('<?xml version="1.0"?>
<Root>
<Id>123456789</Id>
<Element>
<Subelement1><Code>11</Code></Subelement1>
<Subelement2><Code>21</Code></Subelement2>
<Subelement3><Code>31</Code></Subelement3>
</Element>
<Element>
<Subelement1><Code>11</Code></Subelement1>
<Subelement2><Code>21</Code></Subelement2>
<Subelement3><Code>31</Code></Subelement3>
</Element>
<Element>
<Subelement1><Code>11</Code></Subelement1>
<Subelement2><Code>21</Code></Subelement2>
<Subelement3><Code>31</Code></Subelement3>
</Element>
</Root>
for i IN 1..100000 loop
insert into records(ssn, xmlrec) values (i, xmlrec);
end loop;
commit;
END;
-- Some code data like this (ignoring date ranges on codes):
DECLARE
description varchar2(100);
i integer;
BEGIN
description := 'This is the code description ';
for i IN 1..3000 loop
insert into codes(code, description) values (to_char(i), description);
end loop;
commit;
end;
-- Retrieve one record while performing code lookups. Takes about 5-6 seconds...pretty slow.
-- Each additional lookup (times 3 repeating elements in the data) adds about 1 second.
-- A typical real record has 5 Elements and 20 Subelements, meaning more than 20 seconds to display the record
-- Note we are accessing a single XML record based on SSN
-- Note also we are reusing the one test code table multiple times for convenience of this test
select xmlquery('
for $r in Root
return
<Root>
<Id>123456789</Id>
{for $e in $r/Element
 return
 <Element>
 <Subelement1>
 {$e/Subelement1/Code}
<Description>
{ora:view("disaac","codes")/ROW[CODE=$e/Subelement1/Code]/DESCRIPTION/text() }
</Description>
</Subelement1>
<Subelement2>
{$e/Subelement2/Code}
<Description>
{ora:view("disaac","codes")/ROW[CODE=$e/Subelement2/Code]/DESCRIPTION/text()}
</Description>
</Subelement2>
<Subelement3>
{$e/Subelement3/Code}
<Description>
{ora:view("disaac","codes")/ROW[CODE=$e/Subelement3/Code]/DESCRIPTION/text() }
</Description>
</Subelement3>
</Element>
</Root>
' passing xmlrec returning content)
from records
where ssn = '10000';
The plan shows the nested loop access that slows things down.
By contrast, a functionally-similar SQL query on relational data will use a hash join and perform 10x to 100x faster, even for a single record. There seems to be no way for the optimizer to see the regularity in the XML structure and perform a corresponding optimization in joining the code tables. Not sure if registering a schema would help. Using structured storage probably would. But should that be necessary given we’re working with a single record?
Operation Object
|SELECT STATEMENT ()
| SORT (AGGREGATE)
| NESTED LOOPS (SEMI)
| TABLE ACCESS (FULL) CODES
| XPATH EVALUATION ()
| SORT (AGGREGATE)
| NESTED LOOPS (SEMI)
| TABLE ACCESS (FULL) CODES
| XPATH EVALUATION ()
| SORT (AGGREGATE)
| NESTED LOOPS (SEMI)
| TABLE ACCESS (FULL) CODES
| XPATH EVALUATION ()
| SORT (AGGREGATE)
| XPATH EVALUATION ()
| SORT (AGGREGATE)
| XPATH EVALUATION ()
| TABLE ACCESS (BY INDEX ROWID) RECORDS
| INDEX (RANGE SCAN) RECORDS_SSN
With an xmlindex, the same query above runs in about 1 second, so is about 5x faster (0.2 sec/lookup), which is almost good enough. Is this the answer? Or is there a better way? I’m not sure why the optimizer wants to scan the code tables and index into the (one) XML record, rather than the other way around, but maybe that makes sense if the optimizer wants to use the same general plan as when the WHERE clause constraint is relaxed to multiple records.
-- Add an xmlindex. Takes about 2.5 minutes
create index records_record_xml ON records(xmlrec)
indextype IS xdb.xmlindex;
Operation Object
|SELECT STATEMENT ()
| SORT (GROUP BY)
| FILTER ()
| NESTED LOOPS ()
| FAST DUAL ()
| TABLE ACCESS (BY INDEX ROWID) SYS113473_RECORDS_R_PATH_TABLE
| INDEX (RANGE SCAN) SYS113473_RECORDS_R_PATHID_IX
| SORT (AGGREGATE)
| FILTER ()
| TABLE ACCESS (FULL) CODES
| FILTER ()
| NESTED LOOPS ()
| FAST DUAL ()
| TABLE ACCESS (BY INDEX ROWID) SYS113473_RECORDS_R_PATH_TABLE
| INDEX (RANGE SCAN) SYS113473_RECORDS_R_PATHID_IX
| SORT (GROUP BY)
| FILTER ()
| NESTED LOOPS ()
| FAST DUAL ()
| TABLE ACCESS (BY INDEX ROWID) SYS113473_RECORDS_R_PATH_TABLE
| INDEX (RANGE SCAN) SYS113473_RECORDS_R_PATHID_IX
| SORT (AGGREGATE)
| FILTER ()
| TABLE ACCESS (FULL) CODES
| FILTER ()
| NESTED LOOPS ()
| FAST DUAL ()
| TABLE ACCESS (BY INDEX ROWID) SYS113473_RECORDS_R_PATH_TABLE
| INDEX (RANGE SCAN) SYS113473_RECORDS_R_PATHID_IX
| SORT (GROUP BY)
| FILTER ()
| NESTED LOOPS ()
| FAST DUAL ()
| TABLE ACCESS (BY INDEX ROWID) SYS113473_RECORDS_R_PATH_TABLE
| INDEX (RANGE SCAN) SYS113473_RECORDS_R_PATHID_IX
| SORT (AGGREGATE)
| FILTER ()
| TABLE ACCESS (FULL) CODES
| FILTER ()
| NESTED LOOPS ()
| FAST DUAL ()
| TABLE ACCESS (BY INDEX ROWID) SYS113473_RECORDS_R_PATH_TABLE
| INDEX (RANGE SCAN) SYS113473_RECORDS_R_PATHID_IX
| SORT (AGGREGATE)
| FILTER ()
| NESTED LOOPS ()
| FAST DUAL ()
| TABLE ACCESS (BY INDEX ROWID) SYS113473_RECORDS_R_PATH_TABLE
| INDEX (RANGE SCAN) SYS113473_RECORDS_R_PATHID_IX
| SORT (AGGREGATE)
| TABLE ACCESS (BY INDEX ROWID) SYS113473_RECORDS_R_PATH_TABLE
| INDEX (RANGE SCAN) SYS113473_RECORDS_R_PATHID_IX
| TABLE ACCESS (BY INDEX ROWID) RECORDS
| INDEX (RANGE SCAN) RECORDS_SSN
Am I on the right path, or am I totally using the wrong approach? I thought about using XSLT but was unsure how to reference the code tables.
I’ve done the best I can constraining the main record to a single row passed to the XMLQUERY. Given Mark’s post (thanks!) should I be joining and constraining the code tables in the SQL WHERE clause too? That’s going to make the query much more complicated, but right now we’re more concerned about performance than complexity.

[URGENT] Performance problem with BC4J and partioned data

Hi all,
I have a big performance probelm with BC4J and partitioned data. As as partitioned table shouldn't have a primary key like a sequence (or something else) my partitioned table doesn't have any primary key.
When I debug my BC4J application I can see a message showing me "ignoring row with no primary key" from EntityCache. It takes a long time to retrieve my data even if I use the partition keys. A quick & dirty forms application was multiple times faster!
Is this a bug in BC4J, or is BC4J not suitable for partitioned data? Can anyone give me a hint what to do, do make the BC4J application fast even with partitioned data? In a non-partitioned environment the application works quite well. So it seams that it must be an "error" somewhere in this part.
Thanks,
Axel

Here's a SQL statement that creates the table.
CREATE TABLE SEARCH
(SEAR_PARTKEY_DAY              NUMBER(4)        NOT NULL
,SEAR_PARTKEY_EMP            VARCHAR2(2)      NOT NULL
,SEAR_ID                     NUMBER(20)       NOT NULL
,SEAR_ENTRY_DATE             TIMESTAMP        NOT NULL
,SEAR_LAST_MODIFIED            TIMESTAMP             NOT NULL
,SEAR_STATUS                 VARCHAR2(100)    DEFAULT '0'
,SEAR_ITC_DATE               TIMESTAMP        NOT NULL
,SEAR_MESSAGE_CLASS          VARCHAR2(15)     NOT NULL
,SEAR_CHIPHERING_TYPE        VARCHAR2(256)
,SEAR_GMAT                   VARCHAR2(1)      DEFAULT 'U'
,SEAR_NATIONALITY            VARCHAR2(3)      DEFAULT 'XXX'
,SEAR_MESSAGE_ID             VARCHAR2(32)     NOT NULL
,SEAR_COMMENT                VARCHAR2(256)    NOT NULL
,SEAR_NUMBER_OF              NUMBER(3)        NOT NULL
,SEAR_INTERCEPTION_SYSTEM    VARCHAR2(40)
,SEAR_COMM_PRIOD_H           NUMBER(5)        DEFAULT -1
,SEAR_PRIOD_R                  NUMBER(5)        DEFAULT -1
,SEAR_INMARSAT_CES           VARCHAR2(40)
,SEAR_BEAM                   VARCHAR2(10)
,SEAR_DIALED_NUMBER          VARCHAR2(70)
,SEAR_TRANSMIT_NUMBER        VARCHAR2(70)
,SEAR_CALLED_NUMBER          VARCHAR2(40)
,SEAR_CALLER_NUMBER          VARCHAR2(40)
,SEAR_MATERIAL_TYPE          VARCHAR2(3)      NOT NULL
,SEAR_SOURCE                 VARCHAR2(10)
,SEAR_MAPPING                VARCHAR2(100)    DEFAULT '__REST'
,SEAR_DETAIL_MAPPING         VARCHAR2(100)
,SEAR_PRIORITY               NUMBER(3)        DEFAULT 255
,SEAR_LANGUAGE               VARCHAR2(5)      DEFAULT 'XXX'
,SEAR_TRANSMISSION_TYPE      VARCHAR2(40)
,SEAR_INMARSAT_STD           VARCHAR2(1)
,SEAR_FILE_NAME              VARCHAR2(100)    NOT NULL
PARTITION BY RANGE (SEAR_PARTKEY_DAY, SEAR_PARTKEY_EMP)
PARTITION SEARCH_MAX VALUES LESS THAN (MAXVALUE, MAXVALUE) MIRA4_SEARCH_EVEN
);of course SEAR_ID is filled by a sequence but the field is not the primary key as it would decrease the performance of partitioned data.
We moved to native JDBC with our application and the performance is like we never expected to be!

PDF's I'm creating with Group 4 data report Insufficient Data for Image

I am maintaining a software application, which creates PDF files from JPEG and TIFF (with Group 4 compression) image files, which typically come from scanners. It has performed well for many years, but as of the release of Adobe Reader X, we are seeing "Insufficient Data for Image" on some images.
I've read reports of this message on these forums, and I've tried shrinking the image, adjusting zoom, and upgrading to Adobe Reader XI -- none of these have worked.
Our software creates these PDF files by wrapping the raw image data in an XObject, which looks like this:
24 0 obj
<< /Type /XObject
/Subtype /Image
/Name /Im1
/Filter /CCITTFaxDecode
/Width 1704
/Height 2200
/BitsPerComponent 1
/ColorSpace /DeviceGray
/DecodeParms <</K -1 /Columns 1704 /Rows 2200 >>
/Length 25 0 R >>
stream
<<BINARY DATA>>
endstream
endobj
25 0 obj
31512
endobj
Wherein, the <<BINARY DATA>> are 31,512 bytes of raw image data -- the same data which would appear in a Group 4 compressed TIFF file.
Indeed, when we do create a Group 4 TIFF file with the same binary data, it opens successfully in all common viewers.
Any assistance, such as the precise meaning of this error message, could be helpful.

Just a note that the bug appears to relate to JPXDecode, so far as all previous reports seem to suggest (no inside information). JPXDecode supports multiple image resolutions in the same image, so messing with zoom sometimes helps. If you can post a complete file somewhere it might interest someone who could analyse it.
The message simply means that decompressing the filters does not yield enough bytes. This may either reflect that there really isn't enough data, or it could indicate any kind of data error (which causes the filter to silently return EOF).

Select Random Records with Group By?

Hi,
I currently select random records by using this type of SQL:
(SELECT *
FROM ( SELECT *
FROM ( SELECT *
FROM TABLEA
ORDER BY dbms_random.value
WHERE ROWNUM <= 100
Based on the following data, I now want to still pick out random records but I only want 1 record from the same ID and START_DATE but where the FLAG could be Y or N.
So with grouping by ID and START_DATE, I need to still be able to choose the record if it has anything from 1 entry or 10 entries per ID and START_DATE.
Could this be performed with a GROUP BY?
ID START_DATE FLAG
1 12/12/2005 Y
1 12/12/2005 N
1 12/12/2005 N
1 30/12/2005 Y
1 30/12/2005 N
1 30/12/2005 N
2 10/10/2005 Y
2 07/07/2005 Y
Thanks for any help.
Daniel

Try this [not tested]:
SELECT *
 FROM
 (SELECT *
 FROM
 ( SELECT
 ID
 ,START_DATE
 ,FLAG
 ,ROW_NUMBER() OVER
 (PARTITION BY ID,START_DATE ORDER BY dbms_random.value) RN
 FROM TABLEA
 ORDER BY dbms_random.value
 WHERE RN = 1
 AND ROWNUM <= 100
 ;

Performance Decreased with group by

Similar Messages

Maybe you are looking for