Problem with temp space allocation in parallel query
Hello
I've got a query which matches two large result sets (25m+ rows) against each other and does some basic filtering and aggregation. When I run this query in serial it takes about 30 mins and completes successfully. When I specify a parallel degree of 4 for each result set, it also completes successfully in about 20 minutes. However, when I specify that it should be run in parallel but don't specify a degree for each result set, it spawns 10 parallel servers and after a couple of minutes, bombs out from one of the parallel servers with:
ORA-12801: error signaled in parallel query server P000
ORA-01652: unable to extend temp segment by 64 in tablespace TEMPThis appears to be when it is about to perform a large hash join. The execution plan does not change whether the parallel degree is specified or not, and there is several GB of temp space available.
I'm at a bit of a loss as to how to track down specifically what is causing this problem. I've looked at v$sesstat for all of the sessions involved and it hasn't really turned anything up. I've tried tracing the main session and that hasn't really turned up much either. From what I can tell, one of the sessions seems to try to allocate massive amounts of temp space that it just does not need, but I can figure out why.
Any ideas of how to approach finding the cause of the problem?
David
Hello
I've finally resolved this and the resolution was relatively simple - and was also the main thing that Mark Rittman said he did in his article: reduce the size of the hash join.
After querying v$sql_workarea_active I could see what was happening which was that the sum of the temp space for all of the parallel slaves was exceeding the total amount of temp space available on the system. When run in serial, it was virtually at the limit. I guess the extra was just the overhead for each slave maintaining it's own hash table.
I also made the mistake of misreading the exectuion plan - assuming that the data being pushed to the hash join was filtered to eliminate the data that was not of interest. Upon reflection, this was a rather stupid assumption on my part. Anyway, I used sub query factoring with the materialize hint to ensure that the hash join was only working on the data it should have been. This significantly reduced the size of the hash table and therefore the amount of temp space required.
I did speak to oracle support and they suggested using pga_aggregate_target rather than the separate *area_size parameters. I found that this had very little impact as the problem was related to the volume of data rather than whether it was being processed in memory or not. That said, I did try upping the hash_area_size for the session with some initial success, but ultimately it didn't prove to be scalable. We are however now using pga_aggregate_target in prod.
So that's that. Problem sorted. And as the title of Mark Rittman's article suggests, I was trying to be too clever! :-)
HTH
David
Similar Messages
-
Office Web Apps Server 2013 - Word Web App - Problem with Tab space
Hello We have Office Web Apps Server 2013 running with SharePoint 2013. Users Editing a Word document with Office Web Apps, can't use "Tabs", any Word document with Tabs; the tabs are replaced with a single space.
Has anyone noticed this? Is this a bug?
-thanks
thomas
-TomYes, currently the Word Web App does not support
Tab Keyboard shortcut for editing document content .
For more information, you can have a look at
the article:
http://office.microsoft.com/en-us/office-online-help/keyboard-shortcuts-in-word-online-HA010378332.aspx?CTT=5&origin=HA010380212
http://social.technet.microsoft.com/Forums/en-US/3f5978d3-67a1-4c8c-981f-32493d72610b/office-web-apps-server-2013-word-web-app-problem-with-tab-space?forum=sharepointgeneral -
Oracle11g RAC with partitioning and cross instance parallel query problem
I have set up a 300gb TPC-H database using a 4 node RAC environment (8 cpu per node, 16 GB memory, 2 GHz processors) the system is served by 2.5 terabytes of SSD for its IO subsystem managed by a combination of ASM and OCFS2.
When I run a large parallel query (number 9 in the TPCH query set) I get:
ORA-00600: internal error code, arguments [kxfrGraDistNum3],[65535],[4]
with all other arguments blank. There were some reports of this in version 9, but it was supposedly fixed. Has any one seen this behavior or have a work around?
MikeGood Idea! Why didn't I think of that? Oh yea...I did. Unfortunately TMS CSI is an update only partner type CSI so I cannot submit a SR. The 600 lookup was how I found the old stuff, but it didn't have any 11g references. I hoped maybe someone in the community had encountered this and had a workaround. By the way, the querey looks like so:
select
nation,
o_year,
sum(amount) as sum_profit
from
select
n_name as nation,
extract(year from o_orderdate) as o_year,
l_extendedprice * (1 - l_discount) - ps_supplycost * l_quantity as amount
from
h_part,
h_supplier,
h_lineitem,
h_partsupp,
h_order,
h_nation
where
s_suppkey = l_suppkey
and ps_suppkey = l_suppkey
and ps_partkey = l_partkey
and p_partkey = l_partkey
and o_orderkey = l_orderkey
and s_nationkey = n_nationkey
and p_name like '%spring%'
) as profit
group by
nation,
o_year
order by
nation,
o_year desc;
the other 21 queries, all using the same tables and degrees of paralell and cross instance settings, executed ok.
Mike -
Facing problem with a date column in select query
Hi,
I am facing problem with a date column. Below is my query and its fainling with " invalid number format model" .
Query: SELECT *
FROM EMP
WHERE trunc(LAST_UPDATED) >= to_date(to_char(22-05-2009,'dd-mm-yyyy'),'dd-mm-yyyy')
LAST_UPDATED column is "DATE" data type.
Please help me ThanksRadhakrishna Sarma wrote:
SeánMacGC wrote:
WHERE LAST_UPDATED >= to_date('22-05-2009','dd-mm-yyyy');
You do not need the TRUNC here in any case.
I don't think so. What if the user wants only data for 22nd May and the table has records with date later than 22nd also? In that case your query willl not work. In order for the Index to work, I think the query can be written like this I think Sean is right though. Use of TRUNC Function is quiet useless based on the condition given here, since the to_date Function used by OP will always point to midnight of the specified date, in this case 22-05-2009 00:00:00.
Regards,
Jo
Edit: I think Sean proved his point... ;) -
Problem with logical database DDF in ABAP query
Hi All,
I have created an ABAP query via the transaction SQ01 using the logical database DDF to retrieve customer master data from tables KNA1 & KNB1. The selection screen used is the screen 903 of the logical database DDF. But now the problem is that when I execute the query, I am getting a blank value in the tax code4(KNA1-STCD4) field though there is value in the KNA1 table for this particular field. This field was added to the table by means of a customizing include. Can anyone suggest a way to get around this problem so that I can view the content of the field STCD4 in the query.
Thanks and regards,
Sarath.I think it's not used by DDF.
so you must check it with:
check kna1-bran1 in s_bran1.
A. -
Problem with virtual Key Figure in a query
Hi Experts,
I would like to share my problem with the virtual key figure in a query:
I have a query with a virtual key figure and when I eject the query by analyzer, the result sometimes is displayed and sometimes is not displayed (is equal to 0), with the same initials values. And when I eject the query by RSRT always display the result for the virtual key figure.
Why don't always display the result?
Thanks a lotHi,
I think that the problem is the level of support package. We have BI AddOn 7.X (based on 7.20) support package 0.
What do you think?
Regards -
Hi,
I have a problem with spaces. There are constantly two spaces open named Safari. I cannot remove them. See enclosed image. Any ideas how to do it?Further research shows that the spaces are indeed there when I view the page source; however, the spaces are not being rendered.
I removed the css style sheet that was being applied to the shuttle and it made no difference. -
Problem with different execution paths in hierarchical query
Hello,
I have problems with the following query:
SELECT DISTINCT P.ID FROM PRODUCTELEMENTIMPL P WHERE ( ( LABEL = 'SomeLabel' AND PRODUCTELEMENTTYPE = 'SomeText' AND ( STATE = 'created' OR STATE = 'stored' OR STATE = 'archived' OR STATE = 'archivedRestored' ) ) ) START WITH P.ID = 42 CONNECT BY PRIOR P.ID = P.PARENT
We have two databases (an Oracle 10g XE and Oracle10g Enterprise). In the XE Database the query is executed very fast, but in the main installation it takes minutes. If I "explain" the query I get two different execution paths:
The fast:
ID PARENT_ID LEVEL SQL Kosten Anzahl Zeilen
0 - 1 SELECT STATEMENT 20 49
1 0 2 HASH UNIQUE 20 49
2 1 3 FILTER - -
3 2 4 CONNECT BY WITH FILTERING - -
4 3 5 TABLE ACCESS BY INDEX ROWID PRODUCTELEMENTIMPL (TABLE) - -
5 4 6 INDEX UNIQUE SCAN SYS_C0072201 (INDEX (UNIQUE)) 2 1
6 3 5 NESTED LOOPS - -
7 6 6 BUFFER SORT - -
8 7 7 CONNECT BY PUMP - -
9 6 6 TABLE ACCESS BY INDEX ROWID PRODUCTELEMENTIMPL (TABLE) 19 49
10 9 7 INDEX RANGE SCAN PRODUCTELEMENTIMPL_IDX1 (INDEX) 3 49
11 3 5 TABLE ACCESS FULL PRODUCTELEMENTIMPL (TABLE) 19 49
Slow:
ID PARENT_ID LEVEL SQL Kosten Anzahl Zeilen
0 1 SELECT STATEMENT 1 1
1 0 2 HASH UNIQUE 1 1
2 1 3 FILTER
3 2 4 CONNECT BY WITHOUT FILTERING
4 3 5 TABLE ACCESS BY INDEX ROW 3 1
ID PRODUCTELEMENTIMPL (TABLE)
5 4 6 INDEX UNIQUE SCAN SYS_C0 2 1
020528 (INDEX (UNIQUE))
6 3 5 TABLE ACCESS FULL PRODUCT 6628 1100613
ELEMENTIMPL (TABLE)
Any ideas how to avoid this full table scan?
bye
Roland SpatzeneggerHello,
thank you for your replies. The indices and table schemas are the "same", but only the content for the tables was mirrored.
We made some tests with dropping and/or analyzing the tables, but it didn't change anything.
The main problem is that the query takes 33s in the productive environment for searching in a couple of rows. At the moment it's faster to make
SELECT DISTINCT P.ID, P.STATE FROM PRODUCTELEMENTIMPL P WHERE ( ( LABEL = 'SomeLabel' AND PRODUCTELEMENTTYPE = 'SomeText' ) ) START WITH P.ID = 42 CONNECT BY PRIOR P.ID = P.PARENT
and to test in the application if the state-values match ;-)
If I add the hint /*+ no_filtering */ in the test environment, I get the same "slow" execution path as in the production environment. So the question is, what prevents the filtering in "connect by"?
(I think in the fast version it filters only the results of the hierarchical query, in the slow version it first filters the whole table and joins/merge it with the hierachical result).
bye
Roland Spatzenegger -
Problem with 'Filling Space' described in Tim's Blog
Dear Experts,
I am using the Invoice Template described in the excellent Blog by Tim Dexter (Anatomy of a Template) to produce my own version. However, I am having problems with the 'Filling Space' described here: http://blogs.oracle.com/xmlpublisher/2007/03/anatomy_of_a_template_ii_heade.html
My template is almost identical to the example, when I import XML data where there is only one record, the Filling Space does not pad out with enough empty rows so that the Total table is not at the bottom of the page, aligned with the watermark that I have. The code in the 'Filling Space' field is as in the Blog:
<?if:not(count($invLines) mod $lpp=0) and ($start+$lpp>count($invLines))?><?for-each:$invLines?><?if:position()<$lpp - (count($invLines) mod $lpp)?>
If the number of records is greater than half a page or more, then the padding is correct. This is even if, say on the third page there is just one record. Therefore, it seems the padding does not work if the number of records are less than half a page in total.
Is this correct and does anyone have a solution - would be greatly appreciated.
Many thanks,
Mark"If the number of records is greater than half a page or more, then the padding is correct. This is even if, say on the third page there is just one record. Therefore, it seems the padding does not work if the number of records are less than half a page in total."
This statement is wrong.
Leave the if conditions logic,
This is how it works,..
xml contains 43 rows...
each page you need 10 rows..
after every 10th rows [using position mod 10 ,] , you put a page break...
if the mod returns greater than 0, that means row filler has to come into effect now.
what is the logic now to have row fillers is , 10 - 3 [position mod 10] , comes to 7... so seven blank rows should be inserted..
this is how , it works...
so the statement, saying, less than half of page, or something, wont come into this :).
http://winrichman.blogspot.com/2008/09/limit-row-per-page.html -
Problem with free space on itnernal hdd on mac book pro!
On the about this mac menu i storage i get 512 gb of backups and if you add all the numbers given for the internal hd the total is 837.88 gb on a 750 gb hd! When i go to finder my macbook pro and get info on the macintosh hd ->get info i get 582 bg free! i have repaird the disk permitions and repaird the disk! but it doesnt seen to work!
what i want to say is that there is a problem with reading the actual free space from the hard drive....i had 259gb free space and after deleting 100gb + and restarting i had only 100 gb free indicated...!
these are the values i take at the same time!! they are obviusly both wrong!! -
Having problem with extra space below footer (sticky footer already checked)
Hi there, im having this problem after updated my adobe muse cc. For some reason there is extra space below the footer.
It looks fine when i try to view them offline, but when I upload them, the extra space appears. Here's the url for both of the website with extra space below
1. http://unionvisual.web.id/home-page.html
2. PT-SmasHi there
Thx for your reply, is here i upload my screenshot
and here -
Strange Problem With Extra Spaces
Hello,
My development staff is experiencing a very strange problem with our JSP pages which display the values from a JDBC result set. We have two environments here: development and production. We have been creating pages in development which work out fine in our development environment. But when we move the page into production, each value which results from the ResultSet getString() method is having multiple blank spaces appended to the end of the value. So, for example, a 5 digit zip code field is really showing up with a length of 8 or 9 because there are extra spaces added to the end of the value.
We are able to work around this problem by using the trim() method, but this is getting quite tedious to do to every single page in the site. It is very annoying because the problem does not exist in the development environment.
So you figure there must be a difference in our development and production environments. But there really isn't. Here is the details for both environments:
OS: Windows NT 4.0
Web Server: Apache with Tomcat setup as a plugin
Database: Sybase ASA 6.0.3
JDK: 1.3
Apache and Tomcat were installed from the same download on both servers. Sybase was installed from the same CD on both servers - and the JDBC Driver (SybDriver) was copied from one server to the next.
Sorry for the long explanation - but does anyone have any idea what could be occurring? Why are the extra spaces being appended to the production pages and not the development pages?
As always, thanks in advance for any suggestions!
-- ChrisHello Paul,
Thank you for taking the time to analyze this case.
The data in the system comes from two different sources. The first source is a nightly feed from a legacy system. The second source is user input via the JSP application. The records that were inserted from the nightly feed have an accurate length() value (meaning there is no untrimmed spaces trailing along.) But the values coming in from the application have extra spaces appended to them. In the development environment, however, both sources provide properly trimmed data.
If I enter an 8 character last name into the web form in the production app, the value which ends up in the database upon submission will have 2 to 3 extra spaces. I know that this sounds strange - and believe me, it is boggling everyone here. I suspect this problem has to do with the Tomcat installation - even though it was installed identically in each server.
I think we are just going to have live with using the trim() for every database submission. As always, thank you for the help, and all ideas are always welcome.
-- Chris -
11.2.2.4.0 - Problem with temporary space in simple query
ttVersion
TimesTen Release 11.2.2.4.0 (64 bit Linux/x86_64) (timesten:53396) 2012-09-24T08:28:05Z
Instance admin: root
Instance home directory: /opt/TimesTen/timesten
World accessible
Daemon home directory: /var/TimesTen/timesten
I get "TT0802: Database temporary space exhausted" error in simple query with small data amount; Timesten try to allocate *40000312* bytes
describe adm.peer
Table ADM.PEER:
Name Null Type
PEER_ID NOT NULL TT_SMALLINT
CLUSTER_ID NOT NULL TT_TINYINT
DIALECT NOT NULL TT_INTEGER
HOST NOT NULL TT_VARCHAR(256 BYTE)
REALM NOT NULL TT_VARCHAR(256 BYTE)
ADDRESS TT_VARCHAR(256 BYTE)
PORT NOT NULL TT_INTEGER
PROTOCOL NOT NULL TT_INTEGER
AUTO_CONNECT NOT NULL TT_TINYINT
ENABLED NOT NULL TT_TINYINT
PRIORITY NOT NULL TT_TINYINT
MANDATORY NOT NULL TT_TINYINT
TSTAMP BINARY(8)
1 rows selected
describe adm.session
Table ADM.SESSION:
Name Null Type
SESSION_ID NOT NULL TT_VARCHAR(64 BYTE) inline
OBJ_ID NOT NULL TT_BIGINT
PR_OBJ_ID NOT NULL TT_BIGINT
SUBSCRIBER_ID NOT NULL TT_VARCHAR(32 BYTE) inline
IP NOT NULL TT_VARCHAR(15 BYTE) inline
IPV6_PREFIX TT_VARCHAR(39 BYTE) inline
IPV6_PREFIX_LEN NOT NULL TT_TINYINT
CREATE_TIME NOT NULL TT_TIMESTAMP
UPDATE_TIME NOT NULL TT_TIMESTAMP
RULES_SET_ID NOT NULL TT_BIGINT
PEER_ID NOT NULL TT_SMALLINT
MY_PEER_ID NOT NULL TT_SMALLINT
PROFILE_HASHC NOT NULL TT_BIGINT
FLAGS NOT NULL TT_INTEGER
QOS_POLICY_NAME NOT NULL TT_VARCHAR(32 BYTE) inline
BSID NOT NULL TT_BIGINT
CONGESTION_FLAG NOT NULL TT_TINYINT
SERVICE_CATEGORY_ID TT_VARCHAR(32 BYTE) inline
EVENT_CAUSE NOT NULL TT_TINYINT
EVENT_TIME TT_TIMESTAMP
TSTAMP BINARY(8)
1 rows selected
select * from adm.peer;
PEER_ID CLUSTER_ID DIALECT HOST REALM ADDRESS PORT PROTOCOL AUTO_CONNECT ENABLED PRIORITY MANDATORY TSTAMP
21 2 0 ddf1.server.com diameter.realm ddf1.server.com 3868 6 1 1 0 1 (null)
22 2 0 ddf2.server.com diameter.realm ddf2.server.com 3868 6 1 1 1 1 (null)
101 233 0 peer_101 testik.com peer_101.testik.com 3886 0 0 1 101 0 (null)
102 233 0 peer_102 testik.com peer_102.testik.com 3886 0 0 1 102 0 (null)
1 1 0 vr-t500.testik.com diameter.realm vr-t500.testik.com 3868 6 1 1 0 1 (null)
5 rows selected
select * from adm.session;
SESSION_ID OBJ_ID PR_OBJ_ID SUBSCRIBER_ID IP IPV6_PREFIX IPV6_PREFIX_LEN CREATE_TIME UPDATE_TIME RULES_SET_ID PEER_ID MY_PEER_ID PROFILE_HASHC FLAGS QOS_POLICY_NAME BSID CONGESTION_FLAG SERVICE_CATEGORY_ID EVENT_CAUSE EVENT_TIME TSTAMP
TEST_SESSION 13300000000020027 0 TEST_SUBSCRIBER 94.25.209.27 0 2012-10-18 12:56:07.155381000 2012-10-18 12:56:07.155381000 1 101 1 0 0 0 0 DEFAULT 0 (null) (null)
TEST_SESSION2 13300000000020028 13300000000020027 TEST_SUBSCRIBER 94.25.209.27 0 2012-10-18 12:56:07.155687000 2012-10-18 12:56:07.155687000 1 102 1 0 4 0 0 DEFAULT 0 (null) (null)
2 rows selected
SELECT p.address, count(*) as session_count from session s, peer p where p.peer_id = s.peer_id group by p.address failed,
TT0802: Database temporary space exhausted
dssize
PERM_ALLOCATED_SIZE: 307200.0
PERM_IN_USE_SIZE: 61763.0
PERM_IN_USE_HIGH_WATER: 69393.0
TEMP_ALLOCATED_SIZE: 37888.0
TEMP_IN_USE_SIZE: 13494.0
TEMP_IN_USE_HIGH_WATER: 21307.0
This is additional error info when this code exuted inside C code:
[TimesTen][TimesTen 11.2.2.4.0 ODBC Driver][TimesTen]TT0802: Database temporary space exhausted -- file "blk.c", lineno 3477, procedure "sbBlkAlloc"
ODBC Error/Warning = S1000, Additional Error/Warning = 802
[TimesTen][TimesTen 11.2.2.4.0 ODBC Driver][TimesTen]TT6221: Temporary data partition free space insufficient to allocate *40000312* bytes of memory -- file "blk.c", lineno 3477, procedure "sbBlkAlloc"
ODBC Error/Warning = S1000, Additional Error/Warning = 6221
Edited by: Vladimir Romanov on 18.10.2012 13:13
Edited by: Vladimir Romanov on 18.10.2012 13:51This may well be
Bug 14634954 - SELECT WITH GROUP BY REQUESTS LARGE TEMP MEMORY GETS TT0802 / TT6221
The bug is fixed in 11.2.2.4.1 which is hopefully due before the end of October. Can you run your test on 11.2.1 as well? The problem should not reproduce there as it is specific to 11.2.2 -
Problem with available space on bootcamp
Hi everybody, I installed Windows 7 on my MBA to get some specifical applications, and I don't have available memory anymore.
I created a 35go allocation for windows and installed only two softwares : one of 3Go and one of 1Go, and I noticed that everyday I had less memory available.
I definitely can't find what's the problem, I thought that it was the "restoration" process, but it isn't, I installed CC cleaner and it only gave me 365mo more space.
Thanks in advance for your help !
Charles from FranceThere are 2 reliable methods to enlarge a Windows Partition on a Mac (in Boot Camp)
1. Remove the BC created partition using Boot Camp, this will return your drive to as it was prior to partitioning it with BC. Then start over.
2. Paragon Camp Tune can resize the existing partition without removing it, Camp Tune may be purchased here: -
Problem with a Currency field in Adhoc Query - HR
Hi,
I have an Adhoc query that uses Custom infotype fields (Z infotype and z fields).
The currncy field also has a reference field in the infotype (of type waers).
Wehen we try to get the ouput of the Adhoc Query it gives following error:
The report cannot be generated because the internal description is invalid or incomplete, or because the selection screen is too large.
Regenerate the assigned InfoSet, and read the log. If the InfoSet is OK, make sure that at least one field is given as output.
If you used the 'Refresh' icon to start the query, use the 'Output' menu option to execute the query. This gives you a full screen display of the data.
If an output was generated, the query cannot work with actual data in the construction view. In this case, always use the 'Output' function to execute the query.
I have already tried a number of solutions:
1> Regenerate the infoset...
2> make the field as an additional field and write my own code for it (the error comes before the code as i kept a breakpoint but it stopped before that)
if i add other fields of this infoset instead of this field, then those appear in the output.
Any solutions ??
thanks in advance,
AnujHi,
Is the problem not clear or no one has an answer?
Please reply with some suggestions..
Regards,
Anuj.
Maybe you are looking for
-
Iphone cable doesn't work just ipod cable
my iphone only works with a cable of ipod (just charge) does not synchronize does not charge with any other iphone cable... also don't synchronize with a ipod cable... plase help me I have an iphone classic and I can't update the iphone because the c
-
How to set Max size of InternalFrame should be size of JFrame?
Hi, I added the toolbar on the BorderLayout North, JDesktopPane on the BorderLayout Center and the statusBar on the BorderLayout South. Whenever the toolbar gets clicked,the JInternalFrame is added into the JDesktopPane. I want like, If is click the
-
Hello - Does anybody know how to make the Finder show previews of MTS (video) files??
-
Re-Creating the Help button in a Custom Header portlet
I've disabled the Top Bar from my portal, and need to create the Help within my custom Header portlet. I can't find any code on how to have it dynamically work? Is it even possible?
-
Migration assistant on my old PB G4 freezes when trying to use it
Have looked for a solution to this for a few days now. I have tried to use the migration assistant to facilitate the change to my new Macbook Pro. However, each time I try to use it it freezes on the PB when it gets to the point where my ny MB search