Make parallel query (e.g. partition-wise join) evenly distributed cross RAC

How To Make Parallel Query's Slaves (e.g. full partition-wise join) Evenly Distributed Across RAC Nodes?
Environment
* 4-node Oracle 10gR2 (10.2.0.3)
* all instances are included in the same distribution group
* tables are hash-partitioned by the same join key
* 8-CPU per node, 48GB RAM per node
Query
Join 3 big tables (each has DOP=4) based on the hash partition key column.
Problem
The QC is always on one node, and all the slaves are on another node. The slave processes are supposed to be distributed or allocated to multiple nodes/instances, but even the query spawns 16 or more slaves, these slaves are running from only one node. And the QC process is never running on the same node! ? ! ? !
The other 2 nodes are not busy during this time. Is there any configuration wrong or missing here? Why can't the RAC distribute the slaves better, or at least run some slaves together with QC?
Please advise.
Thank you very much!
Eric

Hi,
If your PARALLEL_INSTANCE_GROUP and LOAD_BALANCING is set properly it means the oracle is assuming that intra node parallelism is more beneficial than inter node parallelism, i mean parallelism across multiple nodes. This is very true in scenarios where partition wise joins are involved. intra node parallelism avoids unnecessary interconnect traffic.

Similar Messages

Expected for a partition-wise join?

I have two tables that are partitioned by a hash of the same VARCHAR2(16) strings. When I do a query similar to
select * from table1 a join table2 b
on b.partition_Column = a.partition_Column
I get the following as the "Operation" portion of an explain from Oracle Developer running Oracle 11gR1:
PARTITION HASH(ALL)
HASH JOIN
TABLE ACCESS(FULL) schema.Table1
TABLE ACCESS(FULL) schema.Table2
Is this indicative of a partition-wise hashed join?

pstart/pstop does give partition information but the key to whether this is a "partition-wise" join, is that the partition operation is above the join operation.
Using David's tables above, example of serial partition-wise join:
| Id | Operation           | Name   | Rows | Bytes |TempSpc| Cost (%CPU)| Time     | Pstart| Pstop |
|   0 | SELECT STATEMENT    |        | 2500M| 4670G|       |   111K (61)| 00:09:17 |       |       |
|   1 | PARTITION HASH ALL |        | 2500M| 4670G|       |   111K (61)| 00:09:17 |     1 |     8 |
|* 2 |   HASH JOIN         |        | 2500M| 4670G|    12M|   111K (61)| 00:09:17 |       |       |
|   3 |    TABLE ACCESS FULL| TABLE1 |   100K|    95M|       | 5891   (1)| 00:00:30 |     1 |     8 |
|   4 |    TABLE ACCESS FULL| TABLE2 |   200K|   191M|       | 11808   (1)| 00:01:00 |     1 |     8 |
------------------------------------------------------------------------------------------------------Example of serial non PWJ:
| Id | Operation           | Name   | Rows | Bytes |TempSpc| Cost (%CPU)| Time     | Pstart| Pstop |
|   0 | SELECT STATEMENT    |        | 2500M| 4670G|       |   111K (61)| 00:09:17 |       |       |
|* 1 | HASH JOIN          |        | 2500M| 4670G|    96M|   111K (61)| 00:09:17 |       |       |
|   2 |   PARTITION HASH ALL|        |   100K|    95M|       | 5891   (1)| 00:00:30 |     1 |     8 |
|   3 |    TABLE ACCESS FULL| TABLE1 |   100K|    95M|       | 5891   (1)| 00:00:30 |     1 |     8 |
|   4 |   PARTITION HASH ALL|        |   200K|   191M|       | 11808   (1)| 00:01:00 |     1 |     8 |
|   5 |    TABLE ACCESS FULL| TABLE2 |   200K|   191M|       | 11808   (1)| 00:01:00 |     1 |     8 |
------------------------------------------------------------------------------------------------------Example of parallel PWJ:
| Id | Operation               | Name     | Rows | Bytes | Cost (%CPU)| Time     | Pstart| Pstop |    TQ |IN-OUT| PQ Distrib |
|   0 | SELECT STATEMENT        |          | 2500M| 4670G| 23536 (80)| 00:01:58 |       |       |        |      |            |
|   1 | PX COORDINATOR         |          |       |       |            |          |       |       |        |      |            |
|   2 |   PX SEND QC (RANDOM)   | :TQ10000 | 2500M| 4670G| 23536 (80)| 00:01:58 |       |       | Q1,00 | P->S | QC (RAND) |
|   3 |    PX PARTITION HASH ALL|          | 2500M| 4670G| 23536 (80)| 00:01:58 |     1 |     8 | Q1,00 | PCWC |            |
|* 4 |     HASH JOIN           |          | 2500M| 4670G| 23536 (80)| 00:01:58 |       |       | Q1,00 | PCWP |            |
|   5 |      TABLE ACCESS FULL | TABLE1   |   100K|    95M| 1628   (1)| 00:00:09 |     1 |     8 | Q1,00 | PCWP |            |
|   6 |      TABLE ACCESS FULL | TABLE2   |   200K|   191M| 3263   (1)| 00:00:17 |     1 |     8 | Q1,00 | PCWP |            |
---------------------------------------------------------------------------------------------------------------------------------E.g. parallel non PWJ:
| Id | Operation                  | Name     | Rows | Bytes | Cost (%CPU)| Time     | Pstart| Pstop |    TQ |IN-OUT| PQ Distrib |
|   0 | SELECT STATEMENT           |          | 2500M| 4670G| 23536 (80)| 00:01:58 |       |       |        |      |            |
|   1 | PX COORDINATOR            |          |       |       |            |          |       |       |        |      |            |
|   2 |   PX SEND QC (RANDOM)      | :TQ10001 | 2500M| 4670G| 23536 (80)| 00:01:58 |       |       | Q1,01 | P->S | QC (RAND) |
|* 3 |    HASH JOIN               |          | 2500M| 4670G| 23536 (80)| 00:01:58 |       |       | Q1,01 | PCWP |            |
|   4 |     PART JOIN FILTER CREATE| :BF0000 |   100K|    95M| 1628   (1)| 00:00:09 |       |       | Q1,01 | PCWP |            |
|   5 |      PX RECEIVE            |          |   100K|    95M| 1628   (1)| 00:00:09 |       |       | Q1,01 | PCWP |            |
|   6 |       PX SEND BROADCAST    | :TQ10000 |   100K|    95M| 1628   (1)| 00:00:09 |       |       | Q1,00 | P->P | BROADCAST |
|   7 |        PX BLOCK ITERATOR   |          |   100K|    95M| 1628   (1)| 00:00:09 |     1 |     8 | Q1,00 | PCWC |            |
|   8 |         TABLE ACCESS FULL | TABLE1   |   100K|    95M| 1628   (1)| 00:00:09 |     1 |     8 | Q1,00 | PCWP |            |
|   9 |     PX BLOCK ITERATOR      |          |   200K|   191M| 3263   (1)| 00:00:17 |:BF0000|:BF0000| Q1,01 | PCWC |            |
| 10 |      TABLE ACCESS FULL     | TABLE2   |   200K|   191M| 3263   (1)| 00:00:17 |:BF0000|:BF0000| Q1,01 | PCWP |            |
------------------------------------------------------------------------------------------------------------------------------------Edited by: Dom Brooks on Jul 8, 2011 11:32 AM
Added serial vs parallel

Partition wise join

Hello,
i'm playing with partition. I have read about (full) partition wise join in hash-hash table. The description and the example is in Oracle Database Warehousing guide (b14223.pdf) in chaper 5.
<cite>
A full partition-wise join divides a large join into smaller joins between a pair of
partitions from the two joined tables. To use this feature, you must equipartition both
tables on their join keys. For example, consider a large join between a sales table and a
customer table on the column customerid. The query "find the records of all
customers who bought more than 100 articles in Quarter 3 of 1999" is a typical example
of a SQL statement performing such a join. The following is an example of this:
SELECT c.cust_last_name, COUNT(*)
FROM sales s, customers c
WHERE s.cust_id = c.cust_id AND
s.time_id BETWEEN TO_DATE('01-JUL-1999', 'DD-MON-YYYY') AND
(TO_DATE('01-OCT-1999', 'DD-MON-YYYY'))
GROUP BY c.cust_last_name HAVING COUNT(*) > 100;
</cite>
I have created some sales table with 1M rows and customers table with 1k rows (it means that approximately each customer has one thousand of invoices). Both tables are hash partitioned with 64 partitions. The are analyzed. But there is not any improvement comparing the same table without paritioning and with index on customerid column.
I would just to see that partition wise joins is working. It means is is either faster or it consume less resources.
Thanks
sasa

I don't think that 64 partition is too many ... the problem is that there isn't any gain from partitioning.
Lets see the explain plan:
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | Pstart| Pstop |
| 0 | SELECT STATEMENT | | 50 | 2500 | 11436 (5)| 00:02:18 | | |
|* 1 | FILTER | | | | | | | |
| 2 | HASH GROUP BY | | 50 | 2500 | 11436 (5)| 00:02:18 | | |
|* 3 | HASH JOIN | | 903K| 43M| 11320 (4)| 00:02:16 | | |
| 4 | PARTITION HASH ALL| | 1000 | 34000 | 88 (0)| 00:00:02 | 1 | 64 |
| 5 | TABLE ACCESS FULL| PART_CUSTOMERS | 1000 | 34000 | 88 (0)| 00:00:02 | 1 | 64 |
| 6 | PARTITION HASH ALL| | 903K| 13M| 11219 (4)| 00:02:15 | 1 | 64 |
|* 7 | TABLE ACCESS FULL| PART_INVOICES | 903K| 13M| 11219 (4)| 00:02:15 | 1 | 64 |
And compare this explain plan with the following which is created on non-partition tables (the cost of expalin plan for non partitioned tables is 180 compare to cost 11436 of partitioned tables):
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
| 0 | SELECT STATEMENT | | 1 | 46 | 180 (2)| 00:00:03 |
|* 1 | FILTER | | | | | |
| 2 | HASH GROUP BY | | 1 | 46 | 180 (2)| 00:00:03 |
|* 3 | TABLE ACCESS BY INDEX ROWID | PART_INVOICES2 | 903 | 10836 | 177 (1)| 00:00:03 |
| 4 | NESTED LOOPS | | 903 | 41538 | 179 (1)| 00:00:03 |
| 5 | TABLE ACCESS BY INDEX ROWID| PART_CUSTOMERS2 | 1 | 34 | 2 (0)| 00:00:01 |
|* 6 | INDEX RANGE SCAN | PART_CUSTOMERS2_IDX01 | 1 | | 1 (0)| 00:00:01 |
|* 7 | INDEX RANGE SCAN | PART_INVOCIES2_IDX02 | 9991 | | 22 (0)| 00:00:01 |
---------------------------------------------------------------------------------------------------------

Hint to disable partition wise join

Is there a way to disable partition wise join(serial) in 10gR2? i.e. via hint.. The reason I want to do this is, to use intra-partition parallelism for a very big partition. re-partitioning or subpartitioning is not an option for now. SQL is scanning only one partition so P-W join is not useful and it limit the intra-partition parallelism.
TIA for your answers.

user4529833 wrote:
Above is the plan. Currently there is no prallelism being used but P-W join is used as you can see. Table EC is huge .. (cardinality is screwed up here becasue of IN clause , which has just one vallid part key. [ 3rd party crappy app, so can't change it.] ) . I'd like to enable parallelism here using parallel (EC, 6) hint , it just applied to hash-join and not to table EC because of P-W join, I believe. What I want is to scan EC table via PQ slave.. i.e. PX BLOCK INTERATOR step before TABLE access step... How do I get one? Will PQ_DISTRIBUTE help me there??? or Is there any way to speed up the scan of EC..
The pq_distribute() should do the job. Here's an example
select
     /*+
          parallel(pt_range_1 2)
          parallel(pt_range_2 2)
          ordered
--          pq_distribute(pt_range_2 hash hash)
--          pq_distribute(pt_range_2 broadcast none)
     pt_range_2.grp,
     count(pt_range_1.small_vc)
from
     pt_range_1,
     pt_range_2
where
     pt_range_1.id in (10,20,40)
and     pt_range_2.id = pt_range_1.id
group by
     pt_range_2.grp
| Id | Operation                      | Name       | Rows | Bytes | Cost (%CPU)| Time     | Pstart| Pstop |    TQ |IN-OUT| PQ Distrib |
|   0 | SELECT STATEMENT               |            |     3 |    42 |     6 (34)| 00:00:01 |       |       |        |      |            |
|   1 | PX COORDINATOR                |            |       |       |            |          |       |       |        |      |            |
|   2 |   PX SEND QC (RANDOM)          | :TQ10001   |     3 |    42 |     6 (34)| 00:00:01 |       |       | Q1,01 | P->S | QC (RAND) |
|   3 |    HASH GROUP BY               |            |     3 |    42 |     6 (34)| 00:00:01 |       |       | Q1,01 | PCWP |            |
|   4 |     PX RECEIVE                 |            |     3 |    42 |     5 (20)| 00:00:01 |       |       | Q1,01 | PCWP |            |
|   5 |      PX SEND HASH              | :TQ10000   |     3 |    42 |     5 (20)| 00:00:01 |       |       | Q1,00 | P->P | HASH       |
|   6 |       PX PARTITION RANGE INLIST|            |     3 |    42 |     5 (20)| 00:00:01 |KEY(I) |KEY(I) | Q1,00 | PCWC |            |
|* 7 |        HASH JOIN               |            |     3 |    42 |     5 (20)| 00:00:01 |       |       | Q1,00 | PCWP |            |
|* 8 |         TABLE ACCESS FULL      | PT_RANGE_1 |     3 |    21 |     2   (0)| 00:00:01 |KEY(I) |KEY(I) | Q1,00 | PCWP |            |
|* 9 |         TABLE ACCESS FULL      | PT_RANGE_2 |     3 |    21 |     2   (0)| 00:00:01 |KEY(I) |KEY(I) | Q1,00 | PCWP |            |
------------------------------------------------------------------------------------------------------------------------------------------Unhinted I have a partition-wise parallel join.
The next plan is using hash disrtibution - which may be better for you if the EX_C table is large:
| Id | Operation                  | Name       | Rows | Bytes | Cost (%CPU)| Time     | Pstart| Pstop |    TQ |IN-OUT| PQ Distrib |
|   0 | SELECT STATEMENT           |            |     3 |    42 |     6 (34)| 00:00:01 |       |       |        |      |            |
|   1 | PX COORDINATOR            |            |       |       |            |          |       |       |        |      |            |
|   2 |   PX SEND QC (RANDOM)      | :TQ10003   |     3 |    42 |     6 (34)| 00:00:01 |       |       | Q1,03 | P->S | QC (RAND) |
|   3 |    HASH GROUP BY           |            |     3 |    42 |     6 (34)| 00:00:01 |       |       | Q1,03 | PCWP |            |
|   4 |     PX RECEIVE             |            |     3 |    42 |     5 (20)| 00:00:01 |       |       | Q1,03 | PCWP |            |
|   5 |      PX SEND HASH          | :TQ10002   |     3 |    42 |     5 (20)| 00:00:01 |       |       | Q1,02 | P->P | HASH       |
|* 6 |       HASH JOIN BUFFERED   |            |     3 |    42 |     5 (20)| 00:00:01 |       |       | Q1,02 | PCWP |            |
|   7 |        PX RECEIVE          |            |     3 |    21 |     2   (0)| 00:00:01 |       |       | Q1,02 | PCWP |            |
|   8 |         PX SEND HASH       | :TQ10000   |     3 |    21 |     2   (0)| 00:00:01 |       |       | Q1,00 | P->P | HASH       |
|   9 |          PX BLOCK ITERATOR |            |     3 |    21 |     2   (0)| 00:00:01 |KEY(I) |KEY(I) | Q1,00 | PCWC |            |
|* 10 |           TABLE ACCESS FULL| PT_RANGE_1 |     3 |    21 |     2   (0)| 00:00:01 |KEY(I) |KEY(I) | Q1,00 | PCWP |            |
| 11 |        PX RECEIVE          |            |     3 |    21 |     2   (0)| 00:00:01 |       |       | Q1,02 | PCWP |            |
| 12 |         PX SEND HASH       | :TQ10001   |     3 |    21 |     2   (0)| 00:00:01 |       |       | Q1,01 | P->P | HASH       |
| 13 |          PX BLOCK ITERATOR |            |     3 |    21 |     2   (0)| 00:00:01 |KEY(I) |KEY(I) | Q1,01 | PCWC |            |
|* 14 |           TABLE ACCESS FULL| PT_RANGE_2 |     3 |    21 |     2   (0)| 00:00:01 |KEY(I) |KEY(I) | Q1,01 | PCWP |            |
--------------------------------------------------------------------------------------------------------------------------------------Then the broadcast version if the EC_C data is relatively small (so that the whole set can fit in the memory of each slave)
| Id | Operation                  | Name       | Rows | Bytes | Cost (%CPU)| Time     | Pstart| Pstop |    TQ |IN-OUT| PQ Distrib |
|   0 | SELECT STATEMENT           |            |     3 |    42 |     6 (34)| 00:00:01 |       |       |        |      |            |
|   1 | PX COORDINATOR            |            |       |       |            |          |       |       |        |      |            |
|   2 |   PX SEND QC (RANDOM)      | :TQ10002   |     3 |    42 |     6 (34)| 00:00:01 |       |       | Q1,02 | P->S | QC (RAND) |
|   3 |    HASH GROUP BY           |            |     3 |    42 |     6 (34)| 00:00:01 |       |       | Q1,02 | PCWP |            |
|   4 |     PX RECEIVE             |            |     3 |    42 |     5 (20)| 00:00:01 |       |       | Q1,02 | PCWP |            |
|   5 |      PX SEND HASH          | :TQ10001   |     3 |    42 |     5 (20)| 00:00:01 |       |       | Q1,01 | P->P | HASH       |
|* 6 |       HASH JOIN            |            |     3 |    42 |     5 (20)| 00:00:01 |       |       | Q1,01 | PCWP |            |
|   7 |        PX RECEIVE          |            |     3 |    21 |     2   (0)| 00:00:01 |       |       | Q1,01 | PCWP |            |
|   8 |         PX SEND BROADCAST | :TQ10000   |     3 |    21 |     2   (0)| 00:00:01 |       |       | Q1,00 | P->P | BROADCAST |
|   9 |          PX BLOCK ITERATOR |            |     3 |    21 |     2   (0)| 00:00:01 |KEY(I) |KEY(I) | Q1,00 | PCWC |            |
|* 10 |           TABLE ACCESS FULL| PT_RANGE_1 |     3 |    21 |     2   (0)| 00:00:01 |KEY(I) |KEY(I) | Q1,00 | PCWP |            |
| 11 |        PX BLOCK ITERATOR   |            |     3 |    21 |     2   (0)| 00:00:01 |KEY(I) |KEY(I) | Q1,01 | PCWC |            |
|* 12 |         TABLE ACCESS FULL | PT_RANGE_2 |     3 |    21 |     2   (0)| 00:00:01 |KEY(I) |KEY(I) | Q1,01 | PCWP |            |
--------------------------------------------------------------------------------------------------------------------------------------The "hash join buffered" in the hash/hash distribution might hammer your temporary tablespace though, thanks to [an oddity I discovered |http://jonathanlewis.wordpress.com/2008/11/05/px-buffer/] in parallel hash joins a little while ago.
Regards
Jonathan Lewis
http://jonathanlewis.wordpress.com
http://www.jlcomp.demon.co.uk
"Science is more than a body of knowledge; it is a way of thinking" Carl Sagan

Partition Pruning vs Partition-Wise Join

Hi,
I am not sure if this is the right place for this question, but here it goes.
I am in a situation where in the begining I have to join two big tables without any where clauses. It is pretty much a Caretsian Product where a criteria meets (Internal Code), but I have to join all rows. (Seems like a good place for Partition-Wise Join).
Later I only need to update certain rows based on a key value (Customer ID, Region ID). (Good candidate for Partition Pruning).
What would be the best option. Is there a way to use both?
Assume that following:
Table 1 has the structure of
Customer ID
Internal Code
Other Data
There are about 1000 Customer ID. Each Customer ID has 1000 Internal Codes.
Table 2 has the structure of
Region ID
Internal Code
Other Data
There are about 5000 Region ID. Each Region ID has 1000 Internal Codes(same as Table 1).
I am currently thinking of doing a HASH PARTITION (8 partitions) on Customer ID for Table 1 and HASH PARTITION (8 partitions) on Region ID for Table 2.
The initial insert will take a long time, but when I go to update the joined data based on specific Customer ID, or Region ID atleast from one Table only one Partition will be used.
I would sincerely appreciate some advice from the gurus.
Thanks...

Hi,
I still don't understand what it is that you are trying to do.
Would it be possible for you to create a silly example with just a few rows
to show us what it is that you are trying to accomplish?
Then we can help you solve whatever problem it is that you are having.
create table t1(
   customer_id   number       not null
,internal_code varchar2(20) not null
,<other_columns>
,constraint t1_pk primary key(customer_id, internal_code)
create table t2(
   region_id number not null
,internal_code varchar2(20) not null
,<other_columns>
,constraint t2_pk primary key(region_id, internal_code)
insert into t1(customer_id, internal_code, ...) values(...);
insert into t1(customer_id, internal_code, ...) values(...);
insert into t2(region_id, internal_code, ...) values(...);
insert into t2(region_id, internal_code, ...) values(...);
select <the rating calculation>
   from t1 join t2 using(internal_code);

Hash Partitioning and Partition-Wise Joins

Hi,
For the ETL process of a Data Warehouse project I have to join 2 tables (~10M rows) over their primary key.
I was thinking of hash partitioning these 2 tables over their PK. Through this the database (9.2) should be able to do a Hash-Hash Full Partition-wise Join. For more detail about that you can have a look at:
http://download-west.oracle.com/docs/cd/B10501_01/server.920/a96520/parpart.htm#98291
What I'm looking for are some documents or recommandation concerning the number of hash partitions to create depending on the number of rows of the tables, CPU of the server or any other parameters.
I would be grateful if someone could give some input.
Mike

here you have all papers:
Oracle9i Database List of Books
(Release 2 (9.2))
http://otn.oracle.com/pls/db92/db92.docindex?remark=homepage
Joel P�rez

Partitioning For Optimal Parallel Query Execution

Hi All,
We are trying to design an architecture that benefits from partitioning and parallel query to obtain the best query response times for our system.
Let me start by describing the main table which has five columns:
Columns:
1) DocId ------- Numeric Primary Key Constraint (Unique)
2) Text ------- CLOB of XML Content
3) SCode ------- Varchar 12 ( service code Can be one of 200 values)
4) A_Date ------- Oracle Date ( The arrival date )
5) A_Month ------- Numeric partition key, the month number (1-12)
We insert 100,000 records daily so a month of data will contain 3,000,000 rows. The Text varies from 4k to 200k bytes and on average is around 30k bytes per document. A_Date is obtained when the C++ application receives a client connection. After document transmission is complete the DocId is obtained from an Oracle sequence. It is true that A_Date and DocId increase together. However because of varying document sizes and transmission rates, there is no guarantee that consistent order between the two columns exists.
For Example: If time (t) increases and the connection times are: t1, t2, t3, t4 and the document at t2 took long to transmit, we can have:
A_Date -------- DocId (From Oracle Sequence)
t2 -------------- 1004
t4 -------------- 1003
t3 -------------- 1002
t1 -------------- 1001
A_Month is simply the month number (1-12) extracted from the transmission entry timestamp. It is also our Partition Key (see below). When the year wraps around from 2006 to 2007, data for January will simpy fall into the 1st partition bucket, and so on..
QUERY NEEDS: Our queries are centered around a DateTime interval Where the left endpoint is current day/time. They can query the current day, current to 1 month back, current to 2 months back, .. current to 15 months back. We MUST RETURN a list of DocId's sorted in DESCENDING ORDER for screen display purposes.
In General we need to return sub-second for the 1st month. As we query further back in time longer response times between 1 and 4 seconds are acceptable.
PARTITIONING AND INDEXES:
The table is partitioned by A_Month as shown below:
Create Table IndexTable
PARTITION BY RANGE (A_Month)
( PARTITION p1 VALUES LESS THAN 1.1 ...
PARTITION p2 VALUES LESS THAN 2.1 ...
PARTITION p3 vALUES LESS THAN 3.1 ...
There are GLOBAL INDEXES on DocId, Text(Domain Index), and SCode.
A_Date is a LOCAL INDEX.
QUERY STRUCTURE:
My Query structure looks like this (for a 2 month query):
SELECT DocId from IndexTable WHERE
Contains (Text, 'BUSH and EARNINGS') > 0 AND
SCode in ('S1', 'S2', 'S3', 'S4') AND
A_Date Between '2006-01-15 11:00:00' AND '2006-03-15 11:00:00'
Order By DocId DESC;
QUESTIONS (THERE ARE THREE)
#### QUESTION 1 ####
As I examine various explain plans, the PSTART and PSTOP do not reflect consistency with my A_Date range. It seems to always display:
PStart PStop
RowId RowId
no matter what my A_Date range is.. I don't see it pruning my partitions. I cannot find documentation as to what RowId means or how it affects the optimizer's plan.
#### QUESTION 2 ####
I have tried parallelization hints on the table and indexes such as
/*+ PARALLEL( IndexTable, 4) */ and on the A_Date index
/*+ PARALLEL_INDEX( IndexTable, A_Date_Index, 4) */.
I can't really tell if the parallel hints make a difference.
However, the FIRST_ROWS hint makes a big difference if I nest the query inside a rownum clause as:
SELECT * from
( Select /+* first_rows */ ... WHERE CONTAINS... > 0 AND... )
Where ROWNUM <=20;
#### QUESTION 3 ####
I'm running close to the latest RedHat Linux and Oracle 10g2 and I have read about Parallel Slave Processes in Tom Kyte's Expert Oracle Architecture book in which he says you should see processes like:
ora...p000..
ora...p001..
ora...pnnn..
Which are the parallel slave processes. I have NEVER seen any oracle processes numbered as such running on my system when I run test queries. I have seen some q_ processes and others, but not the pnnn processes..
Can I benefit from parallel query without ever seeing these processes??
I Greatly Appreciate Any Advice To Any Of These Questions..
Joe

Well I can tell you this much. You will never get partition pruning if you don't have the partition key in the where clause.
I'm not sure about the parallel query. There was a time that this wasn't supported with Text indexes, but not sure if that still applies today.
In theory there are two levels of partitioning that can be defined that you may want to test out.
1st, range partition the base table and create a partitioned text index based on that. This is what you are currently doing.
2nd, in the storage preference for the $I table specify a range or hash partition (I've never tried this, but in theory it should work) on the token_text column. Try this out and see if it works.

Oracle11g RAC with partitioning and cross instance parallel query problem

I have set up a 300gb TPC-H database using a 4 node RAC environment (8 cpu per node, 16 GB memory, 2 GHz processors) the system is served by 2.5 terabytes of SSD for its IO subsystem managed by a combination of ASM and OCFS2.
When I run a large parallel query (number 9 in the TPCH query set) I get:
ORA-00600: internal error code, arguments [kxfrGraDistNum3],[65535],[4]
with all other arguments blank. There were some reports of this in version 9, but it was supposedly fixed. Has any one seen this behavior or have a work around?
Mike

Good Idea! Why didn't I think of that? Oh yea...I did. Unfortunately TMS CSI is an update only partner type CSI so I cannot submit a SR. The 600 lookup was how I found the old stuff, but it didn't have any 11g references. I hoped maybe someone in the community had encountered this and had a workaround. By the way, the querey looks like so:
select
nation,
o_year,
sum(amount) as sum_profit
from
select
n_name as nation,
extract(year from o_orderdate) as o_year,
l_extendedprice * (1 - l_discount) - ps_supplycost * l_quantity as amount
from
h_part,
h_supplier,
h_lineitem,
h_partsupp,
h_order,
h_nation
where
s_suppkey = l_suppkey
and ps_suppkey = l_suppkey
and ps_partkey = l_partkey
and p_partkey = l_partkey
and o_orderkey = l_orderkey
and s_nationkey = n_nationkey
and p_name like '%spring%'
) as profit
group by
nation,
o_year
order by
nation,
o_year desc;
the other 21 queries, all using the same tables and degrees of paralell and cross instance settings, executed ok.
Mike

What can I do to make this query run faster

Hi All,
The below query is taking a long time. Is there any thing that I can do to shorten its time.
SELECT C.FOLIO_NO, C.CO_TRANS_NO TRANS_NO, to_char(C.CREATED_DATE, 'dd/mm/yyyy') DOC_DATE, DECODE(PP.NAME, NULL, D.EMP_NAME, PP.NAME) LODGED_BY, decode(sf_fetch_datechange(c.co_trans_no, C.CO_TRANS_ID), Null, '-', sf_fetch_datechange(c.co_trans_no, C.CO_TRANS_ID)) DATE_CHANGE, P.RECEIPT_NO, decode(c.co_trans_id,'A020',(select nvl(base_trans_id,co_trans_id) from co_form5a_trans f where f.co_trans_no=c.co_trans_no),c.co_trans_id) TRANS_ID,(case when decode(c.co_trans_id,'A020',(select nvl(base_trans_id,co_trans_id) from co_form5a_trans f where f.co_trans_no=c.co_trans_no),c.co_trans_id)='AR20' then 1 when decode(c.co_trans_id,'A020',(select nvl(base_trans_id,co_trans_id) from co_form5a_trans f where f.co_trans_no=c.co_trans_no),c.co_trans_id)='AR03' then 2 end) TRANS_TYPE FROM CO_TRANS_MASTER C, PAYMENT_DETAIL P, PEOPLE_PROFILE PP, SC_AGENT_EMP D, M_CAA_TRANS E     where '1' <> TRIM(UPPER('S0750070Z')) and (C.CO_TRANS_ID in TRIM(UPPER('AR20')) OR C.CO_TRANS_ID in TRIM(UPPER('AR03'))OR c.co_trans_id IN TRIM (UPPER ('A020')))and C.CO_TRANS_NO = P.TRANS_NO and (C.VOID_IND = 'N' or C.VOID_IND is Null) and C.CREATED_BY = PP.PP_ID(+) and C.PROF_NO = D.PROF_NO(+) and C.CREATED_BY = D.EMP_ID (+) and TRIM(UPPER(C.CO_NO)) = TRIM(UPPER('200101586W')) and c.co_trans_id = e.trans_id (+) order by FOLIO_NO;
SQL>
SQL> show parameter user_dump_dest
NAME                                 TYPE        VALUE
user_dump_dest                       string      /u01/app/oracle/diag/rdbms/ebi
zfile/EBIZFILE1/trace
SQL> show parameter optimizer
NAME                                 TYPE        VALUE
optimizer_capture_sql_plan_baselines boolean     FALSE
optimizer_dynamic_sampling           integer     2
optimizer_features_enable            string      11.2.0.2
optimizer_index_caching              integer     0
optimizer_index_cost_adj             integer     100
optimizer_mode                       string      ALL_ROWS
optimizer_secure_view_merging        boolean     TRUE
optimizer_use_invisible_indexes      boolean     FALSE
optimizer_use_pending_statistics     boolean     FALSE
optimizer_use_sql_plan_baselines     boolean     TRUE
SQL> show parameter db_file_multi
NAME                                 TYPE        VALUE
db_file_multiblock_read_count        integer     128
SQL> show parameter db_block_size
NAME                                 TYPE        VALUE
db_block_size                        integer     8192
SQL> show parameter cursor_sharing
NAME                                 TYPE        VALUE
cursor_sharing                       string      EXACT
SQL>
SQL> column sname format a20
SQL> column pname format a20
SQL> column pval2 format a20
SQL>
SQL> select
2 sname, pname, pval1, pval2
3 from
4 sys.aux_stats$;
SNAME                PNAME                     PVAL1 PVAL2
SYSSTATS_INFO        STATUS                          COMPLETED
SYSSTATS_INFO        DSTART                          09-11-2010 14:25
SYSSTATS_INFO        DSTOP                           09-11-2010 14:25
SYSSTATS_INFO        FLAGS                         1
SYSSTATS_MAIN        CPUSPEEDNW           739.734748
SYSSTATS_MAIN        IOSEEKTIM                    10
SYSSTATS_MAIN        IOTFRSPEED                 4096
SYSSTATS_MAIN        SREADTIM
SYSSTATS_MAIN        MREADTIM
SYSSTATS_MAIN        CPUSPEED
SYSSTATS_MAIN        MBRC
SYSSTATS_MAIN        MAXTHR
SYSSTATS_MAIN        SLAVETHR
13 rows selected.
Elapsed: 00:00:00.06
SQL>
SQL> explain plan for
2 SELECT C.FOLIO_NO, C.CO_TRANS_NO TRANS_NO, to_char(C.CREATED_DATE, 'dd/mm/yyyy') DOC_DATE, DECODE(PP.NAME, NULL, D.EMP_NAME, PP.NAME) LODGED_BY, decode(sf_fetch_datechange(c.co_trans_no, C.CO_TRANS_ID), Null, '-', sf_fetch_datechange(c.co_trans_no, C.CO_TRANS_ID)) DATE_CHANGE, P.RECEIPT_NO, decode(c.co_trans_id,'A020',(select nvl(base_trans_id,co_trans_id) from co_form5a_trans f where f.co_trans_no=c.co_trans_no),c.co_trans_id) TRANS_ID,(case when decode(c.co_trans_id,'A020',(select nvl(base_trans_id,co_trans_id) from co_form5a_trans f where f.co_trans_no=c.co_trans_no),c.co_trans_id)='AR20' then 1 when decode(c.co_trans_id,'A020',(select nvl(base_trans_id,co_trans_id) from co_form5a_trans f where f.co_trans_no=c.co_trans_no),c.co_trans_id)='AR03' then 2 end) TRANS_TYPE FROM CO_TRANS_MASTER C, PAYMENT_DETAIL P, PEOPLE_PROFILE PP, SC_AGENT_EMP D, M_CAA_TRANS E     where '1' <> TRIM(UPPER('S0750070Z')) and (C.CO_TRANS_ID in TRIM(UPPER('AR20')) OR C.CO_TRANS_ID in TRIM(UPPER('AR03'))OR c.co_trans_id IN TRIM (UPPER ('A020')))and C.CO_TRANS_NO = P.TRANS_NO and (C.VOID_IND = 'N' or C.VOID_IND is Null) and C.CREATED_BY = PP.PP_ID(+) and C.PROF_NO = D.PROF_NO(+) and C.CREATED_BY = D.EMP_ID (+) and TRIM(UPPER(C.CO_NO)) = TRIM(UPPER('200101586W')) and c.co_trans_id = e.trans_id (+) order by FOLIO_NO;
Explained.
Elapsed: 00:00:00.09
SQL>
SQL> set pagesize 1000;
SQL> set linesize 170;
SQL> @/u01/app/oracle/product/11.2.0/rdbms/admin/utlxpls.sql
SQL> Rem
SQL> Rem $Header: utlxpls.sql 26-feb-2002.19:49:37 bdagevil Exp $
SQL> Rem
SQL> Rem utlxpls.sql
SQL> Rem
SQL> Rem Copyright (c) 1998, 2002, Oracle Corporation.     All rights reserved.
SQL> Rem
SQL> Rem    NAME
SQL> Rem      utlxpls.sql - UTiLity eXPLain Serial plans
SQL> Rem
SQL> Rem    DESCRIPTION
SQL> Rem      script utility to display the explain plan of the last explain plan
SQL> Rem      command. Do not display information related to Parallel Query
SQL> Rem
SQL> Rem    NOTES
SQL> Rem      Assume that the PLAN_TABLE table has been created. The script
SQL> Rem      utlxplan.sql should be used to create that table
SQL> Rem
SQL> Rem      With SQL*plus, it is recomended to set linesize and pagesize before
SQL> Rem      running this script. For example:
SQL> Rem      set linesize 100
SQL> Rem      set pagesize 0
SQL> Rem
SQL> Rem    MODIFIED   (MM/DD/YY)
SQL> Rem    bdagevil     02/26/02 - cast arguments
SQL> Rem    bdagevil     01/23/02 - rewrite with new dbms_xplan package
SQL> Rem    bdagevil     04/05/01 - include CPU cost
SQL> Rem    bdagevil     02/27/01 - increase Name column
SQL> Rem    jihuang     06/14/00 - change order by to order siblings by.
SQL> Rem    jihuang     05/10/00 - include plan info for recursive SQL in LE row source
SQL> Rem    bdagevil     01/05/00 - add order-by to make it deterministic
SQL> Rem    kquinn     06/28/99 - 901272: Add missing semicolon
SQL> Rem    bdagevil     05/07/98 - Explain plan script for serial plans
SQL> Rem    bdagevil     05/07/98 - Created
SQL> Rem
SQL>
SQL> set markup html preformat on
SQL>
SQL> Rem
SQL> Rem Use the display table function from the dbms_xplan package to display the last
SQL> Rem explain plan. Force serial option for backward compatibility
SQL> Rem
SQL> select plan_table_output from table(dbms_xplan.display('plan_table',null,'serial'));
PLAN_TABLE_OUTPUT
Plan hash value: 2520189693
| Id | Operation                         | Name                    | Rows | Bytes | Cost (%CPU)| Time     |
|   0 | SELECT STATEMENT                  |                         |   592 | 85248 | 16573   (1)| 00:03:19 |
|   1 | TABLE ACCESS BY INDEX ROWID      | CO_FORM5A_TRANS         |     1 |    20 |     2   (0)| 00:00:01 |
|* 2 |   INDEX UNIQUE SCAN               | SYS_C0059692            |     1 |       |     1   (0)| 00:00:01 |
|   3 | TABLE ACCESS BY INDEX ROWID      | CO_FORM5A_TRANS         |     1 |    20 |     2   (0)| 00:00:01 |
|* 4 |   INDEX UNIQUE SCAN               | SYS_C0059692            |     1 |       |     1   (0)| 00:00:01 |
|   5 |   TABLE ACCESS BY INDEX ROWID     | CO_FORM5A_TRANS         |     1 |    20 |     2   (0)| 00:00:01 |
|* 6 |    INDEX UNIQUE SCAN              | SYS_C0059692            |     1 |       |     1   (0)| 00:00:01 |
|   7 | SORT ORDER BY                    |                         |   592 | 85248 | 16573   (1)| 00:03:19 |
|   8 |   NESTED LOOPS                    |                         |       |       |            |          |
|   9 |    NESTED LOOPS                   |                         |   592 | 85248 | 16572   (1)| 00:03:19 |
| 10 |     NESTED LOOPS OUTER            |                         |   477 | 54855 | 15329   (1)| 00:03:04 |
| 11 |      NESTED LOOPS OUTER           |                         |   477 | 41499 | 14374   (1)| 00:02:53 |
| 12 |       INLIST ITERATOR             |                         |       |       |            |          |
|* 13 |        TABLE ACCESS BY INDEX ROWID| CO_TRANS_MASTER         |   477 | 22896 | 14367   (1)| 00:02:53 |
|* 14 |         INDEX RANGE SCAN          | IDX_CO_TRANS_ID         | 67751 |       |   150   (1)| 00:00:02 |
| 15 |       TABLE ACCESS BY INDEX ROWID | SC_AGENT_EMP            |     1 |    39 |     1   (0)| 00:00:01 |
|* 16 |        INDEX UNIQUE SCAN          | PK_SC_AGENT_EMP         |     1 |       |     0   (0)| 00:00:01 |
| 17 |      TABLE ACCESS BY INDEX ROWID | PEOPLE_PROFILE          |     1 |    28 |     2   (0)| 00:00:01 |
|* 18 |       INDEX UNIQUE SCAN           | SYS_C0063100            |     1 |       |     1   (0)| 00:00:01 |
|* 19 |     INDEX RANGE SCAN              | IDX_PAY_DETAIL_TRANS_NO |     1 |       |     2   (0)| 00:00:01 |
| 20 |    TABLE ACCESS BY INDEX ROWID    | PAYMENT_DETAIL          |     1 |    29 |     3   (0)| 00:00:01 |
Predicate Information (identified by operation id):
   2 - access("F"."CO_TRANS_NO"=:B1)
   4 - access("F"."CO_TRANS_NO"=:B1)
   6 - access("F"."CO_TRANS_NO"=:B1)
13 - filter(TRIM(UPPER("SYS_ALIAS_3"."CO_NO"))='200101586W' AND ("SYS_ALIAS_3"."VOID_IND" IS NULL
              OR "SYS_ALIAS_3"."VOID_IND"='N'))
14 - access("SYS_ALIAS_3"."CO_TRANS_ID"='A020' OR "SYS_ALIAS_3"."CO_TRANS_ID"='AR03' OR
              "SYS_ALIAS_3"."CO_TRANS_ID"='AR20')
16 - access("SYS_ALIAS_3"."PROF_NO"="D"."PROF_NO"(+) AND
              "SYS_ALIAS_3"."CREATED_BY"="D"."EMP_ID"(+))
18 - access("SYS_ALIAS_3"."CREATED_BY"="PP"."PP_ID"(+))
19 - access("SYS_ALIAS_3"."CO_TRANS_NO"="P"."TRANS_NO")
42 rows selected.
Elapsed: 00:00:00.53
SQL>
SQL>
SQL>
SQL> rollback;
Rollback complete.
Elapsed: 00:00:00.01
SQL>
SQL> rem Set the ARRAYSIZE according to your application
SQL> set autotrace traceonly arraysize 100
SQL>
SQL> alter session set tracefile_identifier = 'mytrace1';
Session altered.
Elapsed: 00:00:00.00
SQL>
SQL> rem if you're using bind variables
SQL> rem define them here
SQL>
SQL> rem variable b_var1 number
SQL> rem variable b_var2 varchar2(20)
SQL>
SQL> rem and initialize them
SQL>
SQL> rem exec :b_var1 := 1
SQL> rem exec :b_var2 := 'DIAG'
SQL> set pagesize 1000;
SQL> set linesize 170;
SQL> alter session set events '10046 trace name context forever, level 8';
Session altered.
Elapsed: 00:00:00.01
SQL> SELECT C.FOLIO_NO, C.CO_TRANS_NO TRANS_NO, to_char(C.CREATED_DATE, 'dd/mm/yyyy') DOC_DATE, DECODE(PP.NAME, NULL, D.EMP_NAME, PP.NAME) LODGED_BY, decode(sf_fetch_datechange(c.co_trans_no, C.CO_TRANS_ID), Null, '-', sf_fetch_datechange(c.co_trans_no, C.CO_TRANS_ID)) DATE_CHANGE, P.RECEIPT_NO, decode(c.co_trans_id,'A020',(select nvl(base_trans_id,co_trans_id) from co_form5a_trans f where f.co_trans_no=c.co_trans_no),c.co_trans_id) TRANS_ID,(case when decode(c.co_trans_id,'A020',(select nvl(base_trans_id,co_trans_id) from co_form5a_trans f where f.co_trans_no=c.co_trans_no),c.co_trans_id)='AR20' then 1 when decode(c.co_trans_id,'A020',(select nvl(base_trans_id,co_trans_id) from co_form5a_trans f where f.co_trans_no=c.co_trans_no),c.co_trans_id)='AR03' then 2 end) TRANS_TYPE FROM CO_TRANS_MASTER C, PAYMENT_DETAIL P, PEOPLE_PROFILE PP, SC_AGENT_EMP D, M_CAA_TRANS E     where '1' <> TRIM(UPPER('S0750070Z')) and (C.CO_TRANS_ID in TRIM(UPPER('AR20')) OR C.CO_TRANS_ID in TRIM(UPPER('AR03'))OR c.co_trans_id IN TRIM (UPPER ('A020')))and C.CO_TRANS_NO = P.TRANS_NO and (C.VOID_IND = 'N' or C.VOID_IND is Null) and C.CREATED_BY = PP.PP_ID(+) and C.PROF_NO = D.PROF_NO(+) and C.CREATED_BY = D.EMP_ID (+) and TRIM(UPPER(C.CO_NO)) = TRIM(UPPER('200101586W')) and c.co_trans_id = e.trans_id (+) order by FOLIO_NO;
10 rows selected.
Elapsed: 00:03:42.27
Execution Plan
Plan hash value: 2520189693
| Id | Operation                         | Name                    | Rows | Bytes | Cost (%CPU)| Time     |
|   0 | SELECT STATEMENT                  |                         |   592 | 85248 | 16573   (1)| 00:03:19 |
|   1 | TABLE ACCESS BY INDEX ROWID      | CO_FORM5A_TRANS         |     1 |    20 |     2   (0)| 00:00:01 |
|* 2 |   INDEX UNIQUE SCAN               | SYS_C0059692            |     1 |       |     1   (0)| 00:00:01 |
|   3 | TABLE ACCESS BY INDEX ROWID      | CO_FORM5A_TRANS         |     1 |    20 |     2   (0)| 00:00:01 |
|* 4 |   INDEX UNIQUE SCAN               | SYS_C0059692            |     1 |       |     1   (0)| 00:00:01 |
|   5 |   TABLE ACCESS BY INDEX ROWID     | CO_FORM5A_TRANS         |     1 |    20 |     2   (0)| 00:00:01 |
|* 6 |    INDEX UNIQUE SCAN              | SYS_C0059692            |     1 |       |     1   (0)| 00:00:01 |
|   7 | SORT ORDER BY                    |                         |   592 | 85248 | 16573   (1)| 00:03:19 |
|   8 |   NESTED LOOPS                    |                         |       |       |            |          |
|   9 |    NESTED LOOPS                   |                         |   592 | 85248 | 16572   (1)| 00:03:19 |
| 10 |     NESTED LOOPS OUTER            |                         |   477 | 54855 | 15329   (1)| 00:03:04 |
| 11 |      NESTED LOOPS OUTER           |                         |   477 | 41499 | 14374   (1)| 00:02:53 |
| 12 |       INLIST ITERATOR             |                         |       |       |            |          |
|* 13 |        TABLE ACCESS BY INDEX ROWID| CO_TRANS_MASTER         |   477 | 22896 | 14367   (1)| 00:02:53 |
|* 14 |         INDEX RANGE SCAN          | IDX_CO_TRANS_ID         | 67751 |       |   150   (1)| 00:00:02 |
| 15 |       TABLE ACCESS BY INDEX ROWID | SC_AGENT_EMP            |     1 |    39 |     1   (0)| 00:00:01 |
|* 16 |        INDEX UNIQUE SCAN          | PK_SC_AGENT_EMP         |     1 |       |     0   (0)| 00:00:01 |
| 17 |      TABLE ACCESS BY INDEX ROWID | PEOPLE_PROFILE          |     1 |    28 |     2   (0)| 00:00:01 |
|* 18 |       INDEX UNIQUE SCAN           | SYS_C0063100            |     1 |       |     1   (0)| 00:00:01 |
|* 19 |     INDEX RANGE SCAN              | IDX_PAY_DETAIL_TRANS_NO |     1 |       |     2   (0)| 00:00:01 |
| 20 |    TABLE ACCESS BY INDEX ROWID    | PAYMENT_DETAIL          |     1 |    29 |     3   (0)| 00:00:01 |
Predicate Information (identified by operation id):
   2 - access("F"."CO_TRANS_NO"=:B1)
   4 - access("F"."CO_TRANS_NO"=:B1)
   6 - access("F"."CO_TRANS_NO"=:B1)
13 - filter(TRIM(UPPER("SYS_ALIAS_3"."CO_NO"))='200101586W' AND ("SYS_ALIAS_3"."VOID_IND" IS NULL
              OR "SYS_ALIAS_3"."VOID_IND"='N'))
14 - access("SYS_ALIAS_3"."CO_TRANS_ID"='A020' OR "SYS_ALIAS_3"."CO_TRANS_ID"='AR03' OR
              "SYS_ALIAS_3"."CO_TRANS_ID"='AR20')
16 - access("SYS_ALIAS_3"."PROF_NO"="D"."PROF_NO"(+) AND
              "SYS_ALIAS_3"."CREATED_BY"="D"."EMP_ID"(+))
18 - access("SYS_ALIAS_3"."CREATED_BY"="PP"."PP_ID"(+))
19 - access("SYS_ALIAS_3"."CO_TRANS_NO"="P"."TRANS_NO")
Statistics
         51 recursive calls
          0 db block gets
     651812 consistent gets
      92202 physical reads
          0 redo size
       1594 bytes sent via SQL*Net to client
        524 bytes received via SQL*Net from client
          2 SQL*Net roundtrips to/from client
          1 sorts (memory)
          0 sorts (disk)
         10 rows processed
SQL>
SQL> disconnect
Disconnected from Oracle Database 11g Enterprise Edition Release 11.2.0.2.0 - 64bit Production
With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,
Data Mining and Real Application Testing options
SQL> Thanks in advance!

Hi Raj,
I have given the output below as you requested....
QL> select * from table(dbms_xplan.display_cursor(null, null, 'ALLSTATS LAST'));
PLAN_TABLE_OUTPUT
SQL_ID 0taz7ckjm41yv, child number 1
SELECT C.FOLIO_NO, C.CO_TRANS_NO TRANS_NO, to_char(C.CREATED_DATE,
'dd/mm/yyyy') DOC_DATE, DECODE(PP.NAME, NULL, D.EMP_NAME, PP.NAME)
LODGED_BY, decode(sf_fetch_datechange(c.co_trans_no, C.CO_TRANS_ID),
Null, '-', sf_fetch_datechange(c.co_trans_no, C.CO_TRANS_ID))
DATE_CHANGE, P.RECEIPT_NO, decode(c.co_trans_id,'A020',(select
nvl(base_trans_id,co_trans_id) from co_form5a_trans f where
f.co_trans_no=c.co_trans_no),c.co_trans_id) TRANS_ID,(case when
decode(c.co_trans_id,'A020',(select nvl(base_trans_id,co_trans_id) from
co_form5a_trans f where f.co_trans_no=c.co_trans_no),c.co_trans_id)='AR2
0' then 1 when decode(c.co_trans_id,'A020',(select
nvl(base_trans_id,co_trans_id) from co_form5a_trans f where
f.co_trans_no=c.co_trans_no),c.co_trans_id)='AR03' then 2 end)
TRANS_TYPE FROM CO_TRANS_MASTER C, PAYMENT_DETAIL P, PEOPLE_PROFILE PP,
SC_AGENT_EMP D, M_CAA_TRANS E where '1' <> TRIM(UPPER('S0750070Z')) and
(C.CO_TRANS_ID in TRIM(UPPER('AR20')) OR C.CO_TRANS_ID in
TRIM(UPPER('AR03'))OR c.co
Plan hash value: 4175354585
| Id | Operation                        | Name                    | E-Rows | OMem | 1Mem | Used-Mem |
|   0 | SELECT STATEMENT                 |                         |        |       |       |          |
|   1 | TABLE ACCESS BY INDEX ROWID     | CO_FORM5A_TRANS         |      1 |       |       |          |
|* 2 |   INDEX UNIQUE SCAN              | SYS_C0059692            |      1 |       |       |          |
|   3 | TABLE ACCESS BY INDEX ROWID     | CO_FORM5A_TRANS         |      1 |       |       |          |
|* 4 |   INDEX UNIQUE SCAN              | SYS_C0059692            |      1 |       |       |          |
|   5 |   TABLE ACCESS BY INDEX ROWID    | CO_FORM5A_TRANS         |      1 |       |       |          |
|* 6 |    INDEX UNIQUE SCAN             | SYS_C0059692            |      1 |       |       |          |
|   7 | SORT ORDER BY                   |                         |     12 | 2048 | 2048 | 2048 (0)|
|   8 |   NESTED LOOPS                   |                         |        |       |       |          |
|   9 |    NESTED LOOPS                  |                         |     12 |       |       |          |
| 10 |     NESTED LOOPS OUTER           |                         |     10 |       |       |          |
| 11 |      NESTED LOOPS OUTER          |                         |     10 |       |       |          |
|* 12 |       TABLE ACCESS FULL          | CO_TRANS_MASTER         |     10 |       |       |          |
| 13 |       TABLE ACCESS BY INDEX ROWID| SC_AGENT_EMP            |      1 |       |       |          |
|* 14 |        INDEX UNIQUE SCAN         | PK_SC_AGENT_EMP         |      1 |       |       |          |
| 15 |      TABLE ACCESS BY INDEX ROWID | PEOPLE_PROFILE          |      1 |       |       |          |
|* 16 |       INDEX UNIQUE SCAN          | SYS_C0063100            |      1 |       |       |          |
|* 17 |     INDEX RANGE SCAN             | IDX_PAY_DETAIL_TRANS_NO |      1 |       |       |          |
| 18 |    TABLE ACCESS BY INDEX ROWID   | PAYMENT_DETAIL          |      1 |       |       |          |
Predicate Information (identified by operation id):
   2 - access("F"."CO_TRANS_NO"=:B1)
   4 - access("F"."CO_TRANS_NO"=:B1)
   6 - access("F"."CO_TRANS_NO"=:B1)
12 - filter((INTERNAL_FUNCTION("SYS_ALIAS_3"."CO_TRANS_ID") AND
              TRIM(UPPER("SYS_ALIAS_3"."CO_NO"))='200101586W' AND ("SYS_ALIAS_3"."VOID_IND" IS NULL OR
              "SYS_ALIAS_3"."VOID_IND"='N')))
14 - access("SYS_ALIAS_3"."PROF_NO"="D"."PROF_NO" AND "SYS_ALIAS_3"."CREATED_BY"="D"."EMP_ID")
16 - access("SYS_ALIAS_3"."CREATED_BY"="PP"."PP_ID")
17 - access("SYS_ALIAS_3"."CO_TRANS_NO"="P"."TRANS_NO")
Note
   - cardinality feedback used for this statement
   - Warning: basic plan statistics not available. These are only collected when:
       * hint 'gather_plan_statistics' is used for the statement or
       * parameter 'statistics_level' is set to 'ALL', at session or system level
65 rows selected.

Oracle 10g and parallel query question

Hi Oracle on SAP Gurus!
We are currently thinking of activating parallel query for certain segments (large application tables and indexes). We searched in SAPNet and SDN and have also studied the SAP Note 651060. But we did not find a complete answer to the following question which is very important for us:
Which kinds of queries (despite from full table scan and index scan in partitioned indexes) support parallel queries and which ones do not support parallel queries?
This is important for us to find out whether we have candidates for parallel queries or not.
Thanx for any hint!
Regards,
Volker

But why do you not propose to use parallel query in OLTP systems?
If the queries are accessed very frequently you will just run out of cpu and io ressources. OLTP systems are (historical) typically multi user systems. You can off course use PQ for 'single user' activities, like index rebuilds, some batchjobs, but you shouldn't do for frequent user queries.
If you have time look at this interesting Article [Suck It Dry - Tuning Parallel Execution|http://doug.burns.tripod.com/px.html]
It is quite old, and you don't have to read all tech details, but i recommend having a look at the conclusions at the end.
May it make sense to use partitioning of these tables in conjunction with parallel query?
I know some guys, who do partitioning on OLTP systems, even SAP systems. But they don't use PQ then. The use partitioning to work on a smaller set of data. In your case the range scans, would need to scan only one partition, saving buffer cache and effectively speeding up execution. So you don't need PQ to scan all partitions at all, this would be a typical OLAP approach.
Best regards
Michael

Problem in running parallel query

Hi All,
We have an Oracle Database 11g Enterprise Edition Release 11.1.0.6.0 installed on Microsoft Windows 2000 Advanced Server 5.0.2195 Service Pack 4 Build 2195.
And we have a stored procedure which does lots of works.
In this procedure, there is a select statement which joins a huge table with milions of records(lets call it table X) to some other almost huge tables. This table X
is partitioned and parallel processing has been enabled on it. Every thing has been fine until a week ago. From then on, we are getting the following error in that procedure:
ORA-12801: error signaled in parallel query server P014
ORA-04030: out of process memory when trying to allocate 4194328 bytes (QERHJ hash-joi,QERHJ list array)
We have made some changes in that select statement which are very common changes like adding some additional where clauses and selecting some additional columns. The From List of select statement has not been changed.
I don't exactly know what is realy going on.
The available memory is about 2 GB when the mentioned procedure crashes.
Any Idea?
My Best,
Alireza Vali

Is it a 32bit installation (32bit Windows, 32bit Oracle) ?
What are your MEMORY_TARGET, SGA_TARGET, SGA_MAX_SIZE and PGA_AGGREGATE_TARGET parameters set to ? (If you have MEMORY_TARGET set, it is not necessary that the other parameters be set).
ORA-4030 relates to PGA memory, not SGA memory. But Oracle on Windows is a single, multi-threaded process. Therefore, all Oracle "processes" (threads in the Windows process) share the same memory (address) space, even the SGA and PGA.
Hemant K Chitale

Parallel Query - which actions are possible?

Hello guys,
we have an OLTP system which a huge amount of data... and now i want to try to introduce parallel query in the system and test something around.
I have some experiences with parallel query in an OLAP environment, but not in an OLTP.
What i am searching for?
An overview for Oracle 10g which lists all possible query accesses that are possible in a parallel environment.
For example:
- FTS - possible
- Nested Loops - possible
- Index Range scans - not possible
I know some .. but not all so i am searching for an overview on the oracle documentation page but was not able to found something.
Maybe you can help me.
Regards
Stefan

Hello sybrand (my little senior dba),
regarding to the paper of Doug Burns there are more things that can be executed parallel (but this paper is about Oracle 9i), so please don't post if you don't have any knowledge.
http://www.dbanotes.net/archives/px.pdf
=> As Oracle’s parallel capabilities have been developed, most tasks can be executed in parallel now. According to the 9.2.0.5 documentation, Oracle supports parallel execution of the following operations: -
Access methods
For example, table scans, index fast full scans, and partitioned index range scans.
Join methods
For example, nested loop, sort merge, hash, and star transformation.
I am searching for such an information about Oracle 10g... maybe there are some more things that can be executed parallel wit 10g.
Thanks and Regards
Stefan

Parallel query feature

Hi to utilize the parallel query option in database in 9i 10 g how can me modify the sql staments or the procedures.Whether it will use the parallel query feature without giving the parallel query hint?
with regards
ramya

When we have to handle a lot of data in operations, like full table scans of large tables, or the creation of large indices, we can divide the work. Instead of using a single process for one statement, the work can be spread to multiple processes. This is called parallel execution or parallel processing. Parallel execution normally reduces response time when you are accessing large amount of data, but it is not always faster than a serial operation.
Parallel execution is useful for many types of operations like:
* Queries requiring large table scans, joins, or partitioned index scans
* Creation of large indices and large tables
* Bulk inserts, updates and deletes
* Aggregations
We have the following hints to influence the use of parallel execution
* PARALLEL
* NOPARALLEL
* PQ_DISTRIBUTE
* PARALLEL_INDEX
* NOPARALLEL_INDEX
From the session where the query run, type:
select * from v$pq_sesstat;
SELECT slave_name,status, cpu_secs_total
FROM v$pq_slave;
Performance summary:
It cannot be stressed strongly enough that parallel query is not always quicker than serial. Some queries are more suited to parallel operations than others. If you are using parallel execution, then you should endeavour to maximize I/O throughput to your disk. Ensure that:
* you have enough slaves to retrieve the data efficiently
* you do not have too many slaves (avoid maxing out the CPU)
* you set realistic memory parameters (sort_area_size etc.) so that you do not max out memory and cause swapping
* the data is spread over multiple disk spindles so that slaves do not contend for I/O resource
* the type of query your are attempting to execute in parallel is suitable for parallelisation.
* watch for non-uniform loading on slave processes as this could indicate data skew.
http://www.jlcomp.demon.co.uk/pqcosts.html
courtesy from oracle metalink.
SJH
OCP DBA

ORA-1555 with Parallel query problem

Hi
We are getting many ora-1555 errors with parallel query stuff. Few queries are throwing with hint /* Q98989129 NO_EXPAND INDEX or /* Q908231094 INDEX_RRS. Our applicaiton is like DSS kind with bulk loading and big reports. I have doubled the undo retention and undo size to 200G and set the trace event. But most of the queries are parallel queries. We are not using the following query in applicaiton. seems like it is parallel sub queries. I need to know the exact mechanism how to devide the parallel queries with its degree of parallalism.
SELECT /*+ Q277874009 INDEX_RRS(A1 "PK_TF_UTRAN_UCELL10_TAB") */ A1."TSTAMP" C0,A1."INSTANCE_ID" C1,A1."PMNOFOIRATHOM
ULTIGSMFAILURE" C2,A1."PMNOFOIRATHOCS57GSMFAIL" C3,A1."PMNOFOSPEECHGSMFAILURE" C
4,A1."PMNOFOHOSTANDGSMFAILURE" C5 FROM "FLEXPM"."ERC_TF_UTRAN_UCELL10_TAB" PX_G
RANULE(0, BLOCK_RANGE, DYNAMIC) A1 WHERE A1."TSTAMP"(+)<=TO_DATE('2006-09-25 23:
45:00', 'yyyy-mm-dd hh24:mi:ss')
Thanks

Hi,
probably the error is due to wrong execution plans choosen by the optimizer.
Are the table statistics up to date? Did You use parallel w/o and degree (ex. defining parallel on object using DEFAULT clause instead of a DEGREE)?
Normally, parallel execution can be done in 2 ways, considering the driving object:
1. like in previous versions, like 7.3, w/o partitioning, the object is split, by rowid, directly from the parallel coordinator to the query slaves (normally a number that can be equivalent to the degree of parallelism or the double in case of sort e/o joins).
2. considering a partitioned object, for every partition/subpartition (phisical segment), a query slaves is made in charge.
In both ways, the original session (SID) is the parallel coordinator that coordinates the salves executions.
The hints and statements that You've reported are tipical queries used by slave process.
In my experience, setting the PARALLEL degree to DEFAULT (no degree during CREATE or ALTER of the object after PARALLEL clause) will cause an "explosion" of slaves startup that can conduct to yr errors.
Hope this helps
Max

Problem with temp space allocation in parallel query

Hello
I've got a query which matches two large result sets (25m+ rows) against each other and does some basic filtering and aggregation. When I run this query in serial it takes about 30 mins and completes successfully. When I specify a parallel degree of 4 for each result set, it also completes successfully in about 20 minutes. However, when I specify that it should be run in parallel but don't specify a degree for each result set, it spawns 10 parallel servers and after a couple of minutes, bombs out from one of the parallel servers with:
ORA-12801: error signaled in parallel query server P000
ORA-01652: unable to extend temp segment by 64 in tablespace TEMPThis appears to be when it is about to perform a large hash join. The execution plan does not change whether the parallel degree is specified or not, and there is several GB of temp space available.
I'm at a bit of a loss as to how to track down specifically what is causing this problem. I've looked at v$sesstat for all of the sessions involved and it hasn't really turned anything up. I've tried tracing the main session and that hasn't really turned up much either. From what I can tell, one of the sessions seems to try to allocate massive amounts of temp space that it just does not need, but I can figure out why.
Any ideas of how to approach finding the cause of the problem?
David

Hello
I've finally resolved this and the resolution was relatively simple - and was also the main thing that Mark Rittman said he did in his article: reduce the size of the hash join.
After querying v$sql_workarea_active I could see what was happening which was that the sum of the temp space for all of the parallel slaves was exceeding the total amount of temp space available on the system. When run in serial, it was virtually at the limit. I guess the extra was just the overhead for each slave maintaining it's own hash table.
I also made the mistake of misreading the exectuion plan - assuming that the data being pushed to the hash join was filtered to eliminate the data that was not of interest. Upon reflection, this was a rather stupid assumption on my part. Anyway, I used sub query factoring with the materialize hint to ensure that the hash join was only working on the data it should have been. This significantly reduced the size of the hash table and therefore the amount of temp space required.
I did speak to oracle support and they suggested using pga_aggregate_target rather than the separate *area_size parameters. I found that this had very little impact as the problem was related to the volume of data rather than whether it was being processed in memory or not. That said, I did try upping the hash_area_size for the session with some initial success, but ultimately it didn't prove to be scalable. We are however now using pga_aggregate_target in prod.
So that's that. Problem sorted. And as the title of Mark Rittman's article suggests, I was trying to be too clever! :-)
HTH
David

Make parallel query (e.g. partition-wise join) evenly distributed cross RAC

Similar Messages

Maybe you are looking for