Load Balancing vs PQ processes accros RAC nodes

Question

I have following config.
•     3 node RAC (10.2.0.3 on RHEL)
•     All services (ERP, CRM, MISC and many more) configured for symmetric load-balance. i.e. Load balance across nodes.
•     Each node has Instance_group set to run all the above services. Parallel instance group set to same as service name at the session level.
•     PQ is enabled at the table level for some of the large tables with degree of 4.
Issue :
•     Quite often, QC gets spawned on the one node , but all the slaves gets spawned on a different node. This overloads the interconnect and slow down the fetch when result set is large.
Question :
How do I make all PQ processes (slaves + QC ) to go on the same node ? Is it possible to do without changing the load balancing scheme?

Answer 1

What is your interconnect configuration? Single Gig-E? What number is "large"? What do you do with large result sets? Spool them out?It’s GigE. Mostly extract of Multi-Million rows i.e. spool. Other with similar volumes goes to ETL server for complex downstream processing that subsequently gets fed to other targets.
In order to get this behavior you need to have an instance groups that only contains one node each, then change your parallel_instance_group to be that group.Yes. I have been told that by Timur as well with those two links. The links were helpful to understand the mechanics of Parallel Instance Group. In that link ( Part 2 ) mentions that it’s symmetric load balancing but I see he’s not load balancing connection across the both the nodes as he connects to specific nodes. See below excerpts from his blog site. Did I missed or mis-understood anything there?
Tnsnames.ora: (unchanged)
node1-only =
(DESCRIPTION =
(ADDRESS_LIST =
(ADDRESS = (PROTOCOL = TCP)(HOST = node1)(PORT = 1521))
(LOAD_BALANCING=NO)
(CONNECT_DATA =
(SERVICE_NAME= nodetaf1)
(SID = EMDWH1)
(SERVER = DEDICATED)
$ sqlplus mylogin/mypass@node1-only
alter session set parallel_instance_group=’IG1′;
select /*+ full(orders_part) */ count(*) from orders_part;
QCSID Inst Group Set PROGRAM
984 1 1 1 oracle@node1 (P000)
984 1 1 1 oracle@node1 (P004)
984 1 1 1 oracle@node1 (P002)
984 1 1 1 oracle@node1 (P005)
984 1 1 1 oracle@node1 (P006)
984 1 1 1 oracle@node1 (P001)
984 1 1 1 oracle@node1 (P007)
984 1 1 1 oracle@node1 (P003)
984 1 sqlplus@node1 (TNS V1-V3)In my case , I need connections to be load balanced across all 3 nodes and have all slaves to stay on the same node as QC ( where it gets connected to.. i.e. it could be any of the 3 nodes ). Here is my scenario in detail.
ERP =
(DESCRIPTION =
(ADDRESS_LIST =
(ADDRESS = (PROTOCOL = TCP)(HOST = myvip1)(PORT = 1521))
(ADDRESS = (PROTOCOL = TCP)(HOST = myvip2)(PORT = 1521))
(ADDRESS = (PROTOCOL = TCP)(HOST = myvip3)(PORT = 1521))
(LOAD_BALANCE = yes)
(CONNECT_DATA =(SERVICE_NAME = S_ERP)(FAILOVER_MODE =(TYPE = SELECT)(METHOD = BASIC)(RETRIES = 180)(DELAY = 5))
(SERVER = DEDICATED)
Service S_ERP has been created to  run on each node with Load balance on (server side, REMOTE_LISTENER has been set up on each node )
srvctl add service -d ERPCRMP -s -r  rac1, rac2, rac3….All the session makes connection with above TNS entry. Now how do I set up Instance group so that my connection goes to any of the 3 nodes and slaves are restricted on the same node as QC.
Do I have to set something like this?
Node 1:
INSTANCE_GROUP= IG01
PARALLE_INSTANCE_GROUP=IG01
Node 2:
INSTANCE_GROUP=IG02
PARALLEL_INSTANCE_GROUP=IG02
And so on for the 3rd node.