Service is not relocated to available instance when pref node hangs.

Question

Hi,
I'm fighting strange problem with 4 node 10.2.0.3 RAC on Linux .
We have service defined as follows:
IT PREF: hp3 AVAIL: hp4 hp2 hp1
  1* select name,service_id from dba_services where name = 'IT'
SQL> /
NAME                                                             SERVICE_ID
IT                                                                        8
  1  select name, failover_method, failover_type, failover_retries,goal, clb_goal,aq_ha_notifications
  2* from dba_services where service_id = 8
SQL> /
NAME            METHOD      TYPE        RETRIES GOAL       CLB_GOAL AQNOT
IT              BASIC       SELECT          180 NONE       LONG     YES
lsnrctl service gives
Service "IT" has 1 instance(s).
  Instance "hp3", status READY, has 1 handler(s) for this service...
    Handler(s):
      "DEDICATED" established:36495 refused:0 state:ready
         REMOTE SERVER
         (ADDRESS=(PROTOCOL=TCP)(HOST=_SOME_VIP_)(PORT=1521))When prefered instance hp3 goes down (aborts) ,all is working ok, service is relocated to hp4 and users can connect.
But when prefered instance hangs (local disk controller issue) users are not able to connect to that service .
My goal is to have that service on 1 node only, and in case on failure (hard or soft) redirect all conections to that
service on other node (only 1 not load balancing and so) .
How to configure that ?
Oracle pointed out :
How To Configure Server Side Transparent Application Failover [ID 460982.1]
but I dont want to loadbalance connection .
Please advice .
Regards
GregG

Answer 1

Hi,
actually your database should not hang. Because when it hangs on one node this error will impact all the other instances as well, since this node has database blocks in memory it exclusively accesses, and when these cannot be transferred, your other nodes will "wait". Hence it is crucially important that when your disk controller hangs, the database instance gets aborted, if not even the node should get evicted.
So if your described szenario really is the case, then the service definition will not help you fix the issue. You should instead have a look at the reason, why the node does not react (or does not get an error).
If the clusternode behaves correctly then the service will failover. Just make sure you configure your clients correctly to react to this:
http://www.oracle.com/technetwork/products/clustering/overview/awmrac11g-133673.pdf
Regards
Sebastian