RAC public, VIP subnet

Hi,
I've got question about network configuration in Oracle RAC 11.2. According to documentation http://docs.oracle.com/cd/E11882_01/rac.112/e17264/preparing.htm#TDPRC135
"The virtual IP address must be on the same subnet as your public IP address.",
" The IP addresses used for the SCAN addresses must be on the same subnet as the VIP addresses."
So in ideal configuration we should end up with something similar to configuration example from table 2-1 http://docs.oracle.com/cd/E11882_01/rac.112/e17264/preparing.htm#BCGJBBGE
My question is. Can it be that
public ip will be on separated subnet,
vip and scan will be on different subnet,
and priv will be on separate subnet (dedicated to interconnect)
and is this configuration is supported.
Regards
Jakub

My question is. Can it be that
public ip will be on separated subnet,
vip and scan will be on different subnet,
and priv will be on separate subnet (dedicated to interconnect)
and is this configuration is supported
In a nutshell no.
The SCAN and VIP's will be logical interfaces.
Public, VIP, and SCAN will be on the same subnet.  VIP and SCAN will actually be a logical interface on the public interface.
Private will be in its private network on a different subnet.
I don't think you will even be able to configure the vip to be on a separate subnet from the public.  The installation will just fail.

Similar Messages

  • RAC public IP and VIP

    Hi
    I understand RAC requires 3ips for each node
    VIP -------------------->for near instantaneous failover and clients will be connecting to this
    Private IP ----------->For interconnect traffic
    Public IP ------------>Why is this required ?
    regards

    Public IPs : used to connect to the physical server (Host) OS LEVEL as you can not connect to VIP and Private IP. Public IPs allows you to connect using putty etc for remote connection.
    Private IPs: Should belong to separate class / subnet, should not share Public ip class. Recommendation is not to share for usual traffic.. Used for Heartbeat and Cache fusion.
    Virtual IPs : Used for Failover Management. VIPs are backbone of "RAC" , As Listener serves using these VIPs. After successfull cluster installation you should be able to ping these VIPs. before you cant.. you can also see weather VIPs are up or not.. ifconfig -a , VIPs should be plumbed in with public IPs. one more thing.. should be same subnet as Public IPs.
    It was quick over view really.. If you want more detail then please refer to Documentation.
    Thanks
    Prashant

  • Grid Infrastructure 11.2.0.3 install fails with public & private subnet

    Specify Network Interface Usage
    Interface Name: eth0 Subnet: 192.168.1.0 Public
    Interface Name: eth1 Subnet: 192.168.1.0 Private
    Using the Advanced Installation ->
    No GNS ->
    I'm getting error [ins-41113] Specified public and private interfaces are configured on the same subnet: 192.168.1.0 Why is this an error because Oracle Documentation e17212/typinst.htm says the following:
    A Single Client Acess Name (SCAN) for the cluster , wich the following characteristics:
    * Three static IP address; however I want to use only two static IP address because I am not using DNS I'm using /etc/hosts file.
    * On the same subnet as all other public IP addresses, VIP addresses, and SCAN addresses.
    Please advise because I was able to install Grid Infrastructure 11.2.0.2 using the same configuration with no problems. Now that I am using Oracle 11.2.0.3 I'm encountering this problem.

    Hi,
    I appreciate insight with respect to Why Does it work in Oracle Grid Infrastructure 11.2.0.2 and not in 11.2.0.3 ?Some things that are not supported, are sometimes allowed by Oracle Product Software (I can say it's bug), but still desupported.
    See this example.
    In 11.2.0.1 and 11.2.0.2 it's possible add releases prior to 11.2 of Oracle Database to be managed by Oracle Restart. But It's not certified.
    See this thread.
    {message:id=9270783}
    In 11.2.0.3 Oracle solved this issue don't allowing do that and created a new msg error :
    PRCD-1245 : Addition of database version 11.2.0.3.0 is not allowed using srvctl version 10.2.0.0.0So, like example above in 11.2.0.1 and 11.2.0.2 it's not supported use same subnet to private and public network, but it's allowed by Oracle Product Software, now it's not allowed by Oracle Product Software
    The install guide will tell you the following requirements private IP address must satisfy the following requirements:
    *1. Must be separate from the public network*
    2. Must be accessible on the same network interface on each node
    3. Must have a unique address on each node
    4. Must be specified in the /etc/hosts file on each node
    The Best Pratices recommendation is to use the TCP/IP standard for non-routeable networks. Reserved address ranges for private (non-routed) use (see TCP/IP RFC 1918):
    * 10.0.0.0 -> 10.255.255.255
    * 172.16.0.0 -> 172.31.255.255
    * 192.168.0.0 -> 192.168.255.255
    Cluvfy will give you an error if you do not have your private interconnect in the ranges above.
    You should not ignore this error. If you are using an IP address in the range used for the public network for the private network interfaces, you are pretty much messing up the IP addressing, and possibly the routing tables, for the rest of the corporation. IP addresses are a sparse commodity, use them wisely. If you use them on a non-routable network, there is nothing to prevent someone else to go and use them in the normal corporate network, and then when those RAC nodes find out that there is another path to that address range (through RIP), they just might start sending traffic to those other IP addresses instead of the interconnect. This is just a bad idea.
    This above is described on MoS: *RAC: Frequently Asked Questions [ID 220970.1]*
    Regards,
    Levi Pereira

  • Oracle 10G RAC - Public & Heartbeat on 1 NIC

    Hello all,
    actually I am installing Oracle 10G RAC on rhel 4 (4 node cluster). But the Cluster Verification Utility aborts with errors. I checked the configToolAllCommands and tried to run the failed commands manually:
    #/opt/oracle/crs/bin/oifcfg setif -global eth0.100/172.18.0.0:cluster_interconnect eth0.728/172.16.128.0:public eth0.498/172.17.1.0:cluster_interconnect
    PRIF-50: duplicate interface is given in the input
    PRIF-50: duplicate interface is given in the input
    PRIF-50: duplicate interface is given in the input
    Question:
    Is it possible to put Public & Heartbeat on one NIC (eth0.728 & eth0.498)
    If not is their any workaround for that issue?
    Output /etc/hosts
    # that require network functionality will fail.
    127.0.0.1 localhost.localdomain localhost
    172.18.253.48 eu0266.[company].net cfmaster
    172.16.128.11 eu0200.[company].net eu0200
    172.16.128.12 eu0201.[company].net eu0201
    172.16.128.13 eu0202.[company].net eu0202
    172.16.128.14 eu0203.[company].net eu0203
    172.18.13.11 eu0200m.[company].net eu0200m
    172.18.13.12 eu0201m.[company].net eu0201m
    172.18.13.13 eu0202m.[company].net eu0202m
    172.18.13.14 eu0203m.[company].net eu0203m
    # Private section
    172.17.1.11 eu0200-priv.[company].net eu0200-priv
    172.17.1.12 eu0201-priv.[company].net eu0201-priv
    172.17.1.13 eu0202-priv.[company].net eu0202-priv
    172.17.1.14 eu0203-priv.[company].net eu0203-priv
    # Virtual section
    172.16.128.16 eu0200-vip.[company].net eu0200-vip
    172.16.128.17 eu0201-vip.l[company].net eu0201-vip
    172.16.128.18 eu0202-vip.[company].net eu0202-vip
    172.16.128.19 eu0203-vip.[company].net eu0203-vip
    Output install log:
    Checking existence of VIP node application (required)
    Check failed.
    Check failed on nodes:
    eu0203,eu0202,eu0201,eu0200
    Checking existence of ONS node application (optional)
    Check ignored.
    Checking existence of GSD node application (optional)
    Check ignored.
    Post-check for cluster services setup was unsuccessful on all the nodes.
    Command = /opt/[company]/oracle/crs/bin/cluvfy has failed
    INFO: Configuration assistant "Oracle Cluster Verification Utility" failed
    *** Starting OUICA ***
    Oracle Home set to /opt/[company]/oracle/crs
    Configuration directory is set to /opt/[company]/oracle/crs/cfgtoollogs. All xml files under the directory will be processed
    INFO: The "/opt/[company]/oracle/crs/cfgtoollogs/configToolFailedCommands" script contains all commands that failed, were skipped or were cancelled. This file may be used to run these configuration assistants outside of OUI. Note that you may have to update this script with passwords (if any) before executing the same.
    SEVERE: OUI-25031:Some of the configuration assistants failed. It is strongly recommended that you retry the configuration assistants at this time. Not successfully running any "Recommended" assistants means your system will not be correctly configured.
    1. Check the Details panel on the Configuration Assistant Screen to see the errors resulting in the failures.
    2. Fix the errors causing these failures.
    3. Select the failed assistants and click the 'Retry' button to retry them.
    Thanks in advance for your help!
    Regards

    Tads wrote:
    Hi all,
    I have a question about Oracle RAC and network interface.
    We're using Oracle 10gR2 RAC with two nodes on Linux Red Hat.
    Let's assume that the public network interface goes down.
    I would like to know what happens with existing connections
    on node with network interface with problems.
    Are connections frozen, actives?
    Can the users continue to use theses existing connections using the another node of RAC?If the interface is down? what do you think? All connections to this node will die. How does your application handle fail-over, does it attempt to reconnect or just have a complete application failure?
    You should spend some time in a test lab where you can test this stuff for yourself. Read the documentation and there are tons of sites out there that purport to have all of your RAC/TAF/FAN/FAF questions. - I would read and trust the documentation first.
    >
    I know that the listener goes down and any other connections is allowed.
    Thank you very much!!!!

  • Oracle RAC nodeapp 启动报错-vip:IP:192.168.2.200 is already up in the network

    Linux redhat 5 Oracle RAC 10.2.0.5环境
    启动nodeapp报错如下:
    [oracle@rac1 ~]$ srvctl start nodeapps -n rac1
    CRS-0210: Could not find resource ora.rac1.gsd.
    rac1:ora.rac1.vip:IP:192.168.2.200 is already up in the network (host=rac1)
    rac1:ora.rac1.vip:IP:192.168.2.200 is already up in the network (host=rac1)
    rac1:ora.rac1.vip:IP:192.168.2.200 is already up in the network (host=rac1)
    rac1:ora.rac1.vip:IP:192.168.2.200 is already up in the network (host=rac1)
    rac1:ora.rac1.vip:IP:192.168.2.200 is already up in the network (host=rac1)
    rac1:ora.rac1.vip:IP:192.168.2.200 is already up in the network (host=rac1)
    rac1:ora.rac1.vip:IP:192.168.2.200 is already up in the network (host=rac1)
    rac1:ora.rac1.vip:IP:192.168.2.200 is already up in the network (host=rac1)
    rac1:ora.rac1.vip:IP:192.168.2.200 is already up in the network (host=rac1)
    rac1:ora.rac1.vip:IP:192.168.2.200 is already up in the network (host=rac1)
    rac1:ora.rac1.vip:IP:192.168.2.200 is already up in the network (host=rac1)
    rac1:ora.rac1.vip:IP:192.168.2.200 is already up in the network (host=rac1)
    rac1:ora.rac1.vip:IP:192.168.2.200 is already up in the network (host=rac1)
    rac1:ora.rac1.vip:IP:192.168.2.200 is already up in the network (host=rac1)
    rac1:ora.rac1.vip:IP:192.168.2.200 is already up in the network (host=rac1)
    rac1:ora.rac1.vip:IP:192.168.2.200 is already up in the network (host=rac1)
    rac1:ora.rac1.vip:IP:192.168.2.200 is already up in the network (host=rac1)
    rac1:ora.rac1.vip:IP:192.168.2.200 is already up in the network (host=rac1)
    rac1:ora.rac1.vip:IP:192.168.2.200 is already up in the network (host=rac1)
    rac1:ora.rac1.vip:IP:192.168.2.200 is already up in the network (host=rac1)
    rac1:ora.rac1.vip:IP:192.168.2.200 is already up in the network (host=rac1)
    rac1:ora.rac1.vip:IP:192.168.2.200 is already up in the network (host=rac1)
    crsd.log部分日志如下:
    2012-11-21 20:10:16.831: [  CRSRES][2717907856]0startRunnable: setting CLI values
    2012-11-21 20:10:16.967: [  CRSRES][2717907856]0Attempting to start `ora.rac1.vip` on member `rac1`
    2012-11-21 20:10:44.246: [  CRSAPP][2717907856]0StartResource error for ora.rac1.vip error code = 1
    2012-11-21 20:10:47.007: [  CRSRES][2717907856]0Start of `ora.rac1.vip` on member `rac1` failed.
    2012-11-21 20:10:47.529: [  CRSRES][2717907856]0Attempting to start `ora.rac1.vip` on member `rac2`
    2012-11-21 20:11:18.649: [  CRSRES][2717907856]0Start of `ora.rac1.vip` on member `rac2` failed.
    2012-11-21 20:11:18.897: [  CRSRES][2717907856]0CRS-1006: No more members to consider
    2012-11-21 20:11:20.986: [  CRSRES][2717907856]0startRunnable: setting CLI values
    2012-11-21 20:11:21.190: [  CRSRES][2717907856]0Attempting to start `ora.rac1.vip` on member `rac1`
    2012-11-21 20:11:48.846: [  CRSAPP][2717907856]0StartResource error for ora.rac1.vip error code = 1
    2012-11-21 20:11:51.203: [  CRSRES][2717907856]0Start of `ora.rac1.vip` on member `rac1` failed.
    2012-11-21 20:11:51.492: [  CRSRES][2717907856]0rac2 : CRS-1019: Resource ora.rac1.LISTENER_RAC1.lsnr (application) cannot run on rac2
    请问如何进一步排查“rac1:ora.rac1.vip:IP:192.168.2.200 is already up in the network (host=rac1)”问题?感谢

    ping信息:
    [oracle@rac1 ~]$ ping 192.168.2.200
    PING 192.168.2.200 (192.168.2.200) 56(84) bytes of data.
    64 bytes from 192.168.2.200: icmp_seq=1 ttl=64 time=0.043 ms
    64 bytes from 192.168.2.200: icmp_seq=2 ttl=64 time=0.126 ms
    64 bytes from 192.168.2.200: icmp_seq=3 ttl=64 time=0.059 ms
    64 bytes from 192.168.2.200: icmp_seq=4 ttl=64 time=0.158 ms
    64 bytes from 192.168.2.200: icmp_seq=5 ttl=64 time=0.643 ms
    64 bytes from 192.168.2.200: icmp_seq=6 ttl=64 time=0.034 ms
    64 bytes from 192.168.2.200: icmp_seq=7 ttl=64 time=0.046 ms
    64 bytes from 192.168.2.200: icmp_seq=8 ttl=64 time=0.043 ms
    64 bytes from 192.168.2.200: icmp_seq=9 ttl=64 time=0.048 ms
    64 bytes from 192.168.2.200: icmp_seq=10 ttl=64 time=0.031 ms
    telnet信息
    [oracle@rac1 ~]$ telnet 192.168.2.200
    Trying 192.168.2.200...
    Connected to 192.168.2.200.
    Escape character is '^]'.
    Red Hat Enterprise Linux Server release 5 (Tikanga)
    Kernel 2.6.18-8.el5xen on an i686
    login:
    ssh信息:
    [oracle@rac1 ~]$ ssh 192.168.2.200
    Last login: Sun Nov 18 13:37:10 2012 from rac2-vip
    ifconfig -a信息:
    [root@rac1 ~]# ifconfig -a
    eth0 Link encap:Ethernet HWaddr 00:0C:29:B6:CE:6B
    inet addr:192.168.2.100 Bcast:192.168.2.255 Mask:255.255.255.0
    inet6 addr: fe80::20c:29ff:feb6:ce6b/64 Scope:Link
    UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
    RX packets:142396 errors:0 dropped:0 overruns:0 frame:0
    TX packets:172561 errors:0 dropped:0 overruns:0 carrier:0
    collisions:0 txqueuelen:1000
    RX bytes:41403126 (39.4 MiB) TX bytes:96009307 (91.5 MiB)
    Interrupt:18 Base address:0x1480
    eth1 Link encap:Ethernet HWaddr 00:0C:29:B6:CE:75
    inet addr:192.168.2.200 Bcast:192.168.2.255 Mask:255.255.255.0
    inet6 addr: fe80::20c:29ff:feb6:ce75/64 Scope:Link
    UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
    RX packets:14082 errors:0 dropped:0 overruns:0 frame:0
    TX packets:29 errors:0 dropped:0 overruns:0 carrier:0
    collisions:0 txqueuelen:1000
    RX bytes:9789756 (9.3 MiB) TX bytes:1434 (1.4 KiB)
    Interrupt:19 Base address:0x1800
    eth2 Link encap:Ethernet HWaddr 00:0C:29:B6:CE:7F
    inet addr:192.168.3.100 Bcast:192.168.3.255 Mask:255.255.255.0
    inet6 addr: fe80::20c:29ff:feb6:ce7f/64 Scope:Link
    UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
    RX packets:12665 errors:0 dropped:0 overruns:0 frame:0
    TX packets:32728 errors:0 dropped:0 overruns:0 carrier:0
    collisions:0 txqueuelen:1000
    RX bytes:6590271 (6.2 MiB) TX bytes:28437643 (27.1 MiB)
    Interrupt:16 Base address:0x1880
    lo Link encap:Local Loopback
    inet addr:127.0.0.1 Mask:255.0.0.0
    inet6 addr: ::1/128 Scope:Host
    UP LOOPBACK RUNNING MTU:16436 Metric:1
    RX packets:30893 errors:0 dropped:0 overruns:0 frame:0
    TX packets:30893 errors:0 dropped:0 overruns:0 carrier:0
    collisions:0 txqueuelen:0
    RX bytes:9402131 (8.9 MiB) TX bytes:9402131 (8.9 MiB)
    sit0 Link encap:IPv6-in-IPv4
    NOARP MTU:1480 Metric:1
    RX packets:0 errors:0 dropped:0 overruns:0 frame:0
    TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
    collisions:0 txqueuelen:0
    RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)

  • Oracle 10g RAC - public network interface down

    Hi all,
    I have a question about Oracle RAC and network interface.
    We're using Oracle 10gR2 RAC with two nodes on Linux Red Hat.
    Let's assume that the public network interface goes down.
    I would like to know what happens with existing connections
    on node with network interface with problems.
    Are connections frozen, actives?
    Can the users continue to use theses existing connections using the another node of RAC?
    I know that the listener goes down and any other connections is allowed.
    Thank you very much!!!!

    Tads wrote:
    Hi all,
    I have a question about Oracle RAC and network interface.
    We're using Oracle 10gR2 RAC with two nodes on Linux Red Hat.
    Let's assume that the public network interface goes down.
    I would like to know what happens with existing connections
    on node with network interface with problems.
    Are connections frozen, actives?
    Can the users continue to use theses existing connections using the another node of RAC?If the interface is down? what do you think? All connections to this node will die. How does your application handle fail-over, does it attempt to reconnect or just have a complete application failure?
    You should spend some time in a test lab where you can test this stuff for yourself. Read the documentation and there are tons of sites out there that purport to have all of your RAC/TAF/FAN/FAF questions. - I would read and trust the documentation first.
    >
    I know that the listener goes down and any other connections is allowed.
    Thank you very much!!!!

  • RAC 11GR2 + VIP

    Hi all,
    I have a cluster RAC 11GR2 with 2 nodes on a site A (datacenter).
    I have used 1 IP SCAN et 2 VIP.
    The scan IP is 10.XXX.X.181
    The rac1-vip is 10.XXX.X.165
    The rac2-vip is 10.XXX.X.166
    I used /etc/hosts file and not the dns for the moment.
    These IP are used on a LAN.
    Now, i need to attack this RAC 11GR2 via a WAN (on a site B) and so via other IP adress.
    How can i do to make this? Do you have an idea ?
    Thanks.

    can you be more specific ? You want to setup new environment similar to you have in datacenter you mentioned?

  • RAC 11gR2 VIP and SCAN failover behavior

    I tried to failover testing on 2 node 11gR2 RAC on Linux 5 X86-64 with configure in tnsnames.ora on client side (client side TAF configuration) with the following configuration
    NVSDB =
    (DESCRIPTION =
    (ADDRESS = (PROTOCOL = TCP)(HOST = dbrac-scan)(PORT = 1521))
    (LOAD_BALANCE = yes)
    (CONNECT_DATA =
    (SERVER = DEDICATED)
    (SERVICE_NAME = NVSDB)
    (FAILOVER_MODE =
    (TYPE = SELECT)
    (METHOD = BASIC)
    (RETRIES = 180)
    (DELAY = 5)
    The test scenario
    1. Connect from client to RAC database , and test query with select * from dba_objects
    2. Open new terminal and connect to instance in step 1 , then shutdown abort
    3. I found that , the session in step 1 not failover to other node , and VIP and SCAN not failover to other node too
    In my opinion session , VIP and SCAN should failover to other node.
    Or is this the default behavior of VIP and SCAN on 11gR2 ??
    Can anyone suggest me ?
    Thankyou and Regards

    user10139161 wrote:
    3. I found that , the session in step 1 not failover to other node , Which client/driver did you use to try this? The reason I am asking is that TAF is not supported with jdbc:thin, so if you were trying this with sqldeveloper and the defaul driver that would be the expected behaviour
    and VIP and SCAN not failover to other node tooThat is also expected. You only stopped the instance, there is no reason to relocatethe VIP. In your case, client could still connect to the listener running on this node, the listener would know of the second instance still being up and redirect all connection attempts there.
    What you could try is rebooting the whole server, then you would see IPs being relocated.
    Bjoern

  • RAC crs vip log file ownership

    Hi Experts,
    I am debugging oracl2 11.1 cluster issue and find i am not able to download ora.nodename.vip.log file under log/<hostname>/racg in Linux platform.
    The further finding, ora.nodename.vip.log file file owner is root:root. I think it should be owner as oracle:oinstall
    Experts does this vip log file ownership is correcttion? It seems that i can download these file before by oracle account in linux.
    as ls -ltr commend show
    ora.nodename.vip.log file rw-rw--- root:root
    Thanks
    Jin
    Edited by: user589812 on Feb 3, 2012 4:03 PM

    Thanks for your explaining!

  • Oracle RAC public network bandwidth requirement

    Is there any thumb rule on what should be bandwidth for the public network for a type of workload?

    user458367 wrote:
    Is there any thumb rule on what should be bandwidth for the public network for a type of workload?No. As the workload could mean sending 1KB instruction (SQL) from the client to the cluster, cause a 100 PQs to be created, cluster resources to be maxed out, and a 1KB result (aggregated answer) to be returned to the client.
    What was the workload impact on the network? 2KB. What was the workload impact on the cluster? Significant.
    Network management is also a lot more than simplistically dealing with "+workloads+". It is a shared medium. It is routed. It can implement differentiated services/quality of service. It is managed entirely separately from an application or o/s or database perspective. There are numerous other factors that need to be considered than just the perspective of what the "workload" is.

  • Oracle 10.2.0.4 RAC vip故障漂移导致监听关闭.

    环境:oracle 10.2.0.4 RAC+ASM
    故障现象:vip漂移至另一节点导致监听关闭,vip漂移现象随机在2个节点中出现,数据库实例正常,只是vip漂移至另一节点,同时监听offline。
    # su - oracle
    $ crs_stat -t
    Name Type Target State Host
    ora....SM1.asm application ONLINE ONLINE p770db1
    ora....B1.lsnr application ONLINE OFFLINE
    ora....db1.gsd application ONLINE ONLINE p770db1
    ora....db1.ons application ONLINE ONLINE p770db1
    ora....db1.vip application ONLINE ONLINE p770db2
    ora....SM2.asm application ONLINE ONLINE p770db2
    ora....B2.lsnr application ONLINE ONLINE p770db2
    ora....db2.gsd application ONLINE ONLINE p770db2
    ora....db2.ons application ONLINE ONLINE p770db2
    ora....db2.vip application ONLINE ONLINE p770db2
    ora.zjhorcl.db application ONLINE ONLINE p770db1
    ora....orcl.cs application ONLINE ONLINE p770db1
    ora....cl1.srv application ONLINE OFFLINE
    ora....cl2.srv application ONLINE ONLINE p770db2
    ora....l1.inst application ONLINE ONLINE p770db1
    ora....l2.inst application ONLINE ONLINE p770db2
    然后要用crs_start -all重启后来能恢复,如下:
    $ crs_start -all
    p770db1 : CRS-1018: Resource ora.p770db1.vip (application) is already running on p770db2
    p770db2 : CRS-1019: Resource ora.p770db1.LISTENER_P770DB1.lsnr (application) cannot run on p770db2
    Attempting to start `ora.zjhorcl.orcl.zjhorcl1.srv` on member `p770db1`
    Start of `ora.zjhorcl.orcl.zjhorcl1.srv` on member `p770db1` succeeded.
    CRS-0223: Resource 'ora.p770db1.LISTENER_P770DB1.lsnr' has placement error.
    ========================================================================================
    2012-10-11 06:56:21.830: [  CRSRES][11110]32startRunnable: setting CLI values
    2012-10-12 13:47:28.205: [  CRSRES][11471]32p770db1 : CRS-1019: Resource ora.p770db2.LISTENER_P770DB2.lsnr (application) cannot run
    on p770db1
    p770db2 : CRS-1018: Resource ora.p770db2.vip (application) is already running on p770db1
    2012-10-12 13:47:28.247: [  CRSRES][11728]32Attempting to start `ora.zjhorcl.orcl.zjhorcl2.srv` on member `p770db2`
    2012-10-12 13:47:28.656: [  CRSRES][11986]32StopResource: setting CLI values
    2012-10-12 13:47:31.709: [  CRSRES][11728]32Start of `ora.zjhorcl.orcl.zjhorcl2.srv` on member `p770db2` succeeded.
    2012-10-12 15:34:14.228: [  CRSAPP][11192]32CheckResource error for ora.p770db1.vip error code = 1
    2012-10-12 15:34:14.234: [  CRSRES][11192]32In stateChanged, ora.p770db1.vip target is ONLINE
    2012-10-12 15:34:14.235: [  CRSRES][11192]32ora.p770db1.vip on p770db1 went OFFLINE unexpectedly
    2012-10-12 15:34:14.235: [  CRSRES][11192]32StopResource: setting CLI values
    2012-10-12 15:34:14.238: [  CRSRES][11192]32Attempting to stop `ora.p770db1.vip` on member `p770db1`
    2012-10-12 15:34:14.602: [  CRSRES][11709]32In stateChanged, ora.zjhorcl.orcl.zjhorcl1.srv target is ONLINE
    2012-10-12 15:34:14.602: [  CRSRES][11709]32ora.zjhorcl.orcl.zjhorcl1.srv on p770db1 went OFFLINE unexpectedly
    2012-10-12 15:34:14.603: [  CRSRES][11709]32StopResource: setting CLI values
    2012-10-12 15:34:14.628: [  CRSRES][11709]32Attempting to stop `ora.zjhorcl.orcl.zjhorcl1.srv` on member `p770db1`
    2012-10-12 15:34:14.716: [  CRSRES][11192]32Stop of `ora.p770db1.vip` on member `p770db1` succeeded.
    2012-10-12 15:34:14.717: [  CRSRES][11192]32ora.p770db1.vip RESTART_COUNT=0 RESTART_ATTEMPTS=0
    2012-10-12 15:34:14.722: [  CRSRES][11192]32ora.p770db1.vip failed on p770db1 relocating.
    2012-10-12 15:34:14.750: [  CRSRES][11192]32StopResource: setting CLI values
    2012-10-12 15:34:14.754: [  CRSRES][11192]32Attempting to stop `ora.p770db1.LISTENER_P770DB1.lsnr` on member `p770db1`
    2012-10-12 15:34:15.011: [  CRSRES][11709]32Stop of `ora.zjhorcl.orcl.zjhorcl1.srv` on member `p770db1` succeeded.
    2012-10-12 15:34:15.011: [  CRSRES][11709]32ora.zjhorcl.orcl.zjhorcl1.srv RESTART_COUNT=0 RESTART_ATTEMPTS=0
    2012-10-12 15:34:15.019: [  CRSRES][11709]32ora.zjhorcl.orcl.zjhorcl1.srv failed on p770db1 relocating.
    2012-10-12 15:34:15.115: [  CRSRES][11709]32Cannot relocate ora.zjhorcl.orcl.zjhorcl1.srvStopping dependents
    2012-10-12 15:34:15.321: [  CRSRES][11709]32StopResource: setting CLI values
    2012-10-12 15:35:32.118: [  CRSRES][11192]32Stop of `ora.p770db1.LISTENER_P770DB1.lsnr` on member `p770db1` succeeded.
    2012-10-12 15:35:32.181: [  CRSRES][11192]32Attempting to start `ora.p770db1.vip` on member `p770db2`
    2012-10-12 15:35:33.824: [  CRSRES][11192]32Start of `ora.p770db1.vip` on member `p770db2` succeeded.
    2012-10-13 03:09:02.433: [  CRSEVT][11049]32CAAMonitorHandler :: 0:Could not join /oracle/product/10.2.0/db_2/bin/racgwrap(check)
    category: 1234, operation: scls_process_join, loc: childcrash, OS error: 0, other: Abnormal termination of the child
    2012-10-13 03:09:02.433: [  CRSEVT][11049]32CAAMonitorHandler :: 0:Action Script /oracle/product/10.2.0/db_2/bin/racgwrap(check) tim
    ed out for ora.zjhorcl.orcl.cs! (timeout=600)
    2012-10-13 03:09:02.433: [  CRSAPP][11049]32CheckResource error for ora.zjhorcl.orcl.cs error code = -2
    2012-10-13 03:09:10.452: [  CRSEVT][11562]32CAAMonitorHandler :: 0:Could not join /oracle/product/10.2.0/db_2/bin/racgwrap(check)
    category: 1234, operation: scls_process_join, loc: childcrash, OS error: 0, other: Abnormal termination of the child
    2012-10-13 03:09:10.453: [  CRSEVT][11562]32CAAMonitorHandler :: 0:Action Script /oracle/product/10.2.0/db_2/bin/racgwrap(check) tim
    ed out for ora.p770db1.ASM1.asm! (timeout=600)
    2012-10-13 03:09:10.453: [  CRSAPP][11562]32CheckResource error for ora.p770db1.ASM1.asm error code = -2
    2012-10-13 03:09:10.583: [  CRSEVT][11819]32CAAMonitorHandler :: 0:Could not join /oracle/product/10.2.0/db_2/bin/racgwrap(check)
    category: 1234, operation: scls_process_join, loc: childcrash, OS error: 0, other: Abnormal termination of the child
    2012-10-13 03:09:10.583: [  CRSEVT][11819]32CAAMonitorHandler :: 0:Action Script /oracle/product/10.2.0/db_2/bin/racgwrap(check) tim
    ed out for ora.zjhorcl.zjhorcl1.inst! (timeout=600)
    2012-10-13 03:09:10.583: [  CRSAPP][11819]32CheckResource error for ora.zjhorcl.zjhorcl1.inst error code = -2
    2012-10-13 03:09:14.032: [  CRSEVT][12332]32CAAMonitorHandler :: 0:Could not join /oracle/product/10.2.0/db_1/bin/racgwrap(check)
    category: 1234, operation: scls_process_join, loc: childcrash, OS error: 0, other: Abnormal termination of the child
    2012-10-13 03:09:14.033: [  CRSEVT][12332]32CAAMonitorHandler :: 0:Action Script /oracle/product/10.2.0/db_1/bin/racgwrap(check) tim
    ed out for ora.p770db1.ons! (timeout=600)
    2012-10-13 03:09:14.033: [  CRSAPP][12332]32CheckResource error for ora.p770db1.ons error code = -2
    2012-10-14 19:32:02.637: [  CRSRES][11089]32startRunnable: setting CLI values
    2012-10-14 19:32:03.342: [  CRSRES][11603]32StopResource: setting CLI values
    2012-10-14 19:32:03.353: [  CRSRES][11603]32Attempting to stop `ora.p770db1.vip` on member `p770db2`
    2012-10-14 19:32:03.792: [  CRSRES][11603]32Stop of `ora.p770db1.vip` on member `p770db2` succeeded.
    2012-10-14 19:32:03.799: [  CRSRES][11603]32startRunnable: setting CLI values
    2012-10-14 19:32:03.799: [  CRSRES][11603]32Attempting to start `ora.p770db1.vip` on member `p770db1`
    2012-10-14 19:32:05.520: [  CRSRES][11603]32Start of `ora.p770db1.vip` on member `p770db1` succeeded.
    2012-10-14 19:32:05.688: [  CRSRES][11864]32startRunnable: setting CLI values
    2012-10-14 19:32:05.696: [  CRSRES][11864]32Attempting to start `ora.p770db1.LISTENER_P770DB1.lsnr` on member `p770db1`
    2012-10-14 19:32:06.273: [  CRSRES][12386]32CRS-1002: Resource 'ora.zjhorcl.orcl.cs' is already running on member 'p770db1'
    2012-10-14 19:32:06.576: [  CRSRES][11864]32Start of `ora.p770db1.LISTENER_P770DB1.lsnr` on member `p770db1` succeeded.
    2012-10-21 10:29:19.051: [  CRSAPP][11054]32CheckResource error for ora.p770db1.vip error code = 1
    2012-10-21 10:29:19.059: [  CRSRES][11054]32In stateChanged, ora.p770db1.vip target is ONLINE
    2012-10-21 10:29:19.060: [  CRSRES][11054]32ora.p770db1.vip on p770db1 went OFFLINE unexpectedly
    2012-10-21 10:29:19.060: [  CRSRES][11054]32StopResource: setting CLI values
    2012-10-21 10:29:19.070: [  CRSRES][11054]32Attempting to stop `ora.p770db1.vip` on member `p770db1`
    2012-10-21 10:29:19.526: [  CRSRES][11828]32In stateChanged, ora.zjhorcl.orcl.zjhorcl1.srv target is ONLINE
    2012-10-21 10:29:19.526: [  CRSRES][11828]32ora.zjhorcl.orcl.zjhorcl1.srv on p770db1 went OFFLINE unexpectedly
    2012-10-21 10:29:19.527: [  CRSRES][11828]32StopResource: setting CLI values
    2012-10-21 10:29:19.575: [  CRSRES][11828]32Attempting to stop `ora.zjhorcl.orcl.zjhorcl1.srv` on member `p770db1`
    2012-10-21 10:29:19.673: [  CRSRES][11054]32Stop of `ora.p770db1.vip` on member `p770db1` succeeded.
    2012-10-21 10:29:19.674: [  CRSRES][11054]32ora.p770db1.vip RESTART_COUNT=0 RESTART_ATTEMPTS=0
    2012-10-21 10:29:19.684: [  CRSRES][11054]32ora.p770db1.vip failed on p770db1 relocating.
    2012-10-21 10:29:19.718: [  CRSRES][11054]32StopResource: setting CLI values
    2012-10-21 10:29:19.733: [  CRSRES][11054]32Attempting to stop `ora.p770db1.LISTENER_P770DB1.lsnr` on member `p770db1`
    2012-10-21 10:29:19.932: [  CRSRES][11828]32Stop of `ora.zjhorcl.orcl.zjhorcl1.srv` on member `p770db1` succeeded.
    2012-10-21 10:29:19.933: [  CRSRES][11828]32ora.zjhorcl.orcl.zjhorcl1.srv RESTART_COUNT=0 RESTART_ATTEMPTS=0
    2012-10-21 10:29:19.941: [  CRSRES][11828]32ora.zjhorcl.orcl.zjhorcl1.srv failed on p770db1 relocating.
    2012-10-21 10:29:20.116: [  CRSRES][11828]32Cannot relocate ora.zjhorcl.orcl.zjhorcl1.srvStopping dependents
    2012-10-21 10:29:20.154: [  CRSRES][11828]32StopResource: setting CLI values
    2012-10-21 10:30:36.095: [  CRSRES][11054]32Stop of `ora.p770db1.LISTENER_P770DB1.lsnr` on member `p770db1` succeeded.
    2012-10-21 10:30:36.280: [  CRSRES][11054]32Attempting to start `ora.p770db1.vip` on member `p770db2`
    2012-10-21 10:30:37.942: [  CRSRES][11054]32Start of `ora.p770db1.vip` on member `p770db2` succeeded.
    2012-10-21 10:30:38.289: [  CRSRES][11932]32CRS-1002: Resource 'ora.p770db1.vip' is already running on member 'p770db2'
    2012-10-22 10:51:43.580: [  CRSRES][11262]32startRunnable: setting CLI values
    2012-10-22 10:51:44.110: [  CRSRES][11777]32StopResource: setting CLI values
    2012-10-22 10:51:44.217: [  CRSRES][11777]32Attempting to stop `ora.p770db1.vip` on member `p770db2`
    2012-10-22 10:51:44.976: [  CRSRES][11777]32Stop of `ora.p770db1.vip` on member `p770db2` succeeded.
    2012-10-22 10:51:44.984: [  CRSRES][11777]32startRunnable: setting CLI values
    2012-10-22 10:51:44.984: [  CRSRES][11777]32Attempting to start `ora.p770db1.vip` on member `p770db1`
    2012-10-22 10:51:46.757: [  CRSRES][11777]32Start of `ora.p770db1.vip` on member `p770db1` succeeded.
    2012-10-22 10:51:46.988: [  CRSRES][11534]32startRunnable: setting CLI values
    2012-10-22 10:51:47.006: [  CRSRES][11534]32Attempting to start `ora.p770db1.LISTENER_P770DB1.lsnr` on member `p770db1`
    2012-10-22 10:51:47.217: [  CRSRES][12304]32CRS-1002: Resource 'ora.zjhorcl.orcl.cs' is already running on member 'p770db1'
    2012-10-22 10:51:47.763: [  CRSRES][11534]32Start of `ora.p770db1.LISTENER_P770DB1.lsnr` on member `p770db1` succeeded.
    2012-10-23 04:23:21.417: [  CRSAPP][11219]32CheckResource error for ora.p770db1.vip error code = 1
    2012-10-23 04:23:21.425: [  CRSRES][11219]32In stateChanged, ora.p770db1.vip target is ONLINE
    2012-10-23 04:23:21.425: [  CRSRES][11219]32ora.p770db1.vip on p770db1 went OFFLINE unexpectedly
    2012-10-23 04:23:21.425: [  CRSRES][11219]32StopResource: setting CLI values
    2012-10-23 04:23:21.435: [  CRSRES][11219]32Attempting to stop `ora.p770db1.vip` on member `p770db1`
    2012-10-23 04:23:21.876: [  CRSRES][11993]32In stateChanged, ora.zjhorcl.orcl.zjhorcl1.srv target is ONLINE
    2012-10-23 04:23:21.877: [  CRSRES][11993]32ora.zjhorcl.orcl.zjhorcl1.srv on p770db1 went OFFLINE unexpectedly
    2012-10-23 04:23:21.877: [  CRSRES][11993]32StopResource: setting CLI values
    2012-10-23 04:23:21.888: [  CRSRES][11993]32Attempting to stop `ora.zjhorcl.orcl.zjhorcl1.srv` on member `p770db1`
    2012-10-23 04:23:22.067: [  CRSRES][11219]32Stop of `ora.p770db1.vip` on member `p770db1` succeeded.
    2012-10-23 04:23:22.067: [  CRSRES][11219]32ora.p770db1.vip RESTART_COUNT=0 RESTART_ATTEMPTS=0
    2012-10-23 04:23:22.074: [  CRSRES][11219]32ora.p770db1.vip failed on p770db1 relocating.
    2012-10-23 04:23:22.133: [  CRSRES][11219]32StopResource: setting CLI values
    2012-10-23 04:23:22.157: [  CRSRES][11219]32Attempting to stop `ora.p770db1.LISTENER_P770DB1.lsnr` on member `p770db1`
    2012-10-23 04:23:22.295: [  CRSRES][11993]32Stop of `ora.zjhorcl.orcl.zjhorcl1.srv` on member `p770db1` succeeded.
    2012-10-23 04:23:22.296: [  CRSRES][11993]32ora.zjhorcl.orcl.zjhorcl1.srv RESTART_COUNT=0 RESTART_ATTEMPTS=0
    2012-10-23 04:23:22.309: [  CRSRES][11993]32ora.zjhorcl.orcl.zjhorcl1.srv failed on p770db1 relocating.
    2012-10-23 04:23:22.344: [  CRSRES][11993]32Cannot relocate ora.zjhorcl.orcl.zjhorcl1.srvStopping dependents
    2012-10-23 04:23:22.388: [  CRSRES][11993]32StopResource: setting CLI values
    2012-10-23 04:24:38.655: [  CRSRES][11219]32Stop of `ora.p770db1.LISTENER_P770DB1.lsnr` on member `p770db1` succeeded.
    2012-10-23 04:24:38.734: [  CRSRES][11219]32Attempting to start `ora.p770db1.vip` on member `p770db2`
    2012-10-23 04:24:40.474: [  CRSRES][11219]32Start of `ora.p770db1.vip` on member `p770db2` succeeded.
    2012-10-23 13:50:39.656: [  CRSRES][11257]32startRunnable: setting CLI values
    2012-10-23 13:50:40.032: [  CRSRES][11771]32StopResource: setting CLI values
    2012-10-23 13:50:40.042: [  CRSRES][11771]32Attempting to stop `ora.p770db1.vip` on member `p770db2`
    2012-10-23 13:50:40.607: [  CRSRES][11771]32Stop of `ora.p770db1.vip` on member `p770db2` succeeded.
    2012-10-23 13:50:40.614: [  CRSRES][11771]32startRunnable: setting CLI values
    2012-10-23 13:50:40.614: [  CRSRES][11771]32Attempting to start `ora.p770db1.vip` on member `p770db1`
    2012-10-23 13:50:42.336: [  CRSRES][11771]32Start of `ora.p770db1.vip` on member `p770db1` succeeded.
    2012-10-23 13:50:42.508: [  CRSRES][11776]32startRunnable: setting CLI values
    2012-10-23 13:50:42.517: [  CRSRES][11776]32Attempting to start `ora.p770db1.LISTENER_P770DB1.lsnr` on member `p770db1`
    2012-10-23 13:50:43.251: [  CRSRES][12298]32CRS-1002: Resource 'ora.zjhorcl.orcl.cs' is already running on member 'p770db1'
    2012-10-23 13:50:43.517: [  CRSRES][11776]32Start of `ora.p770db1.LISTENER_P770DB1.lsnr` on member `p770db1` succeeded.
    $
    $ more ora.p770db1.vip.log
    Oracle Database 10g CRS Release 10.2.0.4.0 Production Copyright 1996, 2008 Oracle. All rights reserved.
    2012-08-23 15:07:17.902: [    RACG][1] [15466872][1][ora.p770db1.vip]:
    entstat: 0909-004 Unable to get statistics on device en0, errno = 69
    Invalid parameters, or failed to bring up VIP (host=p770db1)
    2012-08-23 15:07:17.903: [    RACG][1] [15466872][1][ora.p770db1.vip]: clsrcexecut: env ORACLE_CONFIG_HOME=/oracle/product/10.2.0/db
    _1
    2012-08-23 15:07:17.903: [    RACG][1] [15466872][1][ora.p770db1.vip]: clsrcexecut: cmd = /oracle/product/10.2.0/db_1/bin/racgeut -e
    USRORA_DEBUG=0 54 /oracle/product/10.2.0/db_1/bin/racgvip check p770db1
    2012-08-23 15:07:17.903: [    RACG][1] [15466872][1][ora.p770db1.vip]: clsrcexecut: rc = 1, time = 0.509s
    2012-08-23 15:07:17.903: [    RACG][1] [15466872][1][ora.p770db1.vip]: end for resource = ora.p770db1.vip, action = check, status =
    1, time = 0.685s
    2012-09-23 10:43:34.433: [    RACG][1] [6815926][1][ora.p770db1.vip]: Invalid parameters, or failed to bring up VIP (host=p770db1)
    2012-09-23 10:43:34.442: [    RACG][1] [6815926][1][ora.p770db1.vip]: clsrcexecut: env ORACLE_CONFIG_HOME=/oracle/product/10.2.0/db_
    1
    2012-09-23 10:43:34.442: [    RACG][1] [6815926][1][ora.p770db1.vip]: clsrcexecut: cmd = /oracle/product/10.2.0/db_1/bin/racgeut -e
    USRORA_DEBUG=0 54 /oracle/product/10.2.0/db_1/bin/racgvip check p770db1
    2012-09-23 10:43:34.442: [    RACG][1] [6815926][1][ora.p770db1.vip]: clsrcexecut: rc = 1, time = 4.682s
    2012-09-23 10:43:34.442: [    RACG][1] [6815926][1][ora.p770db1.vip]: end for resource = ora.p770db1.vip, action = check, status = 1
    , time = 5.068s
    2012-10-12 15:34:14.222: [    RACG][1] [18809030][1][ora.p770db1.vip]:
    entstat: 0909-004 Unable to get statistics on device en0, errno = 69
    Invalid parameters, or failed to bring up VIP (host=p770db1)
    2012-10-12 15:34:14.222: [    RACG][1] [18809030][1][ora.p770db1.vip]: clsrcexecut: env ORACLE_CONFIG_HOME=/oracle/product/10.2.0/db
    _1
    2012-10-12 15:34:14.222: [    RACG][1] [18809030][1][ora.p770db1.vip]: clsrcexecut: cmd = /oracle/product/10.2.0/db_1/bin/racgeut -e
    USRORA_DEBUG=0 54 /oracle/product/10.2.0/db_1/bin/racgvip check p770db1
    2012-10-12 15:34:14.222: [    RACG][1] [18809030][1][ora.p770db1.vip]: clsrcexecut: rc = 1, time = 0.564s
    2012-10-12 15:34:14.222: [    RACG][1] [18809030][1][ora.p770db1.vip]: end for resource = ora.p770db1.vip, action = check, status =
    1, time = 0.712s
    2012-10-21 10:29:19.043: [    RACG][1] [18546790][1][ora.p770db1.vip]:
    entstat: 0909-004 Unable to get statistics on device en0, errno = 69
    Invalid parameters, or failed to bring up VIP (host=p770db1)
    2012-10-21 10:29:19.043: [    RACG][1] [18546790][1][ora.p770db1.vip]: clsrcexecut: env ORACLE_CONFIG_HOME=/oracle/product/10.2.0/db
    _1
    2012-10-21 10:29:19.043: [    RACG][1] [18546790][1][ora.p770db1.vip]: clsrcexecut: cmd = /oracle/product/10.2.0/db_1/bin/racgeut -e
    USRORA_DEBUG=0 54 /oracle/product/10.2.0/db_1/bin/racgvip check p770db1
    2012-10-21 10:29:19.043: [    RACG][1] [18546790][1][ora.p770db1.vip]: clsrcexecut: rc = 1, time = 0.541s
    2012-10-21 10:29:19.044: [    RACG][1] [18546790][1][ora.p770db1.vip]: end for resource = ora.p770db1.vip, action = check, status =
    1, time = 0.716s
    2012-10-23 04:23:21.410: [    RACG][1] [18415718][1][ora.p770db1.vip]:
    entstat: 0909-004 Unable to get statistics on device en0, errno = 69
    Invalid parameters, or failed to bring up VIP (host=p770db1)
    2012-10-23 04:23:21.410: [    RACG][1] [18415718][1][ora.p770db1.vip]: clsrcexecut: env ORACLE_CONFIG_HOME=/oracle/product/10.2.0/db
    _1
    2012-10-23 04:23:21.410: [    RACG][1] [18415718][1][ora.p770db1.vip]: clsrcexecut: cmd = /oracle/product/10.2.0/db_1/bin/racgeut -e
    USRORA_DEBUG=0 54 /oracle/product/10.2.0/db_1/bin/racgvip check p770db1
    2012-10-23 04:23:21.410: [    RACG][1] [18415718][1][ora.p770db1.vip]: clsrcexecut: rc = 1, time = 0.595s
    2012-10-23 04:23:21.410: [    RACG][1] [18415718][1][ora.p770db1.vip]: end for resource = ora.p770db1.vip, action = check, status =
    1, time = 0.751s
    ===================================================================================================
    2012-09-26 07:45:11.000: [  CRSEVT][11666]32CAAMonitorHandler :: 0:Could not join /oracle/product/10.2.0/db_1/bin/racgwrap(check)
    category: 1234, operation: scls_process_join, loc: childcrash, OS error: 0, other: Abnormal termination of the child
    2012-09-26 07:45:11.000: [  CRSEVT][11666]32CAAMonitorHandler :: 0:Action Script /oracle/product/10.2.0/db_1/bin/racgwrap(check) tim
    ed out for ora.p770db2.vip! (timeout=60)
    2012-09-26 07:45:11.000: [  CRSAPP][11666]32CheckResource error for ora.p770db2.vip error code = -2
    2012-09-26 07:46:44.036: [  CRSEVT][11668]32CAAMonitorHandler :: 0:Could not join /oracle/product/10.2.0/db_1/bin/racgwrap(check)
    category: 1234, operation: scls_process_join, loc: childcrash, OS error: 0, other: Abnormal termination of the child
    2012-09-26 07:46:44.036: [  CRSEVT][11668]32CAAMonitorHandler :: 0:Action Script /oracle/product/10.2.0/db_1/bin/racgwrap(check) tim
    ed out for ora.p770db2.vip! (timeout=60)
    2012-09-26 07:46:44.036: [  CRSAPP][11668]32CheckResource error for ora.p770db2.vip error code = -2
    2012-09-26 07:48:17.067: [  CRSEVT][11670]32CAAMonitorHandler :: 0:Could not join /oracle/product/10.2.0/db_1/bin/racgwrap(check)
    category: 1234, operation: scls_process_join, loc: childcrash, OS error: 0, other: Abnormal termination of the child
    2012-09-26 07:48:17.067: [  CRSEVT][11670]32CAAMonitorHandler :: 0:Action Script /oracle/product/10.2.0/db_1/bin/racgwrap(check) tim
    ed out for ora.p770db2.vip! (timeout=60)
    2012-09-26 07:48:17.067: [  CRSAPP][11670]32CheckResource error for ora.p770db2.vip error code = -2
    2012-09-26 21:27:23.866: [  CRSEVT][11581]32CAAMonitorHandler :: 0:Could not join /oracle/product/10.2.0/db_1/bin/racgwrap(check)
    category: 1234, operation: scls_process_join, loc: childcrash, OS error: 0, other: Abnormal termination of the child
    2012-09-26 21:27:23.866: [  CRSEVT][11581]32CAAMonitorHandler :: 0:Action Script /oracle/product/10.2.0/db_1/bin/racgwrap(check) tim
    ed out for ora.p770db2.vip! (timeout=60)
    2012-09-26 21:27:23.866: [  CRSAPP][11581]32CheckResource error for ora.p770db2.vip error code = -2
    2012-10-03 23:26:47.076: [  CRSEVT][11737]32CAAMonitorHandler :: 0:Could not join /oracle/product/10.2.0/db_1/bin/racgwrap(check)
    category: 1234, operation: scls_process_join, loc: childcrash, OS error: 0, other: Abnormal termination of the child
    2012-10-03 23:26:47.076: [  CRSEVT][11737]32CAAMonitorHandler :: 0:Action Script /oracle/product/10.2.0/db_1/bin/racgwrap(check) tim
    ed out for ora.p770db2.vip! (timeout=60)
    2012-10-03 23:26:47.076: [  CRSAPP][11737]32CheckResource error for ora.p770db2.vip error code = -2
    2012-10-03 23:28:20.116: [  CRSEVT][11739]32CAAMonitorHandler :: 0:Could not join /oracle/product/10.2.0/db_1/bin/racgwrap(check)
    category: 1234, operation: scls_process_join, loc: childcrash, OS error: 0, other: Abnormal termination of the child
    2012-10-03 23:28:20.116: [  CRSEVT][11739]32CAAMonitorHandler :: 0:Action Script /oracle/product/10.2.0/db_1/bin/racgwrap(check) tim
    ed out for ora.p770db2.vip! (timeout=60)
    2012-10-03 23:28:20.116: [  CRSAPP][11739]32CheckResource error for ora.p770db2.vip error code = -2
    2012-10-03 23:29:53.146: [  CRSEVT][11743]32CAAMonitorHandler :: 0:Could not join /oracle/product/10.2.0/db_1/bin/racgwrap(check)
    category: 1234, operation: scls_process_join, loc: childcrash, OS error: 0, other: Abnormal termination of the child
    2012-10-03 23:29:53.146: [  CRSEVT][11743]32CAAMonitorHandler :: 0:Action Script /oracle/product/10.2.0/db_1/bin/racgwrap(check) tim
    ed out for ora.p770db2.vip! (timeout=60)
    2012-10-03 23:29:53.146: [  CRSAPP][11743]32CheckResource error for ora.p770db2.vip error code = -2
    2012-10-03 23:31:26.182: [  CRSEVT][11745]32CAAMonitorHandler :: 0:Could not join /oracle/product/10.2.0/db_1/bin/racgwrap(check)
    category: 1234, operation: scls_process_join, loc: childcrash, OS error: 0, other: Abnormal termination of the child
    2012-10-03 23:31:26.182: [  CRSEVT][11745]32CAAMonitorHandler :: 0:Action Script /oracle/product/10.2.0/db_1/bin/racgwrap(check) tim
    ed out for ora.p770db2.vip! (timeout=60)
    2012-10-03 23:31:26.182: [  CRSAPP][11745]32CheckResource error for ora.p770db2.vip error code = -2
    2012-10-03 23:32:59.221: [  CRSEVT][11748]32CAAMonitorHandler :: 0:Could not join /oracle/product/10.2.0/db_1/bin/racgwrap(check)
    category: 1234, operation: scls_process_join, loc: childcrash, OS error: 0, other: Abnormal termination of the child
    2012-10-03 23:32:59.222: [  CRSEVT][11748]32CAAMonitorHandler :: 0:Action Script /oracle/product/10.2.0/db_1/bin/racgwrap(check) tim
    ed out for ora.p770db2.vip! (timeout=60)
    2012-10-03 23:32:59.222: [  CRSAPP][11748]32CheckResource error for ora.p770db2.vip error code = -2
    2012-10-03 23:34:32.255: [  CRSEVT][11751]32CAAMonitorHandler :: 0:Could not join /oracle/product/10.2.0/db_1/bin/racgwrap(check)
    category: 1234, operation: scls_process_join, loc: childcrash, OS error: 0, other: Abnormal termination of the child
    2012-10-03 23:34:32.255: [  CRSEVT][11751]32CAAMonitorHandler :: 0:Action Script /oracle/product/10.2.0/db_1/bin/racgwrap(check) tim
    ed out for ora.p770db2.vip! (timeout=60)
    2012-10-03 23:34:32.255: [  CRSAPP][11751]32CheckResource error for ora.p770db2.vip error code = -2
    2012-10-06 06:07:13.137: [  CRSEVT][11639]32CAAMonitorHandler :: 0:Could not join /oracle/product/10.2.0/db_1/bin/racgwrap(check)
    category: 1234, operation: scls_process_join, loc: childcrash, OS error: 0, other: Abnormal termination of the child
    2012-10-06 06:07:13.137: [  CRSEVT][11639]32CAAMonitorHandler :: 0:Action Script /oracle/product/10.2.0/db_1/bin/racgwrap(check) tim
    ed out for ora.p770db2.vip! (timeout=60)
    2012-10-06 06:07:13.138: [  CRSAPP][11639]32CheckResource error for ora.p770db2.vip error code = -2
    2012-10-06 06:08:46.175: [  CRSEVT][11641]32CAAMonitorHandler :: 0:Could not join /oracle/product/10.2.0/db_1/bin/racgwrap(check)
    category: 1234, operation: scls_process_join, loc: childcrash, OS error: 0, other: Abnormal termination of the child
    2012-10-06 06:08:46.175: [  CRSEVT][11641]32CAAMonitorHandler :: 0:Action Script /oracle/product/10.2.0/db_1/bin/racgwrap(check) tim
    ed out for ora.p770db2.vip! (timeout=60)
    2012-10-06 06:08:46.176: [  CRSAPP][11641]32CheckResource error for ora.p770db2.vip error code = -2
    2012-10-06 06:10:19.207: [  CRSEVT][11644]32CAAMonitorHandler :: 0:Could not join /oracle/product/10.2.0/db_1/bin/racgwrap(check)
    category: 1234, operation: scls_process_join, loc: childcrash, OS error: 0, other: Abnormal termination of the child
    2012-10-06 06:10:19.207: [  CRSEVT][11644]32CAAMonitorHandler :: 0:Action Script /oracle/product/10.2.0/db_1/bin/racgwrap(check) tim
    ed out for ora.p770db2.vip! (timeout=60)
    2012-10-06 06:10:19.207: [  CRSAPP][11644]32CheckResource error for ora.p770db2.vip error code = -2
    2012-10-06 06:11:52.236: [  CRSEVT][11648]32CAAMonitorHandler :: 0:Could not join /oracle/product/10.2.0/db_1/bin/racgwrap(check)
    category: 1234, operation: scls_process_join, loc: childcrash, OS error: 0, other: Abnormal termination of the child
    2012-10-06 06:11:52.236: [  CRSEVT][11648]32CAAMonitorHandler :: 0:Action Script /oracle/product/10.2.0/db_1/bin/racgwrap(check) tim
    ed out for ora.p770db2.vip! (timeout=60)
    2012-10-06 06:11:52.236: [  CRSAPP][11648]32CheckResource error for ora.p770db2.vip error code = -2
    2012-10-06 06:13:25.268: [  CRSEVT][11652]32CAAMonitorHandler :: 0:Could not join /oracle/product/10.2.0/db_1/bin/racgwrap(check)
    category: 1234, operation: scls_process_join, loc: childcrash, OS error: 0, other: Abnormal termination of the child
    2012-10-06 06:13:25.269: [  CRSEVT][11652]32CAAMonitorHandler :: 0:Action Script /oracle/product/10.2.0/db_1/bin/racgwrap(check) tim
    ed out for ora.p770db2.vip! (timeout=60)
    2012-10-06 06:13:25.269: [  CRSAPP][11652]32CheckResource error for ora.p770db2.vip error code = -2
    2012-10-06 06:14:58.307: [  CRSEVT][11654]32CAAMonitorHandler :: 0:Could not join /oracle/product/10.2.0/db_1/bin/racgwrap(check)
    category: 1234, operation: scls_process_join, loc: childcrash, OS error: 0, other: Abnormal termination of the child
    2012-10-06 06:14:58.307: [  CRSEVT][11654]32CAAMonitorHandler :: 0:Action Script /oracle/product/10.2.0/db_1/bin/racgwrap(check) tim
    ed out for ora.p770db2.vip! (timeout=60)
    2012-10-06 06:14:58.307: [  CRSAPP][11654]32CheckResource error for ora.p770db2.vip error code = -2
    2012-10-11 06:55:01.756: [  CRSAPP][11628]32CheckResource error for ora.p770db2.vip error code = 1
    2012-10-11 06:55:01.761: [  CRSRES][11628]32In stateChanged, ora.p770db2.vip target is ONLINE
    2012-10-11 06:55:01.761: [  CRSRES][11628]32ora.p770db2.vip on p770db2 went OFFLINE unexpectedly
    2012-10-11 06:55:01.761: [  CRSRES][11628]32StopResource: setting CLI values
    2012-10-11 06:55:01.765: [  CRSRES][11628]32Attempting to stop `ora.p770db2.vip` on member `p770db2`
    2012-10-11 06:55:02.390: [  CRSRES][11889]32In stateChanged, ora.zjhorcl.orcl.zjhorcl2.srv target is ONLINE
    2012-10-11 06:55:02.390: [  CRSRES][11889]32ora.zjhorcl.orcl.zjhorcl2.srv on p770db2 went OFFLINE unexpectedly
    2012-10-11 06:55:02.391: [  CRSRES][11889]32StopResource: setting CLI values
    2012-10-11 06:55:02.416: [  CRSRES][11889]32Attempting to stop `ora.zjhorcl.orcl.zjhorcl2.srv` on member `p770db2`
    2012-10-11 06:55:02.532: [  CRSRES][11628]32Stop of `ora.p770db2.vip` on member `p770db2` succeeded.
    2012-10-11 06:55:02.543: [  CRSRES][11628]32ora.p770db2.vip RESTART_COUNT=0 RESTART_ATTEMPTS=0
    2012-10-11 06:55:02.605: [  CRSRES][11628]32ora.p770db2.vip failed on p770db2 relocating.
    2012-10-11 06:55:02.680: [  CRSRES][11628]32StopResource: setting CLI values
    2012-10-11 06:55:02.684: [  CRSRES][11628]32Attempting to stop `ora.p770db2.LISTENER_P770DB2.lsnr` on member `p770db2`
    2012-10-11 06:55:03.613: [  CRSRES][11889]32Stop of `ora.zjhorcl.orcl.zjhorcl2.srv` on member `p770db2` succeeded.
    2012-10-11 06:55:03.614: [  CRSRES][11889]32ora.zjhorcl.orcl.zjhorcl2.srv RESTART_COUNT=0 RESTART_ATTEMPTS=0
    2012-10-11 06:55:03.658: [  CRSRES][11889]32ora.zjhorcl.orcl.zjhorcl2.srv failed on p770db2 relocating.
    2012-10-11 06:55:03.684: [  CRSRES][11889]32Cannot relocate ora.zjhorcl.orcl.zjhorcl2.srvStopping dependents
    2012-10-11 06:55:03.698: [  CRSRES][11889]32StopResource: setting CLI values
    2012-10-11 06:56:19.120: [  CRSRES][11628]32Stop of `ora.p770db2.LISTENER_P770DB2.lsnr` on member `p770db2` succeeded.
    2012-10-11 06:56:19.160: [  CRSRES][11628]32Attempting to start `ora.p770db2.vip` on member `p770db1`
    2012-10-11 06:56:20.894: [  CRSRES][11628]32Start of `ora.p770db2.vip` on member `p770db1` succeeded.
    2012-10-12 13:47:25.650: [  CRSRES][11649]32startRunnable: setting CLI values
    2012-10-12 13:47:26.054: [  CRSRES][11907]32StopResource: setting CLI values
    2012-10-12 13:47:26.068: [  CRSRES][11907]32Attempting to stop `ora.p770db2.vip` on member `p770db1`
    2012-10-12 13:47:26.508: [  CRSRES][11907]32Stop of `ora.p770db2.vip` on member `p770db1` succeeded.
    2012

    vip日志中显示:
    2012-08-23 15:07:17.902: [ RACG][1] [15466872][1][ora.p770db1.vip]:
    entstat: 0909-004 Unable to get statistics on device en0, errno = 69
    Invalid parameters, or failed to bring up VIP (host=p770db1)
    这网卡是否有问题,还有防火墙是否做过切换端口等操作?
    是aix的系统吗?http://www-01.ibm.com/support/docview.wss?uid=isg1IV22840 找到个描述你看看符合不
    Edited by: 小电工II on 2012-10-24 下午11:54

  • Prevent VIP's from other nodes to start on 11gr2 RAC ??

    I have one node which is part of 3 node 11gr2 RAC.
    Other 2 nodes(rac1, rac2) a power down, and on this node(rac3), crs is down.
    Problem is, that, if I switch crs up, he will up all VIP addresses , from
    rac3-vip, but, also rac2-vip, rac1-vip..but, this (rac1-vip, rac2-vip) is
    already VIP address's in production on second RAC.
    1)Can I somehow disable this 2 VIP's BEFORE I start crs on rac3 to prevent
    this other VIP address to start on rac3???
    2) If no, can I start crs, and quick stop down this VIP's,
    crs_stop ora.rac2.vip
    crs_stop ora.rac3.vip
    3) and maybe somehow prevent from future start before I really migrate
    everything on new RAC??
    Tnx

    sauron666 wrote:
    I already do that, because this is preprod, we prepare to switch prod 10g rac to this 11gr2 rac.
    10g rac has 2 nodes. (prod)
    11gr2 rac has 3 nodes. (2 nodes have same VIP as prod).Huh? That's not what I would call a good idea. Using the same IP addresses? What for? Why not re-use the hostnames instead?
    Today the VIP hostnames resolve to the 10g RAC's VIPs. Tomorrow when the switchover to 11g happens, the DNS is updated and these VIP hostnames resolve to the 11g RAC's VIPs.
    So, I want on this third node from 11gr2 RAC to stop those VIP's, which is the same as prod VIP's..so I can have third node up, and first 2 node from 11gr2 rac down (which have same VIP's as 10g rac)Just the idea of purposefully creating duplicate IPs on a network.. Crazy...
    You can try and hide those 11g RAC duplicate IPs from the network. Routing table is perhaps an option. Using iptables is another Or as you want, preventing the VIPs from starting in the first place.
    But you should take this as a lesson in how NOT to configure networking. Duplicating IPs is simply a no-no. There is no reason I've ever seen, nor can think of, why IPs should be duplicated like this. Hostnames resolution exists for addressing such requirements.

  • Please Help Me!!! I have problem while installing 11g RAC on Windows!!!

    Guys,
    I have to configure a test environment on my pc. I need a 2-node RAC system which will run on Windows.
    Problem is this:
    I'm creating two networks on my nodes, public and private but when i come to grid installation, Oracle can not identify my networks. I cant see my public and private while installation.
    I entered my scan ips to hosts file but i am having an error about scan ip also. My host file looks like this:
    127.0.0.1 localhost.localdomain localhost
    # Public
    192.168.2.181 win7rac1.localdomain win7rac1
    192.168.2.181 win7rac2.localdomain win7rac2
    #Private
    192.168.0.191 win7rac1-priv.localdomain win7rac1-priv
    192.168.0.192 win7rac2-priv.localdomain win7rac2-priv
    #Virtual
    192.168.2.183 win7rac1-vip.localdomain win7rac1-vip
    192.168.2.184 win7rac2-vip.localdomain win7rac2-vip
    #SCAN (192.168.0.155 - 192.168.0.157 in DNS)
    192.168.2.155 win7rac-scan.localdomain win7rac-scan
    192.168.2.156 win7rac-scan.localdomain win7rac-scan
    192.168.2.157 win7rac-scan.localdomain win7rac-scan
    INS-40725---- Unable to determine the existence of an interface with a subnet matching the SCAN VIP subnet.
    Really I'm stucked. I need to do this soon. I searched internet but i couldnt find anything.

    Pl do not post duplicates - INS-40725----->Unable to determine the existence of an interface with a su

  • RAC 환경에서 IP 변경에 관련해서 ~

    날씨가 많이 따뜻해 졌습니다 ~ 감기 걸리지 않도록 주의 하시구요 ~
    궁금한 점이 있어 이렇게 글을 올립니다.
    고수님들의 답변을 기다립니다 ~ ^^;
    현재 OS 는 windows2003 에 오라클은 10.2.0.1 RAC 로 구성이 되어 있습니다.
    그런데 아이피 변경을 하는데 있어 이러한 작업을 해보신 분이나 알고 계시분은
    작업 절차를 알려 주셨으면 감사하겠습니다 ^^
    글 수정:
    키노미르

    드디어 IP변경을 성공하였습니다. 3시간 허접질끝에 성공했습니다. 이렇게도 해보고
    저렇게도 해보고 건드리기를 좋아해서 오래 걸렸네요.
    기존에는 SATA2 디스크로 10G RAC를 구성해 놨는데 anygate 공유기 가 말썽을 일으켜서 어쩔 수가 없었지요.
    공유기 gateway IP를 default값으로 변경해야만 무선랜이 정상적으로 되는 버그가 있었네요. 그래서 10G
    rac 구성해 놓은 것도 IP변경을 해야 했습니다.
    그런데 10G RAC를 기존에는 ocfs를 통해서 OCR, voting disk를 구성했는데.. 다시 한번 다른 방식으로 FATA
    디스크로 구성하려고 하니 잘 안되더군요. IP를 변경해야 하는 문제를 해결하다가 실패할 경우를 대비해서
    같은 방법으로 또다시 테스트할 환경을 만들고 싶어졌답니다.
    9i RAC 에서는 Cluster Manager Quorum File과 Shared Configuration File 을 pvcreate, lvcreate를 통해서 logical
    volume을 만들고 그 여러볼륨을 raw device바인딩하고 링크를 걸어서 사용하였습니다.
    9i rac는 logical volume을 통해서 Querom File과 Shared configuration file이 가능하더군요.
    이와 같은 방법으로 FATA 디스크에 10g rac는 CRS 설치시 OCR, voting disk를 logical volume을 통해서 raw device로 바인딩한
    후 링크를 걸었습니다. 권한문제라던지 그런 것도 잘 해주었는데.. root.sh를 하고 나서 crs설치를 실패하더군요.
    Failed to upgrade Oracle Cluster Registry configuration 메세지가 나네요. RH ES3, RH ES4 에서 테스트를 했는데 안되었답니다.
    다시 CentOS에서 해보려고 합니다. 제발 이것은 성공하길...
    이처럼 OCFS를 통해 OCR, voting disk 구성한후 CRS를 설치할 때만 성공하고, logical volume으로 만들면 9i RAC cluster
    manager 설치가 잘 되던 것이 10g에서는 CRS설치시 안되네요.
    하두 답답해서 LG CNS엔지니어한테 물어보니 CRS 설치가 잘 안되는 경우가 있다고 많다고 하더군요. CentOS에서 다시 한번 더
    logical volume으로 OCR voting disk로 만들어보고 안되면 그냥 OCFS로 다시 한번 더 구성해보려고 합니다.
    [root@rac2 root]# /u01/app/oracle/product/crs/root.sh
    WARNING: directory '/u01/app/oracle/product' is not owned by root
    WARNING: directory '/u01/app/oracle' is not owned by root
    WARNING: directory '/u01/app' is not owned by root
    WARNING: directory '/u01' is not owned by root
    Checking to see if Oracle CRS stack is already configured
    Setting the permissions on OCR backup directory
    Setting up NS directories
    Failed to upgrade Oracle Cluster Registry configuration
    ====================================================
    두서가 길어졌죠..~ SATA2 디스크로 기존에 구성했던 것의 IP를 변경해야 할 필요성이 있었고 아래와 같이 구성하였습니다. metalink 내용대로 하였
    습니다.
    # 시스템 구성사항
    OS : CentOS 4.2 32bit
    DB : 10.2.0.1 32bit 2node rac
    shared disk device: 누디앙 CD-503 (ieee1394 2port)
    shared disk : 삼성40G 7200rpm
    server(2노두 모두 동일) : p4 2.8Mhz, 2G RAM , 80G SATA2 disk(7200rpm)
    RAC 구성사항 : OCFS를 통한 OCR, Voting Disk 구성, ASM을 통한 DB구성
    node name (서버명) : rac1, rac2
    instance name: orcl1 , orcl2
    DB name: orcl
    IP구성 :
    기존 : public (192.168.1.101), interconnect (192.168.2.101), VIP (192.168.1.201)
    변경후 : public (192.168.10.101), interconnect (192.168.20.201), VIP (192.168.10.201)
    이 글은 아래 metalink 자료를 통해서 테스트를 하였습니다.
    정확하다고 말씀드릴 수는 없으므로 참고만 하시기 바랍니다.
    metalink note 283684.1 How to Change Interconnect/Public Interface IP Subnet in a 10g Cluster
    metalink note 276434.1 Modifying the VIP of a Cluster Node
    1. vip를 먼저 바꾸었습니다.
    rac1 node vip : 192.168.1.201 (vip-rac1)
    rac2 node vip : 192.168.1.202 (vip-rac2)
    (1) 시스템 정보를 export해서 봅니다. 아래에서와 같이 vip의 IP와 subnet을 변경할
    필요가 있습니다.
         [root@rac1 bin]# srvconfig -exp kkk.txt
         srvconfig successfully exported cluster database configurations to file "kkk.txt"
         [root@rac1 bin]# more kkk.txt
         # PLEASE DO NOT EDIT THIS FILE - it is "srvconfig" generated
         DATABASES = (orcl)
         ########## Configuration of cluster database "orcl" follows ##########
         orcl.ORACLE_HOME = /u01/app/oracle/product/10.2.0/db_1
         orcl.SPFILE = +FLASH_RECOVERY_AREA/orcl/spfileorcl.ora
         orcl.ENABLED = true
         orcl.INSTANCE = (orcl1,orcl2)
         orcl.orcl1.NODE = rac1
         orcl.orcl1.ENABLED = true
         orcl.orcl2.NODE = rac2
         orcl.orcl2.ENABLED = true
         orcl.SERVICE = (orcl)
         orcl.orcl.INSTANCES = (orcl2:PREFERRED:true),(orcl1:PREFERRED:true)
         orcl.orcl.ENABLED = true
         orcl.orcl.TAFPOLICY = basic
         ########## Configuration of nodeapps follows ##########
         NODEAPPS = (rac1,rac2)
         rac1.VIP = 192.168.1.201:255.255.255.0:(eth0,eth1)
         rac1.ORACLE_HOME = /u01/app/oracle/product/crs
         rac2.VIP = 192.168.1.202:255.255.255.0:(eth0,eth1)
         rac2.ORACLE_HOME = /u01/app/oracle/product/crs
         ########## Configuration of vip_range follows ##########
         VIP_RANGE = null
         [root@rac1 oracle]# ocrconfig -export kkk1.txt
         [root@rac1 oracle]# more kkk1.txt
         呈
         SYSTEM
         DATABASE
         CRS
         SYSTEM.css
         SYSTEM.language
         SYSTEM.version
         SYSTEM.versionstring
         SYSTEM.ORA_CRS_HOME
         SYSTEM.local_only
    (2) rac1,rac2 모든 node에서 DB를 내렸습니다.
         [oracle@rac1 ~]$ srvctl stop database -d orcl
    (3) 모든 노드의 asm instance를 내렸습니다.
         [oracle@rac1 ~]$ srvctl stop asm -n rac1
         [oracle@rac1 ~]$ srvctl stop asm -n rac2
    (4) 모든 노드의 nodeapp를 내립니다.
    nodeapp는 VIP, GSD, Listener, and ONS daemon를 말합니다.
         [oracle@rac1 ~]$ srvctl stop nodeapps -n rac1
         [oracle@rac1 ~]$ srvctl stop nodeapps -n rac2
    (5) 양쪽 노드에서 정상적으로 모든 리소스를 내렸는지 crs_stat로 확인합니다.
         [oracle@rac1 bin]$ cd $ORA_CRS_HOME/bin
         [oracle@rac1 bin]$ ./crs_stat
         NAME=ora.ORCL.ORCL.cs
         TYPE=application
         TARGET=OFFLINE
         STATE=OFFLINE
    (6) hosts 파일을 양쪽 노드 모두 변경합니다.
         - 변경전
         127.0.0.1 localhost.localdomain localhost
         # Public Network - (eth0)
         192.168.1.101 rac1
         192.168.1.102 rac2
         # Private Interconnect - (eth1)
         192.168.2.101 int-rac1
         192.168.2.102 int-rac2
         # Public Virtual IP (VIP) addresses for - (eth0)
         192.168.1.201 vip-rac1
         192.168.1.202 vip-rac2
         - 변경후
         127.0.0.1 localhost.localdomain localhost
         # Public Network - (eth0)
         192.168.10.101 rac1
         192.168.10.102 rac2
         # Private Interconnect - (eth1)
         192.168.20.201 int-rac1
         192.168.20.202 int-rac2
         # Public Virtual IP (VIP) addresses for - (eth0)
         192.168.10.201 vip-rac1
         192.168.10.202 vip-rac2
    (7) rac1과 rac2 node의 네트워크를 변경합니다.
    - OS의 네트워크IP를 변경합니다.
    (8) ifconfig 로 현재 상황을 확인해봅니다.
         - rac1 node
         [root@rac1 ~]# ifconfig -a
         eth0 Link encap:Ethernet HWaddr 00:19:21:5D:30:B5
         inet addr:192.168.10.101 Bcast:192.168.10.255 Mask:255.255.255.0
         inet6 addr: fe80::219:21ff:fe5d:30b5/64 Scope:Link
         UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
         RX packets:100273 errors:0 dropped:0 overruns:0 frame:0
         TX packets:68329 errors:0 dropped:0 overruns:0 carrier:0
         collisions:0 txqueuelen:1000
         RX bytes:78294892 (74.6 MiB) TX bytes:35353321 (33.7 MiB)
         Interrupt:10 Base address:0xcc00
         eth1 Link encap:Ethernet HWaddr 00:10:5A:5E:BD:7C
         inet addr:192.168.20.201 Bcast:192.168.20.255 Mask:255.255.255.0
         inet6 addr: fe80::210:5aff:fe5e:bd7c/64 Scope:Link
         UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
         RX packets:44306 errors:0 dropped:0 overruns:0 frame:0
         TX packets:48826 errors:0 dropped:0 overruns:0 carrier:0
         collisions:0 txqueuelen:1000
         RX bytes:23070633 (22.0 MiB) TX bytes:38914552 (37.1 MiB)
         Interrupt:11 Base address:0xe880
         lo Link encap:Local Loopback
         inet addr:127.0.0.1 Mask:255.0.0.0
         inet6 addr: ::1/128 Scope:Host
         UP LOOPBACK RUNNING MTU:16436 Metric:1
         RX packets:35150 errors:0 dropped:0 overruns:0 frame:0
         TX packets:35150 errors:0 dropped:0 overruns:0 carrier:0
         collisions:0 txqueuelen:0
         RX bytes:9869103 (9.4 MiB) TX bytes:9869103 (9.4 MiB)
         sit0 Link encap:IPv6-in-IPv4
         NOARP MTU:1480 Metric:1
         RX packets:0 errors:0 dropped:0 overruns:0 frame:0
         TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
         collisions:0 txqueuelen:0
         RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)
         - rac2 node
         [root@rac2 ~]# ifconfig -a
         eth0 Link encap:Ethernet HWaddr 00:19:21:5D:2B:E8
         inet addr:192.168.10.102 Bcast:192.168.10.255 Mask:255.255.255.0
         inet6 addr: fe80::219:21ff:fe5d:2be8/64 Scope:Link
         UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
         RX packets:67515 errors:0 dropped:0 overruns:0 frame:0
         TX packets:97912 errors:0 dropped:0 overruns:0 carrier:0
         collisions:0 txqueuelen:1000
         RX bytes:35277845 (33.6 MiB) TX bytes:77863535 (74.2 MiB)
         Interrupt:10 Base address:0xcc00
         eth1 Link encap:Ethernet HWaddr 00:50:04:BD:E6:F9
         inet addr:192.168.20.202 Bcast:192.168.20.255 Mask:255.255.255.0
         inet6 addr: fe80::250:4ff:febd:e6f9/64 Scope:Link
         UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
         RX packets:48512 errors:0 dropped:0 overruns:0 frame:0
         TX packets:43953 errors:0 dropped:0 overruns:0 carrier:0
         collisions:0 txqueuelen:1000
         RX bytes:38782288 (36.9 MiB) TX bytes:22830335 (21.7 MiB)
         Interrupt:11 Base address:0xe880
         lo Link encap:Local Loopback
         inet addr:127.0.0.1 Mask:255.0.0.0
         inet6 addr: ::1/128 Scope:Host
         UP LOOPBACK RUNNING MTU:16436 Metric:1
         RX packets:81708 errors:0 dropped:0 overruns:0 frame:0
         TX packets:81708 errors:0 dropped:0 overruns:0 carrier:0
         collisions:0 txqueuelen:0
         RX bytes:12717471 (12.1 MiB) TX bytes:12717471 (12.1 MiB)
         sit0 Link encap:IPv6-in-IPv4
         NOARP MTU:1480 Metric:1
         RX packets:0 errors:0 dropped:0 overruns:0 frame:0
         TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
         collisions:0 txqueuelen:0
         RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)
    (9) vip를 변경합니다.(root유저에서 수행)
    - 저의 경우에는 실수로 /etc/host 파일과 eth1에 설정한 interconnect IP를 다르게 쓰는
    실수를 하였습니다. 그랬더니 아래 vip를 변경하는 명령을 하면서 멈추었고, 차후에 찾아내어
    /etc/hosts를 변경하는 순간 서버가 리부팅되는 상황을 경험하였습니다. 아래 명령을 하기 전에
    꼭 ping rac1, ping int-rac1 을 수행해서 정확하게 변경을 하였는지 꼭 확인해야 합니다.
         [oracle@rac1 ~]$ cd $ORA_CRS_HOME/bin
         [oracle@rac1 bin]$ su
         Password:
         [root@rac1 bin]# srvctl modify nodeapps -n rac1 -A 192.168.10.201/255.255.255.0/eth0
         [root@rac1 bin]# srvctl modify nodeapps -n rac2 -A 192.168.10.202/255.255.255.0/eth0
    2. 두번째로 interconnect/public IP subnet을 변경합니다.
    우리는 위에서 모든 리소스를 내린 후 /etc/hosts파일을 미리 변경하고 OS의 네트워크도 이미
    변경한 상태입니다.
    (1) oifcfg iflist를 보면 희안하게도 자동으로 변경되어 있었습니다.
         [oracle@rac1 ~]$ cd $ORA_CRS_HOME/bin
         [oracle@rac1 bin]$ su
         Password:
         [root@rac1 bin]# oifcfg iflist
         eth0 192.168.10.0
         eth1 192.168.20.0
         - 그런데 rac2 node에서 확인하였을 때에는 에러가 발생하였습니다.
         [root@rac2 bin]# oifcfg iflist
         PRIF-12: failed to initialize cluster support services
    (2) oifcfg getif를 했을 때에는 문서와 다른 결과가 나왔습니다.
         [root@rac1 bin]# oifcfg getif
         아무런 메세지가 없더군요.
         [root@rac2 bin]# oifcfg getif
         PRIF-12: failed to initialize cluster support services
         node2는 위처럼 에러가 났습니다.
    (3) public interface IP를 변경합니다.
    변경전에 oifcfg delif 를 통해서 interface를 삭제합니다.
    저의 경우에는 문서와는 다르게 삭제가 되지 않았습니다.
    하지만 차후에 추가후에는 삭제가 되더군요. 그리고 rac1 node에서는
    명령어가 가능하였으나 rac2 node에서는 불가능하였습니다.
         기존 public interface IP subnet : 192.168.1.0
         변경할 public interface IP subnet : 192.168.10.0
         기존 interconnect interface IP subnet : 192.168.2.0
         변경할 public interface IP subnet : 192.168.20.0
         [root@rac1 bin]# oifcfg delif -global eth1
         PROC-4: The cluster registry key to be operated on does not exist.
         PRIF-11: cluster registry error
         [root@rac1 bin]# oifcfg
         Name:
         oifcfg - Oracle Interface Configuration Tool.
         Usage: oifcfg iflist [-p [-n]]
         oifcfg setif {-node <nodename> | -global} {<if_name>/<subnet>:<if_type>}...
         oifcfg getif [-node <nodename> | -global] [ -if <if_name>[/<subnet>] [-type <if_type>] ]
         oifcfg delif [-node <nodename> | -global] [<if_name>[<subnet>]]
         oifcfg [-help]
         <nodename> - name of the host, as known to a communications network
         <if_name> - name by which the interface is configured in the system
         <subnet> - subnet address of the interface
         <if_type> - type of the interface { cluster_interconnect | public | storage }
         [root@rac1 bin]# oifcfg setif -global eth0/192.168.10.0:public
         [root@rac1 bin]# oifcfg delif -global eth0
         [root@rac1 bin]#
         위처럼 setif를 해야만 delif가 되었습니다.
         다시 설정을 하였습니다.
         [root@rac1 bin]# oifcfg setif -global eth0/192.168.10.0:public
         [root@rac1 bin]# oifcfg getif
         eth0 192.168.10.0 global public
         - 이제 rac2 node에서 명령어를 수행하였으나 실패하였습니다.
         [oracle@rac2 admin]$ oifcfg setif -global eth0/192.168.10.0:public
         PRIF-12: failed to initialize cluster support services
    (4) internal interface IP를 변경합니다.
         [root@rac1 bin]# oifcfg setif -global eth1/192.168.20.0:cluster_interconnect
         [root@rac1 bin]# oifcfg getif
         eth0 192.168.10.0 global public
         eth1 192.168.20.0 global cluster_interconnect
    3. listener.ora 와 tnsnames.ora 파일을 확인/변경합니다.
    (1) listener.ora 파일을 변경합니다.
         - 변경전
         LISTENER_RAC1 =
         (DESCRIPTION_LIST =
         (DESCRIPTION =
         (ADDRESS = (PROTOCOL = TCP)(HOST = vip-rac1)(PORT = 1521)(IP = FIRST))
         (ADDRESS = (PROTOCOL = TCP)(HOST = 192.168.1.101)(PORT = 1521)(IP = FIRST))
         SID_LIST_LISTENER_RAC1 =
         (SID_LIST =
         (SID_DESC =
         (SID_NAME = PLSExtProc)
         (ORACLE_HOME = /u01/app/oracle/product/10.2.0/db_1)
         (PROGRAM = extproc)
         - 변경 후(위에서 192.168.1.101 은 rac1 이라는 hostname의 이름이었습니다. IP를 변경하였습니다.)
         LISTENER_RAC1 =
         (DESCRIPTION_LIST =
         (DESCRIPTION =
         (ADDRESS = (PROTOCOL = TCP)(HOST = vip-rac1)(PORT = 1521)(IP = FIRST))
         (ADDRESS = (PROTOCOL = TCP)(HOST = 192.168.10.102)(PORT = 1521)(IP = FIRST))
         SID_LIST_LISTENER_RAC1 =
         (SID_LIST =
         (SID_DESC =
         (SID_NAME = PLSExtProc)
         (ORACLE_HOME = /u01/app/oracle/product/10.2.0/db_1)
         (PROGRAM = extproc)
         rac2 node도 같은 방법으로 변경하였습니다.
    (2) tnsnames.ora 파일을 확인하였습니다. hostname으로 되어 있어서 변경할 필요가 없었습니다.
         [oracle@rac1 admin]$ cat tnsnames.ora
         # tnsnames.ora Network Configuration File: /u01/app/oracle/product/10.2.0/db_1/network/admin/tnsnames.ora
         # Generated by Oracle configuration tools.
         LISTENERS_ORCL =
         (ADDRESS_LIST =
         (ADDRESS = (PROTOCOL = TCP)(HOST = vip-rac1)(PORT = 1521))
         (ADDRESS = (PROTOCOL = TCP)(HOST = vip-rac2)(PORT = 1521))
         ORCL2 =
         (DESCRIPTION =
         (ADDRESS = (PROTOCOL = TCP)(HOST = vip-rac2)(PORT = 1521))
         (CONNECT_DATA =
         (SERVER = DEDICATED)
         (SERVICE_NAME = orcl)
         (INSTANCE_NAME = orcl2)
         ORCL1 =
         (DESCRIPTION =
         (ADDRESS = (PROTOCOL = TCP)(HOST = vip-rac1)(PORT = 1521))
         (CONNECT_DATA =
         (SERVER = DEDICATED)
         (SERVICE_NAME = orcl)
         (INSTANCE_NAME = orcl1)
         ORCL =
         (DESCRIPTION =
         (ADDRESS = (PROTOCOL = TCP)(HOST = vip-rac1)(PORT = 1521))
         (ADDRESS = (PROTOCOL = TCP)(HOST = vip-rac2)(PORT = 1521))
         (LOAD_BALANCE = yes)
         (CONNECT_DATA =
         (SERVER = DEDICATED)
         (SERVICE_NAME = orcl)
         (FAILOVER_MODE =
         (TYPE = SELECT)
         (METHOD = BASIC)
         (RETRIES = 180)
         (DELAY = 5)
         EXTPROC_CONNECTION_DATA =
         (DESCRIPTION =
         (ADDRESS_LIST =
         (ADDRESS = (PROTOCOL = IPC)(KEY = EXTPROC0))
         (CONNECT_DATA =
         (SID = PLSExtProc)
         (PRESENTATION = RO)
    - rac2 node도 같은 방법으로 점검합니다.
    4. OCFS2 의 디스크 mount정보를 수정하였습니다.
    저는 OCFS를 통해서 OCR과 Voting disk를 공유하였습니다.
    그래서 IP를 변경할 경우 설정을 변경해야 하였습니다.
    /etc/ocfs2/cluster.conf 를 vi로 열어서 IP를 수정하였습니다.
         [root@rac1 ~]# cat /etc/ocfs2/cluster.conf
         node:
         ip_port = 7777
         ip_address = 192.168.10.101
         number = 0
         name = rac1
         cluster = ocfs2
         node:
         ip_port = 7777
         ip_address = 192.168.10.102
         number = 1
         name = rac2
         cluster = ocfs2
         cluster:
         node_count = 2
         name = ocfs2
    5. 서버를 재기동 하여 자동으로 서비스가 올라오는지 보았습니다.
    - 리부팅 후 서비스가 정상적으로 모두 로드되었습니다.
    이제 IP변경을 완료하였습니다.
         [oracle@rac1 ~]$ cd $ORA_CRS_HOME/bin
         [oracle@rac1 bin]$ ./crs_stat
         NAME=ora.ORCL.ORCL.cs
         TYPE=application
         TARGET=OFFLINE
         STATE=OFFLINE
         NAME=ora.orcl.db
         TYPE=application
         TARGET=ONLINE
         STATE=ONLINE on rac1
         NAME=ora.orcl.orcl.cs
         TYPE=application
         TARGET=ONLINE
         STATE=ONLINE on rac1
         NAME=ora.orcl.orcl.orcl1.srv
         TYPE=application
         TARGET=ONLINE
         STATE=ONLINE on rac1
         NAME=ora.orcl.orcl.orcl2.srv
         TYPE=application
         TARGET=ONLINE
         STATE=ONLINE on rac2
         NAME=ora.orcl.orcl1.inst
         TYPE=application
         TARGET=ONLINE
         STATE=ONLINE on rac1
         NAME=ora.orcl.orcl2.inst
         TYPE=application
         TARGET=ONLINE
         STATE=ONLINE on rac2
         NAME=ora.rac1.ASM1.asm
         TYPE=application
         TARGET=ONLINE
         STATE=ONLINE on rac1
         NAME=ora.rac1.LISTENER_RAC1.lsnr
         TYPE=application
         TARGET=ONLINE
         STATE=ONLINE on rac1
         NAME=ora.rac1.gsd
         TYPE=application
         TARGET=ONLINE
         STATE=ONLINE on rac1
         NAME=ora.rac1.ons
         TYPE=application
         TARGET=ONLINE
         STATE=ONLINE on rac1
         NAME=ora.rac1.vip
         TYPE=application
         TARGET=ONLINE
         STATE=ONLINE on rac1
         NAME=ora.rac2.ASM2.asm
         TYPE=application
         TARGET=ONLINE
         STATE=ONLINE on rac2
         NAME=ora.rac2.LISTENER_RAC2.lsnr
         TYPE=application
         TARGET=ONLINE
         STATE=ONLINE on rac2
         NAME=ora.rac2.gsd
         TYPE=application
         TARGET=ONLINE
         STATE=ONLINE on rac2
         NAME=ora.rac2.ons
         TYPE=application
         TARGET=ONLINE
         STATE=ONLINE on rac2
         NAME=ora.rac2.vip
         TYPE=application
         TARGET=ONLINE
         STATE=ONLINE on rac2
    6. VIP 생성여부 확인
    [root@rac1 ~]# ifconfig -a
    eth0 Link encap:Ethernet HWaddr 00:19:21:5D:30:B5
    inet addr:192.168.10.101 Bcast:192.168.10.255 Mask:255.255.255.0
    inet6 addr: fe80::219:21ff:fe5d:30b5/64 Scope:Link
    UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
    RX packets:1076 errors:0 dropped:0 overruns:0 frame:0
    TX packets:1026 errors:0 dropped:0 overruns:0 carrier:0
    collisions:0 txqueuelen:1000
    RX bytes:99825 (97.4 KiB) TX bytes:97112 (94.8 KiB)
    Interrupt:10 Base address:0xcc00
    eth0:2 Link encap:Ethernet HWaddr 00:19:21:5D:30:B5
    inet addr:192.168.10.201 Bcast:192.168.10.255 Mask:255.255.255.0
    UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
    Interrupt:10 Base address:0xcc00
    eth1 Link encap:Ethernet HWaddr 00:10:5A:5E:BD:7C
    inet addr:192.168.20.201 Bcast:192.168.20.255 Mask:255.255.255.0
    inet6 addr: fe80::210:5aff:fe5e:bd7c/64 Scope:Link
    UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
    RX packets:66124 errors:0 dropped:0 overruns:0 frame:0
    TX packets:93483 errors:0 dropped:0 overruns:0 carrier:0
    collisions:0 txqueuelen:1000
    RX bytes:35409232 (33.7 MiB) TX bytes:85751255 (81.7 MiB)
    Interrupt:11 Base address:0xe880
    lo Link encap:Local Loopback
    inet addr:127.0.0.1 Mask:255.0.0.0
    inet6 addr: ::1/128 Scope:Host
    UP LOOPBACK RUNNING MTU:16436 Metric:1
    RX packets:78532 errors:0 dropped:0 overruns:0 frame:0
    TX packets:78532 errors:0 dropped:0 overruns:0 carrier:0
    collisions:0 txqueuelen:0
    RX bytes:8421778 (8.0 MiB) TX bytes:8421778 (8.0 MiB)
    sit0 Link encap:IPv6-in-IPv4
    NOARP MTU:1480 Metric:1
    RX packets:0 errors:0 dropped:0 overruns:0 frame:0
    TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
    collisions:0 txqueuelen:0
    RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)
    글 수정:
    Min Angel (Yeon Hong Min, Korean)
    지금보니 벌써 새벽4시..;; 한번 시작하면 밤을 새워도 해내야만
    직성이 풀리는 이 골치아픈 집착은 언제 없어질지 모르겠네요.
    연인에게 이런 집착을 보여야 결혼도 하게 되는건데 참..오라클하고만
    이러고 있으니..;; 사담이 길어져서 죄송합니다.
    글 수정:
    Min Angel (Yeon Hong Min, Korean)
    도움이 되셨길 바랍니다. ^^

  • RAC: Multiple Databases / Instances with network seperation

    Dear all,
    We are planning to run our oracle 11g DB on RAC, with two instances both on seperate networks. Is this possible?
    here is what i mean
    Instance 1 including RAC DB is configured in e.g VLAN 12 with subnet 192.168.120.0/24. Instance 2 is configured in VLAN 22 with subnet 192.168.122.0/24. Servers of instance 2 send their data also to instance 1 RAC DB at subnet 192.168.120.0/24. Traffic between them is routed by a Cisco router.
    i searched the net for similar implementations and found a couple of people running similar configurations on RAC,
    but i want to know any downsides/pros cons of running such a setup.
    THanks in advance
    Saad

    Hi Saad,
    The problem in my case is clusterware configuration. While creating the cluster CRS asks for public IP,Private IP and VIP for both nodes. I can do it for one database environment which we call here pre-prod.
    The prod environment also uses the same hardware but is in different subnet and will use same cluster. Can i put vips for that environment when clusterware asks for public ip,private ip and vips.
    Or as you suggested,create cluster using ip of one subnet and create database for it and for second network create just database and in listerner.ora enter vips of prod environment. My current hosts file looks like this
    # Do not remove the following line, or various programs
    # that require network functionality will fail.
    127.0.0.1 localhost.localdomain localhost
    ::1 localhost6.localdomain6 localhost6
    #Production
    10.88.118.200 cps-oracle-vip.fedex.com cps-oracle-vip
    10.88.118.201 cps-oracle-1.fedex.com cps-oracle-1
    10.88.118.202 cps-oracle-2.fedex.com cps-oracle-2
    10.88.118.203 cps-oracle-1-vip.fedex.com cps-oracle-1-vip
    10.88.118.204 cps-oracle-2-vip.fedex.com cps-oracle-2-vip
    #Pre-Production
    10.88.119.200 cps-pre-oracle-vip.fedex.com cps-pre-oracle-vip
    10.88.119.201 cps-pre-oracle-1.fedex.com cps-pre-oracle-1
    10.88.119.202 cps-pre-oracle-2.fedex.com cps-pre-oracle-2
    10.88.119.203 cps-pre-oracle-1-vip.fedex.com cps-pre-oracle-1-vip
    10.88.119.204 cps-pre-oracle-2-vip.fedex.com cps-pre-oracle-2-vip
    #Internal
    192.168.101.1 cps-oracle-1-priv.fedex.com cps-oracle-1-priv
    192.168.101.2 cps-oracle-2-priv.fedex.com cps-oracle-2-priv
    Does 11G provide facility to enter multiple vips in CRS.

Maybe you are looking for

  • Error in creating PCO Connector in SAP MII 14.0

    Hello, We have successfully installed SAP PCo 2.3 in one of the servers. This PCo Server is independent of the server on which MII Application is installed. So DCOM had to be done and it was achieved successfully. On PCo Server, in management console

  • ITunes/iPod song order problem

    i have an original 512 shuffle that i load with podcasts and a few songs to listen to while i commute. i drag the podcasts from the library to the ipod in itunes and them arrange them in the order that i want them played. when i disconnect the ipod a

  • HT202358 The Support page contradicts itself! How do I upgrade the later version of my iPhoto Library?

    TELL ME IF THIS DOES NOT CONTRADICT ITSELF? iPhoto '11 or later: About the Library Upgrader You might see this message in iPhoto: "To open your library with this version of iPhoto, it first needs to be prepared." If you do, you can use the iPhoto Lib

  • Movie not on my itunes that i just purchased!

    I was watching a movie I just purchased on the itunes store, "step mom", as part of my term research paper.  I was over half way through the movies, Itunes was no longer responding and I had to close the page and restart.  Everything showed up fine,

  • Evaluation error after installing my app on a new computer

    Hi, I created an application using Visual C++ 6.0 and the ComponentWorks Starter kit V3.0. The app uses the CWGraph ,CWKnob and CWBar. In the app the Graph is displayed on the main window and the other two are added dynamically when the user selects