RAC + oratab + sqlplus

This one should be simple, I'v searched online and just can't seem to find a solid answer for this.
After installing a database with dbca on a 2 node Rac cluster, (db_name = SEED, instances = SEED1, SEED2), I'm left with this in /etc/oratab
SEED:/u01/app/oracle/product/10.2.0/devdb_1:N
I have a net service entry setup with a name of SEED that is load balanced between the two, and then 2 individual service names SEED1, and SEED2 that go to the specific instances.
I modified /usr/local/bin/oratab to include this at the bottom.
TWO_TASK=$ORACLE_SID
I'm running this on RHEL 4.6 (64-bit)
My understanding was that if I set the TWO_TASK variable, then sqlplus would treat the statement 'sqlplus / as sysdba' the same as 'sqlplus sys/password@SEED as sysdba
But I get the following.
[oracle@node-name ~]$ . oraenv
ORACLE_SID = [SEED] ? SEED
[oracle@node-name ~]$ echo $TWO_TASK
SEED
[oracle@node-name ~]$ echo $LOCAL
SEED
[oracle@node-name ~]$ echo $ORACLE_SID
SEED
[oracle@node-name ~]$ echo $ORACLE_HOME
/u01/app/oracle/product/10.2.0/devdb_1
[oracle@node-name ~]$ sqlplus / as sysdba
SQL*Plus: Release 10.2.0.3.0 - Production on Wed Mar 11 08:11:39 2009
Copyright (c) 1982, 2006, Oracle. All Rights Reserved.
Connected to an idle instance.
SQL>
When running 'sqlplus sys/password@SEED as sysdba' works correctly.
If I change ORACLE_SID=SEED1, then 'sqlplus / as sysdba' will work correctly.
I know how to make this work, but I'm not sure what the best practice is here. Should add the following to /etc/oratab, for each instance
SEED1:/u01/app/oracle/product/10.2.0/devdb_1:N
SEED2:/u01/app/oracle/product/10.2.0/devdb_1:N
and then use
. oraenv
SEED1
sqlplus / as sysdba
I see some people have modified oraenv to append the instance number to the end of ORACLE_SID
I would be fine with using the net service name, if the TWO_TASK varaible was working correctly, but it does not seem to work the way the documentation says.
Am I missing something here?

Hello,
It's good idea to add entries for each RAC instance in oratab and setup in a way when you login it sets up env for one instance say rac11 which is the primary instance for the node. Now here in this case , i got 2 different RAC instances and I have setup RAC11 for node1 and RAC12 env as default. So I don't have to set env when i login but i do change env when I go RAC21 and RAC22 instances. As far as the best practices goes, i see more what work for you and how easy is to manage.
+ASM1:/u01/app/oracle/product/10.2.0/database:Y:10.2.0:
rac1:/u01/app/oracle/product/10.2.0/database:Y:10.2.0:
rac11:/u01/app/oracle/product/10.2.0/database:Y:10.2.0:
rac12:/u01/app/oracle/product/10.2.0/database:Y:10.2.0:
rac2:/u01/app/oracle/product/10.2.0/database:Y:10.2.0:
rac21:/u01/app/oracle/product/10.2.0/database:Y:10.2.0:
rac22:/u01/app/oracle/product/10.2.0/database:Y:10.2.0:Regards

Similar Messages

  • Sqlplus is hanging on RAC1 node

    Hi all,
    I have 3 Node RAC DB (11.2.0.3). When i am trying to connect sqlplus to rac 1 , there is no error but i am not getting sqlplus prompt.
    RAC 1 : sqlplus "/as sysdba" -- Hangs
    RAC 2 : sqlplus "/as sysdba" -- getting prompt
    RAC 3 : sqlplus "/as sysdba" -- getting prompt
    from RAC 2 : sqlplus sys/password@rac1 as sysdba -- Hangs
    RAC 3 : sqlplus sys/password@rac1 as sysdba -- Hangs
    RAC 2 : sqlplus "/as sysdba" -- getting prompt but when i query any thing from gv$session, then it hangs.
    there is no error in alert log and cput utilization on RAC 1 is 5-10% only (Which is normal).
    Kindly inform me how i can resolve this issue.
    Thanks
    Krishna

    Hello All,
    My operating system is Linux 64 Bit. and all the mount points have sufficient space. There is no message in DB alert log/listener log.
    Kindly advice me if i can check some else.
    Thanks
    Krishna

  • Apache, PHP, and Oracle 11G

    Hi,
    I have an 11G RAC Setup, and a separate application server that runs php that is presently connecting to MySQL Servers.
    I just configured OCI8 in my application server and can see that my configurations are correct using phpinfo().
    iam using this code to connect to my RAC 11G:
    <?php
    if(!$conn = oci_connect('sys', 'opensesame','callbox.oracle'))
    $err = oci_error();
    trigger_error('Could Not Establish a Connection:'.$err['message'], E_USER_ERROR);
    ?>
    but i get this error:
    Warning: oci_connect() [function.oci-connect]: ORA-12154: TNS:could not resolve the connect identifier specified in /var/www/html/oraconn.php on line 6
    here's tnsnames.ora in my RAC, using sqlplus via command line works perfectly fine in either nodes of the rac.
    LISTENERS_CALLBOX =
    (ADDRESS_LIST =
    (ADDRESS = (PROTOCOL = TCP)(HOST = node01-vip)(PORT = 1521))
    (ADDRESS = (PROTOCOL = TCP)(HOST = node02-vip)(PORT = 1521))
    CALLBOX2 =
    (DESCRIPTION =
    (ADDRESS = (PROTOCOL = TCP)(HOST = node02-vip)(PORT = 1521))
    (CONNECT_DATA =
    (SERVER = DEDICATED)
    (SERVICE_NAME = callbox.oracle)
    (INSTANCE_NAME = callbox2)
    CALLBOX1 =
    (DESCRIPTION =
    (ADDRESS = (PROTOCOL = TCP)(HOST = node01-vip)(PORT = 1521))
    (CONNECT_DATA =
    (SERVER = DEDICATED)
    (SERVICE_NAME = callbox.oracle)
    (INSTANCE_NAME = callbox1)
    CALLBOX =
    (DESCRIPTION =
    (ADDRESS = (PROTOCOL = TCP)(HOST = node01-vip)(PORT = 1521))
    (ADDRESS = (PROTOCOL = TCP)(HOST = node02-vip)(PORT = 1521))
    (LOAD_BALANCE = yes)
    (CONNECT_DATA =
    (SERVER = DEDICATED)
    (SERVICE_NAME = callbox.oracle)
    am I missing something?

    i forgot to include that the tutorial that helped me configure the OCI8 for PHP suggests that i need to create tnsnames.ora in /etc like this:
    SERVICENAME =
    (DESCRIPTION =
    (ADDRESS = (PROTOCOL = TCP)(HOST = ip_of_oracle_server)(PORT = 1521))
    (CONNECT_DATA =
    (SERVER = DEDICATED)
    (SERVICE_NAME = SID)
    (SID = SID)
    so I created this:
    CALLBOX1 =
    (DESCRIPTION =
    (ADDRESS = (PROTOCOL = TCP)(HOST =192.168.50.161)(PORT = 1521))
    (CONNECT_DATA =
    (SERVER = DEDICATED)
    (SERVICE_NAME = callbox.oracle)
    (INSTANCE_NAME = callbox1)
    and did this: export TNS_ADMIN=/etc/
    am not sure which part of my tnsnames.ora in the RAC (posted earlier) be copied in /etc/tnsnames.ora in the application server.

  • Pre db creation

    hi
    I want to create a database using script.for some practice work on 11gR2.
    what to do if :
    [oracle@rac-1 ~]$ sqlplus '/ as sysdba'
    SQL*Plus: Release 11.2.0.1.0 Production on Sun Feb 14 22:06:19 2010
    Copyright (c) 1982, 2009, Oracle.  All rights reserved.
    ERROR:
    ORA-12547: TNS:lost contact
    Enter user-name: sys as sysdba
    Enter password:
    ERROR:
    ORA-12547: TNS:lost contact
    Enter user-name:
    ERROR:
    ORA-12547: TNS:lost contact
    SP2-0157: unable to CONNECT to ORACLE after 3 attempts, exiting SQL*Plus
    [oracle@rac-1 ~]$ listener start:
    [oracle@rac-1 ~]$ lsnrctl start LISTNER
    LSNRCTL for Linux: Version 11.2.0.1.0 - Production on 14-FEB-2010 22:04:39
    Copyright (c) 1991, 2009, Oracle.  All rights reserved.
    Starting /u01/app/oracle/product/11.2.0/db_1/bin/tnslsnr: please wait...
    TNSLSNR for Linux: Version 11.2.0.1.0 - Production
    System parameter file is /u01/app/oracle/product/11.2.0/db_1/network/admin/listener.ora
    Log messages written to /u01/app/oracle/diag/tnslsnr/rac-1/listner/alert/log.xml
    TNS-01151: Missing listener name, LISTNER, in LISTENER.ORA
    Listener failed to start. See the error message(s) above...
    [oracle@rac-1 ~]$ lsnrctl start LISTENER
    LSNRCTL for Linux: Version 11.2.0.1.0 - Production on 14-FEB-2010 22:05:21
    Copyright (c) 1991, 2009, Oracle.  All rights reserved.
    Starting /u01/app/oracle/product/11.2.0/db_1/bin/tnslsnr: please wait...
    TNSLSNR for Linux: Version 11.2.0.1.0 - Production
    System parameter file is /u01/app/oracle/product/11.2.0/db_1/network/admin/listener.ora
    Log messages written to /u01/app/oracle/diag/tnslsnr/rac-1/listener/alert/log.xml
    Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=EXTPROC1521)))
    Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=rac-1.sayantan-chakraborty.de)(PORT=1521)))
    Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=IPC)(KEY=EXTPROC1521)))
    STATUS of the LISTENER
    Alias                     LISTENER
    Version                   TNSLSNR for Linux: Version 11.2.0.1.0 - Production
    Start Date                14-FEB-2010 22:05:23
    Uptime                    0 days 0 hr. 0 min. 11 sec
    Trace Level               off
    Security                  ON: Local OS Authentication
    SNMP                      OFF
    Listener Parameter File   /u01/app/oracle/product/11.2.0/db_1/network/admin/listener.ora
    Listener Log File         /u01/app/oracle/diag/tnslsnr/rac-1/listener/alert/log.xml
    Listening Endpoints Summary...
      (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=EXTPROC1521)))
      (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=rac-1.sayantan-chakraborty.de)(PORT=1521)))
    The listener supports no services
    The command completed successfullyemctl start dbconsole:
    [oracle@rac-1 ~]$ emctl start dbconsole
    Oracle Enterprise Manager 11g Database Control Release 11.2.0.1.0
    Copyright (c) 1996, 2009 Oracle Corporation.  All rights reserved.
    https://rac-1.sayantan-chakraborty.de:1158/em/console/aboutApplication
    Starting Oracle Enterprise Manager 11g Database Control ............................................................................................. failed.
    Logs are generated in directory /u01/app/oracle/product/11.2.0/db_1/rac-1.sayantan-chakraborty.de_orcl/sysman/log
    [oracle@rac-1 ~]$ vi  /u01/app/oracle/product/11.2.0/db_1/rac-1.sayantan-chakraborty.de_orcl/sysman/log
    [oracle@rac-1 ~]$ how to get into sqlplus?
    I have no database now
    regards

    sb92075 wrote:
    post results from following
    env | sort
    [oracle@rac-1 ~]$ env | sort
    _=/bin/env
    COLORTERM=gnome-terminal
    CVS_RSH=ssh
    DBUS_SESSION_BUS_ADDRESS=unix:abstract=/tmp/dbus-KENQ9GTCj0,guid=2840694ab1860f09854fd8004b782547
    DESKTOP_SESSION=default
    DESKTOP_STARTUP_ID=
    DISPLAY=:0.0
    G_BROKEN_FILENAMES=1
    GDMSESSION=default
    GDM_XSERVER_LOCATION=local
    GNOME_DESKTOP_SESSION_ID=Default
    GNOME_KEYRING_SOCKET=/tmp/keyring-YrWn71/socket
    GTK_RC_FILES=/etc/gtk/gtkrc:/home/oracle/.gtkrc-1.2-gnome2
    HISTSIZE=1000
    HOME=/home/oracle
    HOSTNAME=rac-1.sayantan-chakraborty.de
    INPUTRC=/etc/inputrc
    KDEDIR=/usr
    KDE_IS_PRELINKED=1
    KDE_NO_IPV6=1
    LANG=en_US.UTF-8
    LESSOPEN=|/usr/bin/lesspipe.sh %s
    LIBGL_DRIVERS_PATH=/usr/lib/dri:/usr/lib64/dri
    LOGNAME=oracle
    LS_COLORS=no=00:fi=00:di=00;34:ln=00;36:pi=40;33:so=00;35:bd=40;33;01:cd=40;33;01:or=01;05;37;41:mi=01;05;37;41:ex=00;32:*.cmd=00;32:*.exe=00;32:*.com=00;32:*.btm=00;32:*.bat=00;32:*.sh=00;32:*.csh=00;32:*.tar=00;31:*.tgz=00;31:*.arj=00;31:*.taz=00;31:*.lzh=00;31:*.zip=00;31:*.z=00;31:*.Z=00;31:*.gz=00;31:*.bz2=00;31:*.bz=00;31:*.tz=00;31:*.rpm=00;31:*.cpio=00;31:*.jpg=00;35:*.gif=00;35:*.bmp=00;35:*.xbm=00;35:*.xpm=00;35:*.png=00;35:*.tif=00;35:
    MAIL=/var/spool/mail/oracle
    ORACLE_BASE=/u01/app/oracle
    ORACLE_HOME=/u01/app/oracle/product/11.2.0/db_1
    ORACLE_SID=orcl
    PATH=/usr/lib64/qt-3.3/bin:/usr/kerberos/bin:/u01/app/oracle/product/11.2.0/db_1/bin:/bin:/usr/bin:/usr/local/bin:/usr/X11R6/bin:/opt/real/RealPlayer
    PWD=/home/oracle
    QTDIR=/usr/lib64/qt-3.3
    QTINC=/usr/lib64/qt-3.3/include
    QTLIB=/usr/lib64/qt-3.3/lib
    SESSION_MANAGER=local/rac-1.sayantan-chakraborty.de:/tmp/.ICE-unix/7860
    SHELL=/bin/bash
    SHLVL=2
    SSH_AGENT_PID=7899
    SSH_ASKPASS=/usr/libexec/openssh/gnome-ssh-askpass
    SSH_AUTH_SOCK=/tmp/ssh-zKJMpB7860/agent.7860
    TERM=xterm
    USERNAME=oracle
    USER=oracle
    WINDOWID=23068753
    XAUTHORITY=/tmp/.gdm0PE17U
    XMODIFIERS=@im=none
    [oracle@rac-1 ~]$
    cat /etc/hosts
    [oracle@rac-1 ~]$ cat /etc/hosts
    # Do not remove the following line, or various programs
    # that require network functionality will fail.
    127.0.0.1               rac-1.sayantan-chakraborty.de rac-1 localhost.localdomain localhost
    ::1             localhost6.localdomain6 localhost6
    192.168.1.1     router.sayantan-chakraborty.de  router
    192.168.1.2     rac-1.sayantan-chakraborty.de   rac-1
    192.168.1.3     rac-2.sayantan-chakraborty.de   rac-2
    192.168.1.25    rac-1-vip.sayantan-chakraborty.de rac-1
    192.168.1.35    rac-1-vip.sayantan-chakraborty.de rac-1-vip
    192.168.181.10  rac-1-priv.sayantan-chakraborty.de rac-1-vip
    192.168.181.20  rac-2-priv.sayantan-chakraborty.de rac-2-vip
    192.168.1.31    rac-scan.sayantan-chakraborty.de        rac-scan
    192.168.1.23    rac-scan.sayantan-chakraborty.de        rac-scan
    192.168.1.45    rac-scan.sayantan-chakraborty.de        rac-scan
    [oracle@rac-1 ~]$ >
    do not confuse with the name eg .rac-1.I did NOT installed rac till now.
    Does any type of Operating System virtualization exist on this system?YES.but not using for oracle.
    [oracle@rac-1 ~]$ /sbin/ifconfig -a
    eth0      Link encap:Ethernet  HWaddr 00:00:E8:F7:02:B0 
              inet addr:192.168.1.2  Bcast:192.168.1.255  Mask:255.255.255.0
              inet6 addr: fe80::200:e8ff:fef7:2b0/64 Scope:Link
              UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
              RX packets:8515 errors:0 dropped:0 overruns:0 frame:0
              TX packets:9765 errors:0 dropped:0 overruns:0 carrier:0
              collisions:0 txqueuelen:1000
              RX bytes:6183799 (5.8 MiB)  TX bytes:1987002 (1.8 MiB)
              Interrupt:177 Base address:0x8c00
    eth0.5    Link encap:Ethernet  HWaddr 00:00:E8:F7:02:B0 
              inet addr:172.194.193.0  Bcast:172.194.193.255  Mask:255.255.255.0
              inet6 addr: fe80::200:e8ff:fef7:2b0/64 Scope:Link
              UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
              RX packets:0 errors:0 dropped:0 overruns:0 frame:0
              TX packets:64 errors:0 dropped:0 overruns:0 carrier:0
              collisions:0 txqueuelen:0
              RX bytes:0 (0.0 b)  TX bytes:12956 (12.6 KiB)
    eth1      Link encap:Ethernet  HWaddr 00:26:18:59:EE:49 
              inet addr:192.168.181.10  Bcast:192.168.181.63  Mask:255.255.255.192
              UP BROADCAST MULTICAST  MTU:1500  Metric:1
              RX packets:0 errors:0 dropped:0 overruns:0 frame:0
              TX packets:0 errors:0 dropped:0 overruns:0 carrier:1
              collisions:0 txqueuelen:1000
              RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)
              Memory:fe9c0000-fea00000
    lo        Link encap:Local Loopback 
              inet addr:127.0.0.1  Mask:255.0.0.0
              inet6 addr: ::1/128 Scope:Host
              UP LOOPBACK RUNNING  MTU:16436  Metric:1
              RX packets:7268 errors:0 dropped:0 overruns:0 frame:0
              TX packets:7268 errors:0 dropped:0 overruns:0 carrier:0
              collisions:0 txqueuelen:0
              RX bytes:6843826 (6.5 MiB)  TX bytes:6843826 (6.5 MiB)
    sit0      Link encap:IPv6-in-IPv4 
              NOARP  MTU:1480  Metric:1
              RX packets:0 errors:0 dropped:0 overruns:0 frame:0
              TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
              collisions:0 txqueuelen:0
              RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)
    virbr0    Link encap:Ethernet  HWaddr 00:00:00:00:00:00 
              inet addr:192.168.122.1  Bcast:192.168.122.255  Mask:255.255.255.0
              inet6 addr: fe80::200:ff:fe00:0/64 Scope:Link
              UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
              RX packets:0 errors:0 dropped:0 overruns:0 frame:0
              TX packets:60 errors:0 dropped:0 overruns:0 carrier:0
              collisions:0 txqueuelen:0
              RX bytes:0 (0.0 b)  TX bytes:12651 (12.3 KiB)
    [oracle@rac-1 ~]$

  • Primary RAC database hung on sqlplus as sysdba, dgmgrl and rman target

    I need urgent help here, our qa environment is two nodes RAC primary, and two nodes standby.
    Standby environment is ok, so far works, but no entries added to alert log.
    however on primary rac, i cannot sqlplus to the db, and rman target hung, and dgmgrl hung.
    I tried to srvctl stop database -o abort, it just hung there.
    please help me to get that resolved.
    Thanks in advance.

    982335 wrote:
    I need urgent help here, our qa environment is two nodes RAC primary, and two nodes standby.
    Standby environment is ok, so far works, but no entries added to alert log.
    however on primary rac, i cannot sqlplus to the db, and rman target hung, and dgmgrl hung.
    I tried to srvctl stop database -o abort, it just hung there.
    please help me to get that resolved.
    Thanks in advance.check the primary alert log for error and paste here.
    also do network connectivity test....e.g
    lsnrctl status
    from a client system do:
    ping primary_ip
    tnsping primary_tns
    always make sure Abort and killing OS process is your last option!
    Tobi

  • Manual creation of Oracle RAC Database

    Hello Guru's,
    I successfully installed Oracle Grid(clusterware) and Oracle RDBMS 11.2.0.1.0 for RAC containing 2 nodes on Linux 5.3 64 bit.
    All the clusterware services and ASM instance are running fine on both nodes.
    Now im planning to create database manually on node1 (RAC-NODE1).
    1) configure the environmental variables as follows
    export TMP=/tmp
    export ORACLE_HOSTNAME=`hostname`
    export ORACLE_SID=finance1
    export ORACLE_UNQNAME=finance
    export ORACLE_BASE=/rdbms1/app/oracle/
    export ORACLE_HOME=/rdbms1/app/oracle/product/11.2.0/db_home/
    export ORACLE_TERM=xterm
    export PATH=$ORACLE_HOME/bin:$PATH:.
    export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib
    export CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib
    2) created the pfile and modified the parameters as follows
    db_name='finance'
    processes = 200
    audit_trail ='db'
    db_block_size=8192
    db_domain='vod.com'
    diagnostic_dest='/oradump/oradata/finance/dump'
    sessions=200
    remote_login_passwordfile='EXCLUSIVE'
    undo_tablespace='UNDOTBS1'
    control_files = +CTRL
    compatible ='11.2.0'
    job_queue_processes = 10
    undo_management = 'AUTO'
    finance1.instance_name = finance1
    db_create_file_dest = +DBFILE
    db_create_online_log_dest_1 = +REDO1
    db_create_online_log_dest_2 = +REDO2
    3) Created the password file as follows
    $ orapwd file=orapwfinance1 passwors=syspassword entries=10
    4) edit the /etc/oratab file as follows
    finance1:/rdbms1/app/oracle/product/11.2.0/db_home:N
    5) created the rac db script as follows.
    CREATE DATABASE FINANCE
    DATAFILE '+DBFILE' SIZE 500M
    AUTOEXTEND ON NEXT 50M MAXSIZE UNLIMITED
    EXTENT MANAGEMENT LOCAL
    SYSAUX DATAFILE '+DBFILE' SIZE 500M
    AUTOEXTEND ON NEXT 50M MAXSIZE UNLIMITED
    DEFAULT TEMPORARY TABLESPACE TEMP
    TEMPFILE '+DBFILE' SIZE 100M AUTOEXTEND ON NEXT 10M MAXSIZE 500M
    UNDO TABLESPACE UNDOTBS1
    DATAFILE '+DBFILE' SIZE 100M AUTOEXTEND ON NEXT 10M MAXSIZE 500M
    DEFAULT TABLESPACE USERDATA
    DATAFILE '+DBFILE' SIZE 100M AUTOEXTEND ON NEXT 10M MAXSIZE 500M
    LOGFILE
    GROUP 1 ('+REDO1','+REDO2') SIZE 20M,
    GROUP 2 ('+REDO1','+REDO2') SIZE 20M,
    GROUP 3 ('+REDO1','+REDO2') SIZE 20M
    MAXINSTANCES 8
    MAXLOGHISTORY 300
    MAXLOGFILES 64
    MAXLOGMEMBERS 5
    MAXDATAFILES 150
    USER SYS IDENTIFIED BY "sys_123"
    USER SYSTEM IDENTIFIED BY "system_123";
    6) started the instance as follows.
    $ sqlplus / as sysdba
    SQL> startup nomount
    ORACLE instance started.
    Total System Global Area 217157632 bytes
    Fixed Size 2211928 bytes
    Variable Size 159387560 bytes
    Database Buffers 50331648 bytes
    Redo Buffers 5226496 bytes
    SQL> @rac_db.sql
    CREATE DATABASE FINANCE
    ERROR at line 1:
    ORA-01501: CREATE DATABASE failed
    ORA-00200: control file could not be created
    ORA-00202: control file: '+CTRL'
    ORA-15045: ASM file name '+CTRL' is not in reference form
    ORA-17502: ksfdcre:5 Failed to create file +CTRL
    ORA-15081: failed to submit an I/O operation to a disk
    It shows that oracle unable to create control file on +CTRL diskgroup, i found that there is a problem with permissions on disks in diskgroups.
    brw-rw-r-- 1 grid asmadmin 8, 49 Apr 8 04:25 CTRLDISK1
    brw-rw-r-- 1 grid asmadmin 8, 50 Apr 8 04:25 CTRLDISK2
    brw-rw-r-- 1 grid asmadmin 8, 51 Apr 8 04:25 CTRLDISK3
    brw-rw-r-- 1 grid asmadmin 8, 52 Apr 8 04:25 CTRLDISK4
    since all the disks in diskgroup has 664 permision and owned by grid user primary group is asmadmin.
    # id grid
    uid=1001(grid) gid=501(oinstall) groups=501(oinstall),504(asmadmin),505(asmdba),506(asmoper)
    # id oracle
    uid=1002(oracle) gid=501(oinstall) groups=501(oinstall),502(dba),503(oper),505(asmdba)
    please help me to overcome the above problem.
    regards,

    Thanks for comments,
    I successfully created the database, once i restared the database it is unbale to mount because of unbale to open +CTRL group file.
    SQL> startup
    ORACLE instance started.
    Total System Global Area 217157632 bytes
    Fixed Size 2211928 bytes
    Variable Size 159387560 bytes
    Database Buffers 50331648 bytes
    Redo Buffers 5226496 bytes
    ORA-00205: error in identifying control file, check alert log for more info
    alert log:
    ORACLE_BASE from environment = /rdbms1/app/oracle/
    Thu Apr 11 09:36:34 2013
    ALTER DATABASE MOUNT
    ORA-00210: cannot open the specified control file
    ORA-00202: control file: '+CTRL'
    ORA-17503: ksfdopn:2 Failed to open file +CTRL
    ORA-15045: ASM file name '+CTRL' is not in reference form
    ORA-205 signalled during: ALTER DATABASE MOUNT...
    Thu Apr 11 09:36:42 2013
    NOTE: initiating MARK startup
    raw devices:
    # ls -l /dev/sd*
    brw-rw-r-- 1 grid asmadmin 8, 16 Apr 11 09:04 /dev/sdb
    brw-rw-r-- 1 grid asmadmin 8, 17 Apr 11 09:51 /dev/sdb1
    brw-rw-r-- 1 grid asmadmin 8, 18 Apr 11 09:51 /dev/sdb2
    brw-rw-r-- 1 grid asmadmin 8, 19 Apr 11 09:51 /dev/sdb3
    brw-rw-r-- 1 grid asmadmin 8, 20 Apr 11 09:45 /dev/sdb4
    brw-rw-r-- 1 grid asmadmin 8, 32 Apr 11 09:04 /dev/sdc
    brw-rw-r-- 1 grid asmadmin 8, 33 Apr 11 09:30 /dev/sdc1
    brw-rw-r-- 1 grid asmadmin 8, 34 Apr 11 09:30 /dev/sdc2
    brw-rw-r-- 1 grid asmadmin 8, 35 Apr 11 09:30 /dev/sdc3
    brw-rw-r-- 1 grid asmadmin 8, 36 Apr 11 09:05 /dev/sdc4
    brw-rw-r-- 1 grid asmadmin 8, 48 Apr 11 09:04 /dev/sdd
    brw-rw-r-- 1 grid asmadmin 8, 49 Apr 11 09:51 /dev/sdd1
    brw-rw-r-- 1 grid asmadmin 8, 50 Apr 11 09:51 /dev/sdd2
    brw-rw-r-- 1 grid asmadmin 8, 51 Apr 11 09:51 /dev/sdd3
    brw-rw-r-- 1 grid asmadmin 8, 52 Apr 11 09:30 /dev/sdd4
    brw-rw-r-- 1 grid asmadmin 8, 64 Apr 11 09:04 /dev/sde
    brw-rw-r-- 1 grid asmadmin 8, 65 Apr 11 09:51 /dev/sde1
    brw-rw-r-- 1 grid asmadmin 8, 66 Apr 11 09:51 /dev/sde2
    brw-rw-r-- 1 grid asmadmin 8, 67 Apr 11 09:51 /dev/sde3
    brw-rw-r-- 1 grid asmadmin 8, 68 Apr 11 09:30 /dev/sde4
    brw-rw-r-- 1 grid asmadmin 8, 80 Apr 11 09:04 /dev/sdf
    brw-rw-r-- 1 grid asmadmin 8, 81 Apr 11 09:51 /dev/sdf1
    brw-rw-r-- 1 grid asmadmin 8, 82 Apr 11 09:51 /dev/sdf2
    brw-rw-r-- 1 grid asmadmin 8, 83 Apr 11 09:51 /dev/sdf3
    brw-rw-r-- 1 grid asmadmin 8, 84 Apr 11 09:30 /dev/sdf4
    brw-rw-r-- 1 grid asmadmin 8, 96 Apr 11 09:04 /dev/sdg
    brw-rw-r-- 1 grid asmadmin 8, 97 Apr 11 09:51 /dev/sdg1
    brw-rw-r-- 1 grid asmadmin 8, 98 Apr 11 09:51 /dev/sdg2
    brw-rw-r-- 1 grid asmadmin 8, 99 Apr 11 09:51 /dev/sdg3
    brw-rw-r-- 1 grid asmadmin 8, 100 Apr 11 09:30 /dev/sdg4
    brw-rw-r-- 1 grid asmadmin 8, 112 Apr 11 09:04 /dev/sdh
    brw-rw-r-- 1 grid asmadmin 8, 113 Apr 11 09:51 /dev/sdh1
    brw-rw-r-- 1 grid asmadmin 8, 114 Apr 11 09:51 /dev/sdh2
    brw-rw-r-- 1 grid asmadmin 8, 115 Apr 11 09:51 /dev/sdh3
    brw-rw-r-- 1 grid asmadmin 8, 116 Apr 11 09:30 /dev/sdh4
    asm disks:
    [09:52 AM [email protected] disks]# ll
    brw-rw-r-- 1 grid asmadmin 8, 65 Apr 11 09:05 ARCLOGDISK1
    brw-rw-r-- 1 grid asmadmin 8, 66 Apr 11 09:05 ARCLOGDISK2
    brw-rw-r-- 1 grid asmadmin 8, 67 Apr 11 09:05 ARCLOGDISK3
    brw-rw-r-- 1 grid asmadmin 8, 68 Apr 11 09:05 ARCLOGDISK4
    brw-rw-r-- 1 grid asmadmin 8, 49 Apr 11 09:05 CTRLDISK1
    brw-rw-r-- 1 grid asmadmin 8, 50 Apr 11 09:05 CTRLDISK2
    brw-rw-r-- 1 grid asmadmin 8, 51 Apr 11 09:05 CTRLDISK3
    brw-rw-r-- 1 grid asmadmin 8, 52 Apr 11 09:05 CTRLDISK4
    brw-rw-r-- 1 grid asmadmin 8, 33 Apr 11 09:05 DBFILEDISK1
    brw-rw-r-- 1 grid asmadmin 8, 34 Apr 11 09:05 DBFILEDISK2
    brw-rw-r-- 1 grid asmadmin 8, 35 Apr 11 09:05 DBFILEDISK3
    brw-rw-r-- 1 grid asmadmin 8, 36 Apr 11 09:05 DBFILEDISK4
    brw-rw-r-- 1 grid asmadmin 8, 113 Apr 11 09:05 FRADISK1
    brw-rw-r-- 1 grid asmadmin 8, 114 Apr 11 09:05 FRADISK2
    brw-rw-r-- 1 grid asmadmin 8, 115 Apr 11 09:05 FRADISK3
    brw-rw-r-- 1 grid asmadmin 8, 116 Apr 11 09:05 FRADISK4
    brw-rw-r-- 1 grid asmadmin 8, 81 Apr 11 09:05 REDODISK1
    brw-rw-r-- 1 grid asmadmin 8, 82 Apr 11 09:05 REDODISK2
    brw-rw-r-- 1 grid asmadmin 8, 83 Apr 11 09:05 REDODISK3
    brw-rw-r-- 1 grid asmadmin 8, 84 Apr 11 09:05 REDODISK4
    brw-rw-r-- 1 grid asmadmin 8, 97 Apr 11 09:05 REDODISK5
    brw-rw-r-- 1 grid asmadmin 8, 98 Apr 11 09:05 REDODISK6
    brw-rw-r-- 1 grid asmadmin 8, 99 Apr 11 09:05 REDODISK7
    brw-rw-r-- 1 grid asmadmin 8, 100 Apr 11 09:05 REDODISK8
    brw-rw-r-- 1 grid asmadmin 8, 17 Apr 11 09:05 VOTEDISK1
    brw-rw-r-- 1 grid asmadmin 8, 18 Apr 11 09:05 VOTEDISK2
    brw-rw-r-- 1 grid asmadmin 8, 19 Apr 11 09:05 VOTEDISK3
    brw-rw-r-- 1 grid asmadmin 8, 20 Apr 11 09:05 VOTEDISK4
    [09:53 AM [email protected] ~]# ls -l /rdbms1/app/oracle/product/11.2.0/db_home/bin/oracle
    -rwsr-s--x 1 oracle asmadmin 210824720 Apr 8 13:39 /rdbms1/app/oracle/product/11.2.0/db_home/bin/oracle
    I know this problem is coming for improper assiging of permissions, but i could not pin point. kindly clarify the same.
    one more thing i need to know.., whenever i rebooted nodes, both of raw devices and asm disks are come to default permission brw------, i have to change every time to brw-rw-r--. is there any way to make these disks and devices permissions make permenet.
    Thanks in advance.

  • ASM pfile lost in Oracle RAC 11gr2

    Hello Gurus,
    I am new to Oracle RAC 11gr2 and facing some issues. our spfile/pfile for asm instance is lost and I am not able to startup the asm instance.
    Environment is as below:
    RAC 2 nodes
    Oracle RAC 11gr2
    Enterprise Linux Server release 5.5 (Carthage)
    ORA-01078: failure in processing system parameters
    LRM-00109: could not open parameter file '/u01/app/oracle/product/11.2.0/db_1/dbs/init+ASM1.ora'
    Is there any way to recover the spfile/pfile ?
    Also getting following error while try to connect " sqlplus / as sysasm"
    $ sqlplus / as sysasm
    SQL*Plus: Release 11.2.0.1.0 Production on Sun Mar 27 11:26:02 2011
    Copyright (c) 1982, 2009, Oracle. All rights reserved.
    ERROR:
    ORA-01031: insufficient privileges
    Enter user-name:
    Thanks and Regards,

    Hi,
    Do we need to Set variable ORACLE_HOME properly (Grid Home) while starting RDBMS instance?
    e.g ORACLE_HOME=/u01/app/11.2.0/grid*I recommend you set ORACLE_HOME properly for each Oracle Installation when you use SQLPLUS.
    i.e: cat /etc/oratab
    orcl:/u01/app/oracle/product/11.2.0/db_1
    db10g:/u01/app/oracle/product/10.2.0/db_1
    +ASM1:/u01/app/11.2.0/gridAlso you can startup/shutdown our database with SQLPLUS using ORACLE NET*, but it must be configured properly.
    My recommendation <s>is always</s> use SRVCTL and CRSCTL to manage to your environment. Track all the process of initialization processes through the logs using ADRCI utility (only 11.1 or later).
    When use SRVCTL or CRSCTL to manage I recomend you use (GRID HOME).
    SQL> alter database open;
    alter database open
    ERROR at line 1:
    ORA-16038: log 3 sequence# 1067 cannot be archived
    ORA-19809: limit exceeded for recovery files
    ORA-00312: online log 3 thread 2: '+FRA/yyy/onlinelog/group_3.259.738489481'
    SQL>I give to you three options:
    1° - Add more asmdisk on Diskgroup +FRA
    2° - Make backup of all archivelog using RMAN and option delete input (i.e backup archivelog delete all input;)
    3° - If this database is for TEST ONLY and you not need backup or recovery of then you can delete all archivelog using RMAN (i.e delete archivelog all;)
    I recommend you create routine of backups of the database and archivelog to prevent this area becomes exhausted.
    If you need the database in archivelog but it is only to test then create routine to delete archivelogs periodically.
    If you dont need the database in archivelog, then just disable it.
    Regards,
    Levi Pereira

  • Confusion over DBCA script generated for manual RAC DB creation

    Version:11.2.0.4/RHEL 6.3
    We would like to create our 3-node RAC DB manually.  DBCA cannot meet our requirement because our redo log files, datafiles, tempfiles and control files are placed in a complicated manner . If we use DBCA , we will have to spend a lot of time configuring to our requirements after the DB creation.
    I generated the DB creation scripts from DBCA (DB Name = BRCFPRD )
    DBCA placed the db creation scripts in the specified directory in all the 3 nodes !!
    They all have almost the same contents . The only difference being the instance name (BRCFPRD12.sql for Node2, ... etc).
    Scripts in each node have the createDB.sql statement which has CREATE DATABASE "BRCFPRD" statement. Why is this ? The database need to be created only from one node. Then why did DBCA place createDB.sql in all nodes ?
    I just want to run the script from just one node , say Node1 and it should create the 3-Node RAC DB. How can I do this manually?
    -- The scripts genereated by DBCA in Node1
    apex.sql
    BRCFPRD1.sh
    BRCFPRD1.sql
    context.sql
    CreateClustDBViews.sql
    CreateDB.sql
    CreateDBCatalog.sql
    CreateDBFiles.sql
    cwmlite.sql
    emRepository.sql
    init.ora
    interMedia.sql
    JServer.sql
    lockAccount.sql
    ordinst.sql
    owb.sql
    postDBCreation.sql
    spatial.sql
    xdb_protocol.sql
    -- The contents of the main shell script BRCFPRD1.sh
    $ cat BRCFPRD1.sh
    #!/bin/sh
    OLD_UMASK=`umask`
    umask 0027
    mkdir -p /optware/product/admin/BRCFPRD/adump
    mkdir -p /optware/product/admin/BRCFPRD/dpdump
    mkdir -p /optware/product/admin/BRCFPRD/hdump
    mkdir -p /optware/product/admin/BRCFPRD/pfile
    mkdir -p /optware/product/cfgtoollogs/dbca/BRCFPRD
    umask ${OLD_UMASK}
    ORACLE_SID=BRCFPRD1; export ORACLE_SID
    PATH=$ORACLE_HOME/bin:$PATH; export PATH
    echo You should Add this entry in the /etc/oratab: BRCFPRD:/optware/product/oracle/11.2.0:Y
    /optware/product/oracle/11.2.0/bin/sqlplus /nolog @/optware/product/BRCFPRD1.sql
    -- Contents of BRCFSPRD1.sql
    $ cat BRCFPRD1.sql
    set verify off
    ACCEPT sysPassword CHAR PROMPT 'Enter new password for SYS: ' HIDE
    ACCEPT systemPassword CHAR PROMPT 'Enter new password for SYSTEM: ' HIDE
    ACCEPT sysmanPassword CHAR PROMPT 'Enter new password for SYSMAN: ' HIDE
    ACCEPT dbsnmpPassword CHAR PROMPT 'Enter new password for DBSNMP: ' HIDE
    host /optware/product/oracle/11.2.0/bin/orapwd file=/optware/product/oracle/11.2.0/dbs/orapwBRCFPRD1 force=y
    host /grid/product/11.2.0/bin/setasmgidwrap o=/optware/product/oracle/11.2.0/bin/oracle
    host /optware/product/oracle/11.2.0/bin/srvctl add database -d BRCFPRD -o /optware/product/oracle/11.2.0 -p +DATA/BRCFPRD/spfileBRCFPRD.ora -n BRCFPRD -a "DATA,ARCH_DG"
    host /optware/product/oracle/11.2.0/bin/srvctl add instance -d BRCFPRD -i BRCFPRD1 -n cimprd175
    host /optware/product/oracle/11.2.0/bin/srvctl add instance -d BRCFPRD -i BRCFPRD3 -n cimprd177
    host /optware/product/oracle/11.2.0/bin/srvctl add instance -d BRCFPRD -i BRCFPRD2 -n cimprd176
    host /optware/product/oracle/11.2.0/bin/srvctl disable database -d BRCFPRD
    @/optware/product/CreateDB.sql
    @/optware/product/CreateDBFiles.sql
    @/optware/product/CreateDBCatalog.sql
    @/optware/product/JServer.sql
    @/optware/product/context.sql
    @/optware/product/xdb_protocol.sql
    @/optware/product/ordinst.sql
    @/optware/product/interMedia.sql
    @/optware/product/cwmlite.sql
    @/optware/product/spatial.sql
    @/optware/product/emRepository.sql
    @/optware/product/apex.sql
    @/optware/product/owb.sql
    @/optware/product/CreateClustDBViews.sql
    host echo "SPFILE='+DATA/BRCFPRD/spfileBRCFPRD.ora'" > /optware/product/oracle/11.2.0/dbs/initBRCFPRD1.ora
    @/optware/product/lockAccount.sql
    @/optware/product/postDBCreation.sql
    -- Contents of CreateDB.sql in Node1
    $ cat /optware/product/CreateDB.sql
    SET VERIFY OFF
    connect "SYS"/"&&sysPassword" as SYSDBA
    set echo on
    spool /optware/product/CreateDB.log append
    startup nomount pfile="/optware/product/init.ora";
    CREATE DATABASE "BRCFPRD"
    MAXINSTANCES 32
    MAXLOGHISTORY 1
    MAXLOGFILES 192
    MAXLOGMEMBERS 3
    MAXDATAFILES 3000
    DATAFILE SIZE 700M AUTOEXTEND ON NEXT  10240K MAXSIZE UNLIMITED
    EXTENT MANAGEMENT LOCAL
    SYSAUX DATAFILE SIZE 600M AUTOEXTEND ON NEXT  10240K MAXSIZE UNLIMITED
    SMALLFILE DEFAULT TEMPORARY TABLESPACE TEMP TEMPFILE SIZE 20M AUTOEXTEND ON NEXT  640K MAXSIZE UNLIMITED
    SMALLFILE UNDO TABLESPACE "UNDOTBS1" DATAFILE SIZE 200M AUTOEXTEND ON NEXT  5120K MAXSIZE UNLIMITED
    CHARACTER SET AL32UTF8
    NATIONAL CHARACTER SET AL16UTF16
    LOGFILE GROUP 1  SIZE 28672M,
    GROUP 2  SIZE 28672M
    USER SYS IDENTIFIED BY "&&sysPassword" USER SYSTEM IDENTIFIED BY "&&systemPassword";
    set linesize 2048;
    column ctl_files NEW_VALUE ctl_files;
    select concat('control_files=''', concat(replace(value, ', ', ''','''), '''')) ctl_files from v$parameter where name ='control_files';
    host echo &ctl_files >>/optware/product/init.ora;
    spool off

    If you look at scripts generated in Node2 and Node3 , you can see all scripts except the instance specific ones are commented using REM .
    REM host /u01/product/oracle/11.2.0.3/dbhome_1/bin/srvctl add instance -d STOMPER -i STOMPER1 -n ugxtlprd186
    REM host /u01/product/oracle/11.2.0.3/dbhome_1/bin/srvctl add instance -d STOMPER -i STOMPER2 -n ugxtlprd187
    REM host /u01/product/oracle/11.2.0.3/dbhome_1/bin/srvctl disable database -d STOMPER
    REM @/u01/product/CreateDB.sql
    REM @/u01/product/CreateDBFiles.sql
    REM @/u01/product/CreateDBCatalog.sql
    REM @/u01/product/JServer.sql
    REM @/u01/product/context.sql
    <snipped >

  • Steps to drop database in RAC

    Hi
    I m kinda new in RAC environment.I drop database by DROP DATABASE; but other files still left behind like in CRS repository. Here are some issues that I have to deal with:
    . delete all parameter file or password file etc from $ORALCE_HOME/dbs
    . remove detail from /etc/oratab
    . drop ASM diskgroup
    i have to put these steps in .sql script. it would be appreciated if you tell me these three steps. thanks

    When you are dropping a RAC database you need to remember the layers of configuration that support that object. Or, if you don't want to peel back the layers one at a time use DBCA to simplify the process. I still use DBCA because it resolves these dependency issues for me.
    The top most layer in a Cluster Database Configuration is the Cluster Managed Database services (Grid, dbca, or srvctl typically used for maintenance)
    The next layer from the top is the Cluster database and it's instances (Grid, dbca, srvctl, or sqlplus used for maintenance)
    Below the Cluster database is the shared storage. This is typically a cluster file system (ASM, ocfs, QFS, VxFS, etc...) In the case of ASM (grid, dbca, srvctl, or sqlplus may be used for maintenance) other cluster file systems have their own maintenance tools.
    A peer at the level of shared storage is the clusterware Node Apps. These processes (ons,gsd,vip and listener) provide the high availability framework for Oracle Clusterware. Clusterware requires shared storage for Cluster Registry and Voting Disks (raw or cluster file system)
    (srvctl and/or crs_* for maintenance
    (oracle or OSDBA above this level)
    (root or local admin below this level)
    Oracle Clusterware is an extension to the base OS and is the low level interfaces between the Oracle Stack and Operating System. (crs_* utilities and crsctl)
    Hardware & OS: it all needs too start here. OS user, User equivalence, Networking (TCP/IP, UDP), DNS or hosts resolution, Network Time Protocol, Watchdog Timer (linux), Storage Mapping (disks, luns, mounts)
    So, back to your original question about dropping a RAC database. The easiest way is with DBCA. Or, you could look up the commands to unwind each layer and peel it off one step at a time.
    Kevin

  • RAC, ASM failed to start up on second node , ORA-03113: end-of-file on comm

    i'm installing an RAC with 2 nodes on top of ASM
    when creating ASM Diskgroup , it failed and reported error CRS-0215 failed to start asm on node2
    Oracle 10.2.0.1
    linux CentOs 4.x
    u01/app/oracle/product/10.2.0/db_1/bin/dbca  -progress_only   -configureASM -templateName NO_VALUE -gdbName NO -sid NO      -emConf
    iguration NONE    -diskList /dev/raw/raw2,/dev/raw/raw3  -diskGroupName DATA -datafileJarLocation /u01/app/oracle/product/10.2.0/db_
    1/assistants/dbca/templates  -responseFile NO_VALUE  -nodeinfo node1,node2    -obfuscatedPasswords true   -oratabLocation /u01/app/o
    racle/product/10.2.0/db_1/install/oratab   -asmSysPassword 05dbb0be38ecf8cca822cf3cf99e675448  -redundancy EXTERNA
    [oracle@node2 bin]$ ./crs_stat -t -v
    Name           Type           R/RA   F/FT   Target    State     Host       
    ora....SM1.asm application    0/5    0/0    ONLINE    ONLINE    node1      
    ora....E1.lsnr application    0/5    0/0    ONLINE    ONLINE    node1      
    ora.node1.gsd  application    0/5    0/0    ONLINE    ONLINE    node1      
    ora.node1.ons  application    0/3    0/0    ONLINE    ONLINE    node1      
    ora.node1.vip  application    0/0    0/0    ONLINE    ONLINE    node1      
    ora....SM2.asm application    0/5    0/0    OFFLINE   OFFLINE              
    ora....E2.lsnr application    0/5    0/0    ONLINE    ONLINE    node2      
    ora.node2.gsd  application    0/5    0/0    ONLINE    ONLINE    node2      
    ora.node2.ons  application    0/3    0/0    ONLINE    ONLINE    node2      
    ora.node2.vip  application    0/0    0/0    ONLINE    ONLINE    node2  
    i checked the status , asm is able to start on both nodes if not at the same time ,
    when trying to start the second node , with srvctl or sqlplus , each give the error 03113
    can anyone suggest me of how to bring up both instances ,
    thanks~
    [oracle@node2 bin]$ srvctl stop asm -n node1
    [oracle@node2 bin]$ srvctl start asm -n node1
    [oracle@node2 bin]$ srvctl start asm -n node2
    PRKS-1009 : Failed to start ASM instance "+ASM2" on node "node2", [PRKS-1009 : Failed to start ASM instance "+ASM2" on node "node2", [node2:ora.node2.ASM2.asm:
    node2:ora.node2.ASM2.asm:SQL*Plus: Release 10.2.0.1.0 - Production on Wed May 27 16:14:50 2009
    node2:ora.node2.ASM2.asm:
    node2:ora.node2.ASM2.asm:Copyright (c) 1982, 2005, Oracle.  All rights reserved.
    node2:ora.node2.ASM2.asm:
    node2:ora.node2.ASM2.asm:Enter user-name: Connected to an idle instance.
    node2:ora.node2.ASM2.asm:
    node2:ora.node2.ASM2.asm:SQL> ORA-03113: end-of-file on communication channel
    node2:ora.node2.ASM2.asm:SQL> Disconnected
    node2:ora.node2.ASM2.asm:
    [code/]
    Edited by: zs_hzh on May 27, 2009 1:25 AM                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   

    Is it possible to start ASM on second node with SQL*Plus in NOMOUNT state?

  • Oracle ORA_16000 when trying to add standby instance to existing rac node

    I attempted to use dbca to add a new standby instance to an existing cluster. The cluster is 4 nodes, Linux RHEL 5.3 Oracle 11.1.0.7. Also using ASM, asmlib, ocfs2 and shared block devices.
    ASM instances are up and functional on all nodes. current config appears to be running normally and correctly.
    I have a 4 instance database running on the cluster. I also have 3 physical standby active data guard instances running on 3 of the nodes. I wanted to add a new ADG instance to the 4th node.
    While running dbca I received ORA-00604 and ORA-16000.
    The active data guard database was open (read only) and redo apply was on. I am using data guard broker as well, but not grid control.
    Does anyone have a procedure for adding an instance in this environment? Do I need to have the standby in mount state? If dbca won't work does anyone have a manual procedure for adding a new instance?
    Thanks

    zulo
    Let's say you adding node nusclust160## to you existing cluster and dbca is a pain to use.
    Extend clusterware to the nusclust160## server.
    re: Page 64 of Oracle® Clusterware Administration and Deployment Guide 11g Release 1 (11.1)
    1a.
    Add undo tablespace to support additional node.
    Re-check space for DATA1 on nusclust16007 and /dbdata/ORADB on sun16109.
    As of Thursday, May 21, 2009 the DATA1 asm group has 53,584M free.
    As of Thursday, May 21, 2009 the /dbdata/ORADB has 77G free.
    In a separate terminal window on nusclust16007 run the following in sqlplus
    CREATE UNDO TABLESPACE UNDOTBS4 datafile '+DATA1' SIZE 13300M AUTOEXTEND ON ;
    This will take a long time to create this tablespace. Please minimize the window after submitting the ddl and move on to the next step.
    1b.
    Insure .bash_profile on nusclust160## should look like this:
    vi .bash_profile
    export ORACLE_HOSTNAME=nusclust160##
    export ORACLE_SID=ORADB4
    export ORA_CRS_BASE=/apps/ocr/oracle
    export ORACLE_BASE=/apps/dbs/oracle
    export PATH=/usr/ccs/bin:/usr/X/bin:/usr/bin:/usr/sfw/bin:/usr/sbin:/usr/local/bin
    export server=`uname -n`
    export PS1="$ORACLE_SID@$HOSTNAME >"
    alias cls='clear'
    alias More='more'
    alias ll='ls -lt | more'
    Gather IP addresses for fourth node from /etc/hosts:
    222.65.125.### nusclust160##
    222.65.125.### nusclust160##-vip
    10.333.248.### nusclust160##-priv
    2. Start Oracle Universal Installer:
    Go to CRS_home/oui/bin and run the addNode.sh script on one of the existing
    nodes. Oracle Universal Installer runs in add node mode.
    The Oracle inventory on nusclust16007, nusclust16008, and nusclust16036 are found under:
    /home/oracle/oraInventory
    Use a X windows enabled session (The OUI takes 33 minutes)
    cd /apps/ocr/oracle/product/11.1.0/crs/oui/bin
    ./addNode.sh
    a. In the first screen specify a new node as :
    Public Node Name:          nusclust160##
    Private Node Name:     nusclust160##-priv
    Virtual Host Name:     nusclust160##-vip
    If you receive the error:
    " tar. ./bin/racgvip.orig: Permission denied"
    Do the following:
    cd /apps/ocr/oracle/product/11.1.0/crs/bin
    ls -al racgvip.orig
    paste here:
    chown root:oinstall racgvip.orig
    chmod 771 racgvip.orig
    should now show:
    -rwxrwx--x 1 root oinstall 19213 Feb 11 08:36 racgvip.orig
    As root:
    a.
    On nusclust160##:
    cd /home/oracle/oraInventory
    ./orainstRoot.sh
    b.
    On nusclust16007:
    cd /apps/ocr/oracle/product/11.1.0/crs/install
    ./rootaddnode.sh
    clscfg: EXISTING configuration version 4 detected.
    clscfg: version 4 is 11 Release 1.
    Attempting to add 1 new nodes to the configuration
    Using ports: CSS=49895 CRS=49896 EVMC=49898 and EVMR=49897.
    node <nodenumber>: <nodename> <private interconnect name> <hostname>
    node 4: nusclust160## nusclust160##-priv nusclust160##
    Creating OCR keys for user 'root', privgrp 'root'..
    Operation successful.
    /apps/ocr/oracle/product/11.1.0/crs/bin/srvctl add nodeapps -n nusclust160## -A nusclust160##-vip/255.255.255.224/bge0
    c.
    On nusclust160##:
    cd /apps/ocr/oracle/product/11.1.0/crs/
    ./root.sh
    WARNING: directory '/apps/ocr/oracle/product/11.1.0' is not owned by root
    WARNING: directory '/apps/ocr/oracle/product' is not owned by root
    WARNING: directory '/apps/ocr/oracle' is not owned by root
    Checking to see if Oracle CRS stack is already configured
    OCR LOCATIONS = /raw/ocr/ocrconf1,/raw/ocr/ocrconf2
    OCR backup directory '/apps/ocr/oracle/product/11.1.0/crs/cdata/rac_cluster' does not exist. Creating now
    Setting the permissions on OCR backup directory
    Setting up Network socket directories
    Oracle Cluster Registry configuration upgraded successfully
    The directory '/apps/ocr/oracle/product/11.1.0' is not owned by root. Changing owner to root
    The directory '/apps/ocr/oracle/product' is not owned by root. Changing owner to root
    The directory '/apps/ocr/oracle' is not owned by root. Changing owner to root
    clscfg: EXISTING configuration version 4 detected.
    clscfg: version 4 is 11 Release 1.
    Successfully accumulated necessary OCR keys.
    Using ports: CSS=49895 CRS=49896 EVMC=49898 and EVMR=49897.
    node <nodenumber>: <nodename> <private interconnect name> <hostname>
    node 1: nusclust16007 nusclust16007-priv nusclust16007
    node 2: nusclust16008 nusclust16008-priv nusclust16008
    node 3: nusclust16036 nusclust16036-priv nusclust16036
    clscfg: Arguments check out successfully.
    NO KEYS WERE WRITTEN. Supply -force parameter to override.
    -force is destructive and will destroy any previous cluster
    configuration.
    Oracle Cluster Registry for cluster has already been initialized
    Startup will be queued to init within 30 seconds.
    Adding daemons to inittab
    Expecting the CRS daemons to be up within 600 seconds.
    Cluster Synchronization Services is active on these nodes.
    nusclust16007
    nusclust16008
    nusclust16036
    nusclust160##
    Cluster Synchronization Services is active on all the nodes.
    Waiting for the Oracle CRSD and EVMD to start
    Oracle CRS stack installed and running under init(1M)
    4. After this is done crs_stat -t will show nusclust160## in the crs i.e.
    I see:
    Name Type Target State Host
    ora....160##.gsd application ONLINE ONLINE sun...160##
    ora....160##.ons application ONLINE OFFLINE
    ora....160##.vip application ONLINE ONLINE sun...160##
    Do not be concerned about ora.nusclust160##.ons being OFFLINE, as that will be fixed shortly in a step that follows this one.
    5. As oracle :
    On nusclust16007:
    cd /apps/ocr/oracle/product/11.1.0/crs/bin
    ./racgons add_config nusclust160##:6251
    This should take about one second to run.
    If it says that it has already been added to the OCR you are fine.
    If it hangs, you may need to reboot all servers to clear this issue.
    6. Insure new node is properly added to ocr by running
    On nusclust16007:
    ocrdump
    Check for the entries that show:
    [DATABASE.ONS_HOSTS.nusclust160##.PORT]
    ORATEXT : 6251
    7. Check that your cluster is integrated and that the cluster is not divided into
    partitions by completing the following operations:
    On nusclust16007:
    cd /apps/ocr/oracle/product/11.1.0/crs/bin
    ./cluvfy comp clumgr -n all -verbose
    Should see Verification of cluster manager integrity was successful.
    8.
    Use the following command to perform an integrated validation of the Oracle
    Clusterware setup on all of the configured nodes, both the preexisting nodes
    and the nodes that you have added:
    AS oracle on nusclust16007:
    cluvfy stage -post crsinst -n all -verbose
    Post-check for cluster services setup was successful.
    good: Post-check for cluster services setup was successful.
    9.
    On nusclust160## as oracle run the following:
    cd /apps/ocr/oracle/product/11.1.0/crs/bin
    ./crs_stat -t | grep OFFLINE
    If you see this:
    ora.nusclust160##.ons application ONLINE OFFLINE
    then run this:
    ./crs_start -all
    After:
    ./crs_stat -t
    ora.nusclust160##.ons application ONLINE ONLINE nusclust160##
    If you see the above then you can move on the next step.
    Adding database binaries to the nusclust160## server and setting up the listener.
    1.
    From nusclust16007:
    Open an X window (The OUI part takes 13 minutes)
    cd /apps/dbs/oracle/product/11.1.0/db_1/oui/bin
    ./runInstaller -addNode ORACLE_HOME=/apps/dbs/oracle/product/11.1.0/db_1 $*
    You should get a prompt to specify a new node, in this case you should see nusclust160## where you will need to put a check mark beside it.
    2.
    from nusclust160##:
    Eventually you will be prompted to run the following as root on the new node
    On nusclust160##
    cd /apps/dbs/oracle/product/11.1.0/db_1
    ./root.sh
    Running Oracle 11g root.sh script...
    The following environment variables are set as:
    ORACLE_OWNER= oracle
    ORACLE_HOME= /apps/dbs/oracle/product/11.1.0/db_1
    Enter the full pathname of the local bin directory: [usr/local/bin]:
    Copying dbhome to /usr/local/bin ...
    Copying oraenv to /usr/local/bin ...
    Copying coraenv to /usr/local/bin ...
    Creating /var/opt/oracle/oratab file...
    Entries will be added to the /var/opt/oracle/oratab file as needed by
    Database Configuration Assistant when a database is created
    Finished running generic part of root.sh script.
    Now product-specific root actions will be performed.
    Finished product-specific root actions.
    3. verification
    Now set up the .bash_profile and .asm profile to on nusclust160## to support new ORADB4 and +ASM4 instances for the oracle userid.
    On nusclust160##:
    cp .bash_profile .bash_profile.bak
    On nusclust16007:
    sftp nusclust160##
    put .bash_profile
    On nusclust160##:
    vi .bash_profile
    change ORALCE_SID to ORADB4
    cp .bash_profile .asm
    vi .asm
    change ORALCE_SID to +ASM4 in .asm file
    which sqlplus
    Should show the path below is $PATH environmental variable is set correctly.
    /apps/dbs/oracle/product/11.1.0/db_1/bin/sqlplus
    On nusclust160##:
    oifcfg getif
    This should show:
    ce4 10.333.248.192 global cluster_interconnect
    ce5 222.65.125.128 global public
    4.
    Run Netbackup Oracle Agent link script.
    As oracle make sure ORACLE_HOME is fined.
    env | grep ORACLE_HOME
    then
    cd /usr/openv/netbackup/bin/
    ./oracle_link
    ls -al $ORACLE_HOME/lib/libobk.so
    should show:
    /apps/dbs/oracle/product/11.1.0/db_1/lib/libobk.so -> /usr/openv/netbackup/bin/libobk.so64.1
    5.
    On the target node, run the Net Configuration Assistant (NETCA) to add a
    listener. Add a listener to the target node by running NETCA from the target node and
    selecting only the target node on the Node Selection page.
    I shall do the following on nusclust160## using X Windows
    Now before I do this I see:
    crs_stat -t
    ora.nusclust160##.gsd application ONLINE ONLINE nusclust160##
    ora.nusclust160##.ons application ONLINE ONLINE nusclust160##
    ora.nusclust160##.vip application ONLINE ONLINE nusclust160##
    Connect to nusclust160## and open up X windows session.
    netca
    Choose Cluster configuration.
    select nusclust160## as the node to configure.
    Choose Listener configuration, then Add.
    When it prompts you for a listener name choose LISTENER as it will append _NUSCLUST160##(server name) to end of the LISTENER name to make a complete listener name. 
    At this point you will have listener to support the new node in the crs.
    now
    crs_stat -t
    will show:
    ora....0#.lsnr application ONLINE ONLINE nusclust160##
    ora.nusclust160##.gsd application ONLINE ONLINE nusclust160##
    ora.nusclust160##.ons application ONLINE ONLINE nusclust160##
    ora.nusclust160##.vip application ONLINE ONLINE nusclust160##
    At this point the necessary crs entries for gsd, ons, vip, and the listener on nusclust160## all we need now are the ORADB4 and +ASM4 instances added.
    III. 7/11/2009 7:40 AM Sat [120 min] NTTA DBA
    Use NON dbca method to create additional instances on the nusclust160## server. This will involve a complete shutdown of all RAC instances.
    1.
    Undo tablespace creation was taken care of in Step I,1. Check on the progress of the creation of tablespace UNDOTBS4 in the minimized window. Should see tablespace on primary and physical standby databases.
    2. First we shall set up the +ASM4 instance on nusclust160## and add it to the cluster.
    On nusclust160##
    cd $ORACLE_HOME/dbs
    vi init+ASM4.ora
    # Copyright (c) 1991, 2001, 2002 by Oracle Corporation
    # Cluster Database
    cluster_database=true
    cluster_database_instances=6
    # Miscellaneous
    diagnostic_dest=/apps/dbs/oracle
    instance_type=asm
    # Pools
    large_pool_size=12M
    asm_diskgroups='DATA1','ARCH','REDO1','REDO2'
    asm_diskstring='/raw/asm'
    +ASM1.instance_number=1
    +ASM2.instance_number=2
    +ASM3.instance_number=3
    +ASM4.instance_number=4
    3.
    On nusclust16007
    cd $ORACLE_HOME/dbs
    sftp nusclust160##
    put orapw+ASM1 /apps/dbs/oracle/product/11.1.0/db_1/dbs
    put orapwORADB1 /apps/dbs/oracle/product/11.1.0/db_1/dbs
    4.
    On nusclust160##
    cd $ORACLE_HOME/dbs
    cp orapw+ASM1 orapw+ASM4
    cp orapwORADB1 orapwORADB4
    5.
    On nusclust160##
    cd $HOME
    . ./.asm
    sqlplus '/ as sysasm'
    startup
    create spfile from pfile='/apps/dbs/oracle/product/11.1.0/db_1/dbs/init+ASM4.ora' ;
    shutdown immediate ;
    startup
    show parameters spfile
    6. Now that we have a running asm instance add it the cluster.
    On nusclust160##
    srvctl add asm -n nusclust160## -i +ASM4 -o /apps/dbs/oracle/product/11.1.0/db_1
    srvctl enable asm -n nusclust160## -i +ASM4
    7. Now that we have an asm instance let's set up a database instance.
    On nusclust16007/ORADB1 :
    alter system set cluster_database_instances=6 scope=spfile ;
    alter system set instance_name=ORADB4 scope=spfile sid='ORADB4' ;
    alter system set instance_number=4 scope=spfile sid='ORADB4' ;
    alter system set local_listener=LISTENER_ NUSCLUST160## scope=both sid='ORADB4' ;
    alter system set thread=4 scope=both sid='ORADB4' ;
    alter system set undo_tablespace=UNDOTBS4 scope=both sid='ORADB4' ;
    alter database add logfile thread 4 group 28 ('+REDO1', '+REDO2' ) size 100M ;
    alter database add logfile thread 4 group 29 ('+REDO1', '+REDO2' ) size 100M ;
    alter database add logfile thread 4 group 30 ('+REDO1', '+REDO2' ) size 100M ;
    alter database add logfile thread 4 group 31 ('+REDO1', '+REDO2' ) size 100M ;
    alter database enable public thread 4;
    Need to add 5 groups to support standby
    So at the end of the day 900M will be added to REDO1(29,577M free) and 900M will be added to REDO2 (29,577M free).
    8. Set up init.ora, listener.ora, and tnsnames.ora for ORADB4 on nusclust160##.
    a. init.ora set up
    cd $ORACLE_HOME/dbs
    vi initORADB4.ora
    SPFILE='+DATA1/ORADB/spfileORADB.ora'
    b. add entries to tnsnames.ora:
    ORADB4 =
    (DESCRIPTION =
    (ADDRESS = (PROTOCOL = TCP)(HOST = nusclust160##-vip)(PORT = 1521))
    (CONNECT_DATA =
    (SERVER = DEDICATED)
    (SERVICE_NAME = ORADB)
    (INSTANCE_NAME = ORADB4)
    ORADB =
    (DESCRIPTION =
    (ADDRESS = (PROTOCOL = TCP)(HOST = nusclust16007-vip)(PORT = 1521))
    (ADDRESS = (PROTOCOL = TCP)(HOST = nusclust16008-vip)(PORT = 1521))
    (ADDRESS = (PROTOCOL = TCP)(HOST = nusclust16036-vip)(PORT = 1521))
    (ADDRESS = (PROTOCOL = TCP)(HOST = nusclust160##-vip)(PORT = 1521))
    (LOAD_BALANCE = yes)
    (CONNECT_DATA =
    (SERVER = DEDICATED)
    (SERVICE_NAME = ORADB)
    LISTENERS_ORADB =
    (ADDRESS_LIST =
    (ADDRESS = (PROTOCOL = TCP)(HOST = nusclust16007-vip)(PORT = 1521))
    (ADDRESS = (PROTOCOL = TCP)(HOST = nusclust16008-vip)(PORT = 1521))
    (ADDRESS = (PROTOCOL = TCP)(HOST = nusclust16036-vip)(PORT = 1521))
    (ADDRESS = (PROTOCOL = TCP)(HOST = nusclust160##-vip)(PORT = 1521))
    LISTENER_NUSCLUST160## =
    (ADDRESS = (PROTOCOL = TCP)(HOST = nusclust160##-vip)(PORT = 1521))
    ORADB_PRIM =
    (DESCRIPTION =
    (ADDRESS = (PROTOCOL = TCP)(HOST = nusclust16007-vip)(PORT = 1521))
    (ADDRESS = (PROTOCOL = TCP)(HOST = nusclust16008-vip)(PORT = 1521))
    (ADDRESS = (PROTOCOL = TCP)(HOST = nusclust16036-vip)(PORT = 1521))
    (ADDRESS = (PROTOCOL = TCP)(HOST = nusclust160##-vip)(PORT = 1521))
    (LOAD_BALANCE = yes)
    (CONNECT_DATA =
    (SERVER = DEDICATED)
    (SERVICE_NAME = ORADB)
    c. add entries to listener.ora, The entries for most of this file should be set already, just insure modifications that need to be made are made.
    SID_LIST_LISTENER_NUSCLUST160## =
    (SID_LIST =
    (SID_DESC =
    (SID_NAME = PLSExtProc)
    (ORACLE_HOME = /apps/dbs/oracle/product/11.1.0/db_1)
    (PROGRAM = extproc)
    (SID_DESC =
    (GLOBAL_DBNAME = ORADB)
    (ORACLE_HOME = /apps/dbs/oracle/product/11.1.0/db_1)
    (SID_NAME = ORADB4)
    LISTENER_NUSCLUST160## =
    (DESCRIPTION_LIST =
    (DESCRIPTION =
    (ADDRESS = (PROTOCOL = TCP)(HOST = NUSCLUST160##-vip)(PORT = 1521)(IP = FIRST))
    (ADDRESS = (PROTOCOL = TCP)(HOST = 222.65.125.###)(PORT = 1521)(IP = FIRST))
    9. Reload the listener.
    lsnrclt
    set current_listener LISTENER_NUSCLUST160##
    reload
    exit
    10. Check audit trail, add instance to cluster, and start db instance.
    a.
    Check for audit directory and start the instance.
    /apps/dbs/oracle/product/11.1.0/db_1/rdbms/audit
    If this audit trail directory does not exist then create it.
    b.
    srvctl add instance -d ORADB -i ORADB4 -n nusclust160##
    srvctl modify instance -d ORADB -i ORADB4 -s +ASM4
    srvctl enable instance -d ORADB -i ORADB4
    Will probably show: PRKP-1017 : Instance ORADB4 already enabled.
    c.
    sqlplus '/ as sysdba'
    startup
    **Because the cluster_database_instances parameter requires the complete shutdown of all instances in the cluster, you might have an issue when it attempts to start the instance. If you receive an error then run:
    srvctl stop database -d oradb
    sqlplus '/ as sysdba'
    startup
    shutdown
    srvctl start database -d oradb
    shutdown
    srvctl start instance -d ORADB -i ORADB4 -o open
    11.
    Modify spfile of ASM1, ASM2, +ASM3
    On nusclust16007
    . ./.asm
    sqlplus '/ as sysasm'
    alter system set instance_number=4 scope=spfile sid='+ASM4' ;
    On nusclust16008
    . ./.asm
    sqlplus '/ as sysasm'
    alter system set instance_number=4 scope=spfile sid='+ASM4' ;
    On nusclust16036
    . ./.asm
    sqlplus '/ as sysasm'
    alter system set instance_number=4 scope=spfile sid='+ASM4' ;
    b Modify tnsnames.ora on nusclust nusclust16007, nusclust16008, and nusclust16036
    On nusclust16007
    ORADB4 =
    (DESCRIPTION =
    (ADDRESS = (PROTOCOL = TCP)(HOST = nusclust160##-vip)(PORT = 1521))
    (CONNECT_DATA =
    (SERVER = DEDICATED)
    (SERVICE_NAME = ORADB)
    (INSTANCE_NAME = ORADB4)
    Add the following line to the ORADB alias:
    (ADDRESS = (PROTOCOL = TCP)(HOST = nusclust160##-vip)(PORT = 1521))
    Add the following line to the LISTENERS_ORADB alias:
    (ADDRESS = (PROTOCOL = TCP)(HOST = nusclust160##-vip)(PORT = 1521))
    Add the following line to the ORADB_PRIM alias:
    (ADDRESS = (PROTOCOL = TCP)(HOST = nusclust160##-vip)(PORT = 1521))
    On nusclust16008
    ORADB4 =
    (DESCRIPTION =
    (ADDRESS = (PROTOCOL = TCP)(HOST = nusclust160##-vip)(PORT = 1521))
    (CONNECT_DATA =
    (SERVER = DEDICATED)
    (SERVICE_NAME = ORADB)
    (INSTANCE_NAME = ORADB4)
    Add the following line to the ORADB alias:
    (ADDRESS = (PROTOCOL = TCP)(HOST = nusclust160##-vip)(PORT = 1521))
    Add the following line to the LISTENERS_ORADB alias:
    (ADDRESS = (PROTOCOL = TCP)(HOST = nusclust160##-vip)(PORT = 1521))
    Add the following line to the ORADB_PRIM alias:
    (ADDRESS = (PROTOCOL = TCP)(HOST = nusclust160##-vip)(PORT = 1521))
    On nusclust16036
    ORADB4 =
    (DESCRIPTION =
    (ADDRESS = (PROTOCOL = TCP)(HOST = nusclust160##-vip)(PORT = 1521))
    (CONNECT_DATA =
    (SERVER = DEDICATED)
    (SERVICE_NAME = ORADB)
    (INSTANCE_NAME = ORADB4)
    Add the following line to the ORADB alias:
    (ADDRESS = (PROTOCOL = TCP)(HOST = nusclust160##-vip)(PORT = 1521))
    Add the following line to the LISTENERS_ORADB alias:
    (ADDRESS = (PROTOCOL = TCP)(HOST = nusclust160##-vip)(PORT = 1521))
    Add the following line to the ORADB_PRIM alias:
    (ADDRESS = (PROTOCOL = TCP)(HOST = nusclust160##-vip)(PORT = 1521))
    c Add standby logs on primary to support 4th node.
    alter database add standby logfile thread 4 group 32 ('+REDO1', '+REDO2' ) size 100M ;
    alter database add standby logfile thread 4 group 33 ('+REDO1', '+REDO2' ) size 100M ;
    alter database add standby logfile thread 4 group 34 ('+REDO1', '+REDO2' ) size 100M ;
    alter database add standby logfile thread 4 group 35 ('+REDO1', '+REDO2' ) size 100M ;
    alter database add standby logfile thread 4 group 36 ('+REDO1', '+REDO2' ) size 100M ;
    12.
    Test the cluster to make sure everything is set up correctly.
    a. Shutdown resources.
    On nusclust16007:
    emctl stop dbconsole
    ps -ef | grep perl
    ps -ef | grep agent
    ps -ef | grep java
    On nusclust16008:
    emctl stop dbconsole
    On nusclust16036:
    emctl stop dbconsole
    On nusclust16008:
    cd $HOME
    . ./.rman
    cd scripts
    ./go
    shutdown immediate
    cd $HOME
    . ./.bash_profile
    srvctl stop database -d oradb
    crs_stop -all
    crs_stat -t
    b. Startup resources
    On nusclust16007:
    cd $HOME
    . ./.bash_profile
    crs_start -all
    crs_stat -t
    The command above should show everything up and running.
    ocrcheck
    On nusclust16008:
    cd $HOME
    . ./.rman
    cd scripts
    ./go
    startup
    On nusclust16007:
    emctl start dbconsole
    On nusclust16008:
    emctl start dbconsole
    On nusclust16036:
    emctl start dbconsole
    How does that work for you?
    -JR jr

  • Scripting to gather oracle info on a Linux box Finding rac/normal instance?

    Hello all,
    I've been tasked by the disaster recovery team to get a little quick script for them to run, to see what oracle instances are on a box, if it is up, listener is running, etc.
    So far, with normal, one node instances I've had success. I grab data from the /etc/oratab file, and parse it out with awk to get a ORACLE_SID and and ORACLE_HOME.
    This script is run as the oracle user or a user in the dba group, so, that with each instance, I attempt a logon through SQL*Plus as connect / as sysdba and does a query to gather some instance information that I echo to the screen to establish that the instance is up and connectable.
    The problem I have run into, is if there is an instance listed for a node of a RAC cluster....it doesn't see that entries to /etc/oratab are quite the same using dbca for RAC nodes as normal nodes.
    For instance, say I have a cluster RAC which has nodes rac1 rac2 and rac3.
    From what I'm seeing...on the individual nodes, in the /etc/oratab entries only seem to be put in for the cluster RAC, rather that say rac1. My script which tries to set the oracle sid and home and connect / as sysdba fails. If the entry were put in there as rac1, it should work, I'd think.
    Can someone make suggestions for me to use with this script, so that I can find out what node instance is running, if RAC is on the computer...and automatically find and connect similar to how I'm doing with the single databases using /etc/oratab
    Is there a similar file to /etc/oratab that is used by the clusterware on the system?
    Thanks in advance,
    cayenne
    Edited by: cayenne on Apr 15, 2009 12:57 PM

    S2K wrote:
    please post your script here so we can see what exactly is going on...and help you point out stuff...Certainly...that might help indeed.
    Here is the listing:
    #! /bin/bash
    #grab original path so as to reuse on each loop interation to ensure ONLY
    # the oracle binaries from the proper oracle home are used.
    old_path=$PATH
    #set oracle_home/bin path for listener
    #to guess best fit, hit first home that is bootable
    #and not listed as ASM
    #start as empty
    listener_path=
    if [ -e "/etc/oratab" ]
    then
         #grab and clean the entries for oracle instances and their respective oracle homes
         oracle_instances=`cat /etc/oratab | sed '/#/d' | sed '/\*/d' | sed '/^$/d'`
         echo 'Testing connections to Oracle intances as listed in /etc/oratab'
         echo 'and testing connectivity, start times and status'
         echo '----------------------------------------------------------------------------------------------------'
         for oinstance in $oracle_instances;
         do
              oracle_sid=`echo $oinstance | awk -F":" '{print $1}'`
              oracle_home=`echo $oinstance | awk -F":" '{print $2}'`
              oracle_bootable=`echo $oinstance | awk -F":" '{print $3}'`
              echo
              echo 'Using ORACLE_HOME='$oracle_home
              echo
              export ORACLE_HOME=$oracle_home #set oracle home
              echo 'Setting ORACLE_SID='$oracle_sid
              echo
              export ORACLE_SID=$oracle_sid #set oracle sid
              echo 'Adding Oracle bin to path'
              echo
              echo '----------------------------------------------------------------------------------------------------'
              echo
              export PATH=$ORACLE_HOME/bin:$old_path #reset to old path plus new oracle home /bin
              #Test and set listener path
              # do I need this? to exclude asm for listener? && `expr match "$oracle_sid" \+ASM.*` = 0
              if [[ -z "$listener_path" && $oracle_bootable != "N" ]] #Test to see if listener path set yet, if not, is instance set bootable
              then
                   listener_path=$oracle_home/bin:$old_path
              fi
              #Next, connect through sqlplus and query v$instance table to prove connectivity
              output= #Reset output for each iteration
              output=`sqlplus -s "/ as sysdba" <<EOF
              set heading off feedback off verify off
              select 'Instance Name='||instance_name||' Host='||host_name||' Start Time='||to_char (startup_time, 'HH:MI AM MON DD,YYYY')||' Status='||status from v\\$instance;
              exit
              EOF`
              echo 'Connection to Database returns message:'
              # If sqlplus fails, print error message
              echo
              if [[ -z "$output" || `expr match "$output" .*ORA-.*` > 0 ]]
              then
                   echo 'Unable to conect and query database instance = '$oracle_sid
              else
                   echo $output
                   echo
                   echo '----------------------------------------------------------------------------------------------------'
                   echo
                   echo 'Schemas on this Database Instance: '$oracle_sid
                   echo
                   if [[ `expr match "$oracle_sid" \+ASM.*` = 0 ]] #Test to see if an ASM instance
                   then
                        #Hit sqlplus again, this time to grab and display a list of schema/usernames on this instance.
                        sqlplus -s "/ as sysdba" <<EOF
                        set heading off feedback off verify off pagesize 0
                        select username from dba_users order by username;
                        exit
    EOF
                   else
                        echo 'Instance '$oracle_sid' is an Auto Storage Management instance, no user schemas to report'
                   fi #end test for ASM
              fi
              echo
              echo '----------------------------------------------------------------------------------------------------'
         done
         echo
         ###Look into what path is used here for the listener!!
         output=''
         echo 'Checking output for querying the Oracle Listener'
         echo
         if [[ -n "$listener_path" ]]
         then
              lsnrctl status
         else
              echo
              echo 'Oracle Home Not set to bootable path, cannot find path for listener'
         fi
         echo
         echo '----------------------------------------------------------------------------------------------------'
    else
         echo '----------------------------------------------------------------------------------------------------'
         echo 'Unable to test connectivity to oracle database instances due to '
         echo 'the file /etc/oratab NOT being found on this system'
         echo '----------------------------------------------------------------------------------------------------'
    fi #End if statement testing if /etc/oratab present
    echo
    echo 'Listing Oracle processes currently running on this box'
    echo
    ps -ef | grep oracle
    echo
    echo '----------------------------------------------------------------------------------------------------'
    As I mentioned before, the problem comes when a RAC instance is in the /etc/oratab...apparently dbca puts in the RAC sid, not the node sid....and when my script tries to connect...it can't and throws the not started error.
    See other email for details of the problem.
    This is the script I'm working on....any suggestions greatly appreciated!!
    cayenne

  • Failed root.sh on 1st Node of 11.2.0.2.0 RAC on HP-UX 11.31 Itanium 64

    Started with 11.2.0.2.0 Grid Installation for 2 Node RAC on HP-UX 11.31 Itanium 64.
    Copying Software to remote node & linking libraries were successfully without any issue (upto 76%). But got issue while executing root.sh on Node1
    sph1erp:/oracle/11.2.0/grid #sh root.sh
    Running Oracle 11g root script...
    The following environment variables are set as:
    ORACLE_OWNER= oracle
    ORACLE_HOME= /oracle/11.2.0/grid
    Enter the full pathname of the local bin directory: [usr/local/bin]:
    Copying dbhome to /usr/local/bin ...
    Copying oraenv to /usr/local/bin ...
    Copying coraenv to /usr/local/bin ...
    Creating /etc/oratab file...
    Entries will be added to the /etc/oratab file as needed by
    Database Configuration Assistant when a database is created
    Finished running generic part of root script.
    Now product-specific root actions will be performed.
    Using configuration parameter file: /oracle/11.2.0/grid/crs/install/crsconfig_params
    Creating trace directory
    LOCAL ADD MODE
    Creating OCR keys for user 'root', privgrp 'sys'..
    Operation successful.
    OLR initialization - successful
    root wallet
    root wallet cert
    root cert export
    peer wallet
    profile reader wallet
    pa wallet
    peer wallet keys
    pa wallet keys
    peer cert request
    pa cert request
    peer cert
    pa cert
    peer root cert TP
    profile reader root cert TP
    pa root cert TP
    peer pa cert TP
    pa peer cert TP
    profile reader pa cert TP
    profile reader peer cert TP
    peer user cert
    pa user cert
    Adding daemon to inittab
    CRS-2672: Attempting to start 'ora.mdnsd' on 'sph1erp'
    CRS-2676: Start of 'ora.mdnsd' on 'sph1erp' succeeded
    CRS-2672: Attempting to start 'ora.gpnpd' on 'sph1erp'
    CRS-2676: Start of 'ora.gpnpd' on 'sph1erp' succeeded
    CRS-2672: Attempting to start 'ora.cssdmonitor' on 'sph1erp'
    CRS-2672: Attempting to start 'ora.gipcd' on 'sph1erp'
    CRS-2676: Start of 'ora.gipcd' on 'sph1erp' succeeded
    CRS-2676: Start of 'ora.cssdmonitor' on 'sph1erp' succeeded
    CRS-2672: Attempting to start 'ora.cssd' on 'sph1erp'
    CRS-2672: Attempting to start 'ora.diskmon' on 'sph1erp'
    CRS-2676: Start of 'ora.diskmon' on 'sph1erp' succeeded
    CRS-2676: Start of 'ora.cssd' on 'sph1erp' succeeded
    ASM created and started successfully.
    Disk Group OCRVOTE created successfully.
    clscfg: -install mode specified
    Successfully accumulated necessary OCR keys.
    Creating OCR keys for user 'root', privgrp 'sys'..
    Operation successful.
    CRS-4256: Updating the profile
    Successful addition of voting disk ab847ed2b4f04f2dbfb875226d2bb194.
    Successful addition of voting disk 85c05a5b30384f8dbff48cc069de7a7c.
    Successful addition of voting disk 649196fbdd614f9cbf26a9a0e6670a6e.
    Successful addition of voting disk 8815dfcee2e64f64bf00b9c76626ab41.
    Successful addition of voting disk 8ce55fe5534f4f77bfa9f54187592707.
    Successfully replaced voting disk group with +OCRVOTE.
    CRS-4256: Updating the profile
    CRS-4266: Voting file(s) successfully replaced
    ## STATE File Universal Id File Name Disk group
    1. ONLINE ab847ed2b4f04f2dbfb875226d2bb194 (/dev/oracle/ocrvote1) [OCRVOTE]
    2. ONLINE 85c05a5b30384f8dbff48cc069de7a7c (/dev/oracle/ocrvote2) [OCRVOTE]
    3. ONLINE 649196fbdd614f9cbf26a9a0e6670a6e (/dev/oracle/ocrvote3) [OCRVOTE]
    4. ONLINE 8815dfcee2e64f64bf00b9c76626ab41 (/dev/oracle/ocrvote4) [OCRVOTE]
    5. ONLINE 8ce55fe5534f4f77bfa9f54187592707 (/dev/oracle/ocrvote5) [OCRVOTE]
    Located 5 voting disk(s).
    Start of resource "ora.cluster_interconnect.haip" failed
    CRS-2672: Attempting to start 'ora.cluster_interconnect.haip' on 'sph1erp'
    CRS-5017: The resource action "ora.cluster_interconnect.haip start" encountered the following error:
    Start action for HAIP aborted
    CRS-2674: Start of 'ora.cluster_interconnect.haip' on 'sph1erp' failed
    CRS-2679: Attempting to clean 'ora.cluster_interconnect.haip' on 'sph1erp'
    CRS-2681: Clean of 'ora.cluster_interconnect.haip' on 'sph1erp' succeeded
    CRS-4000: Command Start failed, or completed with errors.
    Failed to start Oracle Clusterware stack
    Failed to start High Availability IP at /oracle/11.2.0/grid/crs/install/crsconfig_lib.pm line 1046.
    */oracle/11.2.0/grid/perl/bin/perl -I/oracle/11.2.0/grid/perl/lib -I/oracle/11.2.0/grid/crs/install /oracle/11.2.0/grid/crs/install/rootcrs.pl execution failed*
    sph1erp:/oracle/11.2.0/grid #
    Last few lines from CRS Log for node 1, where error came
    [ctssd(6467)]CRS-2401:The Cluster Time Synchronization Service started on host sph1erp.
    2011-02-25 23:04:16.491
    [oracle/11.2.0/grid/bin/orarootagent.bin(6423)]CRS-5818:Aborted command 'start for resource: ora.cluster_interconnect.haip 1 1' for resource 'ora.cluster_int
    erconnect.haip'. Details at (:CRSAGF00113:) {0:0:178} in */oracle/11.2.0/grid/log/sph1erp/agent/ohasd/orarootagent_root/orarootagent_root.log.*
    2011-02-25 23:04:20.521
    [ohasd(5513)]CRS-2757:Command 'Start' timed out waiting for response from the resource 'ora.cluster_interconnect.haip'. Details at (:CRSPE00111:) {0:0:178} in
    */oracle/11.2.0/grid/log/sph1erp/ohasd/ohasd.log.*
    Few lines from */oracle/11.2.0/grid/log/sph1erp/agent/ohasd/orarootagent_root/orarootagent_root.log.*
    =====================================================================================================
    2011-02-25 23:04:16.823: [ USRTHRD][16] {0:0:178} Starting Probe for ip 169.254.74.54
    2011-02-25 23:04:16.823: [ USRTHRD][16] {0:0:178} Transitioning to Probe State
    2011-02-25 23:04:17.177: [ USRTHRD][15] {0:0:178} [NetHAMain] thread stopping
    2011-02-25 23:04:17.177: [ USRTHRD][15] {0:0:178} Thread:[NetHAMain]isRunning is reset to false here
    2011-02-25 23:04:17.178: [ USRTHRD][12] {0:0:178} Thread:[NetHAMain]stop }
    2011-02-25 23:04:17.178: [ USRTHRD][12] {0:0:178} thread cleaning up
    2011-02-25 23:04:17.178: [ USRTHRD][12] {0:0:178} pausing thread
    2011-02-25 23:04:17.178: [ USRTHRD][12] {0:0:178} posting thread
    2011-02-25 23:04:17.178: [ USRTHRD][12] {0:0:178} Thread:[NetHAWork]stop {
    2011-02-25 23:04:17.645: [ USRTHRD][16] {0:0:178} [NetHAWork] thread stopping
    2011-02-25 23:04:17.645: [ USRTHRD][16] {0:0:178} Thread:[NetHAWork]isRunning is reset to false here
    2011-02-25 23:04:17.645: [ USRTHRD][12] {0:0:178} Thread:[NetHAWork]stop }
    2011-02-25 23:04:17.645: [ USRTHRD][12] {0:0:178} Thread:[NetHAWork]stop {
    2011-02-25 23:04:17.645: [ USRTHRD][12] {0:0:178} Thread:[NetHAWork]stop }
    2011-02-25 23:04:17.891: [ora.cluster_interconnect.haip][12] {0:0:178} [start] Start of HAIP aborted
    2011-02-25 23:04:17.892: [ AGENT][12] {0:0:178} UserErrorException: Locale is
    2011-02-25 23:04:17.893: [ora.cluster_interconnect.haip][12] {0:0:178} [start] clsnUtils::error Exception type=2 string=
    CRS-5017: The resource action "ora.cluster_interconnect.haip start" encountered the following error:
    Start action for HAIP aborted
    2011-02-25 23:04:17.893: [ AGFW][12] {0:0:178} sending status msg [CRS-5017: The resource action "ora.cluster_interconnect.haip start" encountered the foll
    owing error:
    Start action for HAIP aborted
    ] for start for resource: ora.cluster_interconnect.haip 1 1
    2011-02-25 23:04:17.893: [ora.cluster_interconnect.haip][12] {0:0:178} [start] clsn_agent::start }
    2011-02-25 23:04:17.894: [ AGFW][10] {0:0:178} Agent sending reply for: RESOURCE_START[ora.cluster_interconnect.haip 1 1] ID 4098:661
    2011-02-25 23:04:18.552: [ora.diskmon][12] {0:0:154} [check] DiskmonAgent::check {
    2011-02-25 23:04:18.552: [ora.diskmon][12] {0:0:154} [check] DiskmonAgent::check } - 0
    2011-02-25 23:04:19.573: [ AGFW][10] {0:0:154} Agent received the message: AGENT_HB[Engine] ID 12293:669
    2011-02-25 23:04:20.510: [ora.cluster_interconnect.haip][18] {0:0:178} [start] got lock
    2011-02-25 23:04:20.511: [ora.cluster_interconnect.haip][18] {0:0:178} [start] tryActionLock }
    2011-02-25 23:04:20.511: [ora.cluster_interconnect.haip][18] {0:0:178} [start] abort }
    2011-02-25 23:04:20.511: [ora.cluster_interconnect.haip][18] {0:0:178} [start] clsn_agent::abort }
    2011-02-25 23:04:20.511: [ AGFW][18] {0:0:178} Command: start for resource: ora.cluster_interconnect.haip 1 1 completed with status: TIMEDOUT
    2011-02-25 23:04:20.512: [ora.cluster_interconnect.haip][8] {0:0:178} [check] NetworkAgent::init enter {
    2011-02-25 23:04:20.513: [ora.cluster_interconnect.haip][8] {0:0:178} [check] NetworkAgent::init exit }
    2011-02-25 23:04:20.517: [ AGFW][10] {0:0:178} Agent sending reply for: RESOURCE_START[ora.cluster_interconnect.haip 1 1] ID 4098:661
    2011-02-25 23:04:20.519: [ USRTHRD][8] {0:0:178} Ocr Context init default level 23886304
    2011-02-25 23:04:20.519: [ default][8]clsvactversion:4: Retrieving Active Version from local storage.
    [ CLWAL][8]clsw_Initialize: OLR initlevel [70000]
    Few lines from */oracle/11.2.0/grid/log/sph1erp/ohasd/ohasd.log.*
    =====================================================================================================
    2011-02-25 23:04:21.627: [UiServer][30] {0:0:180} Done for ctx=6000000002604ce0
    2011-02-25 23:04:21.642: [UiServer][31] Closed: remote end failed/disc.
    2011-02-25 23:04:26.139: [ CLSINET][33]Returning NETDATA: 1 interfaces
    2011-02-25 23:04:26.139: [ CLSINET][33]# 0 Interface 'lan2',ip='10.10.16.50',mac='3c-4a-92-48-71-be',mask='255.255.255.240',net='10.10.16.48',use='cluster_int
    erconnect'
    2011-02-25 23:04:26.973: [UiServer][31] CS(60000000014b0790)set Properties ( root,60000000012e0260)
    2011-02-25 23:04:26.973: [UiServer][31] SS(6000000001372270)Accepted client connection: saddr =(ADDRESS=(PROTOCOL=ipc)(DEV=92)(KEY=OHASD_UI_SOCKET))daddr = (A
    DDRESS=(PROTOCOL=ipc)(KEY=OHASD_UI_SOCKET))
    2011-02-25 23:04:26.992: [UiServer][30] {0:0:181} processMessage called
    2011-02-25 23:04:26.993: [UiServer][30] {0:0:181} Sending message to PE. ctx= 6000000001b440f0
    2011-02-25 23:04:26.993: [UiServer][30] {0:0:181} Sending command to PE: 67
    2011-02-25 23:04:26.994: [ CRSPE][29] {0:0:181} Processing PE command id=173. Description: [Stat Resource : 600000000135f760]
    2011-02-25 23:04:26.997: [UiServer][30] {0:0:181} Done for ctx=6000000001b440f0
    2011-02-25 23:04:27.012: [UiServer][31] Closed: remote end failed/disc.
    2011-02-25 23:04:31.135: [ CLSINET][33]Returning NETDATA: 1 interfaces
    2011-02-25 23:04:31.135: [ CLSINET][33]# 0 Interface 'lan2',ip='10.10.16.50',mac='3c-4a-92-48-71-be',mask='255.255.255.240',net='10.10.16.48',use='cluster_int
    erconnect'
    2011-02-25 23:04:32.318: [UiServer][31] CS(60000000014b0790)set Properties ( root,60000000012e0260)
    2011-02-25 23:04:32.318: [UiServer][31] SS(6000000001372270)Accepted client connection: saddr =(ADDRESS=(PROTOCOL=ipc)(DEV=92)(KEY=OHASD_UI_SOCKET))daddr = (A
    DDRESS=(PROTOCOL=ipc)(KEY=OHASD_UI_SOCKET))
    2011-02-25 23:04:32.332: [UiServer][30] {0:0:182} processMessage called
    2011-02-25 23:04:32.333: [UiServer][30] {0:0:182} Sending message to PE. ctx= 6000000001b45ef0
    2011-02-25 23:04:32.333: [UiServer][30] {0:0:182} Sending command to PE: 68
    2011-02-25 23:04:32.334: [ CRSPE][29] {0:0:182} Processing PE command id=174. Description: [Stat Resource : 600000000135f760]
    2011-02-25 23:04:32.338: [UiServer][30] {0:0:182} Done for ctx=6000000001b45ef0
    2011-02-25 23:04:32.352: [UiServer][31] Closed: remote end failed/disc.
    2011-02-25 23:04:36.155: [ CLSINET][33]Returning NETDATA: 1 interfaces
    2011-02-25 23:04:36.155: [ CLSINET][33]# 0 Interface 'lan2',ip='10.10.16.50',mac='3c-4a-92-48-71-be',mask='255.255.255.240',net='10.10.16.48',use='cluster_int
    erconnect'
    2011-02-25 23:04:37.683: [UiServer][31] CS(60000000014b0790)set Properties ( root,60000000012e0260)
    2011-02-25 23:04:37.683: [UiServer][31] SS(6000000001372270)Accepted client connection: saddr =(ADDRESS=(PROTOCOL=ipc)(DEV=92)(KEY=OHASD_UI_SOCKET))daddr = (A
    DDRESS=(PROTOCOL=ipc)(KEY=OHASD_UI_SOCKET))
    2011-02-25 23:04:37.702: [UiServer][30] {0:0:183} processMessage called
    2011-02-25 23:04:37.703: [UiServer][30] {0:0:183} Sending message to PE. ctx= 6000000002604ce0
    2011-02-25 23:04:37.703: [UiServer][30] {0:0:183} Sending command to PE: 69
    2011-02-25 23:04:37.704: [ CRSPE][29] {0:0:183} Processing PE command id=175. Description: [Stat Resource : 600000000135f760]
    2011-02-25 23:04:37.708: [UiServer][30] {0:0:183} Done for ctx=6000000002604ce0
    2011-02-25 23:04:37.722: [UiServer][31] Closed: remote end failed/disc.
    2011-02-25 23:04:41.156: [ CLSINET][33]Returning NETDATA: 1 interfaces
    2011-02-25 23:04:41.156: [ CLSINET][33]# 0 Interface 'lan2',ip='10.10.16.50',mac='3c-4a-92-48-71-be',mask='255.255.255.240',net='10.10.16.48',use='cluster_int
    erconnect'
    What could be the issue ????
    Experts Please help me. Doing setup for the PRoduction Env...
    Do response ASAP...... Thanks
    Regards,
    Manish

    Thanks Sebastian for your input.
    yes. my lan2 is used for Cluster_interconnect which is having subnet 255.255.255.240.
    Below are IPs used for RAC
    Public
    Node1: 10.10.1.173/255.255.240.0
    Node2: 10.10.1.174/255.255.240.0
    Private
    Node1: 10.10.16.50/255.255.255.240
    Node2: 10.10.16.51/255.255.255.240
    Virtual
    Node1: 10.10.1.191/255.255.240.0
    Node2: 10.10.1.192/255.255.240.0
    SCAN (Defined in DNS)
    10.10.1.193/255.255.240.0
    10.10.1.194/255.255.240.0
    10.10.1.195/255.255.240.0
    As you said, I will scrap GI Software again & will try with 255.255.255.0.
    I Believe this Redundant Interconnect and ora.cluster_interconnect.haip present in 11.2.0.2.0 Version.
    Oracle says:
    Redundant Interconnect without any 3rd-party IP failover technology (bond, IPMP or similar) is supported natively by Grid Infrastructure starting from 11.2.0.2. Multiple private network adapters can be defined either during the installation phase or afterward using the oifcfg. Oracle Database, CSS, OCR, CRS, CTSS, and EVM components in 11.2.0.2 employ it automatically.
    Grid Infrastructure can activate a maximum of four private network adapters at a time even if more are defined. The ora.cluster_interconnect.haip resource will start one to four link local HAIP on private network adapters for interconnect communication for Oracle RAC, Oracle ASM, and Oracle ACFS etc.
    Grid automatically picks link local addresses from reserved 169.254.*.* subnet for HAIP, and it will not attempt to use any 169.254.*.* address if it's already in use for another purpose. With HAIP, by default, interconnect traffic will be load balanced across all active interconnect interfaces, and corresponding HAIP address will be failed over transparently to other adapters if one fails or becomes non-communicative. .
    The number of HAIP addresses is decided by how many private network adapters are active when Grid comes up on the first node in the cluster . If there's only one active private network, Grid will create one; if two, Grid will create two; and if more than two, Grid will create four HAIPs. The number of HAIPs won't change even if more private network adapters are activated later, a restart of clusterware on all nodes is required for new adapters to become effective.
    In my Setup, I am having Teaming for NIC's for Public & Private Interface. So I am thinking to break teaming of NICs because HAIP internally searching for next available NIC & not getting as all 4 are already in used with OS level NIC teaming.
    My only Concern is, as I am going to change subnet for the Private IPs, should I change Private IP address ????
    Thanks for the Support...
    Regards,
    Manish

  • Database startup after reboot of RAC server

    Hello,
    My config : 2 nodes w2k3 with 15 dbs . Oracle 10.2.0.3
    I started to scheduled my rac server reboot so i follow the oracle doc to shutdown properly all db,asm,service,listener,etc...
    Today one the server reboot, but some instance doesn't start (only five , and the last five in alphabetic order)
    In the log :
    db log : nothing just the shutdown
    crs_log : the start of the 10 db but nothing for the last five !!
    just a trace on one listener service :
    StartResource error for ora.sqyora01.LISTENER_1522_SQYORA01.lsnr error code = 1
    2010-08-11 02:56:08.000: [  CRSRES][3008]32Start of `ora.sqyora01.LISTENER_1522_SQYORA01.lsnr` on member `sqyora01` failed.
    2010-08-11 02:56:08.125: [  CRSRES][3008]32Skip online resource: ora.sqyora01.LISTENER_SQYORA01.lsnr
    On windows services : the OracleServiceSID service isn't started.
    Any help is welcome to understand this problem.
    Thanks.

    user4511076 wrote:
    IMO : (i am not a great oracle dba, because i have not enough time to spend with, and my collegue need really simple think to do, like startup/shutdown a database, no more investigation)Why startup/shutdown a database? I have databases that work 24x7 and uptimes of over a year. The record so far, has been over 24 months uptime (1st downtime were caused by a powerfailure to the rack cabinet). And for a database instance that runs a number of processes and deals with 100's to 1000's of inserts per second.
    it's easier to view that an oracle process take more cpu and so investigate in that database what's appens. Rather than search in my v$sql table to find witch schema doing wrong thinksNot sure what you are trying to say here. Single or multiple database instances do not change how a session looks like, or how SQLs are executed.. and thus not how you troubleshoot problems. Nor does it change resource requirements, or how you address these. But multiple instances do impact resource availability and requires one to split resources amongst instances - the end result of this is less flexibility.
    It's easier to affect cpu ressource to a process, rather than configuring it in oraclePlease explain.
    It's easier to stop a database rather than "put offline a schema"This is a "silly" statement to make IMO. Why do you want to offline a schema? Availability of applications and data are a critical feature in today's information system landscape. So why argue the complete opposite and say it is now more complex to make something less available?
    By the same token, why do you want to offline a database? A down database is even less useful than a downed server - at least you can use the downed server as a doorstop.. ;-)
    And if a database crash it only affect one application.Yeah.. and these happens when? Every few minutes? Every hour? Once a day?
    This is not a valid argument. Oracle instances very seldom crashes just for the heck of it. Most often it is due to incorrect o/s configuration, problematic hardware, uncertified components, old drivers, etc. Or plain old application abuse of Oracle.
    There is the theorical approach and practical one : I'm not alone is that case :
    http://oracledoug.com/serendipity/index.php?/archives/1339-The-Reality-Gap-3-A-Single-Instance-per-Server.html
    http://www.dba-oracle.com/art_dbazine_server_consolidation.htm
    I have never done RDBMS as a theoretical thing. The most dynamic table (ito SQLs against it) I have, grows by more than 450 million rows per day. This is as real world as real world gets. And that is what shapes my experience and opinion. And Burleson and I have never seen eye to eye on a number of subjects - so quoting his views on consolidation does little to persuade me to change my opinion.
    have you any ideas where can i found an error message that explain why my database doesn't start ? (like not enough ressource ;-) )Troubleshooting a problem starts with a very basic principle - isolation. Isolate the layer or moving part where the problem occurs. As the other instances are running, it means that the CRS software layer is up and running, that the storage layer is up and running and so on. So the problem should be with the instance that for some reason refuses to start? Confirm this by using sqlplus to start a down instance and looking at what its alert log file says. It should have some kind of pointer to what dependency that has not been met.

  • Please help with RMAN dup for RAC db

    Happy Holidays!! All,
    I need help with the following issue:
    Both target and auxiliary databases are:
    Database version is 10.2.0.3 with 2-node RAC db
    Servers are MS 2003
    Oracle Clusterware 10.2.0.3
    Oracle ASM for storage
    I have run the rman dup from RAC to a single instance db for a while. This is the first time I ran from RAC to RAC. This auxiliary db was refreshed before and was up for a while and was managed by server control tool. Now I am trying to refresh it with new backup files from production. I did the following as people suggested:
    1)     shutdown instance in the second node.
    2)     Startup nomount mode in the first node
    3)     Run RMAN dup
    It failed in the same place with slightly different errors. I pasted error messages at the bottom. I already created a TAR with Oracle but would like to know if anybody has any info related to this. Also I have several questions to ask:
    1)     Should I stop all services related to Oracle in node-2
    2)     Should I stop all clustware related services in node-1 except oracleCSServie since ASM uses it?
    3)     Is there any way I can finish the dup manually from the place I failed? What going to happen if I don’t finish the rest of the dup and go ahead with “alter database open resetlogs”?
    4)     How do I remove the database from under Server control – I already run “srvctl remove database -d atlrac” successfully. Is this all?
    Thanks a lot for your help and have a great holiday season!!!
    First time run:
    contents of Memory Script:
    shutdown clone;
    startup clone nomount ;
    executing Memory Script
    database dismounted
    Oracle instance shut down
    connected to auxiliary database (not started)
    RMAN-00571: ====================================================
    RMAN-00569: ======= ERROR MESSAGE STACK FOLLOWS ==============
    RMAN-00571: =============================================
    RMAN-03002: failure of Duplicate Db command at 12/21/2009 20:24:47
    RMAN-03015: error occurred in stored script Memory Script
    RMAN-04014: startup failed: ORA-00610: Internal error code
    Second time run:
    contents of Memory Script:
    shutdown clone;
    startup clone nomount ;
    executing Memory Script
    database dismounted
    Oracle instance shut down
    connected to auxiliary database (not started)
    RMAN-00571: ===================================================
    RMAN-00569: ========== ERROR MESSAGE STACK FOLLOWS ==========
    RMAN-00571: ===================================================
    RMAN-03002: failure of Duplicate Db command at 12/22/2009 15:53:27
    RMAN-03015: error occurred in stored script Memory Script
    RMAN-04014: startup failed: ORA-03113: end-of-file on communication channel
    Shirley

    1) Should I stop all services related to Oracle in node-2
    No, you just need to stop the second instance.
    2) Should I stop all clustware related services in node-1 except oracleCSServie since ASM uses it?
    No, you don't need to stop anything. Just have the instance startup nomount.
    3) Is there any way I can finish the dup manually from the place I failed? What going to happen if I don’t finish the rest of the dup and go ahead with “alter database open resetlogs”?
    You have not shown enough information. Did the restore succeed ? Did the recover succeed ? What was it doing when it failed ?
    4) How do I remove the database from under Server control – I already run “srvctl remove database -d atlrac” successfully. Is this all?
    Yes, srvctl remove database is all you need. (unless you also want to remove from listener.ora, tnsnames.ora, oratab etc)

Maybe you are looking for

  • Report not showing the range in from to option

    I am unble to get all records in my report.I am calling a report from form using run_product.I have taken two field varchar using TO and FROM .Som time report retrive right record but some time it spool only one (first one).Then i changes to betwwen

  • Getting Error Code 150:30 when opening Photoshop, how to fix?

    Getting Error Code 150:30 when opening Photoshop CS4, how to fix?

  • Experienced 11i Consultant

    Our national practice, specializing in Oracle Apps expertise has a limited number of experienced 11i consultants, available now. For resumes, skillsets, rates, and additional information, please respond to this message.

  • JSP & ASP / Same Server

    Hello, Is it going to be possible to run both JSP & ASP on the same server. The server is WindowsNT, ASP is already set up, running on IIS I want to move to JSP but I need to prove a few things to my boss before we scrap ASP all together. I am planni

  • Help please with Muvo TX

    I haven't used this for over a year now and it has some serious problems. . It doesn't detect the player when i plug it into the USB port2. When I turn on the Player it will say " File System Error "3. It won't show up as removable disk when I plug t