RAC + Kerberos

Hi !
I have rac 10g (10.1.0.4) RAC and standalone databases in Active Directory domain. All servers have MSWin2003ES SP1 OS. DCs, RAC and standalone DB located on different servers.
Are there any differences between RAC and standalone database enterprise security configuration over kerberos?
At this time:
Snadalone database - OK (sqlplus and ODBC connection)
RAC database - instance is not starting and "Decrypt Integrity Check Failed" error in server trace file when i trying to restart instance service ...
imho (according trace files) the client get ticket of service provider and send it to server but server cannt to decript this ticket ...
ps: also there are MSCS + OFS on my RAC servers.

resolved

Similar Messages

  • KERBEROS + RAC

    Hi !
    I have rac 10g (10.1.0.4) RAC and standalone databases in Active Directory domain. All servers have MSWin2003ES SP1 OS. DCs, RAC and standalone DB located on different servers.
    Are there any differences between RAC and standalone database enterprise security configuration over kerberos?
    At this time:
    Snadalone database - OK (sqlplus and ODBC connection)
    RAC database - instance is not starting and "Decrypt Integrity Check Failed" error in server trace file when i trying to restart instance service ...
    imho (according trace files) the client get ticket of service provider and send it to server but server cannt to decript this ticket ...
    ps: also there are MSCS + OFS on my RAC servers.

    the problem is resolved

  • Connecting to a database using Kerberos authentication

    I have a 10gR2 database for which I want to enable connection using Kerberos authenntication. I receive the following error:
    ORA-12637: Packet receive failed.
    In the sqlnet server trace I get the following error:
    nauk5ky_rd_req_decoded: Returning 31: Decrypt integrity check failed.
    Ideas anyone?

    The problem appears to be that one active directory user can act as service principal for only one database server (or one RAC instance).
    If more than one service principals are created (by ktpass utility) for the same AD user, only the last created one can successfully authenticate. Logins to all other servers are unsuccessful and result in ORA-12637 at the client and ORA-07445 [nauk5a3recvclientauth()+568] in the server alert log.
    The workaround is to create a separate AD user account for each database server (or RAC instance) authenticated by the kerberos.
    Edited by: dbr2 on 27-Feb-2009 07:18

  • Start up problem with Oracle RAC 11g

    Hi all,
    After a power failure I can not start a oracle database it give me the ORA-01031: insufficient privileges error
    [oracle@hostname ~]$ sqlplus /nolog
    SQL*Plus: Release 11.2.0.1.0 Production on Tue Jul 23 19:06:41 2013
    Copyright (c) 1982, 2009, Oracle.  All rights reserved.
    SQL> show sga
    Total System Global Area  283930624 bytes
    Fixed Size            2212656 bytes
    Variable Size          256552144 bytes
    ASM Cache           25165824 bytes
    SQL> disconnect
    Disconnected from Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - 64bit Production
    With the Real Application Clusters and Automatic Storage Management options
    SQL> connect as sysdba
    Enter user-name: system
    Enter password:
    Connected.
    SQL> startup
    ORA-01031: insufficient privileges
    SQL> quit
    Disconnected from Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - 64bit Production
    With the Real Application Clusters and Automatic Storage Management options
    [oracle@hostname ~]$ lsb_release -a
    LSB Version:    :core-4.0-amd64:core-4.0-ia32:core-4.0-noarch:graphics-4.0-amd64:graphics-4.0-ia32:graphics-4.0-noarch:printing-4.0-amd64:printing-4.0-ia32:printing-4.0-noarch
    Distributor ID:    EnterpriseEnterpriseServer
    Description:    Enterprise Linux Enterprise Linux Server release 5.7 (Carthage)
    Release:    5.7
    Codename:    Carthage
    [oracle@hostname ~]$
    [oracle@hostname ~]$ crs_stat -t
    Name           Type           Target    State     Host       
    ora.DATA.dg    ora....up.type ONLINE    ONLINE    sil-rac1   
    ora....ER.lsnr ora....er.type ONLINE    ONLINE    sil-rac1   
    ora....N1.lsnr ora....er.type ONLINE    ONLINE    sil-rac2   
    ora....N2.lsnr ora....er.type ONLINE    ONLINE    sil-rac1   
    ora....N3.lsnr ora....er.type ONLINE    ONLINE    sil-rac1   
    ora.asm        ora.asm.type   ONLINE    ONLINE    sil-rac1   
    ora.eons       ora.eons.type  ONLINE    ONLINE    sil-rac1   
    ora.gsd        ora.gsd.type   OFFLINE   OFFLINE              
    ora....network ora....rk.type ONLINE    ONLINE    sil-rac1   
    ora.oc4j       ora.oc4j.type  ONLINE    ONLINE    sil-rac2   
    ora.ons        ora.ons.type   ONLINE    ONLINE    sil-rac1   
    ora.orcl.db    ora....se.type ONLINE    ONLINE    sil-rac1   
    ora.scan1.vip  ora....ip.type ONLINE    ONLINE    sil-rac2   
    ora.scan2.vip  ora....ip.type ONLINE    ONLINE    sil-rac1   
    ora.scan3.vip  ora....ip.type ONLINE    ONLINE    sil-rac1   
    ora....SM1.asm application    ONLINE    ONLINE    sil-rac1   
    ora....C1.lsnr application    ONLINE    ONLINE    sil-rac1   
    ora....ac1.gsd application    OFFLINE   OFFLINE              
    ora....ac1.ons application    ONLINE    ONLINE    sil-rac1   
    ora....ac1.vip ora....t1.type ONLINE    ONLINE    sil-rac1   
    ora....SM2.asm application    ONLINE    ONLINE    sil-rac2   
    ora....C2.lsnr application    ONLINE    ONLINE    sil-rac2   
    ora....ac2.gsd application    OFFLINE   OFFLINE              
    ora....ac2.ons application    ONLINE    ONLINE    sil-rac2   
    ora....ac2.vip ora....t1.type ONLINE    ONLINE    sil-rac2   
    [oracle@hostname ~]$
    [oracle@hostname ~]$ crsctl check crs
    CRS-4638: Oracle High Availability Services is online
    CRS-4537: Cluster Ready Services is online
    CRS-4529: Cluster Synchronization Services is online
    CRS-4533: Event Manager is online
    [oracle@hostname ~]$
    Does anyone know what can be the problem?
    Thanks,
    Andre Gil

    Thanks for your answer
    The OS user I'm log is ORACLE that own the filesystem, Before you ask ORACLE is part of OINSTALL and DBA group
    Any other suggestion
    [oracle@sil-rac1 ~]$ id
    uid=1100(oracle) gid=1200(dba) groups=4(adm),100(users),1000(oinstall),1200(dba)
    [oracle@sil-rac1 ~]$ env | sort
    BASE_PATH=/usr/sbin:/usr/kerberos/bin:/usr/local/bin:/bin:/usr/bin:/home/oracle/bin
    _=/bin/env
    CLASSPATH=/u01/app/11.2.0/grid/JRE:/u01/app/11.2.0/grid/jlib:/u01/app/11.2.0/grid/rdbms/jlib
    DB_HOME=/u01/app/oracle/product/11.2.0/db_1
    G_BROKEN_FILENAMES=1
    GRID_HOME=/u01/app/11.2.0/grid
    HISTSIZE=1000
    HOME=/home/oracle
    HOSTNAME=sil-rac1
    INPUTRC=/etc/inputrc
    LANG=en_US.UTF-8
    LD_LIBRARY_PATH=/u01/app/11.2.0/grid/lib:/lib:/usr/lib
    LESSOPEN=|/usr/bin/lesspipe.sh %s
    LOGNAME=oracle
    LS_COLORS=no=00:fi=00:di=00;34:ln=00;36:pi=40;33:so=00;35:bd=40;33;01:cd=40;33;01:or=01;05;37;41:mi=01;05;37;41:ex=00;32:*.cmd=00;32:*.exe=00;32:*.com=00;32:*.btm=00;32:*.bat=00;32:*.sh=00;32:*.csh=00;32:*.tar=00;31:*.tgz=00;31:*.arj=00;31:*.taz=00;31:*.lzh=00;31:*.zip=00;31:*.z=00;31:*.Z=00;31:*.gz=00;31:*.bz2=00;31:*.bz=00;31:*.tz=00;31:*.rpm=00;31:*.cpio=00;31:*.jpg=00;35:*.gif=00;35:*.bmp=00;35:*.xbm=00;35:*.xpm=00;35:*.png=00;35:*.tif=00;35:
    MAIL=/var/spool/mail/oracle
    ORACLE_BASE=/u01/app/oracle
    ORACLE_HOME=/u01/app/11.2.0/grid
    ORACLE_HOSTNAME=sil-rac1.silanis.com
    ORACLE_SID=+ASM1
    ORACLE_TERM=xterm
    ORACLE_UNQNAME=RAC
    PATH=/u01/app/11.2.0/grid/bin:/usr/sbin:/usr/kerberos/bin:/usr/local/bin:/bin:/usr/bin:/home/oracle/bin
    PWD=/home/oracle
    SHELL=/bin/bash
    SHLVL=1
    SSH_ASKPASS=/usr/libexec/openssh/gnome-ssh-askpass
    TERM=xterm
    TMPDIR=/tmp
    TMP=/tmp
    USER=oracle
    [oracle@sil-rac1 ~]$

  • RAC Database

    Hello,
    While installing RAC database binaries , Following error was reported into installation logs and installation hang.
    INFO: ORACLE_HOME = /u01/oracle/app/oracle/product/11.2.0
    INFO: PATH = /usr/bin:/usr/ccs/bin:/usr/kerberos/bin:/usr/local/bin:/usr/bin:/bin:/usr/X11R6/bin:/home/oracle/bin:=/usr/kerberos/bin:/usr/local/bin:/usr/bin:/bin:/usr/X11R6/bin:/home/oracle/bin:/home/oracle/bin:/u01/oracle/app/oracle/product/11.2.0/bin:/u01/oracle/app/oracle/product/11.2.0/OPatch:/usr/java/jdk1.6.0_18:/grid/oracle/app/oracle/product/11.2.0/bin:/home/oracle/Desktop/database/install
    INFO: CLASSPATH =
    INFO: Username:oracle
    INFO: Oracle Universal Installer version is 11.2.0.1.0
    INFO: Initializing the OiicInstallAPISession
    INFO: Shutting down OUISetupDriver.JobExecutorThread
    *SEVERE: [FATAL] [INS-00001] Unknown irrecoverable error*
    CAUSE: No additional information available.
    ACTION: Refer to the logs or contact Oracle Support Services
    SUMMARY:
    - java.lang.NullPointerException.
    Refer associated stacktrace #oracle.install.commons.util.exception.DefaultErrorAdvisor:2094
    RAC setup detail :
    Host OS: Windows 7
    Guest os : OEL5
    -> GI installation & root.sh went successfully on both nodes.
    Thank you.

    Whenever you come across this kind of problems, Just raise an SR. Oracle will respond very quickly and could solve your issue in a cleaner way...
    Regards,
    CSM

  • Problem with Oracle RAC with DRCP and persistent connections

    Hello,
    I am having a problem that has me stumped. I cannot get SCAN, and DRCP to work together nicely.
    Right now, I can connect through my scan hostname fine without connection pooling, once I enable it I start having connection time outs. I found if I specified outbound_connection_timeout in sqlnet.ora that it times the connection out to the time I set in there. If I don't set this option I get a "ORA-12170: TNS:Connect timeout occurred".
    The odd thing is I can leave connection pooling on and connect to one of my rac instances individually and it doesn't experience this problem. The only time it pops up is if I combine multiple rac instances (Through the use of SCAN or tnsnames.ora) and I enable connection pooling.
    I don't know which direction to go to even diagnose this, so any and all help is appreciated.
    Thank you.
    Edited by: Rarp on Oct 1, 2011 10:47 PM

    Hi,
    You need to detail the problem. Enable TRACE SQL*NET on the server and client using note below.
    How to Enable Oracle SQLNet Client , Server , Listener , Kerberos and External procedure Tracing from Net Manager [ID 395525.1]
    Also you can try check this note:
    11g: ORA-12170 With Combination RAC, DRCP and a Firewall [ID 953277.1]
    Hope this helps,
    Levi Pereira

  • MULTIPLE USERS 10G RAC ORACLE_HOME INSTALL WITH ASM/CRS

    Hi,
    We need to install multiple 10g RAC databases on a two node Sun servers. Below is our configuration:
    1) Sun Solaris (ver 10) with Sun Cluster 3.2
    2) One ASM/CRS install (by 1 OS account)
    3) Four ORACLE_HOME 10g database install (by 4 different OS user accounts)
    We would like to use one ASM instance for all four databases with appropriate privileges.
    OS User:           OS Group
    ========      =========
    oraasm           dbaasm - (ASM and CRS install owner)
    ora1           dbaora1 - first db owner
    ora2           dbaora2 - second db owner
    ora3           dbaora3 - third db owner
    ora4           dbaora4 - fourth db owner
    I understand that certain privileges need to be shared between ASM/CRS and DB owners. Please let me know the steps to be followed to complete this install.
    Thanks in advance.

    Hi
    Please read that: Documentation http://download.oracle.com/docs/html/B10766_08/intro.htm
    - You can install and operate multiple Oracle homes and different versions of Oracle cluster database software on the same computer as described in the following points:
    -You can install multiple Oracle Database 10g RAC homes on the same node. The multiple homes feature enables you to install one or more releases on the same machine in multiple Oracle home directories. However, each node can have only one CRS home.
    -In addition, you cannot install Oracle Database 10g RAC into an existing single-instance Oracle home. If you have an Oracle home for Oracle Database 10g, then use a different Oracle home, and one that is available across the entire cluster for your new installation. Similarly, if you have an Oracle home for an earlier Oracle cluster database software release, then you must also use a different home for the new installation.
    If the OUI detects an earlier version of a database, then the OUI asks you about your upgrade preferences. You have the option to upgrade one of the previous-version databases with DBUA or to create a new database using DBCA. The information collected during this dialog is passed to DBUA or DBCA after the software is installed.
    - You can use the OUI to complete some of the de-install and re-install steps for Oracle Database 10g Real Application Clusters if needed.
    Note:
    Do not move Oracle binaries from one Oracle home to another because this causes dynamic link failures.
    . If you are using ASM with Oracle database instances from multiple database homes on the same node, then Oracle recommends that you run the ASM instance from an Oracle home that is distinct from the database homes. In addition, the ASM home should be installed on every cluster node. This prevents the accidental removal of ASM instances that are in use by databases from other homes during the de-installation of a database's Oracle home.

  • Error while running runcluvfy.sh(11g RAC on CentOS 5(RHEL 5))

    Oracle Version: 11G
    Operating System: Centos 5 (RHEL 5) : Linux centos51-rac-1 2.6.18-128.1.6.el5 #1 SMP Wed Apr 1 09:19:18 EDT 2009 i686 i686 i386 GNU/Linux
    Question (including full error messages and setup scripts where applicable):
    I am attempting to install oracle 11g in a RAC configuration with Centos 5 (redhat 5) as the operating system. I get the following error
    ERROR : Cannot Identify the operating system. Ensure that the correct software is being executed for this operating system
    Verification cannot complete
    I get this error message when I run runcluvfy.sh, to verify the my configuration is clusterable. I don't know why.
    I edited the /etc/redhat-release and entered echo "Red Hat Enterprise Linux AS release 4 (Nahant Update 7)" to attempt to fool the installer into thinking its red hat 4.
    But still shows the same message.
    Anyone knows how to fix this ?
    Please help me.

    http://www.idevelopment.info/data/Oracle/DBA_tips/Linux/LINUX_20.shtml
    runcluvfy.sh will not work on centos because the cluster verification utility checks the operating system version using the redhat-release packag and centos do this with his packages, so you must install and use redhat-release package
    Get rpm-build to be able to build rpm’s:
    [root@centos5 ~]# yum install rpm-build
    Get source rpm of redhat-release
    [root@centos5 ~]# wget ftp://ftp.redhat.com/pub/redhat/linux/enterprise/5Server/en/os/SRPMS/redhat-release-5Server-5.1.0.2.src.rpm
    Build package:
    [root@centos5 ~]# rpmbuild –rebuild redhat-release-5Server-5.1.0.2.src.rpm
    Install newly generated rpm:
    [root@centos5 ~]# rpm -Uvh –force /usr/src/redhat/RPMS/i386/redhat-release-5Server-5.1.0.2.i386.rpm

  • Error in Creation of Dataguard for RAC

    My pfile of RAC looks like:
    RACDB2.__large_pool_size=4194304
    RACDB1.__large_pool_size=4194304
    RACDB2.__shared_pool_size=92274688
    RACDB1.__shared_pool_size=92274688
    RACDB2.__streams_pool_size=0
    RACDB1.__streams_pool_size=0
    *.audit_file_dest='/u01/app/oracle/admin/RACDB/adump'
    *.background_dump_dest='/u01/app/oracle/admin/RACDB/bdump'
    *.cluster_database_instances=2
    *.cluster_database=true
    *.compatible='10.2.0.1.0'
    *.control_files='+DATA/racdb/controlfile/current.260.627905745','+FLASH/racdb/controlfile/current.256.627905753'
    *.core_dump_dest='/u01/app/oracle/admin/RACDB/cdump'
    *.db_block_size=8192
    *.db_create_file_dest='+DATA'
    *.db_domain=''
    *.db_file_multiblock_read_count=16
    *.db_file_name_convert='+DATA/RACDB','+DATADG/RACDG'
    *.db_name='RACDB'
    *.db_recovery_file_dest='+FLASH'
    *.db_recovery_file_dest_size=2147483648
    *.dispatchers='(PROTOCOL=TCP) (SERVICE=RACDBXDB)'
    *.fal_client='RACDB'
    *.fal_server='RACDG'
    RACDB1.instance_number=1
    RACDB2.instance_number=2
    *.job_queue_processes=10
    *.log_archive_config='DG_CONFIG=(RACDB,RACDG)'
    *.log_archive_dest_1='LOCATION=+FLASH/RACDB/ VALID_FOR=(ALL_LOGFILES,ALL_ROLES) DB_UNIQUE_NAME=RACDB'
    *.log_archive_dest_2='SERVICE=RACDG VALID_FOR=(ONLINE_LOGFILES,PRIMARY_ROLE) DB_UNIQUE_NAME=RACDG'
    *.log_archive_dest_state_1='ENABLE'
    *.log_archive_dest_state_2='DEFER'
    *.log_archive_format='%t_%s_%r.arc'
    *.log_file_name_convert='+DATA/RACDB','+DATADG/RACDG'
    *.open_cursors=300
    *.pga_aggregate_target=16777216
    *.processes=150
    *.remote_listener='LISTENERS_RACDB'
    *.remote_login_passwordfile='exclusive'
    *.service_names='RACDB'
    *.sga_target=167772160
    *.standby_file_management='AUTO'
    RACDB2.thread=2
    RACDB1.thread=1
    *.undo_management='AUTO'
    RACDB2.undo_tablespace='UNDOTBS2'
    RACDB1.undo_tablespace='UNDOTBS1'
    *.user_dump_dest='/u01/app/oracle/admin/RACDB/udump'
    My pfile of Dataguard Instance in nomount state looks like:
    RACDG.__db_cache_size=58720256
    RACDG.__java_pool_size=4194304
    RACDG.__large_pool_size=4194304
    RACDG.__shared_pool_size=96468992
    RACDG.__streams_pool_size=0
    *.audit_file_dest='/u01/app/oracle/admin/RACDG/adump'
    *.background_dump_dest='/u01/app/oracle/admin/RACDG/bdump'
    ##*.cluster_database_instances=2
    ##*.cluster_database=true
    *.compatible='10.2.0.1.0'
    ##*.control_files='+DATA/RACDG/controlfile/current.260.627905745','+FLASH/RACDG/controlfile/current.256.627905753'
    *.core_dump_dest='/u01/app/oracle/admin/RACDG/cdump'
    *.db_block_size=8192
    *.db_create_file_dest='+DATADG'
    *.db_domain=''
    *.db_file_multiblock_read_count=16
    *.db_file_name_convert='+DATADG/RACDG','+DATA/RACDB'
    *.db_name='RACDB'
    *.db_recovery_file_dest='+FLASHDG'
    *.db_recovery_file_dest_size=2147483648
    *.dispatchers='(PROTOCOL=TCP) (SERVICE=RACDGXDB)'
    *.FAL_CLIENT='RACDG'
    *.FAL_SERVER='RACDB'
    *.job_queue_processes=10
    *.LOG_ARCHIVE_CONFIG='DG_CONFIG=(RACDB,RACDG)'
    *.log_archive_dest_1='LOCATION=+FLASHDG/RACDG/ VALID_FOR=(ALL_LOGFILES,ALL_ROLES) DB_UNIQUE_NAME=RACDG'
    *.log_archive_dest_2='SERVICE=RACDB VALID_FOR=(ONLINE_LOGFILES,PRIMARY_ROLE) DB_UNIQUE_NAME=RACDB'
    *.LOG_ARCHIVE_DEST_STATE_1='ENABLE'
    *.LOG_ARCHIVE_DEST_STATE_2='ENABLE'
    *.log_archive_format='%t_%s_%r.arc'
    *.log_file_name_convert='+DATADG/RACDG','+DATA/RACDB'
    *.open_cursors=300
    *.pga_aggregate_target=16777216
    *.processes=150
    ##*.remote_listener='LISTENERS_RACDG'
    *.remote_login_passwordfile='exclusive'
    SERVICE_NAMES='RACDG'
    sga_target=167772160
    standby_file_management='auto'
    undo_management='AUTO'
    undo_tablespace='UNDOTBS1'
    user_dump_dest='/u01/app/oracle/admin/RACDG/udump'
    DB_UNIQUE_NAME=RACDG
    and here is what I am doing on the standby location:
    [oracle@dg01 ~]$ echo $ORACLE_SID
    RACDG
    [oracle@dg01 ~]$ rman
    Recovery Manager: Release 10.2.0.1.0 - Production on Tue Jul 17 21:19:21 2007
    Copyright (c) 1982, 2005, Oracle. All rights reserved.
    RMAN> connect auxiliary /
    connected to auxiliary database: RACDG (not mounted)
    RMAN> connect target sys/xxxxxxx@RACDB
    connected to target database: RACDB (DBID=625522512)
    RMAN> duplicate target database for standby;
    Starting Duplicate Db at 2007-07-17 22:27:08
    using target database control file instead of recovery catalog
    allocated channel: ORA_AUX_DISK_1
    channel ORA_AUX_DISK_1: sid=156 devtype=DISK
    contents of Memory Script:
    restore clone standby controlfile;
    sql clone 'alter database mount standby database';
    executing Memory Script
    Starting restore at 2007-07-17 22:27:10
    using channel ORA_AUX_DISK_1
    channel ORA_AUX_DISK_1: starting datafile backupset restore
    channel ORA_AUX_DISK_1: restoring control file
    channel ORA_AUX_DISK_1: reading from backup piece /software/backup/ctl4.ctl
    channel ORA_AUX_DISK_1: restored backup piece 1
    piece handle=/software/backup/ctl4.ctl tag=TAG20070717T201921
    channel ORA_AUX_DISK_1: restore complete, elapsed time: 00:00:23
    output filename=+DATADG/racdg/controlfile/current.275.628208075
    output filename=+FLASHDG/racdg/controlfile/backup.268.628208079
    Finished restore at 2007-07-17 22:27:34
    sql statement: alter database mount standby database
    released channel: ORA_AUX_DISK_1
    RMAN-00571: ===========================================================
    RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
    RMAN-00571: ===========================================================
    RMAN-03002: failure of Duplicate Db command at 07/17/2007 22:27:43
    RMAN-05501: aborting duplication of target database
    RMAN-05001: auxiliary filename +DATA/racdb/datafile/undotbs2.265.627906771 conflicts with a file used by the target database
    RMAN-05001: auxiliary filename +DATA/racdb/datafile/example.264.627905917 conflicts with a file used by the target database
    RMAN-05001: auxiliary filename +DATA/racdb/datafile/users.259.627905395 conflicts with a file used by the target database
    RMAN-05001: auxiliary filename +DATA/racdb/datafile/sysaux.257.627905385 conflicts with a file used by the target database
    RMAN-05001: auxiliary filename +DATA/racdb/datafile/undotbs1.258.627905395 conflicts with a file used by the target database
    RMAN-05001: auxiliary filename +DATA/racdb/datafile/system.256.627905375 conflicts with a file used by the target database
    RMAN>
    Any help to clear this error will be apprecited.......
    Message was edited by:
    Bal
    null

    Hi
    Thanks everybody for helping me on this issue...........
    As suggested, I had taken the parameter log_file_name_convert and db_file_name_convert out of my RAC primary database but still I am getting the same error.
    Any help will be appriciated..............
    SQL> show parameter convert
    NAME TYPE VALUE
    db_file_name_convert string
    log_file_name_convert string
    SQL>
    oracle@dg01<3>:/u01/app/oracle> rman
    Recovery Manager: Release 10.2.0.1.0 - Production on Wed Jul 18 17:07:49 2007
    Copyright (c) 1982, 2005, Oracle. All rights reserved.
    RMAN> connect auxiliary /
    connected to auxiliary database: RACDB (not mounted)
    RMAN> connect target sys/xxx@RACDB
    connected to target database: RACDB (DBID=625522512)
    RMAN> duplicate target database for standby;
    Starting Duplicate Db at 2007-07-18 17:10:53
    using target database control file instead of recovery catalog
    allocated channel: ORA_AUX_DISK_1
    channel ORA_AUX_DISK_1: sid=156 devtype=DISK
    contents of Memory Script:
    restore clone standby controlfile;
    sql clone 'alter database mount standby database';
    executing Memory Script
    Starting restore at 2007-07-18 17:10:54
    using channel ORA_AUX_DISK_1
    channel ORA_AUX_DISK_1: starting datafile backupset restore
    channel ORA_AUX_DISK_1: restoring control file
    channel ORA_AUX_DISK_1: reading from backup piece /software/backup/ctl5.ctr
    channel ORA_AUX_DISK_1: restored backup piece 1
    piece handle=/software/backup/ctl5.ctr tag=TAG20070718T170529
    channel ORA_AUX_DISK_1: restore complete, elapsed time: 00:00:33
    output filename=+DATADG/racdg/controlfile/current.275.628208075
    output filename=+FLASHDG/racdg/controlfile/backup.268.628208079
    Finished restore at 2007-07-18 17:11:31
    sql statement: alter database mount standby database
    released channel: ORA_AUX_DISK_1
    RMAN-00571: ===========================================================
    RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
    RMAN-00571: ===========================================================
    RMAN-03002: failure of Duplicate Db command at 07/18/2007 17:11:43
    RMAN-05501: aborting duplication of target database
    RMAN-05001: auxiliary filename +DATA/racdb/datafile/undotbs2.265.627906771 conflicts with a file used by the target database
    RMAN-05001: auxiliary filename +DATA/racdb/datafile/example.264.627905917 conflicts with a file used by the target database
    RMAN-05001: auxiliary filename +DATA/racdb/datafile/users.259.627905395 conflicts with a file used by the target database
    RMAN-05001: auxiliary filename +DATA/racdb/datafile/sysaux.257.627905385 conflicts with a file used by the target database
    RMAN-05001: auxiliary filename +DATA/racdb/datafile/undotbs1.258.627905395 conflicts with a file used by the target database
    RMAN-05001: auxiliary filename +DATA/racdb/datafile/system.256.627905375 conflicts with a file used by the target database                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               

  • How to install 11gR2 RAC on 64 bit linux OS

    I am completely new to this topic of RAC and need to be installing and standing up RAC on Linux 64 bit OS . I have good knowledge of installing oracle database ENTERPRISE version 11gR2.
    Can you guide me as to how to start. I am looking for leads. Probably we will have 2 nodes.
    Thank you very much for helping me in advance

    If you are a My Oracle Support (Metalink) user, go check out these two notes created by the Oracle RAC Assurance Team. They are excellent.
    NOTE: 810394.1 RAC Assurance Support Team: RAC Starter Kit and Best Practices (Generic)
    NOTE: 811306.1 RAC Assurance Support Team: RAC Starter Kit (Linux)
    In the Linux note mentioned above there is a link to a Linux Step by Step Instruction Guide. This step by step instruction guide is the best start to finish document I've seen for how to set-up and install Oracle RAC. I believe the guide is written for installing release 11.2.0.2.

  • In oracle rac, If user query a select query and in processing data is fetched but in the duration of fetching the particular node is evicted then how failover to another node internally?

    In oracle rac, If user query a select query and in processing data is fetched but in the duration of fetching the particular node is evicted then how failover to another node internally?

    The query is re-issued as a flashback query and the client process can continue to fetch from the cursor. This is described in the Net Services Administrators Guide, the section on Transparent Application Failover.

  • RMAN, RAC, NFS, and server lock ups

    Good day. My environment is:
    --a 2-node RAC
    --Enterprise Edition 11.2.0.3
    --RHEL 5.1
    The goal is to use RMAN to push backups to a shared NFS mount (on a different server). Both nodes will have access to this location (in the event one node goes down, the other can still run backups). Easy, right?
    Wrong.
    I've tried every NFS mount option in the book. Most work just fine, some don't. When I use the recommended NFS mount options:
    rw,bg,hard,nointr,rsize=32768,wsize=32768,tcp, vers=3,timeo=600, actimeo=0
    or
    rw,bg,hard,nointr,rsize=32768,wsize=32768,proto=tcp,noac,forcedirectio, vers=3,suid
    The mount works normally. I can "ls" and "mkdir" and "touch" and "vi" and "cp" files back and forth from the NFS backup location to the RAC node all day long. No problems. However, when I try to do almost anything in RMAN which requires writing to the NFS backup location such as the command "backup archive all delete input;" (or even things as simple as a Crosscheck or RMAN configuration change which writes any changes back to the autobackup ControlFile) the node locks up. There are no errors (or if there are, I don't know where to find them), even when I use RMAN log.
    Just to recap: I run a Crosscheck (or any RMAN process that writes to the NFS backup location), the node will lock up, and I can let it sit for a day, inaccessible, with CRSCTL on the other node saying it's offline, and the node will never come out of a "frozen" state. It cannot be pinged or connected to.
    I think I can safely rule out NFS mount options at this point.
    I understand (after extensive reading of MOS docs and testing) that RAC RMAN can and does suffer from inefficient I/O when writing to an NFS mount. I don't think that's the culprit either. The autobackup ControlFile is not that big and I cannot see how running a simple Crosscheck would lock an entire node.
    I am hoping someone has encountered this in the past and hopefully it's just a simple misconfiguration somewhere.

    My NFS line in /etc/fstab is (these options are for supporting 11.2.0.3, 11.1.0.7, and 10.2.0.4/5 simultaneously): server.domain:/NFS_Export /backup nfs rw,bg,hard,nointr,rsize=32768,wsize=32768,tcp,actimeo=0,vers=3,timeo=600 0 0
    Before you installed GI, did you by chance do a yum update? I've encountered a similar issue which ended up being due to mkinitrd creating a corrupted kernel; mkinitrd is invoked during the GI installation when the ADVM drivers are added and in my case mkinitrd created a new kernel prior to the new kernel being installed. Second to that, make sure you have the matching kernel headers to your kernel version. If they are different then you could probably get away with just creating a new kernel with mkinitrd and relinking GI/RDBMS homes, but be prepared to wipe GI and reinstall.

  • Gns is getting failed with error CRS-2632  during RAC installation

    hello guys i am new to oracle RAC and i am trying to configure two node ORACLE 11G R2 RAC setup on OEL 5.4 using GNS Every things works great till I execute
    root.sh script on the first node
    It gives me error
    CRS-2674: Start of 'ora.gns' on 'host01' failed
    CRS-2632: There are no more servers to try to place resource 'ora.gns' on that would satisfy its placement policy
    start gns ... failed
    Preparing packages for installation...
    cvuqdisk-1.0.7-1
    Configure Oracle Grid Infrastructure for a Cluster ... failed
    Updating inventory properties for clusterware
    Starting Oracle Universal Installer...
    When i check status of cluster recourse i get this as output
    [root@host01 ~]# crs_stat -t
    Name           Type           Target      State Host
    ora.DATA.dg           ora....up.type      ONLINE ONLINE host01
    ora....N1.lsnr           ora....er.type      OFFLINE OFFLINE
    ora....N2.lsnr           ora....er.type      OFFLINE OFFLINE
    ora....N3.lsnr           ora....er.type      OFFLINE OFFLINE
    ora.asm           ora.asm.type      ONLINE ONLINE host01
    ora.eons           ora.eons.type      ONLINE ONLINE host01
    ora.gns           ora.gns.type      ONLINE OFFLINE
    ora.gns.vip           ora....ip.type      ONLINE OFFLINE
    ora.gsd           ora.gsd.type      OFFLINE OFFLINE
    ora....SM1.asm           application          ONLINE ONLINE host01
    ora.host01.gsd           application      OFFLINE OFFLINE
    ora.host01.ons           application      ONLINE ONLINE host01
    ora.host01.vip           ora....t1.type      ONLINE ONLINE host01
    ora....network           ora....rk.type      ONLINE ONLINE host01
    ora.oc4j           ora.oc4j.type      OFFLINE OFFLINE
    ora.ons           ora.ons.type      ONLINE ONLINE host01
    ora....ry.acfs           ora....fs.type      OFFLINE OFFLINE
    ora.scan1.vip           ora....ip.type      OFFLINE OFFLINE
    ora.scan2.vip           ora....ip.type      OFFLINE OFFLINE
    ora.scan3.vip           ora....ip.type      OFFLINE OFFLINE
    These are my GNS configuration file entries
    vi /var/named/chroot/etc/named.conf
    options {
         listen-on port 53 { 192.9.201.59; };
         listen-on-v6 port 53 { ::1; };
         directory      "/var/named";
         dump-file      "/var/named/data/cache_dump.db";
    statistics-file "/var/named/data/named_stats.txt";
    memstatistics-file "/var/named/data/named_mem_stats.txt";
         allow-query { any; };
         allow-query-cache { any; };
    zone "." IN {
         type hint;
         file "named.ca";
    zone "localdomain" IN {
         type master;
         file "localdomain.zone";
         allow-update { none; };
    zone "localhost" IN {
         type master;
         file "localhost.zone";
         allow-update { none; };
    zone "0.0.127.in-addr.arpa" IN {
         type master;
         file "named.local";
         allow-update { none; };
    zone "0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.ip6.arpa" IN {
    type master;
         file "named.ip6.local";
         allow-update { none; };
    zone "255.in-addr.arpa" IN {
         type master;
         file "named.broadcast";
         allow-update { none; };
    zone "0.in-addr.arpa" IN {
         type master;
         file "named.zero";
         allow-update { none; };
    zone "example.com" IN {
    type master;
    file "forward.zone";
         allow-transfer { 192.9.201.180; };
    zone "201.9.192.in-addr.arpa" IN {
    type master;
    file "reverse.zone";
    zone "0.0.10.in-addr.arpa" IN {
    type master;
    file "reverse1.zone";
    vi /var/named/chroot/var/named/forward.zone
    $TTL     86400
    @          IN SOA     server1.example.com. root.server1.example.com. (
                             42          ; serial (d. adams)
                             3H          ; refresh
                             15M          ; retry
                             1W          ; expiry
                             1D )          ; minimum
         IN NS          server1
              IN A          192.9.201.59
    server1          IN A          192.9.201.59
    host01          IN A          192.9.201.181
    host02          IN A          192.9.201.182
    host03          IN A          192.9.201.183
    openfiler     IN A          192.9.201.184
    host01-priv     IN A          10.0.0.2
    host02-priv     IN A          10.0.0.3
    host03-priv     IN A          10.0.0.4
    vi /var/named/chroot/var/named/reverse.zone
    $ORIGIN cluster01.example.com.
    @     IN     NS     cluster01-gns.cluster01.example.com.
    cluster01-gns     IN     A     192.9.201.180
    $TTL     86400
    @ IN SOA server1.example.com. root.server1.example.com. (
    1997022700 ; Serial
    28800 ; Refresh
    14400 ; Retry
    3600000 ; Expire
    86400 ) ; Minimum
    IN NS server1.example.com.
    59 IN PTR server1.example.com.
    184 IN PTR openfiler.example.com.
    181 IN PTR host01.example.com.
    182 IN PTR host02.example.com.
    183 IN PTR host03.example.com.
    vi /var/named/chroot/var/named/reverse1.zone
    $TTL     86400
    @ IN SOA server1.example.com. root.server1.example.com. (
    1997022700 ; Serial
    28800 ; Refresh
    14400 ; Retry
    3600000 ; Expire
    86400 ) ; Minimum
    IN NS server1.example.com.
    2 IN PTR host01-priv.example.com.
    3 IN PTR host02-priv.example.com.
    4 IN PTR host03-priv.example.com.
    Please suggest me what i am doing wrong
    Edited by: 1001408 on Apr 21, 2013 9:17 AM
    Edited by: 1001408 on Apr 21, 2013 9:22 AM

    Hello guys finally i find mistake i was doing
    while configuring Public Ip for the nodes i was not giving Default Gateway .I was assuming as all these machine is in same network with same Ip range so they would not be needing Gateway but here my assumption mismatch with oracle well finally happy to see 11G r2 with GNS running on my personal laptop.
    cheers
    Rahul

  • Oracel 11gR1 RAC Cluster issue

    We have 2-node Oracle 11gR2 RAC on HP-UX 11.31 environment. It was running lase 2 month without any issue.
    We got some netconfig issue, and node-1 got rebooted today. after the reboot cluster didn't not start on node-1, database is running on node-2.
    grid@hublhp4:/app/oracle/grid/product/11.2.0.1/log/hublhp4/crsd$ crsctl check crs
    CRS-4638: Oracle High Availability Services is online
    CRS-4535: Cannot communicate with Cluster Ready Services
    CRS-4529: Cluster Synchronization Services is online
    CRS-4534: Cannot communicate with Event Manager
    grid@hublhp4:/app/oracle/grid/product/11.2.0.1/log/hublhp4/crsd$ crs_stat -t
    CRS-0184: Cannot communicate with the CRS daemon.
    grid@hublhp4:/app/oracle/grid/product/11.2.0.1/log/hublhp4/crsd$ ocrcheck
    PROT-602: Failed to retrieve data from the cluster registry
    PROC-26: Error while accessing the physical storage ASM error [SLOS: cat=8, opn=kgfolclcpi1, dep=301, loc=kgfokge
    AMDU-00301: Unable to open file tmp-AMIPOCR01.ocr
    AMDU-00204: Disk N0002 is in currently mounted diskgroup AMIPOCR01
    AMDU-00201: Disk N0002: '/dev/rdisk/ora_OCR
    ] [8]
    grid@hublhp4:/app/oracle/grid/product/11.2.0.1/log/hublhp4/crsd$ olsnodes -n
    hublhp4 1
    hublhp5 2
    any idea please.
    Edited by: ManoRangasamy on Jul 5, 2011 6:38 PM

    Hi,
    Please post the alertlog ASM from node 1, crsd.log and ocssd.log from node 1
    It might be because node 1 can't see asm disk or permission accidentally changed when the node rebooted
    Cheers

  • Failed root.sh on 1st Node of 11.2.0.2.0 RAC on HP-UX 11.31 Itanium 64

    Started with 11.2.0.2.0 Grid Installation for 2 Node RAC on HP-UX 11.31 Itanium 64.
    Copying Software to remote node & linking libraries were successfully without any issue (upto 76%). But got issue while executing root.sh on Node1
    sph1erp:/oracle/11.2.0/grid #sh root.sh
    Running Oracle 11g root script...
    The following environment variables are set as:
    ORACLE_OWNER= oracle
    ORACLE_HOME= /oracle/11.2.0/grid
    Enter the full pathname of the local bin directory: [usr/local/bin]:
    Copying dbhome to /usr/local/bin ...
    Copying oraenv to /usr/local/bin ...
    Copying coraenv to /usr/local/bin ...
    Creating /etc/oratab file...
    Entries will be added to the /etc/oratab file as needed by
    Database Configuration Assistant when a database is created
    Finished running generic part of root script.
    Now product-specific root actions will be performed.
    Using configuration parameter file: /oracle/11.2.0/grid/crs/install/crsconfig_params
    Creating trace directory
    LOCAL ADD MODE
    Creating OCR keys for user 'root', privgrp 'sys'..
    Operation successful.
    OLR initialization - successful
    root wallet
    root wallet cert
    root cert export
    peer wallet
    profile reader wallet
    pa wallet
    peer wallet keys
    pa wallet keys
    peer cert request
    pa cert request
    peer cert
    pa cert
    peer root cert TP
    profile reader root cert TP
    pa root cert TP
    peer pa cert TP
    pa peer cert TP
    profile reader pa cert TP
    profile reader peer cert TP
    peer user cert
    pa user cert
    Adding daemon to inittab
    CRS-2672: Attempting to start 'ora.mdnsd' on 'sph1erp'
    CRS-2676: Start of 'ora.mdnsd' on 'sph1erp' succeeded
    CRS-2672: Attempting to start 'ora.gpnpd' on 'sph1erp'
    CRS-2676: Start of 'ora.gpnpd' on 'sph1erp' succeeded
    CRS-2672: Attempting to start 'ora.cssdmonitor' on 'sph1erp'
    CRS-2672: Attempting to start 'ora.gipcd' on 'sph1erp'
    CRS-2676: Start of 'ora.gipcd' on 'sph1erp' succeeded
    CRS-2676: Start of 'ora.cssdmonitor' on 'sph1erp' succeeded
    CRS-2672: Attempting to start 'ora.cssd' on 'sph1erp'
    CRS-2672: Attempting to start 'ora.diskmon' on 'sph1erp'
    CRS-2676: Start of 'ora.diskmon' on 'sph1erp' succeeded
    CRS-2676: Start of 'ora.cssd' on 'sph1erp' succeeded
    ASM created and started successfully.
    Disk Group OCRVOTE created successfully.
    clscfg: -install mode specified
    Successfully accumulated necessary OCR keys.
    Creating OCR keys for user 'root', privgrp 'sys'..
    Operation successful.
    CRS-4256: Updating the profile
    Successful addition of voting disk ab847ed2b4f04f2dbfb875226d2bb194.
    Successful addition of voting disk 85c05a5b30384f8dbff48cc069de7a7c.
    Successful addition of voting disk 649196fbdd614f9cbf26a9a0e6670a6e.
    Successful addition of voting disk 8815dfcee2e64f64bf00b9c76626ab41.
    Successful addition of voting disk 8ce55fe5534f4f77bfa9f54187592707.
    Successfully replaced voting disk group with +OCRVOTE.
    CRS-4256: Updating the profile
    CRS-4266: Voting file(s) successfully replaced
    ## STATE File Universal Id File Name Disk group
    1. ONLINE ab847ed2b4f04f2dbfb875226d2bb194 (/dev/oracle/ocrvote1) [OCRVOTE]
    2. ONLINE 85c05a5b30384f8dbff48cc069de7a7c (/dev/oracle/ocrvote2) [OCRVOTE]
    3. ONLINE 649196fbdd614f9cbf26a9a0e6670a6e (/dev/oracle/ocrvote3) [OCRVOTE]
    4. ONLINE 8815dfcee2e64f64bf00b9c76626ab41 (/dev/oracle/ocrvote4) [OCRVOTE]
    5. ONLINE 8ce55fe5534f4f77bfa9f54187592707 (/dev/oracle/ocrvote5) [OCRVOTE]
    Located 5 voting disk(s).
    Start of resource "ora.cluster_interconnect.haip" failed
    CRS-2672: Attempting to start 'ora.cluster_interconnect.haip' on 'sph1erp'
    CRS-5017: The resource action "ora.cluster_interconnect.haip start" encountered the following error:
    Start action for HAIP aborted
    CRS-2674: Start of 'ora.cluster_interconnect.haip' on 'sph1erp' failed
    CRS-2679: Attempting to clean 'ora.cluster_interconnect.haip' on 'sph1erp'
    CRS-2681: Clean of 'ora.cluster_interconnect.haip' on 'sph1erp' succeeded
    CRS-4000: Command Start failed, or completed with errors.
    Failed to start Oracle Clusterware stack
    Failed to start High Availability IP at /oracle/11.2.0/grid/crs/install/crsconfig_lib.pm line 1046.
    */oracle/11.2.0/grid/perl/bin/perl -I/oracle/11.2.0/grid/perl/lib -I/oracle/11.2.0/grid/crs/install /oracle/11.2.0/grid/crs/install/rootcrs.pl execution failed*
    sph1erp:/oracle/11.2.0/grid #
    Last few lines from CRS Log for node 1, where error came
    [ctssd(6467)]CRS-2401:The Cluster Time Synchronization Service started on host sph1erp.
    2011-02-25 23:04:16.491
    [oracle/11.2.0/grid/bin/orarootagent.bin(6423)]CRS-5818:Aborted command 'start for resource: ora.cluster_interconnect.haip 1 1' for resource 'ora.cluster_int
    erconnect.haip'. Details at (:CRSAGF00113:) {0:0:178} in */oracle/11.2.0/grid/log/sph1erp/agent/ohasd/orarootagent_root/orarootagent_root.log.*
    2011-02-25 23:04:20.521
    [ohasd(5513)]CRS-2757:Command 'Start' timed out waiting for response from the resource 'ora.cluster_interconnect.haip'. Details at (:CRSPE00111:) {0:0:178} in
    */oracle/11.2.0/grid/log/sph1erp/ohasd/ohasd.log.*
    Few lines from */oracle/11.2.0/grid/log/sph1erp/agent/ohasd/orarootagent_root/orarootagent_root.log.*
    =====================================================================================================
    2011-02-25 23:04:16.823: [ USRTHRD][16] {0:0:178} Starting Probe for ip 169.254.74.54
    2011-02-25 23:04:16.823: [ USRTHRD][16] {0:0:178} Transitioning to Probe State
    2011-02-25 23:04:17.177: [ USRTHRD][15] {0:0:178} [NetHAMain] thread stopping
    2011-02-25 23:04:17.177: [ USRTHRD][15] {0:0:178} Thread:[NetHAMain]isRunning is reset to false here
    2011-02-25 23:04:17.178: [ USRTHRD][12] {0:0:178} Thread:[NetHAMain]stop }
    2011-02-25 23:04:17.178: [ USRTHRD][12] {0:0:178} thread cleaning up
    2011-02-25 23:04:17.178: [ USRTHRD][12] {0:0:178} pausing thread
    2011-02-25 23:04:17.178: [ USRTHRD][12] {0:0:178} posting thread
    2011-02-25 23:04:17.178: [ USRTHRD][12] {0:0:178} Thread:[NetHAWork]stop {
    2011-02-25 23:04:17.645: [ USRTHRD][16] {0:0:178} [NetHAWork] thread stopping
    2011-02-25 23:04:17.645: [ USRTHRD][16] {0:0:178} Thread:[NetHAWork]isRunning is reset to false here
    2011-02-25 23:04:17.645: [ USRTHRD][12] {0:0:178} Thread:[NetHAWork]stop }
    2011-02-25 23:04:17.645: [ USRTHRD][12] {0:0:178} Thread:[NetHAWork]stop {
    2011-02-25 23:04:17.645: [ USRTHRD][12] {0:0:178} Thread:[NetHAWork]stop }
    2011-02-25 23:04:17.891: [ora.cluster_interconnect.haip][12] {0:0:178} [start] Start of HAIP aborted
    2011-02-25 23:04:17.892: [ AGENT][12] {0:0:178} UserErrorException: Locale is
    2011-02-25 23:04:17.893: [ora.cluster_interconnect.haip][12] {0:0:178} [start] clsnUtils::error Exception type=2 string=
    CRS-5017: The resource action "ora.cluster_interconnect.haip start" encountered the following error:
    Start action for HAIP aborted
    2011-02-25 23:04:17.893: [ AGFW][12] {0:0:178} sending status msg [CRS-5017: The resource action "ora.cluster_interconnect.haip start" encountered the foll
    owing error:
    Start action for HAIP aborted
    ] for start for resource: ora.cluster_interconnect.haip 1 1
    2011-02-25 23:04:17.893: [ora.cluster_interconnect.haip][12] {0:0:178} [start] clsn_agent::start }
    2011-02-25 23:04:17.894: [ AGFW][10] {0:0:178} Agent sending reply for: RESOURCE_START[ora.cluster_interconnect.haip 1 1] ID 4098:661
    2011-02-25 23:04:18.552: [ora.diskmon][12] {0:0:154} [check] DiskmonAgent::check {
    2011-02-25 23:04:18.552: [ora.diskmon][12] {0:0:154} [check] DiskmonAgent::check } - 0
    2011-02-25 23:04:19.573: [ AGFW][10] {0:0:154} Agent received the message: AGENT_HB[Engine] ID 12293:669
    2011-02-25 23:04:20.510: [ora.cluster_interconnect.haip][18] {0:0:178} [start] got lock
    2011-02-25 23:04:20.511: [ora.cluster_interconnect.haip][18] {0:0:178} [start] tryActionLock }
    2011-02-25 23:04:20.511: [ora.cluster_interconnect.haip][18] {0:0:178} [start] abort }
    2011-02-25 23:04:20.511: [ora.cluster_interconnect.haip][18] {0:0:178} [start] clsn_agent::abort }
    2011-02-25 23:04:20.511: [ AGFW][18] {0:0:178} Command: start for resource: ora.cluster_interconnect.haip 1 1 completed with status: TIMEDOUT
    2011-02-25 23:04:20.512: [ora.cluster_interconnect.haip][8] {0:0:178} [check] NetworkAgent::init enter {
    2011-02-25 23:04:20.513: [ora.cluster_interconnect.haip][8] {0:0:178} [check] NetworkAgent::init exit }
    2011-02-25 23:04:20.517: [ AGFW][10] {0:0:178} Agent sending reply for: RESOURCE_START[ora.cluster_interconnect.haip 1 1] ID 4098:661
    2011-02-25 23:04:20.519: [ USRTHRD][8] {0:0:178} Ocr Context init default level 23886304
    2011-02-25 23:04:20.519: [ default][8]clsvactversion:4: Retrieving Active Version from local storage.
    [ CLWAL][8]clsw_Initialize: OLR initlevel [70000]
    Few lines from */oracle/11.2.0/grid/log/sph1erp/ohasd/ohasd.log.*
    =====================================================================================================
    2011-02-25 23:04:21.627: [UiServer][30] {0:0:180} Done for ctx=6000000002604ce0
    2011-02-25 23:04:21.642: [UiServer][31] Closed: remote end failed/disc.
    2011-02-25 23:04:26.139: [ CLSINET][33]Returning NETDATA: 1 interfaces
    2011-02-25 23:04:26.139: [ CLSINET][33]# 0 Interface 'lan2',ip='10.10.16.50',mac='3c-4a-92-48-71-be',mask='255.255.255.240',net='10.10.16.48',use='cluster_int
    erconnect'
    2011-02-25 23:04:26.973: [UiServer][31] CS(60000000014b0790)set Properties ( root,60000000012e0260)
    2011-02-25 23:04:26.973: [UiServer][31] SS(6000000001372270)Accepted client connection: saddr =(ADDRESS=(PROTOCOL=ipc)(DEV=92)(KEY=OHASD_UI_SOCKET))daddr = (A
    DDRESS=(PROTOCOL=ipc)(KEY=OHASD_UI_SOCKET))
    2011-02-25 23:04:26.992: [UiServer][30] {0:0:181} processMessage called
    2011-02-25 23:04:26.993: [UiServer][30] {0:0:181} Sending message to PE. ctx= 6000000001b440f0
    2011-02-25 23:04:26.993: [UiServer][30] {0:0:181} Sending command to PE: 67
    2011-02-25 23:04:26.994: [ CRSPE][29] {0:0:181} Processing PE command id=173. Description: [Stat Resource : 600000000135f760]
    2011-02-25 23:04:26.997: [UiServer][30] {0:0:181} Done for ctx=6000000001b440f0
    2011-02-25 23:04:27.012: [UiServer][31] Closed: remote end failed/disc.
    2011-02-25 23:04:31.135: [ CLSINET][33]Returning NETDATA: 1 interfaces
    2011-02-25 23:04:31.135: [ CLSINET][33]# 0 Interface 'lan2',ip='10.10.16.50',mac='3c-4a-92-48-71-be',mask='255.255.255.240',net='10.10.16.48',use='cluster_int
    erconnect'
    2011-02-25 23:04:32.318: [UiServer][31] CS(60000000014b0790)set Properties ( root,60000000012e0260)
    2011-02-25 23:04:32.318: [UiServer][31] SS(6000000001372270)Accepted client connection: saddr =(ADDRESS=(PROTOCOL=ipc)(DEV=92)(KEY=OHASD_UI_SOCKET))daddr = (A
    DDRESS=(PROTOCOL=ipc)(KEY=OHASD_UI_SOCKET))
    2011-02-25 23:04:32.332: [UiServer][30] {0:0:182} processMessage called
    2011-02-25 23:04:32.333: [UiServer][30] {0:0:182} Sending message to PE. ctx= 6000000001b45ef0
    2011-02-25 23:04:32.333: [UiServer][30] {0:0:182} Sending command to PE: 68
    2011-02-25 23:04:32.334: [ CRSPE][29] {0:0:182} Processing PE command id=174. Description: [Stat Resource : 600000000135f760]
    2011-02-25 23:04:32.338: [UiServer][30] {0:0:182} Done for ctx=6000000001b45ef0
    2011-02-25 23:04:32.352: [UiServer][31] Closed: remote end failed/disc.
    2011-02-25 23:04:36.155: [ CLSINET][33]Returning NETDATA: 1 interfaces
    2011-02-25 23:04:36.155: [ CLSINET][33]# 0 Interface 'lan2',ip='10.10.16.50',mac='3c-4a-92-48-71-be',mask='255.255.255.240',net='10.10.16.48',use='cluster_int
    erconnect'
    2011-02-25 23:04:37.683: [UiServer][31] CS(60000000014b0790)set Properties ( root,60000000012e0260)
    2011-02-25 23:04:37.683: [UiServer][31] SS(6000000001372270)Accepted client connection: saddr =(ADDRESS=(PROTOCOL=ipc)(DEV=92)(KEY=OHASD_UI_SOCKET))daddr = (A
    DDRESS=(PROTOCOL=ipc)(KEY=OHASD_UI_SOCKET))
    2011-02-25 23:04:37.702: [UiServer][30] {0:0:183} processMessage called
    2011-02-25 23:04:37.703: [UiServer][30] {0:0:183} Sending message to PE. ctx= 6000000002604ce0
    2011-02-25 23:04:37.703: [UiServer][30] {0:0:183} Sending command to PE: 69
    2011-02-25 23:04:37.704: [ CRSPE][29] {0:0:183} Processing PE command id=175. Description: [Stat Resource : 600000000135f760]
    2011-02-25 23:04:37.708: [UiServer][30] {0:0:183} Done for ctx=6000000002604ce0
    2011-02-25 23:04:37.722: [UiServer][31] Closed: remote end failed/disc.
    2011-02-25 23:04:41.156: [ CLSINET][33]Returning NETDATA: 1 interfaces
    2011-02-25 23:04:41.156: [ CLSINET][33]# 0 Interface 'lan2',ip='10.10.16.50',mac='3c-4a-92-48-71-be',mask='255.255.255.240',net='10.10.16.48',use='cluster_int
    erconnect'
    What could be the issue ????
    Experts Please help me. Doing setup for the PRoduction Env...
    Do response ASAP...... Thanks
    Regards,
    Manish

    Thanks Sebastian for your input.
    yes. my lan2 is used for Cluster_interconnect which is having subnet 255.255.255.240.
    Below are IPs used for RAC
    Public
    Node1: 10.10.1.173/255.255.240.0
    Node2: 10.10.1.174/255.255.240.0
    Private
    Node1: 10.10.16.50/255.255.255.240
    Node2: 10.10.16.51/255.255.255.240
    Virtual
    Node1: 10.10.1.191/255.255.240.0
    Node2: 10.10.1.192/255.255.240.0
    SCAN (Defined in DNS)
    10.10.1.193/255.255.240.0
    10.10.1.194/255.255.240.0
    10.10.1.195/255.255.240.0
    As you said, I will scrap GI Software again & will try with 255.255.255.0.
    I Believe this Redundant Interconnect and ora.cluster_interconnect.haip present in 11.2.0.2.0 Version.
    Oracle says:
    Redundant Interconnect without any 3rd-party IP failover technology (bond, IPMP or similar) is supported natively by Grid Infrastructure starting from 11.2.0.2. Multiple private network adapters can be defined either during the installation phase or afterward using the oifcfg. Oracle Database, CSS, OCR, CRS, CTSS, and EVM components in 11.2.0.2 employ it automatically.
    Grid Infrastructure can activate a maximum of four private network adapters at a time even if more are defined. The ora.cluster_interconnect.haip resource will start one to four link local HAIP on private network adapters for interconnect communication for Oracle RAC, Oracle ASM, and Oracle ACFS etc.
    Grid automatically picks link local addresses from reserved 169.254.*.* subnet for HAIP, and it will not attempt to use any 169.254.*.* address if it's already in use for another purpose. With HAIP, by default, interconnect traffic will be load balanced across all active interconnect interfaces, and corresponding HAIP address will be failed over transparently to other adapters if one fails or becomes non-communicative. .
    The number of HAIP addresses is decided by how many private network adapters are active when Grid comes up on the first node in the cluster . If there's only one active private network, Grid will create one; if two, Grid will create two; and if more than two, Grid will create four HAIPs. The number of HAIPs won't change even if more private network adapters are activated later, a restart of clusterware on all nodes is required for new adapters to become effective.
    In my Setup, I am having Teaming for NIC's for Public & Private Interface. So I am thinking to break teaming of NICs because HAIP internally searching for next available NIC & not getting as all 4 are already in used with OS level NIC teaming.
    My only Concern is, as I am going to change subnet for the Private IPs, should I change Private IP address ????
    Thanks for the Support...
    Regards,
    Manish

Maybe you are looking for