ASM : CSSD

Hi,
I am reading oracle 10g Rac-grid-service,Murali vallath book.
The page number 125 says that " When installing ASM on a single-instance configuration, ensure that the cluster synchronization services (CSS) module is installed."
Does it mean voting disk is mandatory for a single instance database.
and
Page number 48 says that "
Using a vacuous monitoring over the voting disk locations, CSS
performs state changes to bring the voting disk online. This is to
determine if CSS has a registered MASTER node already active. The
various states of the voting disk are
1 - Not configured and no thread has been spawned
2 - Threads are spawned
3 - Thread started and disk is offline
4 - The voting disk is online "
what is vacuous monitoring?
Is there any way to identify  voting disk state?
Please help me.
Thanks & Regards,

If you are installing single instance ASM then CSSD daemon is used to communicate with ASM disks. In single instance environment, Voting disk is not needed. Instead, CSS will communicate with OLR which is oracle local registry. This daemon is mandatory to start ASM instance for non RAC environment. If you kill this daemon process then node will reboot so please be careful with this process.

Similar Messages

  • Cssd does not start in non-RAC environment Thus we can not bring up ASM

    Non-RAC environment
    ASM version = 11.1.0.7
    HP-UX Itanium 11.23
    After power outage, CSSD does not start on non-RAC environment
    Running as root "/sbin/init.d/init.cssd start" does not start cssd
    Oracle support tried "$ASM_HOME/bin/localconfig delete" and "$ASM_HOME/bin/localconfig add"
    but it did not start CSS
    Oracle support tried "$ASM_HOME/bin/localconfig reset $ASM_HOME"
    It started the CSSD and the "crsctl check css" came back with CSS is healthy
    But around 1 minute later it rebooted the server and when it came up again CSS does not start.
    They checked /etc/inittab and it looked fine.
    Before the reboot we saw this message in the /var/adm/syslog/OLDsyslog.log:
    Cluster Ready Services completed waiting on dependencies
    Again it is a NON-RAC environment. We only need CSSD for ASM. We do not have CRS installed on this server.
    Our test system has been down for a week and we did not get the resolution from Oracle support yet !
    Any pointers are greately appriciated.
    Thanks,
    Dzung

    Here is the message in $ASM_HOME/log/<hostname>/alert<hostname>.log :
    2010-07-16 09:42:02.956
    [client(11930)]CRS-1006:The OCR location /db/app/oracle/product/11.1/cdata/localhost/local.ocr is inacce
    ssible. Details in /db/app/oracle/product/11.1/log/rmodbd01/client/clscfg10.log.
    2010-07-16 09:42:02.971
    [client(11930)]CRS-1006:The OCR location /db/app/oracle/product/11.1/cdata/localhost/local.ocr is inacce
    ssible. Details in /db/app/oracle/product/11.1/log/rmodbd01/client/clscfg10.log.
    2010-07-16 09:42:03.054
    [client(11930)]CRS-1013:The OCR at /db/app/oracle/product/11.1/cdata/localhost/local.ocr was successfull
    y formatted using version 2. Ignore earlier CRS-1006 messages if any.
    2010-07-16 09:42:46.379
    [cssd(12297)]CRS-1601:CSSD Reconfiguration complete. Active nodes are rmodbd01 .
    Here is the message in $ASM_HOME/log/<hostname>/cssd/cssdOUT.log:
    setsid: failed with -1/1
    s0clssscGetEnvOracleUser: calling getpwnam_r for user oracle
    s0clssscGetEnvOracleUser: info for user oracle complete
    07/16/10 09:42:36: CSSD starting
    Here is the message in $ASM_HOME/log/<hostname>/cssd/ocssd.log:
    [    CSSD]2010-07-16 09:42:46.378 [18] >TRACE: clssgmCompareSwapEventValue: changed CmInfo State val 7, from 6, changes 6
    [    CSSD]2010-07-16 09:42:46.378 [18] >TRACE: clssgmSetVersions: properties common to all peers: 1,2,3,4,5,6,7
    [    CSSD]2010-07-16 09:42:46.378 [18] >TRACE: clssgmEstablishMasterNode: MASTER for 174732166 is node(0) birth(174732166)
    [    CSSD]2010-07-16 09:42:46.378 [18] >TRACE: clssgmChangeMasterNode: requeued 0 RPCs
    [    CSSD]2010-07-16 09:42:46.378 [18] >TRACE: clssgmCompareSwapEventValue: changed CmInfo State val 8, from 7, changes 7
    [    CSSD]2010-07-16 09:42:46.378 [18] >TRACE: clssgmMasterCMSync: Synchronizing group/lock status
    [    CSSD]2010-07-16 09:42:46.378 [18] >TRACE: clssgmMasterSendDBDone: group/lock status synchronization complete
    [    CSSD]2010-07-16 09:42:46.378 [18] >TRACE: clssgmCompareSwapEventValue: changed CmInfo State val 9, from 8, changes 8
    [    CSSD]2010-07-16 09:42:46.378 [18] >TRACE: clssgmCompareSwapEventValue: changed CmInfo State val 10, from 9, changes 9
    [    CSSD]CLSS-3000: reconfiguration successful, incarnation 174732166 with 1 nodes
    [    CSSD]CLSS-3001: local node number 0, master node number 0
    [    CSSD]2010-07-16 09:42:46.378 [18] >TRACE: clssscSAGEInitFenceCompl: Completing kgzf fence initialization
    [    CSSD]2010-07-16 09:42:46.394 [12] >TRACE: clssgmUpdateEventValue: Client listener incarn val 174732166, changes 1
    [    CSSD]2010-07-16 09:42:46.395 [12] >TRACE: clssgmAllocProc: (60000000003c7120) allocated
    [    CSSD]2010-07-16 09:42:46.395 [12] >TRACE: clssgmAllocProc: (60000000003c73a0) allocated
    [    CSSD]2010-07-16 09:42:46.396 [14] >TRACE: Connect request from user oracle
    [    CSSD]2010-07-16 09:42:46.396 [14] >TRACE: Connect request from user root
    [    CSSD]2010-07-16 09:42:46.397 [12] >TRACE: clssgmClientConnectMsg: properties of cmProc 60000000003c7120 - 1,2,3
    [    CSSD]2010-07-16 09:42:46.397 [12] >TRACE: clssgmClientConnectMsg: Connect from con(60000000003b2810) proc(60000000003c7120) pid(12350) version 11:1:1:4
    [    CSSD]2010-07-16 09:42:46.397 [12] >TRACE: clssgmClientConnectMsg: properties of cmProc 60000000003c73a0 - 1,2,3
    [    CSSD]2010-07-16 09:42:46.397 [12] >TRACE: clssgmClientConnectMsg: Connect from con(60000000003b2990) proc(60000000003c73a0) pid(12131) version 11:1:1:4
    [    CSSD]2010-07-16 09:42:46.402 [12] >TRACE: clssgmRegisterClient: proc(1/60000000003c7120), client(1/600000000096e7c0)
    [    CSSD]2010-07-16 09:42:46.402 [12] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (600000000096e7c0)
    [    CSSD]2010-07-16 09:42:46.402 [12] >TRACE: clssgmJoinGrock: local grock CSS_INTERNAL_NODE_GROUP new client 600000000096e7c0 with con 60000000003b2b10, requested num 0
    [    CSSD]2010-07-16 09:42:46.402 [12] >TRACE: clssgmAddNodeGrpMember: member (60000000009e0030) added
    [    CSSD]2010-07-16 09:42:46.403 [12] >TRACE: clssgmGroupState: requested group state of group localhost_NG, member count 0
    [    CSSD]2010-07-16 09:42:46.403 [12] >TRACE: clssgmGroupState: requested group state of group localhost_NG, member count 0
    [    CSSD]2010-07-16 09:42:46.404 [12] >TRACE: clssgmDeadProc: proc 60000000003c73a0
    [    CSSD]2010-07-16 09:42:46.404 [12] >TRACE: clssgmDestroyProc: cleaning up proc(60000000003c73a0) con(60000000003b2990) skgpid ospid 12131 with 0 clients, refcount 0
    [    CSSD]2010-07-16 09:42:46.425 [12] >TRACE: clssgmGroupState: requested group state of unknown group MASTER#DISKMON#GROUP
    [    CSSD]2010-07-16 09:42:46.425 [12] >TRACE: clssgmGroupState: requested group state of group MASTER#DISKMON#GROUP, member count 0
    [    CSSD]2010-07-16 09:42:46.425 [18] >TRACE: KGZF: context successfully initialized, API version 1.4, using pipe default
    [    CSSD]2010-07-16 09:42:46.425 [18] >TRACE: clssscSAGEInitFenceCompl: kgzf fence initialization successfully completed
    [    CSSD]2010-07-16 09:42:46.425 [18] >TRACE: clssgmReconfigThread: CSS/GM open for global group registrations
    [    CSSD]2010-07-16 09:42:46.425 [18] >TRACE: clssgmReconfigThread: completed for reconfig(174732166), with status(1)
    [    CSSD]2010-07-16 09:42:46.425 [18] >TRACE: clssgmUpdateEventValue: Reconfig Event val 2, changes 2
    [    CSSD]2010-07-16 09:42:46.425 [1] >TRACE: clssgmWaitOnEventValue: after Reconfig Event val 2, eval 2 waited 47
    [    CSSD]2010-07-16 09:42:46.425 [12] >TRACE: clssgmRegisterClient: proc(1/60000000003c7120), client(2/600000000096e870)
    [    CSSD]2010-07-16 09:42:46.425 [1] >TRACE: clssgmUpdateEventValue: Reconfig Event val 0, changes 3
    [    CSSD]2010-07-16 09:42:46.425 [1] >TRACE: clssgmStartNMMon: previous reconfig complete, incarnation(174732166)
    [    CSSD]2010-07-16 09:42:46.425 [12] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 2 (600000000096e870)
    [    CSSD]2010-07-16 09:42:46.425 [12] >TRACE: clssgmJoinGrock: global grock MASTER#DISKMON#GROUP#MX new client 600000000096e870 with con 60000000003b2990, requested num -1
    [    CSSD]2010-07-16 09:42:46.425 [12] >TRACE: clssgmAddGrockMember: adding member to grock MASTER#DISKMON#GROUP#MX
    [    CSSD]2010-07-16 09:42:46.425 [12] >TRACE: clssgmAddMember: granted member(0) flags(0x2) node(0) grock (6000000000989e50/MASTER#DISKMON#GROUP#MX)
    [    CSSD]2010-07-16 09:42:46.425 [12] >TRACE: clssgmQueueGrockEvent: lockName(MASTER#DISKMON#GROUP#MX) type(3) count (1/1) xwaiters(1) event(1) to memberNo(0)
    [    CSSD]2010-07-16 09:42:46.425 [12] >TRACE: clssgmCommonAddMember: global lock grock MASTER#DISKMON#GROUP#MX member(0/Local) node(0) flags 0x2 0x2
    [    CSSD]2010-07-16 09:42:46.425 [12] >TRACE: clssgmRegisterClient: proc(1/60000000003c7120), client(3/600000000096e920)
    [    CSSD]2010-07-16 09:42:46.425 [12] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 3 (600000000096e920)
    [    CSSD]2010-07-16 09:42:46.425 [12] >TRACE: clssgmJoinGrock: global grock MASTER#DISKMON#GROUP new client 600000000096e920 with con 60000000003b2bd0, requested num 0
    [    CSSD]2010-07-16 09:42:46.425 [12] >TRACE: clssgmAddGrockMember: adding member to grock MASTER#DISKMON#GROUP
    [    CSSD]2010-07-16 09:42:46.425 [12] >TRACE: clssgmAddMember: new master 0 for group(MASTER#DISKMON#GROUP)
    [    CSSD]2010-07-16 09:42:46.426 [12] >TRACE: clssgmAddMember: Adding fencing for member 0, group MASTER#DISKMON#GROUP, death 1, SAGE 0
    [    CSSD]2010-07-16 09:42:46.426 [12] >TRACE: clssgmAddMember: member (0/60000000009e0230) added. pbsz(72) prsz(0) flags 0x0 to grock (600000000098a170/MASTER#DISKMON#GROUP)
    [    CSSD]2010-07-16 09:42:46.426 [12] >TRACE: clssgmQueueGrockEvent: groupName(MASTER#DISKMON#GROUP) count(1) master(0) event(1), incarn 1, mbrc 1, to member 0, events 0x78, state 0x0
    [    CSSD]2010-07-16 09:42:46.426 [12] >TRACE: clssgmCommonAddMember: global group grock MASTER#DISKMON#GROUP member(0/Local) node(0) flags 0x0 0x1e00
    [    CSSD]2010-07-16 09:42:46.426 [12] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 2 (600000000096e870)
    [    CSSD]2010-07-16 09:42:46.426 [12] >TRACE: clssgmExitGrock: client 2 (600000000096e870), grock MASTER#DISKMON#GROUP#MX, member 0
    [    CSSD]2010-07-16 09:42:46.426 [12] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (60000000009e0130) in global grock MASTER#DISKMON#GROUP#MX
    [    CSSD]2010-07-16 09:42:46.426 [12] >TRACE: clssgmRemoveMember: grock MASTER#DISKMON#GROUP#MX, member number 0 (60000000009e0130) node number 0 state 0x14 member refcnt 0 grock type 3
    [    CSSD]2010-07-16 09:42:48.405 [8] >TRACE: clssnmSendingThread: sending status msg to all nodes
    [    CSSD]2010-07-16 09:42:48.405 [8] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes
    [    CSSD]2010-07-16 09:42:52.444 [8] >TRACE: clssnmSendingThread: sending status msg to all nodes
    [    CSSD]2010-07-16 09:42:52.444 [8] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes
    [    CSSD]2010-07-16 09:42:56.484 [8] >TRACE: clssnmSendingThread: sending status msg to all nodes
    [    CSSD]2010-07-16 09:42:56.484 [8] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes
    [    CSSD]2010-07-16 09:43:00.516 [8] >TRACE: clssnmSendingThread: sending status msg to all nodes
    [    CSSD]2010-07-16 09:43:00.516 [8] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes
    [    CSSD]2010-07-16 09:43:04.563 [8] >TRACE: clssnmSendingThread: sending status msg to all nodes
    [    CSSD]2010-07-16 09:43:04.563 [8] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes
    [    CSSD]2010-07-16 09:43:08.603 [8] >TRACE: clssnmSendingThread: sending status msg to all nodes
    [    CSSD]2010-07-16 09:43:08.603 [8] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes
    [    CSSD]2010-07-16 09:43:12.643 [8] >TRACE: clssnmSendingThread: sending status msg to all nodes
    [    CSSD]2010-07-16 09:43:12.643 [8] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes
    [    CSSD]2010-07-16 09:43:16.676 [8] >TRACE: clssnmSendingThread: sending status msg to all nodes
    [    CSSD]2010-07-16 09:43:16.676 [8] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes
    [    CSSD]2010-07-16 09:43:20.723 [8] >TRACE: clssnmSendingThread: sending status msg to all nodes
    [    CSSD]2010-07-16 09:43:20.723 [8] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes
    [    CSSD]2010-07-16 09:43:24.762 [8] >TRACE: clssnmSendingThread: sending status msg to all nodes
    [    CSSD]2010-07-16 09:43:24.762 [8] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes
    [    CSSD]2010-07-16 09:43:28.802 [8] >TRACE: clssnmSendingThread: sending status msg to all nodes
    [    CSSD]2010-07-16 09:43:28.802 [8] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes
    [    CSSD]2010-07-16 09:43:32.842 [8] >TRACE: clssnmSendingThread: sending status msg to all nodes
    [    CSSD]2010-07-16 09:43:32.842 [8] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes
    [    CSSD]2010-07-16 09:43:36.882 [8] >TRACE: clssnmSendingThread: sending status msg to all nodes
    [    CSSD]2010-07-16 09:43:36.882 [8] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes
    [    CSSD]2010-07-16 09:43:40.922 [8] >TRACE: clssnmSendingThread: sending status msg to all nodes
    [    CSSD]2010-07-16 09:43:40.922 [8] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes
    [    CSSD]2010-07-16 09:43:44.964 [8] >TRACE: clssnmSendingThread: sending status msg to all nodes
    [    CSSD]2010-07-16 09:43:44.964 [8] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes
    [    CSSD]2010-07-16 09:43:49.002 [8] >TRACE: clssnmSendingThread: sending status msg to all nodes
    [    CSSD]2010-07-16 09:43:49.002 [8] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes
    [    CSSD]2010-07-16 09:43:53.043 [8] >TRACE: clssnmSendingThread: sending status msg to all nodes
    [    CSSD]2010-07-16 09:43:53.043 [8] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes
    [    CSSD]2010-07-16 09:43:57.085 [8] >TRACE: clssnmSendingThread: sending status msg to all nodes
    [    CSSD]2010-07-16 09:43:57.085 [8] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes
    [    CSSD]2010-07-16 09:43:58.354 [12] >TRACE: clssgmAllocProc: (60000000003c7ee0) allocated
    [    CSSD]2010-07-16 09:43:58.355 [14] >TRACE: Connect request from user oracle
    [    CSSD]2010-07-16 09:43:58.356 [12] >TRACE: clssgmClientConnectMsg: properties of cmProc 60000000003c7ee0 - 1,2,3
    [    CSSD]2010-07-16 09:43:58.356 [12] >TRACE: clssgmClientConnectMsg: Connect from con(60000000003b2990) proc(60000000003c7ee0) pid(13157) version 11:1:1:4
    [    CSSD]2010-07-16 09:43:58.360 [12] >TRACE: clssgmRegisterClient: proc(2/60000000003c7ee0), client(1/600000000096e870)
    [    CSSD]2010-07-16 09:43:58.360 [12] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 1 (600000000096e870)
    [    CSSD]2010-07-16 09:43:58.360 [12] >TRACE: clssgmJoinGrock: global grock CLSSSCHECK_GROUP new client 600000000096e870 with con 60000000003b2c90, requested num -1
    [    CSSD]2010-07-16 09:43:58.360 [12] >TRACE: clssgmAddGrockMember: adding member to grock CLSSSCHECK_GROUP
    [    CSSD]2010-07-16 09:43:58.360 [12] >TRACE: clssgmAddMember: new master 0 for group(CLSSSCHECK_GROUP)
    [    CSSD]2010-07-16 09:43:58.360 [12] >TRACE: clssgmAddMember: Adding fencing for member 0, group CLSSSCHECK_GROUP, death 1, SAGE 0
    [    CSSD]2010-07-16 09:43:58.360 [12] >TRACE: clssgmAddMember: member (0/60000000009e0130) added. pbsz(8) prsz(8) flags 0x0 to grock (6000000000989e50/CLSSSCHECK_GROUP)
    [    CSSD]2010-07-16 09:43:58.360 [12] >TRACE: clssgmQueueGrockEvent: groupName(CLSSSCHECK_GROUP) count(1) master(0) event(1), incarn 1, mbrc 1, to member 0, events 0x0, state 0x0
    [    CSSD]2010-07-16 09:43:58.360 [12] >TRACE: clssgmCommonAddMember: global group grock CLSSSCHECK_GROUP member(0/Local) node(0) flags 0x0 0x0
    [    CSSD]2010-07-16 09:43:58.361 [12] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 1 (600000000096e870)
    [    CSSD]2010-07-16 09:43:58.361 [12] >TRACE: clssgmExitGrock: client 1 (600000000096e870), grock CLSSSCHECK_GROUP, member 0
    [    CSSD]2010-07-16 09:43:58.361 [12] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (60000000009e0130) in global grock CLSSSCHECK_GROUP
    [    CSSD]2010-07-16 09:43:58.361 [12] >TRACE: clssgmRemoveMember: grock CLSSSCHECK_GROUP, member number 0 (60000000009e0130) node number 0 state 0x14 member refcnt 0 grock type 2
    [    CSSD]2010-07-16 09:43:58.361 [12] >TRACE: clssgmRegisterClient: proc(2/60000000003c7ee0), client(2/600000000096e870)
    [    CSSD]2010-07-16 09:43:58.361 [12] >TRACE: clssgmExecuteClientRequest: GRKJOIN recvd from client 2 (600000000096e870)
    [    CSSD]2010-07-16 09:43:58.361 [12] >TRACE: clssgmJoinGrock: global grock CLSSSCHECK_LOCK new client 600000000096e870 with con 60000000003b2c90, requested num -1
    [    CSSD]2010-07-16 09:43:58.361 [12] >TRACE: clssgmAddGrockMember: adding member to grock CLSSSCHECK_LOCK
    [    CSSD]2010-07-16 09:43:58.361 [12] >TRACE: clssgmAddMember: granted member(0) flags(0x2) node(0) grock (6000000000989e50/CLSSSCHECK_LOCK)
    [    CSSD]2010-07-16 09:43:58.361 [12] >TRACE: clssgmQueueGrockEvent: lockName(CLSSSCHECK_LOCK) type(3) count (1/1) xwaiters(1) event(1) to memberNo(0)
    [    CSSD]2010-07-16 09:43:58.361 [12] >TRACE: clssgmCommonAddMember: global lock grock CLSSSCHECK_LOCK member(0/Local) node(0) flags 0x2 0x2
    [    CSSD]2010-07-16 09:43:58.361 [12] >TRACE: clssgmExecuteClientRequest: GRKEXIT recvd from client 2 (600000000096e870)
    [    CSSD]2010-07-16 09:43:58.361 [12] >TRACE: clssgmExitGrock: client 2 (600000000096e870), grock CLSSSCHECK_LOCK, member 0
    [    CSSD]2010-07-16 09:43:58.361 [12] >TRACE: clssgmUnregisterPrimary: Unregistering member 0 (60000000009e0130) in global grock CLSSSCHECK_LOCK
    [    CSSD]2010-07-16 09:43:58.361 [12] >TRACE: clssgmRemoveMember: grock CLSSSCHECK_LOCK, member number 0 (60000000009e0130) node number 0 state 0x14 member refcnt 0 grock type 3
    [    CSSD]2010-07-16 09:43:58.362 [12] >TRACE: clssgmDeadProc: proc 60000000003c7ee0
    [    CSSD]2010-07-16 09:43:58.362 [12] >TRACE: clssgmDestroyProc: cleaning up proc(60000000003c7ee0) con(60000000003b2990) skgpid ospid 13157 with 0 clients, refcount 0
    [    CSSD]2010-07-16 09:44:01.125 [8] >TRACE: clssnmSendingThread: sending status msg to all nodes
    [    CSSD]2010-07-16 09:44:01.125 [8] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes
    [    CSSD]2010-07-16 09:44:05.164 [8] >TRACE: clssnmSendingThread: sending status msg to all nodes
    [    CSSD]2010-07-16 09:44:05.164 [8] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes
    [    CSSD]2010-07-16 09:44:09.196 [8] >TRACE: clssnmSendingThread: sending status msg to all nodes
    [    CSSD]2010-07-16 09:44:09.196 [8] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes
    [    CSSD]2010-07-16 09:44:13.236 [8] >TRACE: clssnmSendingThread: sending status msg to all nodes
    [    CSSD]2010-07-16 09:44:13.236 [8] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes
    [    CSSD]2010-07-16 09:44:17.284 [8] >TRACE: clssnmSendingThread: sending status msg to all nodes
    [    CSSD]2010-07-16 09:44:17.284 [8] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes
    [    CSSD]2010-07-16 09:44:21.322 [8] >TRACE: clssnmSendingThread: sending status msg to all nodes
    [    CSSD]2010-07-16 09:44:21.322 [8] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes
    [    CSSD]2010-07-16 09:44:25.316 [8] >TRACE: clssnmSendingThread: sending status msg to all nodes
    [    CSSD]2010-07-16 09:44:25.316 [8] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes
    [    CSSD]2010-07-16 09:44:30.333 [8] >TRACE: clssnmSendingThread: sending status msg to all nodes
    [    CSSD]2010-07-16 09:44:30.333 [8] >TRACE: clssnmSendingThread: sent 5 status msgs to all nodes
    [    CSSD]2010-07-16 09:44:34.324 [8] >TRACE: clssnmSendingThread: sending status msg to all nodes
    [    CSSD]2010-07-16 09:44:34.324 [8] >TRACE: clssnmSendingThread: sent 4 status msgs to all nodes
    Here is the message in $ASM_HOME/log/<hostname>/diskmon/client.log:
    [ DISKMON] 07/16/2010 18:33:32.050 dskm_send_command: process 23246 sending command 8 to master diskmon listening on default pipe
    [ DISKMON] 07/16/2010 18:33:32.080 dskm_send_command3: skgznp_connect failed with error 56815
    [ DISKMON] 07/16/2010 18:33:32.080 dskm_send_command3: error 56815 at location skgznpcon6 - connect() - Connection refused
    Here is the message in $ASM_HOME/log/<hostname>/diskmon/diskmonOUT.log:
    setsid: failed with -1/1
    dskm_getenv_oracle_user: calling getpwnam_r for user oracle
    dskm_getenv_oracle_user: info for user oracle complete
    07/16/10 18:33:31: Master Diskmon starting
    Here is the message in $ASM_HOME/log/<hostname>/diskmon/diskmon.log:
    [ DISKMON] 07/16/2010 09:42:35.573 dskm main: starting up
    [ DISKMON] 07/16/2010 09:42:35.588 [12350:3] dskm_rac_thrd_main: running
    [ DISKMON] 07/16/2010 09:42:35.588 [12350:1] dskm_rac_thrd_creat2: got the post from the css event handling thread
    [ DISKMON] 07/16/2010 09:42:35.589 [12350:1] dskm main: startup complete
    [ DISKMON] 07/16/2010 09:42:35.589 [12350:1] listening on -> default pipe
    [ DISKMON] 07/16/2010 09:42:35.792 clsc_connect: (6000000000700420) no listener at (ADDRESS=(PROTOCOL=ipc)(KEY=OCSSD_LL_rmodbd01_))
    [ DISKMON] 07/16/2010 09:42:36.385 clsc_connect: (6000000000700420) no listener at (ADDRESS=(PROTOCOL=ipc)(KEY=OCSSD_LL_rmodbd01_))
    [ DISKMON] 07/16/2010 09:42:36.906 clsc_connect: (6000000000700420) no listener at (ADDRESS=(PROTOCOL=ipc)(KEY=OCSSD_LL_rmodbd01_))
    [ DISKMON] 07/16/2010 09:42:37.426 clsc_connect: (6000000000700420) no listener at (ADDRESS=(PROTOCOL=ipc)(KEY=OCSSD_LL_rmodbd01_))
    [ DISKMON] 07/16/2010 09:42:37.945 clsc_connect: (6000000000700420) no listener at (ADDRESS=(PROTOCOL=ipc)(KEY=OCSSD_LL_rmodbd01_))
    [ DISKMON] 07/16/2010 09:42:38.465 clsc_connect: (6000000000700420) no listener at (ADDRESS=(PROTOCOL=ipc)(KEY=OCSSD_LL_rmodbd01_))
    [ DISKMON] 07/16/2010 09:42:46.376 [12350:1] dskm_slave_thrd_creat: thread created
    [ DISKMON] 07/16/2010 09:42:46.376 [12350:11] dskm_slave_thrd_main1: slave 0 running
    [ DISKMON] 07/16/2010 09:42:46.376 [12350:11] dskm_process_msg5: received msg type KGZM_IDENTIFY (0x0001)
    [ DISKMON] 07/16/2010 09:42:46.389 [12350:11] dskm_proc_identify8: client kgzf/12297, version 0x01020000, slave 0, reid cid=3e0391f05e06cfafbf7419d7cf085a44,icin=174732166,nmn=0,lnid=174732166,gid=0,gin=0,gmn=0,umemid=0,opid=0,opsn=0,lvl=node
    [ DISKMON] 07/16/2010 09:42:46.389 [12350:11] dskm_send_version1:
    [ DISKMON] 07/16/2010 09:42:46.389 [12350:11] dskm_send_version4: done
    [ DISKMON] 07/16/2010 09:42:46.389 [12350:11] dskm_process_msg7: processed msg 0 type KGZM_IDENTIFY (0x0001), retcode 0
    [ DISKMON] 07/16/2010 09:42:46.389 [12350:11] dskm_process_msg5: received msg type KGZM_KGZF_HANDSHAKE (0x0010)
    [ DISKMON] 07/16/2010 09:42:46.389 [12350:11] dskm_proc_kgzf_handshake3: client kgzf/12297, kgzf version 0x00010004, slave 0
    [ DISKMON] 07/16/2010 09:42:46.401 [12350:3] dskm_clss_ini2: successful clsssinit(), clssvers 2.1
    [ DISKMON] 07/16/2010 09:42:46.402 [12350:3] dskm_clss_ini12: node rmodbd01 (0) registered in cluster
    [ DISKMON] 07/16/2010 09:42:46.403 [12350:3] dskm_reid_ini12: diskmon reid cid=3e0391f05e06cfafbf7419d7cf085a44,icin=174732166,nmn=0,lnid=174732166,gid=-1,gin=-1,gmn=-1,umemid=-1,opid=12350,opsn=1279291355,lvl=process
    [ DISKMON] 07/16/2010 09:42:46.424 [12350:3] dskm_sage_config: CELL storage configuration file cellinit.ora not found
    [ DISKMON] 07/16/2010 09:42:46.425 [12350:3] dskm_nfy_kgzf1: notified thread kgzf enabled
    [ DISKMON] 07/16/2010 09:42:46.425 [12350:11] dskm_proc_kgzf_handshake5: got the post from the hb thread
    [ DISKMON] 07/16/2010 09:42:46.425 [12350:11] dskm_proc_kgzf_handshake9: done, kgzf enabled
    [ DISKMON] 07/16/2010 09:42:46.425 [12350:11] dskm_process_msg7: processed msg 0 type KGZM_KGZF_HANDSHAKE (0x0010), retcode 0
    [ DISKMON] 07/16/2010 09:42:46.426 [12350:3] dskm_rac_ini22: CELL storage not configured in the cluster; registered in group MASTER#DISKMON#GROUP as memno 0 (GSDGRPSZ 512)

  • Switching from one ASM installation to another ASM installation

    Hi,
    We've a server Red Hat Enterprise Linux Server release 5.3 (Tikanga) with installed two 11.1.0.7.0 installation: one for ASM and the other for rdmbs.
    Now, we want to install the 11.1.0.7.4 PSU.
    For reducing downtime we'd like to create two new 11.1.0.7.0 installation; then patching them with the 11.1.0.7.4 PSU.
    Then we'd like to stop ASM and all database running and switching the two old installation with the new one.
    For the rdbms installation it's easy: just to modify the oratab and then starting the databases with the new patched installation and then follow the post installation steps (atbundle.sql, etc.)
    For ASM instead, how can I do it ? How can I reconfigure the ocssd for using the new ASM installation ? And do I need to do some other steps ?
    Thanks
    massi

    Hi,
    Probably my post was not clear enough. Sorry.
    Here our scenario:
    - we have one linux server with two Oracle11g installations. We use one installation for ASM and the other one for the rdbms. We planned to install the PSU 4 on both the installations.
    We could simply shutdown all databases and ASM. Then install PSU4 on both ASM and rdmbs installation. Then startup AMS and databases and proceed with post PSU installation tasks.
    There is another possibility. To install in advance two new Oracle11g: one for ASM and the other one for the rdbms. Then apply PSU4 on these two new Oracle11g installations. Then shutdown the databases and the ASM. Then execute:
    $ORACLE_HOME/bin/localconfig reset <new_ASM_oracle_home>
    stop cssd (/etc/init.d/init.cssd stop)
    start cssd (/etc/init.d/init.cssd start)
    Then change in /etc/oratab so that databases will point to the new installation (the new one installed for rdbms)
    Then copy spfile and password files to the new oracle_home/dbs (the new one installed for rdbms)
    Then change in /etc/oratab so that ASM is using the new installation (the new one installed forASM)
    Then copy spfile and password file to the new oracle_home/dbs used by ASM (the new one installed forASM)
    Finally restart ASM and databases
    This way, the ASM will point to the new installation - and the databases will use the other new installation. This way the downtime will be shorter. It will be only necessary to apply the post PSU installation tasks
    I've already did it last weekend. It worked
    Thanks
    massi
    Thanks
    massi

  • Can't create database using ASM (SOLVED)

    Hi all
    I'm trying to use ASM for the first time, on Oracle 10.2.0.1 on Solaris x64.
    I have installed the ASM instance into /opt/oracle/asm/10.2.0 and created disk groups. I have cssd running OK. I am able to start and stop the ASM instance without problems, and I can select from v$asm_diskgroup to confirm that disks are mounted OK.
    I have then installed Oracle EE separately into /opt/oracle/server/10.2.0. I first did a software only install, and now I am trying to create a DB.
    The problems come when I try to use this ASM instance to host a new database. I first tried to use DBCA to create a new database, but on database creation I got the following errors:
    ORA-00200: control file could not be created
    ORA-00202: control file: '+DBLIVE1'
    ORA-17502: ksfdcre:4 Failed to create file +DBLIVE1
    ORA-15001: diskgroup "DBLIVE1" does not exist or is not mounted
    ORA-15055: Message 15055 not found; No message file for product=RDBMS, facility=ORA
    ORA-01031: insufficient privileges
    I then told DBCA just to create the DB creation scripts, and I tried manually running these with SQL*Plus.
    When doing it with SQL*PLus, I initially got the same error as shown above. But then something changed (sorry, not sure what), and now the error I get is:
    CREATE DATABASE "NEONREL1"
    ERROR at line 1:
    ORA-01501: CREATE DATABASE failed
    ORA-00349: failure obtaining block size for '+DBLIVE1'
    ORA-01031: insufficient privileges
    I've put some debug info below, showing me succesfully connecting to the ASM instance and then attempting to create the DB using the db creation scripts, showing the error at the end. You can see that the oracle OS user is able to connect fine to ASM, then I swithc ORACLE_SID and ORACLE_HOME to the EE install and try to create the DB, at which point it apparently can't connect to ASM any more.
    I've tried the DB creation many times, and in between attempts I completely empty $ORACLE_HOME/admin/<dbname> and delete the files related to the attempted install from $ORACLE_HOME/dbs/ . I've also stopping/starting ASM, rebooting, and I've done the install of ASM and EE a couple of times over in case I made any mistakes in my earlier attempts.
    Any help would be much appreciated!
    Tom
    ##### CHECKING ASM
    oracle@neonrcom-db1:~$ uname -a
    SunOS neonrcom-db1 5.10 Generic_127128-11 i86pc i386 i86pc
    # css is running
    oracle@neonrcom-db1:~$ ps -ef | grep css
    oracle 498 1 0 21:46:40 ? 0:01 /opt/oracle/asm/10.2.0/bin/ocssd.bin
    # listener is running in the ASM instance
    oracle@neonrcom-db1:~$ ps -ef | grep tnsl
    oracle 1332 1 0 21:49:59 ? 0:00 /opt/oracle/asm/10.2.0/bin/tnslsnr LISTENER -inherit
    # ASM is only entry in /var/opt/oracle/oratab
    oracle@neonrcom-db1:~$ grep -v "^#" /var/opt/oracle/oratab
    +ASM:/opt/oracle/asm/10.2.0:N
    # I can connect to ASM fine, and it has diskgroups mounted.
    oracle@neonrcom-db1:~$ export ORACLE_HOME=/opt/oracle/asm/10.2.0
    oracle@neonrcom-db1:~$ export ORACLE_SID='+ASM'
    oracle@neonrcom-db1:~$ sqlplus "sys as sysdba"
    SQL*Plus: Release 10.2.0.1.0 - Production on Mon Jul 21 20:53:10 2008
    Copyright (c) 1982, 2005, Oracle. All rights reserved.
    Enter password:
    Connected to:
    Oracle Database 10g Enterprise Edition Release 10.2.0.1.0 - Production
    With the Partitioning, OLAP and Data Mining options
    SQL> set line 150
    SQL> select name, block_size, state, type, total_mb, free_mb from v$asm_diskgroup;
    NAME BLOCK_SIZE STATE TYPE TOTAL_MB FREE_MB
    DBARCH1 4096 MOUNTED EXTERN 2096856 2096784
    DBLIVE1 4096 MOUNTED EXTERN 4193904 4193812
    #### Contents of init.ora for new DB
    db_create_file_dest=+DBLIVE1
    db_recovery_file_dest=+DBARCH1
    db_recovery_file_dest_size=2147483648
    ##### DB INSTALLATION ATTEMPT
    oracle@neonrcom-db1:~$ export ORACLE_HOME=/opt/oracle/server/10.2.0
    oracle@neonrcom-db1:~$ export ORACLE_SID='NEONREL1'
    oracle@neonrcom-db1:~$ export PATH=$ORACLE_HOME/bin:$PATH
    oracle@neonrcom-db1:~$ /opt/oracle/server/10.2.0/admin/NEONREL1/scripts/NEONREL1.sh
    You should Add this entry in the /var/opt/oracle/oratab: NEONREL1:/opt/oracle/server/10.2.0:Y
    SQL*Plus: Release 10.2.0.1.0 - Production on Mon Jul 21 22:10:54 2008
    Copyright (c) 1982, 2005, Oracle. All rights reserved.
    specify a password for sys as parameter 1
    Enter value for 1: xxx
    specify a password for system as parameter 2
    Enter value for 2: xxx
    specify a password for sysman as parameter 3
    Enter value for 3: xxx
    specify a password for dbsnmp as parameter 4
    Enter value for 4: xxx
    specify ASM SYS user password as parameter 6
    Enter value for 6: xxx
    Connected to an idle instance.
    SQL> spool /opt/oracle/server/10.2.0/admin/NEONREL1/scripts/CreateDB.log
    SQL> startup nomount pfile="/opt/oracle/server/10.2.0/admin/NEONREL1/scripts/init.ora";
    ORACLE instance started.
    Total System Global Area 1.9294E+10 bytes
    Fixed Size 2054976 bytes
    Variable Size 2264925376 bytes
    Database Buffers 1.7012E+10 bytes
    Redo Buffers 14721024 bytes
    SQL> CREATE DATABASE "NEONREL1"
    2 MAXINSTANCES 8
    3 MAXLOGHISTORY 1
    4 MAXLOGFILES 16
    5 MAXLOGMEMBERS 3
    6 MAXDATAFILES 100
    7 DATAFILE SIZE 300M AUTOEXTEND ON NEXT 10240K MAXSIZE UNLIMITED
    8 EXTENT MANAGEMENT LOCAL
    9 SYSAUX DATAFILE SIZE 120M AUTOEXTEND ON NEXT 10240K MAXSIZE UNLIMITED
    10 SMALLFILE DEFAULT TEMPORARY TABLESPACE TEMP TEMPFILE SIZE 20M AUTOEXTEND ON NEXT 640K MAXSIZE UNLIMITED
    11 SMALLFILE UNDO TABLESPACE "UNDOTBS1" DATAFILE SIZE 200M AUTOEXTEND ON NEXT 5120K MAXSIZE UNLIMITED
    12 CHARACTER SET AL32UTF8
    13 NATIONAL CHARACTER SET UTF8
    14 LOGFILE GROUP 1 SIZE 51200K,
    15 GROUP 2 SIZE 51200K,
    16 GROUP 3 SIZE 51200K
    17 USER SYS IDENTIFIED BY "&&sysPassword" USER SYSTEM IDENTIFIED BY "&&systemPassword";
    CREATE DATABASE "NEONREL1"
    ERROR at line 1:
    ORA-01501: CREATE DATABASE failed
    ORA-00349: failure obtaining block size for '+DBLIVE1'
    ORA-01031: insufficient privileges
    Message was edited by:
    tjobbins

    Update: I've worked out the difference between the two sets of errors I get.
    The basic error is this:
    ORA-00200: control file could not be created
    ORA-00202: control file: '+DBLIVE1'
    ORA-17502: ksfdcre:4 Failed to create file +DBLIVE1
    ORA-15001: diskgroup "DBLIVE1" does not exist or is not mounted
    ORA-15055: Message 15055 not found; No message file for product=RDBMS, facility=ORA
    ORA-01031: insufficient privileges
    However if my init.ora contains the line:
    control_files=/opt/oracle/server/10.2.0/dbs/cntrlNEONREL1.dbf
    then I instead get the second error:
    CREATE DATABASE "NEONREL1"
    ERROR at line 1:
    ORA-01501: CREATE DATABASE failed
    ORA-00349: failure obtaining block size for '+DBLIVE1'
    ORA-01031: insufficient privileges
    So basically these must be the same error, just in the second case I'm not trying to put the control file on the ASM so it fails at a different point.
    But both errors must be because of the same cause, I suppose.

  • ASM instance crash due  to error ORA-27506: IPC error connecting to a port

    Hi All,
    Today the ASM instance goes down.
    When i checked the alert log I found the below error.
    ORA-27506: IPC error connecting to a port
    ORA-27300: OS system dependent operation:sendmsg failed with status: 22
    ORA-27301: OS failure message: Invalid argument
    ORA-27302: failure occurred at: sskgxpsnd1
    Please find the environment details.
    OS : RHEL-5
    DB: 11.1.0.7 2-node RAC
    I want to know the root cause of this issue.
    Please suggest.
    Thanks and Regards,

    Hi,
    Could you please upload cluster alert log and cssd.log?
    regards,
    Kishore

  • Root.sh fails on 11.2.0.3 clusterware while starting 'ora.asm' resource

    Dear all,
    I am trying to install clean Oracle 11.2.0.3 grid infrastructure on a two node cluster running on Solaris 5.10.
    - Cluster verification was successfully on both nodes; No warning or issues;
    - I am using 2 network cards for the public and 2 for the private interconnect;
    - OCR is stored on ASM
    - Firewall is disabled on both nodes
    - SCAN is being configured on the DNS (not added in /etc/hosts)
    - GNS is not used
    - hosts file is identical (except the primary hostname)
    The problem: root.sh fails on the 2nd (remote) node, because it fails to start the "ora.asm" resource. However, the root.sh has completed successfully on the 1st node.. Somehow, root.sh doesn't create +ASM2 instance on the remote (host2) node.
    root.sh was executed first on the local node (host1) and after the successful execution was started on the remote (host2) node.
    Output from host1 (working):
    ===================
    Adding Clusterware entries to inittab
    CRS-2672: Attempting to start 'ora.mdnsd' on 'host1'
    CRS-2676: Start of 'ora.mdnsd' on 'host1' succeeded
    CRS-2672: Attempting to start 'ora.gpnpd' on 'host1'
    CRS-2676: Start of 'ora.gpnpd' on 'host1' succeeded
    CRS-2672: Attempting to start 'ora.cssdmonitor' on 'host1'
    CRS-2672: Attempting to start 'ora.gipcd' on 'host1'
    CRS-2676: Start of 'ora.cssdmonitor' on 'host1' succeeded
    CRS-2676: Start of 'ora.gipcd' on 'host1' succeeded
    CRS-2672: Attempting to start 'ora.cssd' on 'host1'
    CRS-2672: Attempting to start 'ora.diskmon' on 'host1'
    CRS-2676: Start of 'ora.diskmon' on 'host1' succeeded
    CRS-2676: Start of 'ora.cssd' on 'host1' succeeded
    ASM created and started successfully.
    Disk Group CRS created successfully.
    clscfg: -install mode specified
    Successfully accumulated necessary OCR keys.
    Creating OCR keys for user 'root', privgrp 'root'..
    Operation successful.
    CRS-4256: Updating the profile
    Successful addition of voting disk 4373be34efab4f01bf79f6c5362acfd3.
    Successful addition of voting disk 7fd725fa4d904f07bf76cecf96791547.
    Successful addition of voting disk a9c85297bdd74f3abfd86899205aaf17.
    Successfully replaced voting disk group with +CRS.
    CRS-4256: Updating the profile
    CRS-4266: Voting file(s) successfully replaced
    ## STATE File Universal Id File Name Disk group
    1. ONLINE 4373be34efab4f01bf79f6c5362acfd3 (/dev/rdsk/c4t600A0B80006E2CC40000C6674E82AA57d0s4) [CRS]
    2. ONLINE 7fd725fa4d904f07bf76cecf96791547 (/dev/rdsk/c4t600A0B80006E2CC40000C6694E82AADDd0s4) [CRS]
    3. ONLINE a9c85297bdd74f3abfd86899205aaf17 (/dev/rdsk/c4t600A0B80006E2F100000C7744E82AC7Ad0s4) [CRS]
    Located 3 voting disk(s).
    CRS-2672: Attempting to start 'ora.asm' on 'host1'
    CRS-2676: Start of 'ora.asm' on 'host1' succeeded
    CRS-2672: Attempting to start 'ora.CRS.dg' on 'host1'
    CRS-2676: Start of 'ora.CRS.dg' on 'host1' succeeded
    CRS-2672: Attempting to start 'ora.registry.acfs' on 'host1'
    CRS-2676: Start of 'ora.registry.acfs' on 'host1' succeeded
    Configure Oracle Grid Infrastructure for a Cluster ... succeeded
    Name Type Target State Host
    ora.CRS.dg ora....up.type ONLINE ONLINE host1
    ora....ER.lsnr ora....er.type ONLINE ONLINE host1
    ora....N1.lsnr ora....er.type ONLINE ONLINE host1
    ora....N2.lsnr ora....er.type ONLINE ONLINE host1
    ora....N3.lsnr ora....er.type ONLINE ONLINE host1
    ora.asm ora.asm.type ONLINE ONLINE host1
    ora....SM1.asm application ONLINE ONLINE host1
    ora....B1.lsnr application ONLINE ONLINE host1
    ora....db1.gsd application OFFLINE OFFLINE
    ora....db1.ons application ONLINE ONLINE host1
    ora....db1.vip ora....t1.type ONLINE ONLINE host1
    ora.cvu ora.cvu.type ONLINE ONLINE host1
    ora.gsd ora.gsd.type OFFLINE OFFLINE
    ora....network ora....rk.type ONLINE ONLINE host1
    ora.oc4j ora.oc4j.type ONLINE ONLINE host1
    ora.ons ora.ons.type ONLINE ONLINE host1
    ora....ry.acfs ora....fs.type ONLINE ONLINE host1
    ora.scan1.vip ora....ip.type ONLINE ONLINE host1
    ora.scan2.vip ora....ip.type ONLINE ONLINE host1
    ora.scan3.vip ora....ip.type ONLINE ONLINE host1
    Output from host2 (failing):
    ===================
    OLR initialization - successful
    Adding Clusterware entries to inittab
    CRS-4402: The CSS daemon was started in exclusive mode but found an active CSS daemon on node billdb1, number 1, and is terminating
    An active cluster was found during exclusive startup, restarting to join the cluster
    Start of resource "ora.asm" failed
    CRS-2672: Attempting to start 'ora.drivers.acfs' on 'host2'
    CRS-2676: Start of 'ora.drivers.acfs' on 'host2' succeeded
    CRS-2672: Attempting to start 'ora.asm' on 'host2'
    CRS-5017: The resource action "ora.asm start" encountered the following error:
    ORA-03113: end-of-file on communication channel
    Process ID: 0
    Session ID: 0 Serial number: 0
    *. For details refer to "(:CLSN00107:)" in "/u01/11.2.0/grid/log/host2/agent/ohasd/oraagent_grid/oraagent_grid.log".*
    CRS-2674: Start of 'ora.asm' on 'host2' failed
    CRS-2679: Attempting to clean 'ora.asm' on 'host2'
    CRS-2681: Clean of 'ora.asm' on 'host2' succeeded
    CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'host2'
    CRS-2677: Stop of 'ora.drivers.acfs' on 'host2' succeeded
    CRS-4000: Command Start failed, or completed with errors.
    Failed to start Oracle Grid Infrastructure stack
    Failed to start ASM at /u01/11.2.0/grid/crs/install/crsconfig_lib.pm line 1272.
    /u01/11.2.0/grid/perl/bin/perl -I/u01/11.2.0/grid/perl/lib -I/u01/11.2.0/grid/crs/install /u01/11.2.0/grid/crs/install/rootcrs.pl execution failed
    Contents of "/u01/11.2.0/grid/cfgtoollogs/crsconfig/rootcrs_host2.log"
    =============================================
    CRS-2672: Attempting to start 'ora.asm' on 'host2'
    CRS-5017: The resource action "ora.asm start" encountered the following error:
    ORA-03113: end-of-file on communication channel
    Process ID: 0
    Session ID: 0 Serial number: 0
    . For details refer to "(:CLSN00107:)" in "/u01/11.2.0/grid/log/host2/agent/ohasd/oraagent_grid/oraagent_grid.log".
    CRS-2674: Start of 'ora.asm' on 'host2' failed
    CRS-2679: Attempting to clean 'ora.asm' on 'host2'
    CRS-2681: Clean of 'ora.asm' on 'host2' succeeded
    CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'host2'
    CRS-2677: Stop of 'ora.drivers.acfs' on 'host2' succeeded
    CRS-4000: Command Start failed, or completed with errors.
    2011-10-24 19:36:54: Failed to start Oracle Grid Infrastructure stack
    2011-10-24 19:36:54: ###### Begin DIE Stack Trace ######
    2011-10-24 19:36:54: Package File Line Calling
    2011-10-24 19:36:54: --------------- -------------------- ---- ----------
    2011-10-24 19:36:54: 1: main rootcrs.pl 375 crsconfig_lib::dietrap
    2011-10-24 19:36:54: 2: crsconfig_lib crsconfig_lib.pm 1272 main::__ANON__
    2011-10-24 19:36:54: 3: crsconfig_lib crsconfig_lib.pm 1171 crsconfig_lib::start_cluster
    2011-10-24 19:36:54: 4: main rootcrs.pl 803 crsconfig_lib::perform_start_cluster
    2011-10-24 19:36:54: ####### End DIE Stack Trace #######
    Shortened output from "/u01/11.2.0/grid/log/host2/agent/ohasd/oraagent_grid/oraagent_grid.log"
    2011-10-24 19:35:48.726: [ora.asm][9] {0:0:224} [start] clean {
    2011-10-24 19:35:48.726: [ora.asm][9] {0:0:224} [start] InstAgent::stop_option stop mode immediate option 1
    2011-10-24 19:35:48.726: [ora.asm][9] {0:0:224} [start] InstAgent::stop {
    2011-10-24 19:35:48.727: [ora.asm][9] {0:0:224} [start] InstAgent::stop original reason system do shutdown abort
    2011-10-24 19:35:48.727: [ora.asm][9] {0:0:224} [start] ConnectionPool::resetConnection s_statusOfConnectionMap 00ab1948
    2011-10-24 19:35:48.727: [ora.asm][9] {0:0:224} [start] ConnectionPool::resetConnection sid +ASM2 status  2
    2011-10-24 19:35:48.728: [ora.asm][9] {0:0:224} [start] Gimh::check OH /u01/11.2.0/grid SID +ASM2
    2011-10-24 19:35:48.728: [ora.asm][9] {0:0:224} [start] Gimh::check condition changes to (GIMH_NEXT_NUM) 0,1,7 exists
    2011-10-24 19:35:48.729: [ora.asm][9] {0:0:224} [start] (:CLSN00006:)AsmAgent::check failed gimh state 0
    2011-10-24 19:35:48.729: [ora.asm][9] {0:0:224} [start] AsmAgent::check ocrCheck 1 m_OcrOnline 0 m_OcrTimer 0
    2011-10-24 19:35:48.729: [ora.asm][9] {0:0:224} [start] DgpAgent::initOcrDgpSet { entry
    2011-10-24 19:35:48.730: [ora.asm][9] {0:0:224} [start] DgpAgent::initOcrDgpSet procr_get_conf: retval [0] configured [1] local only [0] error buffer []
    2011-10-24 19:35:48.730: [ora.asm][9] {0:0:224} [start] DgpAgent::initOcrDgpSet procr_get_conf: OCR loc [0], Disk Group : [+CRS]
    2011-10-24 19:35:48.730: [ora.asm][9] {0:0:224} [start] DgpAgent::initOcrDgpSet m_ocrDgpSet 015fba90 dgName CRS
    2011-10-24 19:35:48.731: [ora.asm][9] {0:0:224} [start] DgpAgent::initOcrDgpSet ocrret 0 found 1
    2011-10-24 19:35:48.731: [ora.asm][9] {0:0:224} [start] DgpAgent::initOcrDgpSet ocrDgpSet CRS
    2011-10-24 19:35:48.731: [ora.asm][9] {0:0:224} [start] DgpAgent::initOcrDgpSet exit }
    2011-10-24 19:35:48.731: [ora.asm][9] {0:0:224} [start] DgpAgent::ocrDgCheck Entry {
    2011-10-24 19:35:48.732: [ora.asm][9] {0:0:224} [start] DgpAgent::getConnxn new pool
    2011-10-24 19:35:48.732: [ora.asm][9] {0:0:224} [start] DgpAgent::getConnxn new pool m_oracleHome:/u01/11.2.0/grid m_oracleSid:+ASM2 m_usrOraEnv:
    2011-10-24 19:35:48.732: [ora.asm][9] {0:0:224} [start] ConnectionPool::ConnectionPool 2 m_oracleHome:/u01/11.2.0/grid, m_oracleSid:+ASM2, m_usrOraEnv:
    2011-10-24 19:35:48.733: [ora.asm][9] {0:0:224} [start] ConnectionPool::addConnection m_oracleHome:/u01/11.2.0/grid m_oracleSid:+ASM2 m_usrOraEnv: pConnxn:
    01fcdf10
    2011-10-24 19:35:48.733: [ora.asm][9] {0:0:224} [start] Utils::getCrsHome crsHome /u01/11.2.0/grid
    2011-10-24 19:35:51.969: [ora.asm][14] {0:0:224} [check] makeConnectStr = (DESCRIPTION=(ADDRESS=(PROTOCOL=beq)(PROGRAM=/u01/11.2.0/grid/bin/oracle)(ARGV0=o
    racle+ASM2)(ENVS='ORACLE_HOME=/u01/11.2.0/grid,ORACLE_SID=+ASM2')(ARGS='(DESCRIPTION=(LOCAL=YES)(ADDRESS=(PROTOCOL=beq)))'))(CONNECT_DATA=(SID=+ASM2)))
    2011-10-24 19:35:51.971: [ora.asm][14] {0:0:224} [check] ConnectionPool::getConnection 260 pConnxn 013e40a0
    2011-10-24 19:35:51.971: [ora.asm][14] {0:0:224} [check] DgpAgent::getConnxn connected
    2011-10-24 19:35:51.971: [ora.asm][14] {0:0:224} [check] InstConnection::connectInt: server not attached
    2011-10-24 19:35:52.190: [ora.asm][14] {0:0:224} [check] ORA-01034: ORACLE not available
    ORA-27101: shared memory realm does not exist
    SVR4 Error: 2: No such file or directory
    Process ID: 0
    Session ID: 0 Serial number: 0
    2011-10-24 19:35:52.190: [ora.asm][14] {0:0:224} [check] InstConnection::connectInt (2) Exception OCIException
    2011-10-24 19:35:52.190: [ora.asm][14] {0:0:224} [check] InstConnection:connect:excp OCIException OCI error 1034
    2011-10-24 19:35:52.190: [ora.asm][14] {0:0:224} [check] DgpAgent::queryDgStatus excp ORA-01034: ORACLE not available
    ORA-27101: shared memory realm does not exist
    SVR4 Error: 2: No such file or directory
    Process ID: 0
    Session ID: 0 Serial number: 0
    2011-10-24 19:35:52.190: [ora.asm][14] {0:0:224} [check] DgpAgent::queryDgStatus asm inst is down or going down
    2011-10-24 19:35:52.191: [ora.asm][14] {0:0:224} [check] DgpAgent::queryDgStatus dgName CRS ret 1
    2011-10-24 19:35:52.191: [ora.asm][14] {0:0:224} [check] (:CLSN00100:)DgpAgent::ocrDgCheck OCR dgName CRS state 1
    2011-10-24 19:35:52.192: [ora.asm][14] {0:0:224} [check] ConnectionPool::releaseConnection InstConnection 013e40a0
    2011-10-24 19:35:52.192: [ora.asm][14] {0:0:224} [check] AsmAgent::check ocrCheck 2 m_OcrOnline 0 m_OcrTimer 0
    2011-10-24 19:35:52.193: [ora.asm][14] {0:0:224} [check] CrsCmd::ClscrsCmdData::stat entity 1 statflag 32 useFilter 0
    2011-10-24 19:35:52.197: [ COMMCRS][23]clsc_connect: (1020d39d0) no listener at (ADDRESS=(PROTOCOL=IPC)(KEY=CRSD_UI_SOCKET))
    Please advice for any workaround or a metalink note.
    Thanks in advance!

    Thanks for the fast reply!
    - Yes, the shared storage is accessible.
    - The alert log for the +ASM2 clearly shows that ASM instance has started normally using default parameters and at one point PMON process dumped.
    - The system logs just shows that there is an error executing "crswrapexece.pl"
    System Log
    ===================
    *Oct 24 19:25:03 host2 root: [ID 702911 user.error] exec /u01/11.2.0/grid/perl/bin/perl -I/u01/11.2.0/grid/perl/lib /u01/11.2.0/grid/bin/crswrapexece.pl /*
    u01/11.2.0/grid/crs/install/s_crsconfig_host2_env.txt /u01/11.2.0/grid/bin/ohasd.bin "reboot"
    Oct 24 19:26:33 host2 oracleoks: [ID 902884 kern.notice] [Oracle OKS] mallocing log buffer, size=10485760
    Oct 24 19:26:33 host2 oracleoks: [ID 714332 kern.notice] [Oracle OKS] log buffer = 0x301780fcb50, size 10485760
    Oct 24 19:26:33 host2 oracleoks: [ID 400061 kern.notice] NOTICE: [Oracle OKS] ODLM hash size 16384
    Oct 24 19:26:33 host2 oracleoks: [ID 160659 kern.notice] NOTICE: OKSK-00004: Module load succeeded. Build information: (LOW DEBUG) USM_11.2.0.3.0_SOLAR
    IS.SPARC64_110803.1 2011/08/11 02:38:30
    Oct 24 19:26:33 host2 pseudo: [ID 129642 kern.info] pseudo-device: oracleadvm0
    Oct 24 19:26:33 host2 genunix: [ID 936769 kern.info] oracleadvm0 is /pseudo/oracleadvm@0
    Oct 24 19:26:33 host2 oracleoks: [ID 141287 kern.notice] NOTICE: ADVMK-00001: Module load succeeded. Build information: (LOW DEBUG) - USM_11.2.0.3.0_SOL
    ARIS.SPARC64_110803.1 built on 2011/08/11 02:40:17.
    Oct 24 19:26:33 host2 oracleacfs: [ID 202941 kern.notice] NOTICE: [Oracle ACFS] FCB hash size 16384
    Oct 24 19:26:33 host2 oracleacfs: [ID 671725 kern.notice] NOTICE: [Oracle ACFS] buffer cache size 511MB (79884 buckets)
    Oct 24 19:26:33 host2 oracleacfs: [ID 730054 kern.notice] NOTICE: [Oracle ACFS] DLM hash size 16384
    Oct 24 19:26:33 host2 oracleoks: [ID 617314 kern.notice] NOTICE: ACFSK-0037: Module load succeeded. Build information: (LOW DEBUG) USM_11.2.0.3.0_SOLAR
    IS.SPARC64_110803.1 2011/08/11 02:42:45
    Oct 24 19:26:33 host2 pseudo: [ID 129642 kern.info] pseudo-device: oracleacfs0
    Oct 24 19:26:33 host2 genunix: [ID 936769 kern.info] oracleacfs0 is /pseudo/oracleacfs@0
    Oct 24 19:26:36 host2 oracleoks: [ID 621795 kern.notice] NOTICE: OKSK-00010: Persistent OKS log opened at /u01/11.2.0/grid/log/host2/acfs/acfs.log.0.
    Oct 24 19:31:37 host2 last message repeated 1 time
    Oct 24 19:33:05 host2 CLSD: [ID 770310 daemon.notice] The clock on host host2 has been updated by the Cluster Time Synchronization Service to be synchr
    onous with the mean cluster time.
    ASM alert log
    ====================================================================
    <msg time='2011-10-24T19:35:48.776+01:00' org_id='oracle' comp_id='asm'
    client_id='' type='UNKNOWN' level='16'
    host_id='host2' host_addr='10.172.16.200' module=''
    pid='26406'>
    <txt>System state dump requested by (instance=2, osid=26396 (PMON)), summary=[abnormal instance termination].
    </txt>
    </msg>
    <msg time='2011-10-24T19:35:48.778+01:00' org_id='oracle' comp_id='asm'
    client_id='' type='UNKNOWN' level='16'
    host_id='host2' host_addr='10.172.16.200' module=''
    pid='26406'>
    <txt>System State dumped to trace file /u01/app/oracle/diag/asm/+asm/+ASM2/trace/+ASM2_diag_26406.trc
    </txt>
    </msg>
    <msg time='2011-10-24T19:35:48.927+01:00' org_id='oracle' comp_id='asm'
    type='UNKNOWN' level='16' host_id='host2'
    host_addr='10.172.16.200' pid='26470'>
    <txt>ORA-1092 : opitsk aborting process
    </txt>
    </msg>
    <msg time='2011-10-24T19:35:49.128+01:00' org_id='oracle' comp_id='asm'
    type='UNKNOWN' level='16' host_id='host2'
    host_addr='10.172.16.200' pid='26472'>
    <txt>ORA-1092 : opitsk aborting process
    </txt>
    </msg>
    Output from "/u01/app/oracle/diag/asm/+asm/+ASM2/trace/+ASM2_diag_26406.trc"
    REQUEST:system state dump at level 10, requested by (instance=2, osid=26396 (PMON)), summary=[abnormal instance termination].
    kjzdattdlm: Can not attach to DLM (LMON up=[TRUE], DB mounted=[FALSE]).
    ===================================================
    SYSTEM STATE (level=10)
    Orapids on dead process list: [count = 0]
    PROCESS 1:
    SO: 0x3df098b50, type: 2, owner: 0x0, flag: INIT/-/-/0x00 if: 0x3 c: 0x3
    proc=0x3df098b50, name=process, file=ksu.h LINE:12616 ID:, pg=0
    (process) Oracle pid:1, ser:0, calls cur/top: 0x0/0x0
    flags : (0x20) PSEUDO
    flags2: (0x0), flags3: (0x10)
    intr error: 0, call error: 0, sess error: 0, txn error 0
    intr queue: empty
    ksudlp FALSE at location: 0
    (post info) last post received: 0 0 0
    last post received-location: No post
    last process to post me: none
    last post sent: 0 0 0
    last post sent-location: No post
    last process posted by me: none
    (latch info) wait_event=0 bits=0
    O/S info: user: , term: , ospid: (DEAD)
    OSD pid info: Unix process pid: 0, image: PSEUDO
    SO: 0x38000cef0, type: 5, owner: 0x3df098b50, flag: INIT/-/-/0x00 if: 0x3 c: 0x3
    proc=0x0, name=kss parent, file=kss2.h LINE:138 ID:, pg=0
    PSO child state object changes :
    Dump of memory from 0x00000003DF722AC0 to 0x00000003DF722CC8
    3DF722AC0 00000000 00000000 00000000 00000000 [................]
    Repeat 31 times
    3DF722CC0 00000000 00000000 [........]
    PROCESS 2: PMON
    SO: 0x3df099bf8, type: 2, owner: 0x0, flag: INIT/-/-/0x00 if: 0x3 c: 0x3
    proc=0x3df099bf8, name=process, file=ksu.h LINE:12616 ID:, pg=0
    (process) Oracle pid:2, ser:1, calls cur/top: 0x3db6c8d30/0x3db6c8d30
    flags : (0xe) SYSTEM
    flags2: (0x0), flags3: (0x10)
    intr error: 0, call error: 0, sess error: 0, txn error 0
    intr queue: empty
    ksudlp FALSE at location: 0
    (post info) last post received: 0 0 136
    last post received-location: kjm.h LINE:1228 ID:kjmdmi: pmon to attach
    last process to post me: 3df0a2138 1 6
    last post sent: 0 0 137
    last post sent-location: kjm.h LINE:1230 ID:kjiath: pmon attached
    last process posted by me: 3df0a2138 1 6
    (latch info) wait_event=0 bits=0
    Process Group: DEFAULT, pseudo proc: 0x3debbbf40
    O/S info: user: grid, term: UNKNOWN, ospid: 26396
    OSD pid info: Unix process pid: 26396, image: oracle@host2 (PMON)
    SO: 0x3d8800c18, type: 30, owner: 0x3df099bf8, flag: INIT/-/-/0x00 if: 0x3 c: 0x3
    proc=0x3df099bf8, name=ges process, file=kji.h LINE:3669 ID:, pg=0
    GES MSG BUFFERS: st=emp chunk=0x0 hdr=0x0 lnk=0x0 flags=0x0 inc=0
    outq=0 sndq=0 opid=0 prmb=0x0
    mbg=(0 0) mbg=(0 0) mbg[r]=(0 0)
    fmq=(0 0) fmq=(0 0) fmq[r]=(0 0)
    mop[s]=0 mop[q]=0 pendq=0 zmbq=0
    nonksxp_recvs=0
    ------------process 3d8800c18--------------------
    proc version : 0
    Local inst : 2
    pid : 26396
    lkp_inst : 2
    svr_mode : 0
    proc state : KJP_FROZEN
    Last drm hb acked : 0
    flags : x50
    ast_rcvd_svrmod : 0
    current lock op : 0
    Total accesses : 1
    Imm. accesses : 0
    Locks on ASTQ : 0
    Locks Pending AST : 0
    Granted locks : 0
    AST_Q:
    PENDING_Q:
    GRANTED_Q:
    SO: 0x3d9835198, type: 14, owner: 0x3df099bf8, flag: INIT/-/-/0x00 if: 0x1 c: 0x1
    proc=0x3df099bf8, name=channel handle, file=ksr2.h LINE:367 ID:, pg=0
    (broadcast handle) 3d9835198 flag: (2) ACTIVE SUBSCRIBER,
    owner: 3df099bf8 - ospid: 26396
    event: 1, last message event: 1,
    last message waited event: 1,
    next message: 0(0), messages read: 0
    channel: (3d9934df8) PMON actions channel [name: 2]
    scope: 7, event: 1, last mesage event: 0,
    publishers/subscribers: 0/1,
    messages published: 0
    heuristic msg queue length: 0
    SO: 0x3d9835008, type: 14, owner: 0x3df099bf8, flag: INIT/-/-/0x00 if: 0x1 c: 0x1
    proc=0x3df099bf8, name=channel handle, file=ksr2.h LINE:367 ID:, pg=0
    (broadcast handle) 3d9835008 flag: (2) ACTIVE SUBSCRIBER,
    owner: 3df099bf8 - ospid: 26396
    event: 1, last message event: 1,
    last message waited event: 1,
    next message: 0(0), messages read: 0
    channel: (3d9941e40) scumnt mount lock [name: 157]
    scope: 1, event: 12, last mesage event: 0,
    publishers/subscribers: 0/12,
    messages published: 0
    heuristic msg queue length: 0
    SO: 0x3de4a2b80, type: 4, owner: 0x3df099bf8, flag: INIT/-/-/0x00 if: 0x3 c: 0x3
    proc=0x3df099bf8, name=session, file=ksu.h LINE:12624 ID:, pg=0
    (session) sid: 33 ser: 1 trans: 0x0, creator: 0x3df099bf8
    flags: (0x51) USR/- flags_idl: (0x1) BSY/-/-/-/-/-
    flags2: (0x409) -/-/INC
    DID: , short-term DID:
    txn branch: 0x0
    oct: 0, prv: 0, sql: 0x0, psql: 0x0, user: 0/SYS
    ksuxds FALSE at location: 0
    service name: SYS$BACKGROUND
    Current Wait Stack:
    Not in wait; last wait ended 0.666415 sec ago
    Wait State:
    fixed_waits=0 flags=0x21 boundary=0x0/-1
    Session Wait History:
    elapsed time of 0.666593 sec since last wait
    0: waited for 'pmon timer'
    duration=0x12c, =0x0, =0x0
    wait_id=63 seq_num=64 snap_id=1
    wait times: snap=3.000089 sec, exc=3.000089 sec, total=3.000089 sec
    wait times: max=3.000000 sec
    wait counts: calls=1 os=1
    occurred after 0.002067 sec of elapsed time
    1: waited for 'pmon timer'
    duration=0x12c, =0x0, =0x0
    wait_id=62 seq_num=63 snap_id=1
    wait times: snap=3.010111 sec, exc=3.010111 sec, total=3.010111 sec
    wait times: max=3.000000 sec
    wait counts: calls=1 os=1
    occurred after 0.001926 sec of elapsed time
    2: waited for 'pmon timer'
    duration=0x12c, =0x0, =0x0
    wait_id=61 seq_num=62 snap_id=1
    wait times: snap=3.125286 sec, exc=3.125286 sec, total=3.125286 sec
    wait times: max=3.000000 sec
    wait counts: calls=1 os=1
    occurred after 0.003361 sec of elapsed time
    3: waited for 'pmon timer'
    duration=0x12c, =0x0, =0x0
    wait_id=60 seq_num=61 snap_id=1
    wait times: snap=3.000081 sec, exc=3.000081 sec, total=3.000081 sec
    wait times: max=3.000000 sec
    wait counts: calls=1 os=1
    occurred after 0.002102 sec of elapsed time
    4: waited for 'pmon timer'
    duration=0x12c, =0x0, =0x0

  • The Script root.sh problem - ora.asm and ASM and Clusterware Stack failed

    Folks,
    Hello. I am installing Oracle 11gR2 RAC using 2 VMs (rac1 and rac2) whose OS are Oracle Linux 5.6 in VMPlayer according to the website http://appsdbaworkshop.blogspot.com/2011/10/11gr2-rac-on-linux-56-using-vmware.html
    I am installing Grid infrastructure. On step 9 of 10 - execute script /u01/app/grid/root.sh for 2 VMs rac1 and rac2.
    After run root.sh in rac1 successfully. I run root.sh in rac2 and get an error as below:
    [root@rac2 grid]# ./root.sh
    Running Oracle 11g root.sh script...
    The following environment variables are set as:
    ORACLE_OWNER= ora11g
    ORACLE_HOME= /u01/app/grid
    Enter the full pathname of the local bin directory: [usr/local/bin]: /usr/local/bin
    Copying dbhome to /usr/local/bin ...
    Copying oraenv to /usr/local/bin ...
    Copying coraenv to /usr/local/bin ...
    Creating /etc/oratab file...
    Entries will be added to the /etc/oratab file as needed by
    Database Configuration Assistant when a database is created
    Finished running generic part of root.sh script.
    Now product-specific root actions will be performed.
    2012-03-05 16:32:52: Parsing the host name
    2012-03-05 16:32:52: Checking for super user privileges
    2012-03-05 16:32:52: User has super user privileges
    Using configuration parameter file: /u01/app/grid/crs/install/crsconfig_params
    Creating trace directory
    LOCAL ADD MODE
    Creating OCR keys for user 'root', privgrp 'root'..
    Operation successful.
    Adding daemon to inittab
    CRS-4123: Oracle High Availability Services has been started.
    ohasd is starting
    CRS-4402: The CSS daemon was started in exclusive mode but found an active CSS daemon on node rac1, number 1, and is terminating
    An active cluster was found during exclusive startup, restarting to join the cluster
    CRS-2672: Attempting to start 'ora.mdnsd' on 'rac2'
    CRS-2676: Start of 'ora.mdnsd' on 'rac2' succeeded
    CRS-2672: Attempting to start 'ora.gipcd' on 'rac2'
    CRS-2676: Start of 'ora.gipcd' on 'rac2' succeeded
    CRS-2672: Attempting to start 'ora.gpnpd' on 'rac2'
    CRS-2676: Start of 'ora.gpnpd' on 'rac2' succeeded
    CRS-2672: Attempting to start 'ora.cssdmonitor' on 'rac2'
    CRS-2676: Start of 'ora.cssdmonitor' on 'rac2' succeeded
    CRS-2672: Attempting to start 'ora.cssd' on 'rac2'
    CRS-2672: Attempting to start 'ora.diskmon' on 'rac2'
    CRS-2676: Start of 'ora.diskmon' on 'rac2' succeeded
    CRS-2676: Start of 'ora.cssd' on 'rac2' succeeded
    CRS-2672: Attempting to start 'ora.ctssd' on 'rac2'
    Start action for octssd aborted
    CRS-2676: Start of 'ora.ctssd' on 'rac2' succeeded
    CRS-2672: Attempting to start 'ora.drivers.acfs' on 'rac2'
    CRS-2672: Attempting to start 'ora.asm' on 'rac2'
    CRS-2676: Start of 'ora.drivers.acfs' on 'rac2' succeeded
    CRS-2676: Start of 'ora.asm' on 'rac2' succeeded
    CRS-2664: Resource 'ora.ctssd' is already running on 'rac2'
    CRS-4000: Command Start failed, or completed with errors.
    Command return code of 1 (256) from command: /u01/app/grid/bin/crsctl start resource ora.asm -init
    Start of resource "ora.asm -init" failed
    Failed to start ASM
    Failed to start Oracle Clusterware stack
    [root@rac2 grid]#
    As we see the output above, at the end of the output
    1) Start of resource ora.asm -init failed
    2) Failed to start ASM
    3) Failed to start Oracle Clusterware stack
    The runInstaller is in the first VM rac1. My question is:
    Do any folk understand how to solve the script root.sh in rac2 problem ( 3 fails of ora.asm, ASM and Clusterware stack as above) ?
    Thanks.

    Please check there is no firewall exist:
    try this like:
    root.sh fails on second node
    MOS note:
    11gR2 Grid: root.sh Fails to Start the Clusterware on the Second Node Due to Firewall on Private Network [ID 981357.1]
    Grid Infrastructure 11.2.0.2 Installation or Upgrade may fail due to Multicasting Requirement [ID 1212703.1] (Most probabily this issue)

  • Failed to restart the CSSD during the interconnect failure

    Hi all,
    I run a small ATP on my LAB where i have
    - 2x nodes RAC 11.2.0.2 & ASM (my OCR & Voting files are stored on ASM)
    - 1 public interface <> eth0
    - 1 private interface <> eth1
    - 1 SCAN IP defined in the /etc/hosts file (i'm not using DNS or GNS)
    The test i run was to shutdown the private interface (eth1) on node 1 and i saw that
    1) all cluster services and cluster daemons on node 2 were killed and node 2 was evicted from the cluster by node 1
    2) all new connections were redirected to the survived node
    3) Oracle OHASD daemon was restarted on node 2 and tried to start the cluster services without success because private network between cluster nodes was down
    Up to here everything worked as expected but once i turn on eth1 it took ~ 9 minutes for the CSSD to startup and bring all the components up & running.
    The node2 alert logs showes
    [ctssd(12949)]CRS-2402:The Cluster Time Synchronization Service aborted on host node2. Details at (:ctss_css_init1:) in /u01/oracle/installed/oracle_cluster-11.2.0.2-1/log/node2/ctssd/octssd.log.
    2011-04-13 08:09:40.978
    [ohasd(5058)]CRS-2765:Resource 'ora.cssd' has failed on server 'node2'.
    2011-04-13 08:09:40.985
    [/u01/oracle/installed/oracle_cluster-11.2.0.2-1/bin/oraagent.bin(5764)]CRS-5011:Check of resource "+ASM" failed: details at "(:CLSN00006:)" in "/u01/oracle/installed/oracle_cluster-11.2.0.2-1/log/node2/agent/ohasd/oraagent_oracle/oraagent_oracle.log";
    2011-04-13 08:09:41.169
    [ohasd(5058)]CRS-2765:Resource 'ora.asm' has failed on server 'node2'.
    2011-04-13 08:09:50.337
    [cssd(13103)]CRS-1713:CSSD daemon is started in clustered mode
    2011-04-13 08:10:05.833
    [cssd(13103)]CRS-1707:Lease acquisition for node node2 number 2 completed
    2011-04-13 08:10:07.119
    [cssd(13103)]CRS-1605:CSSD voting file is online: ORCL:CRS_DISK1_2G; details in /u01/oracle/installed/oracle_cluster-11.2.0.2-1/log/node2/cssd/ocssd.log.
    2011-04-13 08:10:07.121
    [cssd(13103)]CRS-1605:CSSD voting file is online: ORCL:CRS_DISK2_2G; details in /u01/oracle/installed/oracle_cluster-11.2.0.2-1/log/node2/cssd/ocssd.log.
    2011-04-13 08:10:07.143
    [cssd(13103)]CRS-1605:CSSD voting file is online: ORCL:CRS_DISK1_2G; details in /u01/oracle/installed/oracle_cluster-11.2.0.2-1/log/node2/cssd/ocssd.log.
    2011-04-13 08:19:49.386
    [/u01/oracle/installed/oracle_cluster-11.2.0.2-1/bin/cssdagent(13091)]CRS-5818:Aborted command 'start for resource: ora.cssd 1 1' for resource 'ora.cssd'. Details at (:CRSAGF00113:) {0:6:7} in /u01/oracle/installed/oracle_cluster-11.2.0.2-1/log/node2/agent/ohasd/oracssdagent_root/oracssdagent_root.log.
    2011-04-13 08:19:49.387
    [cssd(13103)]CRS-1656:The CSS daemon is terminating due to a fatal error; Details at (:CSSSC00012:) in /u01/oracle/installed/oracle_cluster-11.2.0.2-1/log/node2/cssd/ocssd.log
    2011-04-13 08:19:49.387
    [cssd(13103)]CRS-1603:CSSD on node node2 shutdown by user.
    2011-04-13 08:19:54.501
    [ohasd(5058)]CRS-2765:Resource 'ora.cssdmonitor' has failed on server 'node2'.
    2011-04-13 08:19:57.723
    [cssd(17068)]CRS-1713:CSSD daemon is started in clustered mode
    2011-04-13 08:20:01.177
    [ohasd(5058)]CRS-2765:Resource 'ora.diskmon' has failed on server 'node2'.
    2011-04-13 08:20:13.167
    [cssd(17068)]CRS-1707:Lease acquisition for node node2 number 2 completed pay attention at the timestamp 08:10:07.143 & 08:19:49.386
    The error in the oracssdagent_root.log is
    2011-04-13 08:09:49.286: [CLSFRAME][3014212592] New Framework state: 2
    2011-04-13 08:09:49.286: [CLSFRAME][3014212592] M2M is starting...
    2011-04-13 08:09:49.288: [ CRSCOMM][3014212592] Ipc: Starting send thread
    2011-04-13 08:09:49.288: [ CRSCOMM][1092061504] Ipc: sendWork thread started.
    2011-04-13 08:09:49.289: [ CRSCOMM][1105643840] IpcC: IPC Client thread started listening
    2011-04-13 08:09:49.289: [ CRSCOMM][1105643840] IpcC: Received member number of 10
    2011-04-13 08:09:49.290: [CLSFRAME][3014212592] New IPC Member:{Relative|Node:0|Process:0|Type:2}:OHASD:node2
    2011-04-13 08:09:49.290: [CLSFRAME][3014212592] New process connected to us ID:{Relative|Node:0|Process:0|Type:2} Info:OHASD:node2
    2011-04-13 08:09:49.291: [CLSFRAME][3014212592] Tints initialized with nodeId: 0 procId: 10
    2011-04-13 08:09:49.291: [CLSFRAME][3014212592] Starting thread model named: MultiThread
    2011-04-13 08:09:49.292: [CLSFRAME][3014212592] Starting thread model named: TimerSharedTM
    2011-04-13 08:09:49.293: [CLSFRAME][3014212592] New Framework state: 3
    2011-04-13 08:09:49.293: [    AGFW][3014212592] Agent Framework started successfully
    2011-04-13 08:09:49.293: [    AGFW][1116150080] {0:10:2} Agfw engine module has enabled...
    2011-04-13 08:09:49.293: [CLSFRAME][1116150080] {0:10:2} Module Enabling is complete
    2011-04-13 08:09:49.293: [CLSFRAME][1116150080] {0:10:2} New Framework state: 6
    2011-04-13 08:09:49.294: [CLSFRAME][3014212592] M2M is now powered by a doWork() thread.
    2011-04-13 08:09:49.294: [    AGFW][1116150080] {0:10:2} Agent is started with userid: root , expected user: root
    2011-04-13 08:09:49.294: [   AGENT][1116150080] {0:10:2} Static Version 11.2.0.2.0
    2011-04-13 08:09:49.294: [    AGFW][1116150080] {0:10:2} Agent sending message to PE: AGENT_HANDSHAKE[Proxy] ID 20484:11
    2011-04-13 08:09:49.302: [    AGFW][1116150080] {0:10:2} Agent received the message: RESTYPE_ADD[ora.cssd.type] ID 8196:12358
    2011-04-13 08:09:49.302: [    AGFW][1116150080] {0:10:2} Added new restype: ora.cssd.type
    2011-04-13 08:09:49.303: [    AGFW][1116150080] {0:10:2} Agent sending last reply for: RESTYPE_ADD[ora.cssd.type] ID 8196:12358
    2011-04-13 08:09:49.305: [    AGFW][1116150080] {0:10:2} Agent received the message: RESOURCE_ADD[ora.cssd 1 1] ID 4356:12359
    2011-04-13 08:09:49.305: [    AGFW][1116150080] {0:10:2} Added new resource: ora.cssd 1 1 to the agfw
    2011-04-13 08:09:49.306: [    AGFW][1116150080] {0:10:2} Agent sending last reply for: RESOURCE_ADD[ora.cssd 1 1] ID 4356:12359
    2011-04-13 08:09:49.308: [    AGFW][1116150080] {0:6:7} Agent received the message: RESOURCE_START[ora.cssd 1 1] ID 4098:12360
    2011-04-13 08:09:49.308: [    AGFW][1116150080] {0:6:7} Preparing START command for: ora.cssd 1 1
    2011-04-13 08:09:49.308: [    AGFW][1116150080] {0:6:7} ora.cssd 1 1 state changed from: UNKNOWN to: STARTING
    2011-04-13 08:09:49.309: [ora.cssd][1114048832] {0:6:7} [start] clsncssd_cssdstart: Start action called
    2011-04-13 08:09:49.309: [ora.cssd][1114048832] {0:6:7} [start] clsncssd_getattr: attr OMON_INITRATE, value 1000
    2011-04-13 08:09:49.309: [ora.cssd][1114048832] {0:6:7} [start] clsncssd_getattr: attr OMON_POLLRATE, value 500
    2011-04-13 08:09:49.309: [ora.cssd][1114048832] {0:6:7} [start] clsncssd_getattr: attr ORA_OPROCD_MODE, value
    2011-04-13 08:09:49.310: [ora.cssd][1114048832] {0:6:7} [start] clsncssd_getattr: attr PROCD_TIMEOUT, value 1000
    2011-04-13 08:09:49.310: [ora.cssd][1114048832] {0:6:7} [start] clsncssd_getattr: attr LOGGING_LEVEL, value 1
    2011-04-13 08:09:49.310: [ora.cssd][1114048832] {0:6:7} [start] clsncssd_cssdstart: loglevels CSSD=2,GIPCNM=2,GIPCGM=2,GIPCCM=2,CLSF=0,SKGFD=0,GPNP=1,OLR=0
    2011-04-13 08:09:49.313: [ora.cssd][1114048832] {0:6:7} [start] clsncssd_cssdstart: START action for resource /u01/oracle/installed/oracle_cluster-11.2.0.2-1/bin/ocssd: SUCCESS
    2011-04-13 08:09:49.313: [ora.cssd][1114048832] {0:6:7} [start] clsncssd_waitomon: start waiting
    2011-04-13 08:09:49.313: [ CSSCLNT][1098377536]clsssInitNative: Init for agent
    2011-04-13 08:09:50.317: [ CSSCLNT][1098377536]clsssInitNative: Init for agent
    2011-04-13 08:09:51.319: [ CSSCLNT][1098377536]clsssInitNative: Init for agent
    2011-04-13 08:09:51.322: [ CSSCLNT][1098377536]clssnsqueryfatal: css is fatal = 0
    2011-04-13 08:09:51.322: [ USRTHRD][1098377536] clsncssd_thrdspawn: spawn OPROCD succ
    2011-04-13 08:09:51.322: [ USRTHRD][1098377536] clsncssd_thrdspawn: spawn POLLMSG succ
    2011-04-13 08:09:51.323: [ USRTHRD][1099954496] clsnpollmsg_main: starting pollmsg thread
    2011-04-13 08:09:51.323: [ USRTHRD][1107745088] clsnproc_main: timeout of procd cannot be 0, now we set to default 1000.
    2011-04-13 08:09:51.323: [ USRTHRD][1117727040] clsnwork_main: starting worker thread
    2011-04-13 08:09:51.323: [ USRTHRD][1098377536] clsncssd_thrdspawn: spawn WORKER succ
    2011-04-13 08:09:51.323: [ USRTHRD][1107745088] clsnproc_main: starting oprocd
    2011-04-13 08:09:51.323: [ USRTHRD][1098377536] clsncssd_thrdspawn: spawn KILL succ
    2011-04-13 08:10:07.151: [ USRTHRD][1098377536] clsnomon_init: css init done, nodenum 2
    2011-04-13 08:10:07.151: [ USRTHRD][1098377536] clsnomon_WaitToRegister: waiting for first reconfiguration and kgzf initialization
    2011-04-13 08:19:49.385: [CLSFRAME][3014212592] TM [MultiThread] is changing desired thread # to 3. Current # is 2
    2011-04-13 08:19:49.387: [    AGFW][1111947584] {0:6:7} Created alert : (:CRSAGF00113:) :  Aborting the command: start for resource: ora.cssd 1 1
    2011-04-13 08:19:49.387: [ora.cssd][1111947584] {0:6:7} [start] clsncssd_cssdabort: sending shutdown abort to CSS with new ctx
    2011-04-13 08:19:49.387: [ CSSCLNT][1098377536]clsssRecvMsg: wrong type request (0) on 0xc9 ret 0
    2011-04-13 08:19:49.387: [ CSSCLNT][1098377536]clssnskgzfdone: RPC failed rc 1
    2011-04-13 08:19:49.387: [ USRTHRD][1098377536] clsnomon_WaitToRegister: exadata initialization completed with rc=1
    2011-04-13 08:19:49.387: [ USRTHRD][1098377536] clsnomon_init: problems in the CSS to allow OMON registration 2
    2011-04-13 08:19:49.387: [ USRTHRD][1098377536] clsnomon_cleanup: to exit status = 2
    2011-04-13 08:19:49.387: [ USRTHRD][1098377536] clsnomon_cleanup: failure, sending shutdown immediate to CSS
    2011-04-13 08:19:49.387: [ USRTHRD][1098377536] CHECK action is in progress, Rejecting the check action requested by entry point for ora.cssd
    2011-04-13 08:19:49.426: [    AGFW][2008402928] Starting the agent: /u01/oracle/installed/oracle_cluster-11.2.0.2-1/log/node2/agent/ohasd/oracssdagent_root/
    2011-04-13 08:19:49.426: [   AGENT][2008402928] Agent framework initialized, Process Id = 17013
    2011-04-13 08:19:49.426: [ USRTHRD][2008402928] to enter agent main
    2011-04-13 08:19:49.426: [ USRTHRD][2008402928] clsscssd_main: New soft limit for stack size is 1572864, hard limit is 4294967295
    2011-04-13 08:19:49.434: [ USRTHRD][2008402928] clsncssd_main: setting priority to 4
    2011-04-13 08:19:49.434: [ USRTHRD][2008402928]  *** Agent Framework Started *** Do you have any idea why it took so long to bring all the components up & running?
    Thanks a lot!!
    G

    Hi,
    there is an internal timer for the clusterware ressources regarding restarting the ressources.
    In case of a node eviction or clusterstack reboot the clusterware tries to startup again.
    If the issue still persists, CRS will wait for some time to start the stack again. This "restart" try is based on a timer, which is set to 600 seconds (note this is not the ORA_CHECK_TIMEOUT) but the STARTUP_TIMEOUT.
    Since a missing interconnect does have some implications (not only on the network but on the whole stack) it is expected, that the cluster does not start so fast automatically (because it still has the first start running.
    There is even another "issue" connected to this - Oracle will only try several times (FAILURE_COUNT/FAILURE_THRESHOLD) to restart ressources. If he cannot restart cssd/crsd for several times, OCW will not try to startup automatically, but expects the administrator to solve the error and then startup again.
    But actually this does make sense:
    We have to give some time for an error to be resolved, before we start automatically. It does not matter if the restart of the node is delayed by this, because
    => If the error is fixed automatically, it will normally be fixed after a cluster/node reboot and hence cluster will come up
    => If the error is not fixed automatically, but manually, it can be expected that the administrator tells clusterware the issue is resolved. He does that by simply starting the stack (crsctl start crs)
    => If the error is fixed automaticall, but fixing took a while (lets say 15 minutes), it does not really matter if clusterware needs 10 more minutes to come up.
    So what you see is expected, and wanted.
    It would cost way too much to monitor all ressources regarding cluster problems and trigger a startup....
    Sebastian

  • Asm diskgroup does not exist or not mounted

    Hi,
    iam have configured a 2 node cluster and now when i rebooted and again started my rac database so all the applications are offline so i stop them with crs_stop and then restarted but my databases asm and instance service is not up so in that case i tried to connect to database or to instance but i was not able to so then i thought of connecting to asm instance and then the status is as follows
    SQL> select name ,state ,mount_status from v$asm_disk ;
    NAME STATE MOUNT_S
    NORMAL CLOSED
    NORMAL CLOSED
    FG_0000 NORMAL CACHED
    SQL> alter database open ;
    alter database open
    ERROR at line 1:
    ORA-15000: command disallowed by current instance type
    SQL> alter diskgroup all mount ;
    alter diskgroup all mount
    ERROR at line 1:
    ORA-15032: not all alterations performed
    ORA-15130: diskgroup "DG1" is being dismounted
    ORA-15066: offlining disk "DG1_0001" may result in a data loss
    SQL>
    SQL>
    so in that cse the contents are as follows
    on Jul 11 21:58:46 2011
    Dirty detach reconfiguration started (old inc 2, new inc 2)
    List of nodes:
    0 1
    Global Resource Directory partially frozen for dirty detach
    * dirty detach - domain 1 invalid = TRUE
    0 GCS resources traversed, 0 cancelled
    Dirty Detach Reconfiguration complete
    Mon Jul 11 21:58:46 2011
    freeing rdom 1
    Mon Jul 11 21:58:47 2011
    WARNING: dirty detached from domain 1
    Mon Jul 11 21:58:47 2011
    ERROR: diskgroup DG1 was not mounted
    Mon Jul 11 21:58:47 2011
    WARNING: PST-initiated MANDATORY DISMOUNT of group DG1 not performed - group not mounted
    Mon Jul 11 21:58:47 2011
    Errors in file /u01/app/oracle/admin/+ASM/bdump/+asm1_b000_18030.trc:
    ORA-15001: diskgroup "DG1" does not exist or is not mounted
    ORA-15001: diskgroup "DG1" does not exist or is not mounted
    so iam not able to figure out please help me in solving this
    Thanks in Advance
    Regards
    Kavita

    Hi,
    Sir i wait for many times but the major issue is iam facing that all aplications when i start my machine are offline or unknown........ then when i exec crsctl stop crs and crsctl start crs i wait a lot and then i exec crrs_stat t that time all applications are either unknown or offline............ then i stop forcily all applications which are unknown finally bring all to offline status and then restart but in this case i exec srvctl start nodeapps - n rac1 and nrac2 these things work but tried with database or service or instnace that ownt work it give me the following error
    and then i sart instance or database through sqlplus and if i do any transcations then i get the follwoing error .........
    PRKP-1001 : Error starting instance devdb1 on node rac1
    CRS-0215: Could not start resource 'ora.devdb.devdb1.inst'
    and now if iam trying to start my asm instance iam have already posted through what iam getting and this i have tried for 2 diffemt 2 node cluster but facing the same issue
    2011-07-12 22:02:02.500: [  CRSRES][3772144560]0startRunnable: setting CLI values
    2011-07-12 22:02:03.235: [  CRSRES][3772144560]0Attempting to start `ora.prod1.LISTENER_PROD1.lsnr` on member `prod1`
    2011-07-12 22:02:06.197: [  CRSRES][3772144560]0Start of `ora.prod1.LISTENER_PROD1.lsnr` on member `prod1` succeeded.
    Sir this from
    (0x8364680) pid() proto(10:2:1:1)
    [    CSSD]2011-07-12 22:46:09.669 [91069360] >TRACE: clssgmClientConnectMsg: Connect from con(0x8350880) proc(0x8364680) pid() proto(10:2:1:1)
    [    CSSD]2011-07-12 22:47:03.826 [91069360] >TRACE: clssgmClientConnectMsg: Connect from con(0x8352a08) proc(0x8364680) pid() proto(10:2:1:1)
    [    CSSD]2011-07-12 22:47:04.385 [91069360] >TRACE: clssgmClientConnectMsg: Connect from con(0x835bf18) proc(0x835ed88) pid() proto(10:2:1:1)
    [    CSSD]2011-07-12 22:47:17.938 [91069360] >TRACE: clssgmClientConnectMsg: Connect from con(0x835bf18) proc(0x8364680) pid() proto(10:2:1:1)
    [    CSSD]2011-07-12 22:48:27.641 [91069360] >TRACE: clssgmClientConnectMsg: Connect from con(0x8350880) proc(0x8364680) pid() proto(10:2:1:1)
    Thanks in Advance
    Regards
    Kavita

  • Asm 11.1.0.6 root.sh  fails to start css

    We have Oracle ASM and installing 11.1.0.6 on a new machine (eventually will apply 11.1.0.7 patchset). So we are on 11g R1 in production environments.
    When installing Oracle ASM 11.1.0.6, the root.sh fails
    I checked various metalink notes, all settings are OK. We have ASMLib and i have configured ASMLib with the new disks prior to the ASM installation
    Just to rule out the rootcause is with ASMLib , I have also disabled it. Still same problem. I ran the usual localconfig delete and localconfig add too.
    Startup will be queued to init within 30 seconds.
    Checking the status of new Oracle init process...
    Expecting the CRS daemons to be up within 600 seconds.
    Giving up: Oracle CSS stack appears NOT to be running.+
    Oracle CSS service would not start as installed+
    Automatic Storage Management(ASM) cannot be used until Oracle CSS service is started+
    Finished product-specific root actions.+

    I applied the 11.1.0.7 patchset. Still same problem.
    Strangely, there are no logfiles in $ASM_HOME/log/<hostname>/cssd/ - This directory is empty
    The below are the final messages in root.sh
    Startup will be queued to init within 30 seconds.
    Checking the status of new Oracle init process...
    Expecting the CRS daemons to be up within 600 seconds.
    Giving up: Oracle CSS stack appears NOT to be running.
    Oracle CSS service would not start as installed
    Automatic Storage Management(ASM) cannot be used until Oracle CSS service is started

  • Not able to start the oracle database with ASM

    HI all,
    WE are using oracle10.2.0.1 on OEL5.
    I have mounted ASM but when i try to start the oracle database, it is asking "db is already started shut it down first"
    [root@localhost ~]# /etc/init.d/oracleasm listdisks
    VOL1
    VOL2
    [root@localhost ~]#
    [root@localhost ~]# su - oracle
    -bash-3.2$ sqlplus /nolog
    SQL*Plus: Release 10.2.0.1.0 - Production on Mon May 31 02:12:06 2010
    Copyright (c) 1982, 2005, Oracle.  All rights reserved.
    SQL> shutdown
    ORA-01012: not logged on
    SQL> startp mount
    SP2-0734: unknown command beginning "startp mou..." - rest of line ignored.
    SQL> conn / as sysdba
    Connected.
    SQL> shutdown
    ASM diskgroups dismounted
    ASM instance shutdown
    SQL> startup mount
    ASM instance started
    Total System Global Area  130023424 bytes
    Fixed Size                  2019032 bytes
    Variable Size             102838568 bytes
    ASM Cache                  25165824 bytes
    ASM diskgroups mounted
    SQL> exit
    Disconnected from Oracle Database 10g Enterprise Edition Release 10.2.0.1.0 - 64bit Production
    With the Partitioning, OLAP and Data Mining options
    -bash-3.2$ export ORACLE_SID=dbtest
    -bash-3.2$ sqlplus /nolog
    SQL*Plus: Release 10.2.0.1.0 - Production on Mon May 31 02:13:30 2010
    Copyright (c) 1982, 2005, Oracle.  All rights reserved.
    SQL> exit
    -bash-3.2$ cd /u01/app/oracle/admin/dbtest/pfile/
    -bash-3.2$ ls -l
    total 8
    -rw-r----- 1 oracle oinstall 2442 May 27 23:07 init.ora.427201023913
    -rw-r--r-- 1 oracle oinstall 1406 May 31 01:44 sqlnet.log
    -bash-3.2$ pwd
    /u01/app/oracle/admin/dbtest/pfile
    -bash-3.2$ sqlplus /nolog
    SQL*Plus: Release 10.2.0.1.0 - Production on Mon May 31 02:14:00 2010
    Copyright (c) 1982, 2005, Oracle.  All rights reserved.
    SQL> exit
    -bash-3.2$ sqlplus "/as sysdba"
    SQL*Plus: Release 10.2.0.1.0 - Production on Mon May 31 02:14:09 2010
    Copyright (c) 1982, 2005, Oracle.  All rights reserved.
    Connected to:
    Oracle Database 10g Enterprise Edition Release 10.2.0.1.0 - 64bit Production
    With the Partitioning, OLAP and Data Mining options
    SQL> startup pfile="/u01/app/oracle/admin/dbtest/pfile/init.ora.427201023913';
    ORA-01081: cannot start already-running ORACLE - shut it down first
    SQL>
    -bash-3.2$ ps -ef|grep ora
    root      5501  5483  0 00:32 ?        00:00:00 hald-addon-storage: polling /dev/hdc
    root      6333     1  0 00:32 ?        00:00:00 /bin/su -l oracle -c sh -c 'cd /u01/app/oracle/product/10.1.0/db_1/log/localhost/cssd;  ulimit -c unlimited; exec /u01/app/oracle/product/10.1.0/db_1/bin/ocssd '
    oracle    6527  6333  0 00:33 ?        00:00:00 /u01/app/oracle/product/10.1.0/db_1/bin/ocssd.bin
    oracle    6849     1  0 00:49 ?        00:00:00 /u01/app/oracle/product/10.1.0/db_1/bin/tnslsnr LISTENER -inherit
    root      7637  7617  0 01:53 pts/2    00:00:00 su - oracle
    oracle    7638  7637  0 01:53 pts/2    00:00:00 -bash
    root      7751  6763  0 02:11 pts/1    00:00:00 su - oracle
    oracle    7752  7751  0 02:11 pts/1    00:00:00 -bash
    oracle    7773     1  0 02:12 ?        00:00:00 oracledbtest (DESCRIPTION=(LOCAL=YES)(ADDRESS=(PROTOCOL=beq)))
    oracle    7777     1  0 02:13 ?        00:00:00 asm_pmon_dbtest
    oracle    7779     1  0 02:13 ?        00:00:00 asm_psp0_dbtest
    oracle    7781     1  0 02:13 ?        00:00:00 asm_mman_dbtest
    oracle    7783     1  0 02:13 ?        00:00:00 asm_dbw0_dbtest
    oracle    7785     1  0 02:13 ?        00:00:00 asm_lgwr_dbtest
    oracle    7787     1  0 02:13 ?        00:00:00 asm_ckpt_dbtest
    oracle    7789     1  0 02:13 ?        00:00:00 asm_smon_dbtest
    oracle    7791     1  0 02:13 ?        00:00:00 asm_rbal_dbtest
    oracle    7793     1  0 02:13 ?        00:00:00 asm_gmon_dbtest
    oracle    7805  7752  0 02:15 pts/1    00:00:00 ps -ef
    oracle    7806  7752  0 02:15 pts/1    00:00:00 grep ora
    -bash-3.2$
    {code}
    so anybody can help me how to start the oracle database....                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               

    -bash-3.2$ sqlplus "/as sysdba"
    SQL*Plus: Release 10.2.0.1.0 - Production on Mon May 31 02:26:11 2010
    Copyright (c) 1982, 2005, Oracle.  All rights reserved.
    Connected to:
    Oracle Database 10g Enterprise Edition Release 10.2.0.1.0 - 64bit Production
    With the Partitioning, OLAP and Data Mining options
    SQL> show parameter instance
    NAME                                 TYPE        VALUE
    active_instance_count                integer
    cluster_database_instances           integer     1
    instance_groups                      string
    instance_name                        string      dbtest
    instance_number                      integer     0
    instance_type                        string      asm
    open_links_per_instance              integer     4
    parallel_instance_group              string
    parallel_server_instances            integer     1
    SQL>
    -bash-3.2$ sqlplus "/as sysdba"
    SQL*Plus: Release 10.2.0.1.0 - Production on Mon May 31 02:26:11 2010
    Copyright (c) 1982, 2005, Oracle.  All rights reserved.
    Connected to:
    Oracle Database 10g Enterprise Edition Release 10.2.0.1.0 - 64bit Production
    With the Partitioning, OLAP and Data Mining options
    SQL> show paramete instance_name
    SP2-0158: unknown SHOW option "paramete"
    instance "local"
    SP2-0158: unknown SHOW option "_name"
    SQL> show parameter instance
    NAME                                 TYPE        VALUE
    active_instance_count                integer
    cluster_database_instances           integer     1
    instance_groups                      string
    instance_name                        string      dbtest
    instance_number                      integer     0
    instance_type                        string      asm
    open_links_per_instance              integer     4
    parallel_instance_group              string
    parallel_server_instances            integer     1
    SQL>
    -bash-3.2$ ps -ef|grep pmon
    oracle    7777     1  0 02:13 ?        00:00:00 asm_pmon_dbtest
    oracle    7851  7752  0 02:32 pts/1    00:00:00 grep pmon
    -bash-3.2$
    -bash-3.2$

  • Installing ASM in it's own home (non-RAC environment)

    I am installing ASM
    When I run root.sh I get the following error:Expecting the CRS daemons to be up within 600 seconds.
    Giving up: Oracle CSS stack appears NOT to be running.
    Oracle CSS service would not start as installed
    Automatic Storage Management(ASM) cannot be used until Oracle CSS service is started.
    The install is on Centos 5 (aka RHEL 5). I have set up a user and a group specifically for this install (user: oracleasm group: asmdba) all of which are the owners of the disks (dev/oracleasm/disks) and the home which i am installing it on. But cannot get past the CSS stack appears not be running.
    Any assistance would be greatly appreciated...I have been all over OTN and metalink but no luck.
    Regards,
    Matt

    Check if daemon ocssd.bin is running by executing:
    ps -ef|grep ocssd
    If it not running then execute
    /etc/init.d/init.cssd run
    sleep 60
    /etc/init.d/init.cssd start
    sleep 120
    /etc/init.d/dbora start
    If that works you had problem because of Oracle bug and the solution in metalink note id 264235.1
    You need to fix the /etc/inittab file moving the line that had init.cssd run to line before level 3 scripts executed.

  • Found the errors in CSSD logs of RAC node

    Found the below error in CSSD logs in One of RAC nodes from 5:15 to 5:18 PM, after this the error got disappeared. Could anyone please have an idea what could be the reason of this error.
    Also, at that time we didn't find any errors in the alert log.
    [    CSSD]2009-07-19 17:15:51.048 [3600] >TRACE: Authorization failed (112bd2a70), timed out, start 17:13:51.041, duration 120009
    [    CSSD]2009-07-19 17:15:51.048 [3600] >TRACE: Authorization prepare time: 2 ms
    [    CSSD]2009-07-19 17:15:51.233 [3086] >TRACE: clssgmClientConnectMsg: Connect from con(112b67930) proc(112b680b0) pid(1049540) proto(10:2:1:1)
    [    CSSD]2009-07-19 17:15:51.268 [3600] >TRACE: Authorization failed (112bd4a10), timed out, start 17:13:51.268, duration 120003
    [    CSSD]2009-07-19 17:15:51.268 [3600] >TRACE: Authorization prepare time: 3 ms
    [    CSSD]2009-07-19 17:15:52.544 [3086] >TRACE: clssgmClientConnectMsg: Connect from con(112b67930) proc(112b680b0) pid(786918) proto(10:2:1:1)
    [    CSSD]2009-07-19 17:15:53.297 [3600] >TRACE: Authorization failed (112c38af0), timed out, start 17:13:53.290, duration 120009
    [    CSSD]2009-07-19 17:15:53.297 [3600] >TRACE: Authorization prepare time: 3 ms
    [    CSSD]2009-07-19 17:15:53.317 [3600] >TRACE: Authorization failed (112d356f0), timed out, start 17:13:53.320, duration 120000
    [    CSSD]2009-07-19 17:15:53.317 [3600] >TRACE: Authorization prepare time: 2 ms
    [    CSSD]2009-07-19 17:16:02.342 [3086] >TRACE: clssgmClientConnectMsg: Connect from con(112b932b0) proc(112b67d10) pid(1336252) proto(10:2:1:1)
    [    CSSD]2009-07-19 17:16:02.977 [3600] >TRACE: Authorization failed (112d04f70), timed out, start 17:14:02.978, duration 120001
    [    CSSD]2009-07-19 17:16:02.977 [3600] >TRACE: Authorization prepare time: 2 ms
    [    CSSD]2009-07-19 17:16:03.007 [3600] >TRACE: Authorization failed (112d38210), timed out, start 17:14:03.006, duration 120002
    [    CSSD]2009-07-19 17:16:03.007 [3600] >TRACE: Authorization prepare time: 2 ms
    [    CSSD]2009-07-19 17:16:10.447 [3600] >TRACE: Authorization failed (112bd7e30), timed out, start 17:14:10.441, duration 120007
    [    CSSD]2009-07-19 17:16:10.447 [3600] >TRACE: Authorization prepare time: 2 ms
    [    CSSD]2009-07-19 17:16:10.847 [3600] >TRACE: Authorization failed (112d3ee70), timed out, start 17:14:10.840, duration 120008
    [    CSSD]2009-07-19 17:16:10.847 [3600] >TRACE: Authorization prepare time: 2 ms
    Thanks,
    Mahi

    Check the metalink note:
    6996694-OCSSD.BIN CONSUMING 100% CPU AND ASM/DB HANGING

  • Unable to create ASM instance in Sol 10 with oracle 10g

    Hi
    I am trying to create ASM instance in oracle 10g, getting an error will try to add localconfig add command
    "bash-3.00# /export/home/oracle/oracle/product/10.2.0/db_1/bin/localconfig add reset
    Failure at scls_scr_create with code 1
    Internal Error Information:
    Category: 1234
    Operation: scls_scr_create
    Location: mkdir
    Other: Unable to make user dir
    Dep: 2
    Successfully accumulated necessary OCR keys.
    Creating OCR keys for user 'root', privgrp 'root'..
    Operation successful.
    Configuration for local CSS has been initialized
    Adding to inittab
    /etc/init.d/init.cssd: /var/opt/oracle/scls_scr/Sun/root/cssrun: cannot create
    Startup will be queued to init within 30 seconds.
    Checking the status of new Oracle init process...
    Expecting the CRS daemons to be up within 600 seconds.
    Giving up: Oracle CSS stack appears NOT to be running.
    Oracle CSS service would not start as installed
    Automatic Storage Management(ASM) cannot be used until Oracle CSS service is started "
    initcssd has been installed and unable to start also getting an error
    # svcs -x svc:/system/initcssd:default
    svc:/system/initcssd:default (system activity reporting package)
    State: maintenance since Wed Nov 16 10:39:29 2011
    Reason: Start method failed repeatedly, last exited with status 2.
    See: http://sun.com/msg/SMF-8000-KS
    See: sar(1M)
    See: /var/svc/log/system-initcssd:default.log
    Impact: This service is not running.
    Can some please help me to create this instance, alos need a initcssd.zip file for 10g

    Hi thanks,
    I have passed more steps, CSS is started after changing the hostname, after that I created two drive and mounted properly
    when I try to create a ASM disk its failing with following error and idea
    SQL> CREATE DISKGROUP DB_DATA NORMAL REDUNDANCY
    2 FAILGROUP controller1 DISK '/dev/dsk/c0d1s0'
    3 FAILGROUP controller2 DISK '/dev/dsk/c1d1s0';
    CREATE DISKGROUP DB_DATA NORMAL REDUNDANCY
    ERROR at line 1:
    ORA-15018: diskgroup cannot be created
    ORA-15031: disk specification '/dev/dsk/c1d1s0' matches no disks
    ORA-15025: could not open disk '/dev/dsk/c1d1s0'
    ORA-15056: additional error message
    Intel SVR4 UNIX Error: 13: Permission denied
    Additional information: 42
    Additional information: 134497888
    Additional information: -809278080
    ORA-15031: disk specification '/dev/dsk/c0d1s0' matches no disks
    ORA-15025: could not open disk '/dev/dsk/c0d1s0'
    ORA-27037: unable to obtain file status
    Intel SVR4 UNIX Error: 25: Inappropriate ioctl for device
    Additional information: 16
    Additional information: 134497888
    Additional information: -809278080

  • Oracle restart and ASM

    Hi,
    I have noticed following 'strange' behaviour of Oracle Restart and ASM.
    starting position:
    -bash-3.2 $ crsctl status resource -t
    NAME           TARGET  STATE        SERVER                   STATE_DETAILS
    Local Resources
    ora.DATA.dg
                   ONLINE  ONLINE       oracle-restart
    ora.LISTENERASM.lsnr
                   ONLINE  ONLINE       oracle-restart
    ora.asm
                   ONLINE  ONLINE       oracle-restart           Started
    Cluster Resources
    ora.cssd
          1        ONLINE  ONLINE       oracle-restart
    ora.diskmon
          1        ONLINE  ONLINE       oracle-restartstep 1:
    -bash-3.2 $ srvctl stop asm
    -bash-3.2 $ srvctl stop diskgroup -g data
    -bash-3.2 $ srvctl disable diskgroup -g datastep 2:
    via sqlplus start ASM instance
    SQL> startup
    ASM instance started
    Total System Global Area  283930624 bytes
    Fixed Size                  2212656 bytes
    Variable Size             256552144 bytes
    ASM Cache                  25165824 bytes
    ASM diskgroups mounted
    ASM diskgroups volume enabled
    SQL> select * from v$asm_diskgroup;
    GROUP_NUMBER NAME                           SECTOR_SIZE BLOCK_SIZE
    ALLOCATION_UNIT_SIZE STATE       TYPE     TOTAL_MB    FREE_MB HOT_USED_MB
    COLD_USED_MB REQUIRED_MIRROR_FREE_MB USABLE_FILE_MB OFFLINE_DISKS
    COMPATIBILITY
    DATABASE_COMPATIBILITY                                       V
               1 DATA                                   512       4096
                 1048576 MOUNTED     EXTERN      10236      10177           0
              59                       0          10177             0
    GROUP_NUMBER NAME                           SECTOR_SIZE BLOCK_SIZE
    ALLOCATION_UNIT_SIZE STATE       TYPE     TOTAL_MB    FREE_MB HOT_USED_MB
    COLD_USED_MB REQUIRED_MIRROR_FREE_MB USABLE_FILE_MB OFFLINE_DISKS
    COMPATIBILITY
    DATABASE_COMPATIBILITY                                       V
    11.2.0.0.0
    10.1.0.0.0                                                   N
    -bash-3.2 $ crsctl status resource -t
    NAME           TARGET  STATE        SERVER                   STATE_DETAILS
    Local Resources
    ora.DATA.dg
                   OFFLINE OFFLINE      oracle-restart          <== funny !!!
    ora.LISTENERASM.lsnr
                   ONLINE  ONLINE       oracle-restart
    ora.asm
                   ONLINE  ONLINE       oracle-restart           Started
    Cluster Resources
    ora.cssd
          1        ONLINE  ONLINE       oracle-restart
    ora.diskmon
          1        ONLINE  ONLINE       oracle-restartIs this behaviour a 'feature' or bug?
    Anyone had similar experience?
    thanks,
    goran

    Hi,
    asm resource is depending on diskgroup resource ... if diskgroup res. is not available, crsctl status shows offline, I would expect asm should be also shown as 'offline' (and brought offline) as they are dependent.
    What is the point of managing resources via srvctl when it doesn't take care of dependencies? For me it's wrong.ora.asm : is ASM Instance
    ora.*.dg : is Diskgroup
    ora.*.dg is dependent of ora.asm, not to the contrary.
    I can have more than one diskgroup and want only one diskgroup disabled, so I need the ASM Instance (ora.asm) online.
    Important:
    If you shut down the database with SQL*Plus, Oracle Restart does not interpret this as a database failure and does not attempt to restart the database.
    Similarly, if you shut down the Oracle ASM instance with SQL*Plus or ASMCMD, Oracle Restart does not attempt to restart it.
    An important difference between starting a component with SRVCTL and starting it with SQL*Plus (or another utility) is the following:
    When you start a component with SRVCTL, any components on which this component depends are automatically started first, and in the proper order.
    When you start a component with SQL*Plus (or another utility), other components in the dependency chain are not automatically started; you must ensure that any components on which this component depends are started.
    Oracle Restart also manages the weak dependency between database instances and the Oracle Net listener (the listener): When a database instance is started, Oracle Restart attempts to start the listener. If the listener startup fails, then the database is still started. If the listener later fails, Oracle Restart does not shut down and restart any database instances.
    It makes no sense Oracle Restart to shut down all environment (databases) because the listener down.
    Regards,
    Levi Pereira

Maybe you are looking for

  • Clearing of Balance in GR/IR clearing account due to date problem

    >Hi to All, >The scenario is like this :- >The client is making back dated entries for quarter between 01.04.2008 to 30.06.2008 in SAP in July 2008 as the system is getting implemented now. He wants to regularise all the transactions in SAP from 01/0

  • Tds line item not simulating while posting as reference with FBR2

    Dear All When I am posting an Invoice using previous document as reference with Tcode FBR2 WTax line Item not simulating, but while I am posting  without reference  i.e. with Tcode FB60 it is simulating, this problem i am facing in production server

  • Is it possible to create reports in 1 month without any experience?

    I'm on a project where we need to make some custom reports in the span of about a month. I'm trying to figure out what is feasible for a person, with no reporting background but plenty of SQL experience,to handle in that time frame. The format of all

  • I can't log into FaceTime any suggestions?

    Having trouble logging into facetime. I get a message that I cannot connect with the FaceTime server. I can connect with everything else .

  • Find thread that opened a session in AIX

    Hello. As stated in "Determining which Oracle database session is being used by a specific Siebel user session [ID 863771.1]": On the Oracle database side, you can query the V$SESSION table (make sure you are logged in as the tableowner) using the fo