Steps to stop / start CRS & ASM in RAC 11gr2

Hello Gurus,
Environment is as below:
RAC 2 nodes
Oracle RAC 11gr2
Enterprise Linux Server release 5.5 (Carthage)
I am new to Oracle RAC 11gr2 and facing some issues while stop / start crs & asm.
We want to reboot both nodes and for that I have done following steps.
1. srvctl stop database -d <db name> -- from one node
Databse is down now.2. srvctl stop nodeapps -n <hostname> -- from both nodes
services are down now.3. Now I want to stop asm
$ export ORACLE_HOME=/u01/app/11.2.0/grid ( Grid Home)
$ srvctl stop asm - This is not stopping ASM instance.
4. crsctl stop crs -- this is also not working.
I would appreciate if you please give me full steps.
Thanks and Regards,

In 11gr2, gsd remains offline. When I had checked with metalink, they said that it is fine.
In 11gr2 you can stop all resouces as below
<CRS_HOME>/bin/crsctl stop has
or you can stop in the following sequence
stop or relocate service using srvctl
stop instnace using srvctl
stop asm using srvctl
stop nodeapps using srvctl
stop cluster resources as root - <CRS_HOME>/bin/crsctl stop has
To start
<CRS_HOME>/bin/crsctl start has - it should bring all resources automatically
verify the services status and start or relocate it back.

Similar Messages

  • Steps to stop & Start servicesservices

    Hi ,
    we r doing SAN migration next week,
    i'll giv u a brief about my setup. It is 2 node RAC connected to a SAN storage and size of DB is 500GB on Windows 2003. So now they r asking me to move this database to a new SAN storage box with 3TB and they asked to migrate this entire setup to new one. So please provide me the steps with scripts to full fill this.
    So please provide me the steps(commands on windows cmd prompt) to stop & start services like cluster,asm and instances....etc what are the services to stop and copy database to new one and how to start the services with scripts to complete this task.
    Actually some of our frns told me that there is no much downtime required but my comp asked me to do like stop services and shut down down DB and migrate then start services.
    Pls help me out on this. i am very new t pls pls pls pls.. help me out
    Regards
    Poorna

    Pls help me out on this. i am very new t pls pls pls pls.. help me outthen maybe someone else should do it
    sorry but providing scripts for this task is just impossible, there are too many factors
    its all written in the documentation
    one possible scenario, that we did last week on a 11gR2 OEL5 RAC:
    1. attach the new storage, create disks/volumes, make the hosts recognize them
    2. add the new disks to the ASM diskgroup, and drop the old disks from the ASM diskgroup - this will cause a rebalance, that will take some time depending on the rebalance power and the size of the database
    [3. you can do the same with OCR/votedisk, with the proper commands, if they are located in ASM]
    4. remove the old storage
    zero downtime needed

  • Can't create 11gR1 Database with 11gR2 CRS/ASM non RAC, dbca ASM password

    Hi,
    try to create a 11gR1 SE Database on a single server with 11gR2 ASM up and running.
    when dbca ask where to store the datafiles and i choose ASM, i have to give the asm sys password which i know (can connect with sql+).
    dbca replies wrong password and i can NOT go further.
    is it possible to use 11gR2 ASM with 11gR1 database and create a database with dbca?
    any hint for this situation?
    thanks
    Michael
    Edited by: user12132314 on 18.07.2010 22:24

    Hello,
    i tried again - still don't work
    it is dbca step 6 from 16, storage option (or something simiular - dbca lang here is german)
    the dialog looks like this
    ASM-ID-Data (Titel)
    enter ASM specific SYS password (prompt)
    entering the correct pwd and push OK
    dbca shows a error dialog with something like: password for user SYS not valid. Enter a valid password.
    Any help?
    Michael

  • Stopping Oracle 11.2.1 RAC

    I am using Oracle RAC 11.2.0.1 two node cluster on OEL 5 64-bit.
    I have few questions about the stopping and starting of Oracle RAC 11.2, please see my steps and questions:
    Steps to Stop Oracle RAC:
    /u01/app/oracle/product/11.2.0/dbhome_1/bin/emctl stop agent
    /u01/app/oracle/product/11.2.0/dbhome_1/bin/srvctl stop database –d TESTDB
    Change Oracle home to grid home:
    export ORACLE_HOME=/u01/app/11.2.0/grid
    /u01/app/11.2.0/grid/bin/srvctl stop asm
    The following needs to be run on each of the nodes:
    export ORACLE_HOME=/u01/app/oracle/product/11.2.0/dbhome_1
    /u01/app/oracle/product/11.2.0/dbhome_1/bin/srvctl stop nodeapps -n test1
    Then as root
    Shut down the Oracle Clusterware or CRS process by entering the following command on all nodes:
    (In grid home)
    /u01/app/11.2.0/grid/bin/crsctl stop crsSteps to Start:
    As root:
    /u01/app/11.2.0/grid/bin/crsctl start crs
    CRS-4123: Oracle High Availability Services has been started.
    As oracle user from database home:
    export ORACLE_HOME=/u01/app/oracle/product/11.2.0/dbhome_1
    srvctl start nodeapps
    Check services-
    ps -ef | grep LISTENER ensure the listener and scan listeners are up.
    ps -ef | grep <sid> ps -ef | grep asm
    start agent on all nodes....
    export ORACLE_HOME=/u01/app/oracle/product/11.2.0/dbhome_1
    /u01/app/oracle/product/11.2.0/dbhome_1/bin/emctl start agent Are above steps right and complete? Am I mentioning the correct homes?
    Thanks.

    Shut down the Oracle Clusterware or CRS process by entering the following command on all nodes:
    (In grid home)
    do the following(as root user):
    cd $GRID_HOME/bin
    ./crsctl stop cluster -all
    In 11gr2 you can stop all resouces as below
    <CRS_HOME>/bin/crsctl stop has
    or you can stop in the following sequence
    stop or relocate service using srvctl
    stop instnace using srvctl
    stop asm using srvctl
    stop nodeapps using srvctl
    stop cluster resources as root - <CRS_HOME>/bin/crsctl stop has
    To start
    <CRS_HOME>/bin/crsctl start has - it should bring all resources automatically.
    refer the forum thread:-
    Thread: Steps to stop / start CRS & ASM in RAC 11gr2
    Steps to stop / start CRS & ASM in RAC 11gr2
    Edited by: raJEYsh on Apr 14, 2011 11:56 AM

  • Automatic ASM & Database up After starting CRS

    Hi Experts,
    My CRS version is 11.1.0
    Due to certain problems, CRS unable to communicate with daemons.
    Later after stopping & starting CRS daemons, ALL the ASM & Database came up automatically. I like to know how it will start all these automatically?

    784585 wrote:
    I can verify and trouleshot.
    My question is how the ASM & Database came up automatically?#crsctl start crs
    will start all the resources available in the clusterware
    that why ASM and Database Instances came up.
    check your crs logs. you will understand.
    after restarting or reboot the system you have to wait for sometime ,
    #crs_stat -t
    if the resources status is either offline or unknown you should check your crs logs.
    you can stop and start the services as you have done.
    #crsctl stop crs
    #crsctl start crs
    as you written, by seeing logs, you can verify and troubleshoot

  • How to bounce (stop/start) trex servers

    HELP!
    We need to bounce trex servers for our client. What are the steps in stopping/starting Text Retrieval and Information Extraction (TREX) servers?
    We need immediate help on this. Thanks!

    have you tried to :
    Go to trex server ...
    START and STOP the TREX servive ...
    Hadar

  • Step by step procedure to start and stop ASM database.

    Hi All,
    could any one tell me the steps to start an ASM database manually in windows ?
    derick

    I just want to start the data base.....
    C:\Documents and Settings\Administrator>set ORACLE_SID=ASM1
    C:\Documents and Settings\Administrator>sqlplus
    SQL*Plus: Release 10.2.0.1.0 - Production on Mon May 3 17:13:04 2010
    Copyright (c) 1982, 2005, Oracle. All rights reserved.
    Enter user-name: /as sysdba
    Connected to an idle instance.
    SQL> startup
    ORA-01078: failure in processing system parameters
    ORA-01565: error in identifying file '+ASMDG/ASM1/spfileASM1.ora'
    ORA-17503: ksfdopn:2 Failed to open file +ASMDG/ASM1/spfileASM1.ora
    ORA-15077: could not locate ASM instance serving a required diskgroup
    SQL>

  • Could Not start CRS or HAS in RAC 11.2.0.1

    Dear Legends,
    We have the following environment in SOLARIS 2 Node Database. Not sure why the 2nd node went down today morning but 1st Node is still Up and Running.
    My Tries so for
    1. Changed the following line as per Doc 1368382.1 from
       h1:3:respawn:/etc/init.d/init.ohasd run >/dev/null 2>&1 </dev/null
    To
       h1:35:respawn:/etc/init.d/init.ohasd run >/dev/null 2>&1 </dev/null
    2. Provided a Restart of Entire Linux Box
    3. Tried issuing the following command ./crsctl start crs but Error out as follows
    CRS-4124: Oracle High Availability Services startup failed.
    CRS-4000: Command Start failed, or completed with errors.
    4. Tried checking in Database, ASM alert logs nothing related with OHASD.
    5. Also checked in ohasd.log and ohasdOUT.log not able to find any thing.
    Please help me. This is our Production Environment.
    Thanks,
    Karthik

    Karthik,
    I encountered the same error few days back and below is what I did to resolve the issue as mentioned in metalink 1368382.1. Did you see anything in the crs logs?
    did you follow all the steps in the solution section below like killiing remaining rc3 scripts?
    checks :
    1. Command '$GRID_HOME/bin/crsctl check crs' returns error:
         CRS-4639: Could not contact Oracle High Availability Services
    2. Command 'ps -ef | grep init' does not show a line similar to:
         root 4878 1 0 Sep12 ? 00:00:02 /bin/sh /etc/init.d/init.ohasd run
    3. Command 'ps -ef | grep d.bin' does not show a line similar to:
         root 21350 1 6 22:24 ? 00:00:01 /u01/app/11.2.0/grid/bin/ohasd.bin reboot
        Or it may only show "ohasd.bin reboot" process without any other processes
    4. ohasd.log report:
           2013-11-04 09:09:15.541: [ default][2609911536] Created alert : (:OHAS00117:) :  TIMED OUT WAITING FOR OHASD MONITOR
    5. ohasOUT.log report:
           2013-11-04 08:59:14
           Changing directory to /u01/app/11.2.0/grid/log/lc1n1/ohasd
           OHASD starting
           Timed out waiting for init.ohasd script to start; posting an alert
    Solutions:
    1. Add the following line to /etc/inittab
        h1:35:respawn:/etc/init.d/init.ohasd run >/dev/null 2>&1 </dev/null
       and then run "init q" as the root user.
    2. Run command 'ps -ef | grep rc' and kill any remaining rc3 scripts that appear to be stuck.
    3. Remove the bad entry before init.ohasd. Consult with OS vendor if "init q" does not spawn "init.ohasd run" process. As a workaround,
       start the init.ohasd manually, eg: as root user, run "/etc/init.d/init.ohasd run >/dev/null 2>&1 </dev/null &"
    4. Enable CRS autostart:
       # crsctl enable crs
       # crsctl start crs
    5. Restore OLR from backup, as root user:
       # touch $GRID_HOME/cdata/<node>.olr
      # chown root:oinstall $GRID_HOME/cdata/<node>.olr
      # ocrconfig -local -restore$GRID_HOME/cdata/<node>/backup_<date>_<num>.olr
      # crsctl start crs
    If OLR backup does not exist for any reason, perform deconfig and rerun root.sh is required to recreate OLR, as root user:
       # $GRID_HOME/crs/install/rootcrs.pl -deconfig -force
       # $GRID_HOME/root.sh
    6. If above does not help, check OS messages for ohasd.bin logger message and manually execute crswrapexece.pl command mentioned in the OS message with LD_LIBRARY_PATH set to <GRID_HOME/lib to continue debug.

  • RAC (Linux) Security: "Unknown" Passwords for CRS, ASM, Oracle?

    We're going through an audit (PCI) and the question of these shared accounts has come up. I do realize that ssh connectivity, with key exchange, is used in the maintenance of Oracle RAC clusters. What I'm thinking/wondering is if it's possible to have an "unknown" password for one or all of these accounts. This would mean that a 'sudo su' would still be allowed to the account if needed, and ssh key exchange would still work, but it would tighten up the security a bit (and make writing compensating controls a lot easier).
    I'm wondering if anyone can help identify areas where actually having the crs, asm and oracle passwords will be required in the care and feeding of RAC? Also, or alternately, if someone can point me to the documentation where it shows having the password is a prerequisite, that would make my life a lot easier.
    Thanks in advance for any assistance.

    Hi,
    If I understand your requirement correctly, yes, I believe it's possible.
    If you're doing RAC, you must have "user equivalence" (i.e. ssh via password-less key based authentication) set up on all nodes in the cluster. In addition, it's possible to disable password authentication, once you have key authentication set up. Finally, once you're logged in, you still need sudo to work, correct? If password will not be known to you, you can set up sudo to work without a password, (look at NOPASSWD option) but consider that, you really need to set things up correctly, or you may jeopardize your system security.
    Ideally, you would:
    1.) set up ssh keys with passphrase.
    2.) disable password authentication.
    3.) set up ssh-agent and authentication forwarding.
    I'm not an expert at all, I've just recently been reading up to try to do something similar in my environment.
    I found this three-part article that seems to cover the subject pretty thoroughly:
    http://www.ibm.com/developerworks/linux/library/l-keyc.html
    Hope that helps,
    -Mark

  • Can't start CRS -- giving error about OCR device but ....?? Please help.

    I am running CentOS 4.5 64-bit with kernel 2.6.9-55.EL on 2 AMD64 single CPU machines. Installed Oracle Clusterware 10.2.0.1 on both of these (pushing from node1). Then created RAC database using ASM. Everything worked fine & CRS/RAC were up (on both) for sometime. Then I stopped the resources & CRS/RAC. But, after sometime I can't start CRS on any of these! Getting error message (in /tmp/crsctl.pid):
    OCR initialization failed accessing OCR device: PROC-26: Error while accessing the physical storage.
    But I can see the FireWire drive/RAW partitions using "fdisk -l" and the permission/ownership are correct on the OCR/Voting Disk RAW partitions.
    I also don't see any socket opened under /var/tmp/.oracle!
    The is no other message in any other error/log file.
    What is this & how to solve?
    Thanks.

    Hi,
    Pls refer to the following solution:
    1. Please check the rights and owners of the ocr devices
    2. Check if the ocrconfig_loc is pointing to the correct ocr in /var/opt/oracle/ocr.loc.
    The ocrconfig_loc parameter specifies the location of the Oracle Cluster
    Registry (OCR) used by the CSS daemon. The path up to the cdata directory is the Oracle home directory where the CSS daemon is running.
    3. Reconfiguring Oracle Cluster Synchronization Services
    a. $/u01/app/oracle/product/10G/bin/localconfig delete
    Unexpected parameter: css
    $
    b - I killed init.cssd process
    $ps -ef |grep css
    root 1559 1 0.1 09:34:53 ?? 0:07.52 /sbin/init.d/init.cssd startcheck
    root 16390 16070 0.0 12:33:53 pts/1 0:00.01 grep css
    $kill -9 1559
    $ps -ef |grep css
    root 16407 16070 0.0 12:34:15 pts/1 0:00.01 grep css
    c - Set up ORACLE_HOME
    $export ORACLE_HOME=/u01/app/oracle/product/10G
    $/u01/app/oracle/product/10G/bin/localconfig delete
    Enter the following command to reconfigure the CSS daemon to run from this Oracle home:
    # $ORACLE_HOME/bin/localconfig reset $ORACLE_HOME
    The script stops the Oracle CSS daemon, reconfigures it in the new Oracle home, and then restarts it. When the system boots, the CSS daemon starts automatically from the new Oracle home.
    Terry liu

  • Root.sh fails on 11.2.0.3 clusterware while starting 'ora.asm' resource

    Dear all,
    I am trying to install clean Oracle 11.2.0.3 grid infrastructure on a two node cluster running on Solaris 5.10.
    - Cluster verification was successfully on both nodes; No warning or issues;
    - I am using 2 network cards for the public and 2 for the private interconnect;
    - OCR is stored on ASM
    - Firewall is disabled on both nodes
    - SCAN is being configured on the DNS (not added in /etc/hosts)
    - GNS is not used
    - hosts file is identical (except the primary hostname)
    The problem: root.sh fails on the 2nd (remote) node, because it fails to start the "ora.asm" resource. However, the root.sh has completed successfully on the 1st node.. Somehow, root.sh doesn't create +ASM2 instance on the remote (host2) node.
    root.sh was executed first on the local node (host1) and after the successful execution was started on the remote (host2) node.
    Output from host1 (working):
    ===================
    Adding Clusterware entries to inittab
    CRS-2672: Attempting to start 'ora.mdnsd' on 'host1'
    CRS-2676: Start of 'ora.mdnsd' on 'host1' succeeded
    CRS-2672: Attempting to start 'ora.gpnpd' on 'host1'
    CRS-2676: Start of 'ora.gpnpd' on 'host1' succeeded
    CRS-2672: Attempting to start 'ora.cssdmonitor' on 'host1'
    CRS-2672: Attempting to start 'ora.gipcd' on 'host1'
    CRS-2676: Start of 'ora.cssdmonitor' on 'host1' succeeded
    CRS-2676: Start of 'ora.gipcd' on 'host1' succeeded
    CRS-2672: Attempting to start 'ora.cssd' on 'host1'
    CRS-2672: Attempting to start 'ora.diskmon' on 'host1'
    CRS-2676: Start of 'ora.diskmon' on 'host1' succeeded
    CRS-2676: Start of 'ora.cssd' on 'host1' succeeded
    ASM created and started successfully.
    Disk Group CRS created successfully.
    clscfg: -install mode specified
    Successfully accumulated necessary OCR keys.
    Creating OCR keys for user 'root', privgrp 'root'..
    Operation successful.
    CRS-4256: Updating the profile
    Successful addition of voting disk 4373be34efab4f01bf79f6c5362acfd3.
    Successful addition of voting disk 7fd725fa4d904f07bf76cecf96791547.
    Successful addition of voting disk a9c85297bdd74f3abfd86899205aaf17.
    Successfully replaced voting disk group with +CRS.
    CRS-4256: Updating the profile
    CRS-4266: Voting file(s) successfully replaced
    ## STATE File Universal Id File Name Disk group
    1. ONLINE 4373be34efab4f01bf79f6c5362acfd3 (/dev/rdsk/c4t600A0B80006E2CC40000C6674E82AA57d0s4) [CRS]
    2. ONLINE 7fd725fa4d904f07bf76cecf96791547 (/dev/rdsk/c4t600A0B80006E2CC40000C6694E82AADDd0s4) [CRS]
    3. ONLINE a9c85297bdd74f3abfd86899205aaf17 (/dev/rdsk/c4t600A0B80006E2F100000C7744E82AC7Ad0s4) [CRS]
    Located 3 voting disk(s).
    CRS-2672: Attempting to start 'ora.asm' on 'host1'
    CRS-2676: Start of 'ora.asm' on 'host1' succeeded
    CRS-2672: Attempting to start 'ora.CRS.dg' on 'host1'
    CRS-2676: Start of 'ora.CRS.dg' on 'host1' succeeded
    CRS-2672: Attempting to start 'ora.registry.acfs' on 'host1'
    CRS-2676: Start of 'ora.registry.acfs' on 'host1' succeeded
    Configure Oracle Grid Infrastructure for a Cluster ... succeeded
    Name Type Target State Host
    ora.CRS.dg ora....up.type ONLINE ONLINE host1
    ora....ER.lsnr ora....er.type ONLINE ONLINE host1
    ora....N1.lsnr ora....er.type ONLINE ONLINE host1
    ora....N2.lsnr ora....er.type ONLINE ONLINE host1
    ora....N3.lsnr ora....er.type ONLINE ONLINE host1
    ora.asm ora.asm.type ONLINE ONLINE host1
    ora....SM1.asm application ONLINE ONLINE host1
    ora....B1.lsnr application ONLINE ONLINE host1
    ora....db1.gsd application OFFLINE OFFLINE
    ora....db1.ons application ONLINE ONLINE host1
    ora....db1.vip ora....t1.type ONLINE ONLINE host1
    ora.cvu ora.cvu.type ONLINE ONLINE host1
    ora.gsd ora.gsd.type OFFLINE OFFLINE
    ora....network ora....rk.type ONLINE ONLINE host1
    ora.oc4j ora.oc4j.type ONLINE ONLINE host1
    ora.ons ora.ons.type ONLINE ONLINE host1
    ora....ry.acfs ora....fs.type ONLINE ONLINE host1
    ora.scan1.vip ora....ip.type ONLINE ONLINE host1
    ora.scan2.vip ora....ip.type ONLINE ONLINE host1
    ora.scan3.vip ora....ip.type ONLINE ONLINE host1
    Output from host2 (failing):
    ===================
    OLR initialization - successful
    Adding Clusterware entries to inittab
    CRS-4402: The CSS daemon was started in exclusive mode but found an active CSS daemon on node billdb1, number 1, and is terminating
    An active cluster was found during exclusive startup, restarting to join the cluster
    Start of resource "ora.asm" failed
    CRS-2672: Attempting to start 'ora.drivers.acfs' on 'host2'
    CRS-2676: Start of 'ora.drivers.acfs' on 'host2' succeeded
    CRS-2672: Attempting to start 'ora.asm' on 'host2'
    CRS-5017: The resource action "ora.asm start" encountered the following error:
    ORA-03113: end-of-file on communication channel
    Process ID: 0
    Session ID: 0 Serial number: 0
    *. For details refer to "(:CLSN00107:)" in "/u01/11.2.0/grid/log/host2/agent/ohasd/oraagent_grid/oraagent_grid.log".*
    CRS-2674: Start of 'ora.asm' on 'host2' failed
    CRS-2679: Attempting to clean 'ora.asm' on 'host2'
    CRS-2681: Clean of 'ora.asm' on 'host2' succeeded
    CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'host2'
    CRS-2677: Stop of 'ora.drivers.acfs' on 'host2' succeeded
    CRS-4000: Command Start failed, or completed with errors.
    Failed to start Oracle Grid Infrastructure stack
    Failed to start ASM at /u01/11.2.0/grid/crs/install/crsconfig_lib.pm line 1272.
    /u01/11.2.0/grid/perl/bin/perl -I/u01/11.2.0/grid/perl/lib -I/u01/11.2.0/grid/crs/install /u01/11.2.0/grid/crs/install/rootcrs.pl execution failed
    Contents of "/u01/11.2.0/grid/cfgtoollogs/crsconfig/rootcrs_host2.log"
    =============================================
    CRS-2672: Attempting to start 'ora.asm' on 'host2'
    CRS-5017: The resource action "ora.asm start" encountered the following error:
    ORA-03113: end-of-file on communication channel
    Process ID: 0
    Session ID: 0 Serial number: 0
    . For details refer to "(:CLSN00107:)" in "/u01/11.2.0/grid/log/host2/agent/ohasd/oraagent_grid/oraagent_grid.log".
    CRS-2674: Start of 'ora.asm' on 'host2' failed
    CRS-2679: Attempting to clean 'ora.asm' on 'host2'
    CRS-2681: Clean of 'ora.asm' on 'host2' succeeded
    CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'host2'
    CRS-2677: Stop of 'ora.drivers.acfs' on 'host2' succeeded
    CRS-4000: Command Start failed, or completed with errors.
    2011-10-24 19:36:54: Failed to start Oracle Grid Infrastructure stack
    2011-10-24 19:36:54: ###### Begin DIE Stack Trace ######
    2011-10-24 19:36:54: Package File Line Calling
    2011-10-24 19:36:54: --------------- -------------------- ---- ----------
    2011-10-24 19:36:54: 1: main rootcrs.pl 375 crsconfig_lib::dietrap
    2011-10-24 19:36:54: 2: crsconfig_lib crsconfig_lib.pm 1272 main::__ANON__
    2011-10-24 19:36:54: 3: crsconfig_lib crsconfig_lib.pm 1171 crsconfig_lib::start_cluster
    2011-10-24 19:36:54: 4: main rootcrs.pl 803 crsconfig_lib::perform_start_cluster
    2011-10-24 19:36:54: ####### End DIE Stack Trace #######
    Shortened output from "/u01/11.2.0/grid/log/host2/agent/ohasd/oraagent_grid/oraagent_grid.log"
    2011-10-24 19:35:48.726: [ora.asm][9] {0:0:224} [start] clean {
    2011-10-24 19:35:48.726: [ora.asm][9] {0:0:224} [start] InstAgent::stop_option stop mode immediate option 1
    2011-10-24 19:35:48.726: [ora.asm][9] {0:0:224} [start] InstAgent::stop {
    2011-10-24 19:35:48.727: [ora.asm][9] {0:0:224} [start] InstAgent::stop original reason system do shutdown abort
    2011-10-24 19:35:48.727: [ora.asm][9] {0:0:224} [start] ConnectionPool::resetConnection s_statusOfConnectionMap 00ab1948
    2011-10-24 19:35:48.727: [ora.asm][9] {0:0:224} [start] ConnectionPool::resetConnection sid +ASM2 status  2
    2011-10-24 19:35:48.728: [ora.asm][9] {0:0:224} [start] Gimh::check OH /u01/11.2.0/grid SID +ASM2
    2011-10-24 19:35:48.728: [ora.asm][9] {0:0:224} [start] Gimh::check condition changes to (GIMH_NEXT_NUM) 0,1,7 exists
    2011-10-24 19:35:48.729: [ora.asm][9] {0:0:224} [start] (:CLSN00006:)AsmAgent::check failed gimh state 0
    2011-10-24 19:35:48.729: [ora.asm][9] {0:0:224} [start] AsmAgent::check ocrCheck 1 m_OcrOnline 0 m_OcrTimer 0
    2011-10-24 19:35:48.729: [ora.asm][9] {0:0:224} [start] DgpAgent::initOcrDgpSet { entry
    2011-10-24 19:35:48.730: [ora.asm][9] {0:0:224} [start] DgpAgent::initOcrDgpSet procr_get_conf: retval [0] configured [1] local only [0] error buffer []
    2011-10-24 19:35:48.730: [ora.asm][9] {0:0:224} [start] DgpAgent::initOcrDgpSet procr_get_conf: OCR loc [0], Disk Group : [+CRS]
    2011-10-24 19:35:48.730: [ora.asm][9] {0:0:224} [start] DgpAgent::initOcrDgpSet m_ocrDgpSet 015fba90 dgName CRS
    2011-10-24 19:35:48.731: [ora.asm][9] {0:0:224} [start] DgpAgent::initOcrDgpSet ocrret 0 found 1
    2011-10-24 19:35:48.731: [ora.asm][9] {0:0:224} [start] DgpAgent::initOcrDgpSet ocrDgpSet CRS
    2011-10-24 19:35:48.731: [ora.asm][9] {0:0:224} [start] DgpAgent::initOcrDgpSet exit }
    2011-10-24 19:35:48.731: [ora.asm][9] {0:0:224} [start] DgpAgent::ocrDgCheck Entry {
    2011-10-24 19:35:48.732: [ora.asm][9] {0:0:224} [start] DgpAgent::getConnxn new pool
    2011-10-24 19:35:48.732: [ora.asm][9] {0:0:224} [start] DgpAgent::getConnxn new pool m_oracleHome:/u01/11.2.0/grid m_oracleSid:+ASM2 m_usrOraEnv:
    2011-10-24 19:35:48.732: [ora.asm][9] {0:0:224} [start] ConnectionPool::ConnectionPool 2 m_oracleHome:/u01/11.2.0/grid, m_oracleSid:+ASM2, m_usrOraEnv:
    2011-10-24 19:35:48.733: [ora.asm][9] {0:0:224} [start] ConnectionPool::addConnection m_oracleHome:/u01/11.2.0/grid m_oracleSid:+ASM2 m_usrOraEnv: pConnxn:
    01fcdf10
    2011-10-24 19:35:48.733: [ora.asm][9] {0:0:224} [start] Utils::getCrsHome crsHome /u01/11.2.0/grid
    2011-10-24 19:35:51.969: [ora.asm][14] {0:0:224} [check] makeConnectStr = (DESCRIPTION=(ADDRESS=(PROTOCOL=beq)(PROGRAM=/u01/11.2.0/grid/bin/oracle)(ARGV0=o
    racle+ASM2)(ENVS='ORACLE_HOME=/u01/11.2.0/grid,ORACLE_SID=+ASM2')(ARGS='(DESCRIPTION=(LOCAL=YES)(ADDRESS=(PROTOCOL=beq)))'))(CONNECT_DATA=(SID=+ASM2)))
    2011-10-24 19:35:51.971: [ora.asm][14] {0:0:224} [check] ConnectionPool::getConnection 260 pConnxn 013e40a0
    2011-10-24 19:35:51.971: [ora.asm][14] {0:0:224} [check] DgpAgent::getConnxn connected
    2011-10-24 19:35:51.971: [ora.asm][14] {0:0:224} [check] InstConnection::connectInt: server not attached
    2011-10-24 19:35:52.190: [ora.asm][14] {0:0:224} [check] ORA-01034: ORACLE not available
    ORA-27101: shared memory realm does not exist
    SVR4 Error: 2: No such file or directory
    Process ID: 0
    Session ID: 0 Serial number: 0
    2011-10-24 19:35:52.190: [ora.asm][14] {0:0:224} [check] InstConnection::connectInt (2) Exception OCIException
    2011-10-24 19:35:52.190: [ora.asm][14] {0:0:224} [check] InstConnection:connect:excp OCIException OCI error 1034
    2011-10-24 19:35:52.190: [ora.asm][14] {0:0:224} [check] DgpAgent::queryDgStatus excp ORA-01034: ORACLE not available
    ORA-27101: shared memory realm does not exist
    SVR4 Error: 2: No such file or directory
    Process ID: 0
    Session ID: 0 Serial number: 0
    2011-10-24 19:35:52.190: [ora.asm][14] {0:0:224} [check] DgpAgent::queryDgStatus asm inst is down or going down
    2011-10-24 19:35:52.191: [ora.asm][14] {0:0:224} [check] DgpAgent::queryDgStatus dgName CRS ret 1
    2011-10-24 19:35:52.191: [ora.asm][14] {0:0:224} [check] (:CLSN00100:)DgpAgent::ocrDgCheck OCR dgName CRS state 1
    2011-10-24 19:35:52.192: [ora.asm][14] {0:0:224} [check] ConnectionPool::releaseConnection InstConnection 013e40a0
    2011-10-24 19:35:52.192: [ora.asm][14] {0:0:224} [check] AsmAgent::check ocrCheck 2 m_OcrOnline 0 m_OcrTimer 0
    2011-10-24 19:35:52.193: [ora.asm][14] {0:0:224} [check] CrsCmd::ClscrsCmdData::stat entity 1 statflag 32 useFilter 0
    2011-10-24 19:35:52.197: [ COMMCRS][23]clsc_connect: (1020d39d0) no listener at (ADDRESS=(PROTOCOL=IPC)(KEY=CRSD_UI_SOCKET))
    Please advice for any workaround or a metalink note.
    Thanks in advance!

    Thanks for the fast reply!
    - Yes, the shared storage is accessible.
    - The alert log for the +ASM2 clearly shows that ASM instance has started normally using default parameters and at one point PMON process dumped.
    - The system logs just shows that there is an error executing "crswrapexece.pl"
    System Log
    ===================
    *Oct 24 19:25:03 host2 root: [ID 702911 user.error] exec /u01/11.2.0/grid/perl/bin/perl -I/u01/11.2.0/grid/perl/lib /u01/11.2.0/grid/bin/crswrapexece.pl /*
    u01/11.2.0/grid/crs/install/s_crsconfig_host2_env.txt /u01/11.2.0/grid/bin/ohasd.bin "reboot"
    Oct 24 19:26:33 host2 oracleoks: [ID 902884 kern.notice] [Oracle OKS] mallocing log buffer, size=10485760
    Oct 24 19:26:33 host2 oracleoks: [ID 714332 kern.notice] [Oracle OKS] log buffer = 0x301780fcb50, size 10485760
    Oct 24 19:26:33 host2 oracleoks: [ID 400061 kern.notice] NOTICE: [Oracle OKS] ODLM hash size 16384
    Oct 24 19:26:33 host2 oracleoks: [ID 160659 kern.notice] NOTICE: OKSK-00004: Module load succeeded. Build information: (LOW DEBUG) USM_11.2.0.3.0_SOLAR
    IS.SPARC64_110803.1 2011/08/11 02:38:30
    Oct 24 19:26:33 host2 pseudo: [ID 129642 kern.info] pseudo-device: oracleadvm0
    Oct 24 19:26:33 host2 genunix: [ID 936769 kern.info] oracleadvm0 is /pseudo/oracleadvm@0
    Oct 24 19:26:33 host2 oracleoks: [ID 141287 kern.notice] NOTICE: ADVMK-00001: Module load succeeded. Build information: (LOW DEBUG) - USM_11.2.0.3.0_SOL
    ARIS.SPARC64_110803.1 built on 2011/08/11 02:40:17.
    Oct 24 19:26:33 host2 oracleacfs: [ID 202941 kern.notice] NOTICE: [Oracle ACFS] FCB hash size 16384
    Oct 24 19:26:33 host2 oracleacfs: [ID 671725 kern.notice] NOTICE: [Oracle ACFS] buffer cache size 511MB (79884 buckets)
    Oct 24 19:26:33 host2 oracleacfs: [ID 730054 kern.notice] NOTICE: [Oracle ACFS] DLM hash size 16384
    Oct 24 19:26:33 host2 oracleoks: [ID 617314 kern.notice] NOTICE: ACFSK-0037: Module load succeeded. Build information: (LOW DEBUG) USM_11.2.0.3.0_SOLAR
    IS.SPARC64_110803.1 2011/08/11 02:42:45
    Oct 24 19:26:33 host2 pseudo: [ID 129642 kern.info] pseudo-device: oracleacfs0
    Oct 24 19:26:33 host2 genunix: [ID 936769 kern.info] oracleacfs0 is /pseudo/oracleacfs@0
    Oct 24 19:26:36 host2 oracleoks: [ID 621795 kern.notice] NOTICE: OKSK-00010: Persistent OKS log opened at /u01/11.2.0/grid/log/host2/acfs/acfs.log.0.
    Oct 24 19:31:37 host2 last message repeated 1 time
    Oct 24 19:33:05 host2 CLSD: [ID 770310 daemon.notice] The clock on host host2 has been updated by the Cluster Time Synchronization Service to be synchr
    onous with the mean cluster time.
    ASM alert log
    ====================================================================
    <msg time='2011-10-24T19:35:48.776+01:00' org_id='oracle' comp_id='asm'
    client_id='' type='UNKNOWN' level='16'
    host_id='host2' host_addr='10.172.16.200' module=''
    pid='26406'>
    <txt>System state dump requested by (instance=2, osid=26396 (PMON)), summary=[abnormal instance termination].
    </txt>
    </msg>
    <msg time='2011-10-24T19:35:48.778+01:00' org_id='oracle' comp_id='asm'
    client_id='' type='UNKNOWN' level='16'
    host_id='host2' host_addr='10.172.16.200' module=''
    pid='26406'>
    <txt>System State dumped to trace file /u01/app/oracle/diag/asm/+asm/+ASM2/trace/+ASM2_diag_26406.trc
    </txt>
    </msg>
    <msg time='2011-10-24T19:35:48.927+01:00' org_id='oracle' comp_id='asm'
    type='UNKNOWN' level='16' host_id='host2'
    host_addr='10.172.16.200' pid='26470'>
    <txt>ORA-1092 : opitsk aborting process
    </txt>
    </msg>
    <msg time='2011-10-24T19:35:49.128+01:00' org_id='oracle' comp_id='asm'
    type='UNKNOWN' level='16' host_id='host2'
    host_addr='10.172.16.200' pid='26472'>
    <txt>ORA-1092 : opitsk aborting process
    </txt>
    </msg>
    Output from "/u01/app/oracle/diag/asm/+asm/+ASM2/trace/+ASM2_diag_26406.trc"
    REQUEST:system state dump at level 10, requested by (instance=2, osid=26396 (PMON)), summary=[abnormal instance termination].
    kjzdattdlm: Can not attach to DLM (LMON up=[TRUE], DB mounted=[FALSE]).
    ===================================================
    SYSTEM STATE (level=10)
    Orapids on dead process list: [count = 0]
    PROCESS 1:
    SO: 0x3df098b50, type: 2, owner: 0x0, flag: INIT/-/-/0x00 if: 0x3 c: 0x3
    proc=0x3df098b50, name=process, file=ksu.h LINE:12616 ID:, pg=0
    (process) Oracle pid:1, ser:0, calls cur/top: 0x0/0x0
    flags : (0x20) PSEUDO
    flags2: (0x0), flags3: (0x10)
    intr error: 0, call error: 0, sess error: 0, txn error 0
    intr queue: empty
    ksudlp FALSE at location: 0
    (post info) last post received: 0 0 0
    last post received-location: No post
    last process to post me: none
    last post sent: 0 0 0
    last post sent-location: No post
    last process posted by me: none
    (latch info) wait_event=0 bits=0
    O/S info: user: , term: , ospid: (DEAD)
    OSD pid info: Unix process pid: 0, image: PSEUDO
    SO: 0x38000cef0, type: 5, owner: 0x3df098b50, flag: INIT/-/-/0x00 if: 0x3 c: 0x3
    proc=0x0, name=kss parent, file=kss2.h LINE:138 ID:, pg=0
    PSO child state object changes :
    Dump of memory from 0x00000003DF722AC0 to 0x00000003DF722CC8
    3DF722AC0 00000000 00000000 00000000 00000000 [................]
    Repeat 31 times
    3DF722CC0 00000000 00000000 [........]
    PROCESS 2: PMON
    SO: 0x3df099bf8, type: 2, owner: 0x0, flag: INIT/-/-/0x00 if: 0x3 c: 0x3
    proc=0x3df099bf8, name=process, file=ksu.h LINE:12616 ID:, pg=0
    (process) Oracle pid:2, ser:1, calls cur/top: 0x3db6c8d30/0x3db6c8d30
    flags : (0xe) SYSTEM
    flags2: (0x0), flags3: (0x10)
    intr error: 0, call error: 0, sess error: 0, txn error 0
    intr queue: empty
    ksudlp FALSE at location: 0
    (post info) last post received: 0 0 136
    last post received-location: kjm.h LINE:1228 ID:kjmdmi: pmon to attach
    last process to post me: 3df0a2138 1 6
    last post sent: 0 0 137
    last post sent-location: kjm.h LINE:1230 ID:kjiath: pmon attached
    last process posted by me: 3df0a2138 1 6
    (latch info) wait_event=0 bits=0
    Process Group: DEFAULT, pseudo proc: 0x3debbbf40
    O/S info: user: grid, term: UNKNOWN, ospid: 26396
    OSD pid info: Unix process pid: 26396, image: oracle@host2 (PMON)
    SO: 0x3d8800c18, type: 30, owner: 0x3df099bf8, flag: INIT/-/-/0x00 if: 0x3 c: 0x3
    proc=0x3df099bf8, name=ges process, file=kji.h LINE:3669 ID:, pg=0
    GES MSG BUFFERS: st=emp chunk=0x0 hdr=0x0 lnk=0x0 flags=0x0 inc=0
    outq=0 sndq=0 opid=0 prmb=0x0
    mbg=(0 0) mbg=(0 0) mbg[r]=(0 0)
    fmq=(0 0) fmq=(0 0) fmq[r]=(0 0)
    mop[s]=0 mop[q]=0 pendq=0 zmbq=0
    nonksxp_recvs=0
    ------------process 3d8800c18--------------------
    proc version : 0
    Local inst : 2
    pid : 26396
    lkp_inst : 2
    svr_mode : 0
    proc state : KJP_FROZEN
    Last drm hb acked : 0
    flags : x50
    ast_rcvd_svrmod : 0
    current lock op : 0
    Total accesses : 1
    Imm. accesses : 0
    Locks on ASTQ : 0
    Locks Pending AST : 0
    Granted locks : 0
    AST_Q:
    PENDING_Q:
    GRANTED_Q:
    SO: 0x3d9835198, type: 14, owner: 0x3df099bf8, flag: INIT/-/-/0x00 if: 0x1 c: 0x1
    proc=0x3df099bf8, name=channel handle, file=ksr2.h LINE:367 ID:, pg=0
    (broadcast handle) 3d9835198 flag: (2) ACTIVE SUBSCRIBER,
    owner: 3df099bf8 - ospid: 26396
    event: 1, last message event: 1,
    last message waited event: 1,
    next message: 0(0), messages read: 0
    channel: (3d9934df8) PMON actions channel [name: 2]
    scope: 7, event: 1, last mesage event: 0,
    publishers/subscribers: 0/1,
    messages published: 0
    heuristic msg queue length: 0
    SO: 0x3d9835008, type: 14, owner: 0x3df099bf8, flag: INIT/-/-/0x00 if: 0x1 c: 0x1
    proc=0x3df099bf8, name=channel handle, file=ksr2.h LINE:367 ID:, pg=0
    (broadcast handle) 3d9835008 flag: (2) ACTIVE SUBSCRIBER,
    owner: 3df099bf8 - ospid: 26396
    event: 1, last message event: 1,
    last message waited event: 1,
    next message: 0(0), messages read: 0
    channel: (3d9941e40) scumnt mount lock [name: 157]
    scope: 1, event: 12, last mesage event: 0,
    publishers/subscribers: 0/12,
    messages published: 0
    heuristic msg queue length: 0
    SO: 0x3de4a2b80, type: 4, owner: 0x3df099bf8, flag: INIT/-/-/0x00 if: 0x3 c: 0x3
    proc=0x3df099bf8, name=session, file=ksu.h LINE:12624 ID:, pg=0
    (session) sid: 33 ser: 1 trans: 0x0, creator: 0x3df099bf8
    flags: (0x51) USR/- flags_idl: (0x1) BSY/-/-/-/-/-
    flags2: (0x409) -/-/INC
    DID: , short-term DID:
    txn branch: 0x0
    oct: 0, prv: 0, sql: 0x0, psql: 0x0, user: 0/SYS
    ksuxds FALSE at location: 0
    service name: SYS$BACKGROUND
    Current Wait Stack:
    Not in wait; last wait ended 0.666415 sec ago
    Wait State:
    fixed_waits=0 flags=0x21 boundary=0x0/-1
    Session Wait History:
    elapsed time of 0.666593 sec since last wait
    0: waited for 'pmon timer'
    duration=0x12c, =0x0, =0x0
    wait_id=63 seq_num=64 snap_id=1
    wait times: snap=3.000089 sec, exc=3.000089 sec, total=3.000089 sec
    wait times: max=3.000000 sec
    wait counts: calls=1 os=1
    occurred after 0.002067 sec of elapsed time
    1: waited for 'pmon timer'
    duration=0x12c, =0x0, =0x0
    wait_id=62 seq_num=63 snap_id=1
    wait times: snap=3.010111 sec, exc=3.010111 sec, total=3.010111 sec
    wait times: max=3.000000 sec
    wait counts: calls=1 os=1
    occurred after 0.001926 sec of elapsed time
    2: waited for 'pmon timer'
    duration=0x12c, =0x0, =0x0
    wait_id=61 seq_num=62 snap_id=1
    wait times: snap=3.125286 sec, exc=3.125286 sec, total=3.125286 sec
    wait times: max=3.000000 sec
    wait counts: calls=1 os=1
    occurred after 0.003361 sec of elapsed time
    3: waited for 'pmon timer'
    duration=0x12c, =0x0, =0x0
    wait_id=60 seq_num=61 snap_id=1
    wait times: snap=3.000081 sec, exc=3.000081 sec, total=3.000081 sec
    wait times: max=3.000000 sec
    wait counts: calls=1 os=1
    occurred after 0.002102 sec of elapsed time
    4: waited for 'pmon timer'
    duration=0x12c, =0x0, =0x0

  • Service ora. database .db not start datbase automaticaly (RAC 11gR2)

    Hi all,
    after install Grid Infraestructure, RAC database and created database. All services are ok, database is ok. Everything are running fine.
    So, when I make a test to reboot my servers, i see that the service ora.<database>.db are OFFLINE and not start automaticaly after reboot.
    I´m using 11g R2 on Solaris 10.
    Does anybody can help me ?
    Thanks
    Wander (Brazil)
    Edited by: Wander(Brazil) on Oct 15, 2010 6:43 AM
    Edited by: Wander(Brazil) on Oct 15, 2010 6:46 AM

    Hi,
    You have to follow some order for starting services in RAC database.
    Check the below steps can help you:
    The ORDER IS AS FOLLOWS
    For Starting things:
    Step 1 Start Clusterware
    Note
    When any host in the cluster boots,Clusterware should be started.
    The scripts listed below help make this happen:
    /etc/inittab
    /etc/rc5.d/S96init.crs
    /etc/init.d/init.crs
    /etc/init.d/init.crsd
    /etc/init.d/init.cssd
    /etc/init.d/init.evmd
    Be aware of this for 2 reasons:
    1. It may be unneccesary for you to start Clusterware. It might already be started.
    2. The script, /etc/init.d/init.crs, can be used to start Clusterware if you want to use it.
    To start Clusterware manually, maybe use this script instead of init.crs:
    To start Clusterware manually, maybe use this script instead of init.crs:
    #!/bin/sh
    # Run as root
    # start
    ssh dbcl1n1 /home/app/crs/bin/crsctl start crs
    or
    $ sudo /home/app/crs/bin/crsctl start crs
    Step 2: Start nodeapps
    After you start Clusterware, you may start nodeapps.
    Use this script to start nodeapps:
    #! /bin/sh
    # start
    srvctl start nodeapps -n dbcl1n1
    Step 3:Start Database
    After you start the nodeapps, you may start the Databases.
    Use this script to start the databases:
    #! /bin/sh
    # ps
    ssh dbcl1n1 ps -ef | grep dbw
    # status
    srvctl status database -d AUCD
    # start
    srvctl start database -d AUCD
    Step 4:Start the application after starting the Database
    Best regards,
    Rafi.
    http://rafioracledba.blogspot.com

  • Unable to start CRS

    Hi experts..we are ahving 2 node RAC in 10g..Last week we took a node down and today when I was trying to bring the node back up I am facing problems with the CRS...Here are the few lines...
    root@dcaxogc2[u01/crs/bin]: ./crsctl check crs
    CSS appears healthy
    Cannot communicate with CRS
    EVM appears healthy
    root@dcaxogc2[u01/crs/bin]: ./crsctl start crs
    Attempting to start CRS stack
    The CRS stack will be started shortly
    root@dcaxogc2[u01/crs/bin]: ./crsctl stop crs
    Stopping resources. This could take several minutes.
    Error while stopping resources. Possible cause: CRSD is down.
    root@dcaxogc2[u01/crs/bin]: ./crsctl start resources
    Starting resources.
    Error while starting resources. Possible cause: CRSD is down.
    root@dcaxogc2[u01/crs/bin]: ./srvctl start instance -d ogcdb -i ogcdb2
    PRKP-1001 : Error starting instance OGCDB2 on node dcaxogc2
    root@dcaxogc2[u01/crs/bin]: ./crs_stat
    IOT/Abort trap(coredump)
    Alert log content
    2011-05-16 12:33:41.245
    [crsd(651330)]CRS-1012:The OCR service started on node dcaxogc2.
    2011-05-16 12:33:42.041
    [crsd(651330)]CRS-1201:CRSD started on node dcaxogc2.
    2011-05-16 12:34:01.675
    [crsd(651364)]CRS-1012:The OCR service started on node dcaxogc2.
    2011-05-16 12:34:02.432
    [crsd(651364)]CRS-1201:CRSD started on node dcaxogc2.
    I dont know where the problem is coming from...how to start the CRSD? i think this might be the error.....If I am wrong please correct me...Any suggestions please....
    Thanks for your time...

    do it on both nodes
    ./crsctl start crs
    wait for few minutes and check
    ./crsctl check crs
    if all services comes up
    check
    ./crs_stat -t
    if resources not coming up, check the clusterware file logs
    The Cluster Ready Services Daemon (crsd) Log Files
    Log files for the CRSD process (crsd) can be found in the following directories:
    CRS home/log/hostname/crsd
    Oracle Cluster Registry (OCR) Log Files
    The Oracle Cluster Registry (OCR) records log information in the following location:
    CRS Home/log/hostname/client
    Cluster Synchronization Services (CSS) Log Files
    You can find CSS information that the OCSSD generates in log files in the following locations:
    CRS Home/log/hostname/cssd
    Event Manager (EVM) Log Files
    Event Manager (EVM) information generated by evmd is recorded in log files in the following locations:
    CRS Home/log/hostname/evmd
    RACG Log Files
    The Oracle RAC high availability trace files are located in the following two locations:
    CRS home/log/hostname/racg
    $ORACLE_HOME/log/hostname/racg
    Core files are in the sub-directories of the log directories. Each RACG executable has a sub-directory assigned exclusively for that executable. The name of the RACG executable sub-directory is the same as the name of the executable.
    refer the link:-
    http://download.oracle.com/docs/cd/B19306_01/rac.102/b14197/appsupport.htm

  • NOt able to start CRS

    Hi Support ,
    We are having two node RAC, due to some hardware issue both servers goes down. Both servers came up now but RAC Services/Cluster has bee started in only one node where as in other node we are facing issue.
    OS: Solaris.
    version Oracle: 10.2.0
    Refer the logs:
    oracle$ crsctl check crs
    Failure 1 contacting CSS daemon
    Cannot communicate with CRS
    Cannot communicate with EVM
    oracle$ crsctl start crs
    Insufficient user privileges.   {Using Oracle user}
    root$  crsctl start crs
    crsctl: not found                    {Using root user}
    root$/etc/init.d/init.crs start
    Startup will be queued to init within 30 seconds.
    CRS logs:
    2014-04-04 18:20:02.741: [  CRSRTI][1] CSS is not ready. Received status 3 from CSS. Waiting for good status ..
    2014-04-04 18:20:03.748: [ COMMCRS][2058]clsc_connect: (100a513d0) no listener at (ADDRESS=(PROTOCOL=ipc)(KEY=OCSSD_LL_ch1d5bl0_localhost))
    2014-04-04 18:20:03.748: [ CSSCLNT][1]clsssInitNative: connect failed, rc 9
    CSS logs: Are not getting generated after march 2013.
    I have verified the permissions and ownership of crs/bin, it seems ok. /etc/inittab all enteries are availble. OCR and voting disk are in sync/same with correct permissions.
    Permissions:
    drwxr-xr-x   2 root     oinstall    3072 Mar 12  2013 bin
    $ id
    uid=102(oracle) gid=200(oinstall)
    root$ id
    uid=0(root) gid=0(root)
    thanks....

    Hi Thomas,
    Thanks for the revert .
    I have executed the sharable points :
    root# cd /u01/app/oracle/product/10.2.0/crs_1/bin
    root # ./crsctl stat res -t -init
    Unknown parameter: stat
    root # ./crsctl stop crs -f
    Stopping CSSD.
    Unable to communicate with the CSS daemon.
    root # ./crsctl start crs
    Attempting to start CRS stack
    The CRS stack will be started shortly
    I have check the logs available in CRSD & whereas CSSD logs are not generated:
    2014-04-09 10:45:31.898: [ COMMCRS][586]clsc_connect: (100a513d0) no listener at (ADDRESS=(PROTOCOL=ipc)(KEY=OCSSD_LL_ch1d5bl0_localhost))
    2014-04-09 10:45:31.898: [ CSSCLNT][1]clsssInitNative: connect failed, rc 9
    2014-04-09 10:45:31.899: [  CRSRTI][1] CSS is not ready. Received status 3 from CSS. Waiting for good status ..

  • Stop/start cluster

    Which is the better one to stop/start the cluster 11.2.0.3;
    What is the difference b/n these commands below ?
    #crsctl start cluster
    # crsctl start has
    #crsctl start crs

    Hello,
    To understand the basic difference b/w all three commands, first you need to understand the daemon process architecture:
    There are four levels of daemons are there from 11.2 onward.
    Level-1 : OHASD'S Spawn_
    Here OHASD(Super Daemon process) will spawn itself and it will start :
    a) CSSDAGENT, b) CSSDMONITOR, c) ORAROOTAGENT and d) ORAAGENT
    Level-2: OHASD cssdagent, cssdmonitor, orarootagent and oraagent spawn_
    Here cssdagent, cssdmonitor, orarootagent and oraagent, they all will start:
    a) CSSD, b) CRSD, c) CTSSD, d) DISKMON, e) GIPCD, f) GPNPD, g) EVMD, h) MDNSD etc.     
    Level-3: CRSD spawns:_
    In this level CRSD will start again:
    a) ORAROOTAGENT and b) ORAAGENT
    Level-4: CRSD orarootagent and oraagent SPAWNS:_
    Both orarootagent and oraagent both will start all the resources like ASM, DB, LISTENER, VIPs etc.
    Now Answer for your question:
    *# CRSCTL STOP CLUSTER:*
    This command will stop all the resources being managed by CRSD's Oragent & Orarootagent.
    *# CRSCTL STOP CRS:*
    This command will stop entire gridInfra i.e all daemons.
    *# CRSCTL STOP HAS:*
    This command is similar like CRSCTL STOP CRS but better to use this command in standalone server having GridInfra.
    Thanks,
    Ashish Shukla
    Edited by: Ashish Shukla on May 20, 2013 3:39 PM

Maybe you are looking for

  • Rented TMNT 2014 no longer available

    Hello, Friday night (Oct 24th) I rented the new Teenage Mutant Ninja Turtles (2014 - Michael Bay) movie. 3/4 of the way through, the movie stopped playing. I blamed my Internet (line of sight) and figured I would finish the movie later this weekend.

  • Updating to Acrobat Pro 7.1.0  Solution to error 1304 in Vista

    I have spent considerable time finding a solution to this problem and thought I'd post a new message with the solution. Sometimes it is very difficult to find real solutions on this site, and "you must fork out megabucks for a new CSx version because

  • I need help with upgrade troubleshooting

    I have a g4 powerbook 17" running on 10.3. When i tried to install 10.4 it started out fine. My laptop restarted then i chose a language but shortly then after a split second after an error message pops up with "This software cannot be installed on t

  • Print driver problem with OS 10.4.6

    I recently installed 10.4.6 and ever since have been unable to print via a network on a Lexmark Optra T614 (duplex). All was working fine until the upgrade to 10.4.6. I have downloaded a driver from Lexmark's website and successfully installed it to

  • Advantages of New G/L activation

    Hi Experts, If i enable new G/L account what all benefit can get it ? regards Sanju M S Moderator: Please, avoid asking basic questions