Database Hang Analysis

DB:11.2.0.3
OS :Linux 5
My database is in hung state now. I did the below steps
sqlplus -prelim '/as sysdba'
SQL*Plus: Release 11.2.0.3.0 Production on Thu Oct 24 08:18:17 2013
Copyright (c) 1982, 2011, Oracle.  All rights reserved.
SQL> oradebug setmypid
Statement processed.
SQL> oradebug unlimit
Statement processed.
SQL> oradebug hanganalyze 3
Statement processed.
SQL> oradebug hanganalyze 3
Statement processed.
SQL> oradebug tracefile_name
/oemdwlk1/oracle/diag/rdbms/eioemd1/eioemd1/trace/eioemd1_ora_14132.trc
SQL> exit
Getting the below output in trace file
Processing Oradebug command 'hanganalyze 3'
===============================================================================
HANG ANALYSIS:
ERROR: Can not perform hang analysis dump without a process
       state object and a session state object.
  ( process=(nil), sess=(nil) )
As per oracle doc How to Collect Diagnostics for Database Hanging Issues (Doc ID 452358.1).
Note: From 11.2.0.2 onwards, hanganalyze will not produce output under a sqlplus "preliminary connection" since it requires a process state object and a session state object. If a hanganalyze is attempted, the trace will output:
HANG ANALYSIS:
ERROR: Can not perform hang analysis dump without a process state object and a session state object.
( process=(nil), sess=(nil) )
Is something i am missing or there is some different steps.
===============================================================================

Level 3  --> will collect HANGANALYZE OUTPUT + DUMPS ONLY THE PROCESSES THOUGH TO BE IN HANG.
This many not be a case of process hang. So check with LEVEL 10 .
Still if you are getting the same error than , you need to contact oracle support.
Thanks,
<mod. action: blog ref. removed, please link to one of your blog entries when it's appropriate>
Message was edited by: Nicolas.Gasparotto

Similar Messages

  • Facing error while performing hang analysis in RAC database.

    Hi,
    We have been performing hang analysis in one of our RAC databases. While doing this we faced the below error after executing the systemstate command.
    SQL> oradebug -g all dump systemstate 266
    Error 601 encountered by local DIAG [ospid=11984]
    Error 601 encountered by remote DIAG [ospid=25384][instance=2]
    Error 601 encountered by remote DIAG [ospid=688][instance=3]
    Please help us to troubleshoot this and guide us on how to proceed.
    Database version is as follows.
    11.1.0.7
    OS version is as follows,
    $ uname -a
    SunOS isdtarhp001 5.10 Generic_144488-06 sun4v sparc SUNW,T5240
    $ isainfo -vk
    64-bit sparcv9 kernel modules

    Hi..
    Check the note # SAP Note 1121978
    SAP Note 1121978 - Recommended settings to improve peformance risk analysis.
    Check for the following...
    CONFIGTOOL>SERVER>MANAGERS>THREADMANAGER
    ChangeThreadCountStep =50
    InitialThreadCount= 100
    MaxThreadCount =200
    MinThreadCount =50
    Regards
    Gangadhar

  • Shutting down database hangs

    Dear Gurus,
    I m facing a database hang issue while performing the scheduled offline backup.I m able to take the online backup.When the backup starts , the database starts to shutdown.When checked in the log of backup it shows in tail as shutting down database.When checked for user trace , it shows message as "Attempting to kill pid xxxxxxx" and keeps on retrying for n no. of times.Then I have to manually kill the backup and restart the application.Then again try for an offline backup.This time it happens smoothly.This activity we did for couple of weeks .But now from past 3 days even after restarting the backup, it goes to hang mode.In other words unable to perform the backup activity.
    The message had been raised to SAP from almost 15 days at very high.They every time ask the logs , and say that the data in log is nt providing much info for analysing the issue or cause of hang.Also enabled the traces of 10400 for trace level 1 ,10 some more traces also.But still the output is not helping even them to analyse the root cause.
    Its my observation that this issue is occuring only on ABAP+JAVA systems(CRM &BIW)not in ECC.We are on oracle database 10.2.0.0.4 with latest patch set as suggested by SAP for the same issue.
    This issue is h
    Please suggest me at the earliest.As need to resolve this at the earliest.
    Bye.
    Amit

    Hi,
    No "archive process shutdown avoided" message is there in the log file.
    In usertrace i get messages as mentioned below.
    bslcrmprddb #more bcp_ora_1257502.trc
    Dump file /oracle/BCP/saptrace/usertrace/bcp_ora_1257502.trc
    Oracle Database 10g Enterprise Edition Release 10.2.0.4.0 - 64bit Production
    With the Partitioning, Data Mining and Real Application Testing options
    ORACLE_HOME = /oracle/BCP/102_64
    System name:    AIX
    Node name:      bslcrmprddb
    Release:        3
    Version:        5
    Machine:        00C6C4024C00
    Instance name: BCP
    Redo thread mounted by this instance: 1
    Oracle process number: 0
    Unix process pid: 1257502, image: oracle@bslcrmprddb
    2009-06-05 08:01:36.875
    Oracle Instance Startup operation failed. Another process may be attempting to startup or shutdown
    Failed to acquire instance startup/shutdown serialization primitive
    bslcrmprddb #more bcp_ora_2453676.trc
    Dump file /oracle/BCP/saptrace/usertrace/bcp_ora_2453676.trc
    Oracle Database 10g Enterprise Edition Release 10.2.0.4.0 - 64bit Production
    With the Partitioning, Data Mining and Real Application Testing options
    ORACLE_HOME = /oracle/BCP/102_64
    System name:    AIX
    Node name:      bslcrmprddb
    Release:        3
    Version:        5
    Machine:        00C6C4024C00
    Instance name: BCP
    Redo thread mounted by this instance: 0 <none>
    Oracle process number: 0
    Unix process pid: 2453676, image: oracle@bslcrmprddb
    2009-06-06 07:04:22.494
    Dynamic strand is set to TRUE
    Running with 2 shared and 88 private strand(s). Zero-copy redo is FALSE
    bslcrmprddb #more bcp_ora_1495266.trc
    Dump file /oracle/BCP/saptrace/usertrace/bcp_ora_1495266.trc
    Oracle Database 10g Enterprise Edition Release 10.2.0.4.0 - 64bit Production
    With the Partitioning, Data Mining and Real Application Testing options
    ORACLE_HOME = /oracle/BCP/102_64
    System name:    AIX
    Node name:      bslcrmprddb
    Release:        3
    Version:        5
    Machine:        00C6C4024C00
    Instance name: BCP
    Redo thread mounted by this instance: 1
    Oracle process number: 213
    Unix process pid: 1495266, image: oracle@bslcrmprddb (TNS V1-V3)
    ACTION NAME:() 2009-06-06 03:00:22.223
    MODULE NAME:(sqlplus@bslcrmprddb (TNS V1-V3)) 2009-06-06 03:00:22.223
    SERVICE NAME:(SYS$USERS) 2009-06-06 03:00:22.223
    SESSION ID:(517.13229) 2009-06-06 03:00:22.223
    ksukia: Starting kill, force = 0
    ksukia: killed 274 out of 274 processes.
    ksukia: Starting kill, force = 0
    ksukia: killed 0 out of 2 processes.
    ksukia: Starting kill, force = 0
    ksukia: killed 0 out of 1 processes.
    ksukia: Starting kill, force = 0
    ksukia: killed 0 out of 1 processes.
    ksukia: Starting kill, force = 0
    Please suggest.
    Regards,
    Amit

  • 11gR2 Hang Analysis

    Does anyone know any resources from where we can learn to interpret hang analysis? I read DOc 215858.1 and
    http://arup.blogspot.com/2008/08/diagnosing-library-cache-latch.html and
    http://wiki.oracle.com/page/Hanganalyze.
    But I think those are for 10g and prior and looks different from the hang analysis trace report I generated. I am looking for something very detailed.

    Oracle Database 10g introduced the Hang Analysis tool in Enterprise Manager, and Oracle Database 11g now expands this concept with the Hang Manager. So read and search on Hang Manager from 11g.

  • Oracle 10g Database Hanging twice a day

    My database server has 10g with MS 2003 server as OS.
    Twice a day the database hangs and does disconnects all the users from local and remote locations.
    Current SYSTEM tablespace has a size of 470 MB and is 98.86 in use, when I try to add another datafile as 'SYSTEM02.DBF' in the same path where other system data files are, I kept this operation running during weekends but datafile was not added.
    Has it anything to do with the MS 2003 server no. of users.
    Any other suggestions for SYSTEM Tablespace to be free.
    Please reply.
    Thanks
    Ajaz Ahmed

    Hi friends / sameer,
    Sorry for the delay. Here is the attached alert.log file Since the data from foxpro was loaded into oracle on to oracle 10 g from 7th may the problem of database hanging started from the next day. I joined the organization on 22nd july.
    Current SYSTEM tablespace has a size of 470 MB and is 98.86 in use, when I try to add another datafile as 'SYSTEM02.DBF' in the same path where other system data file is, it does not add. I kept this operation running during weekends but datafile was not added.
    Has it anything to do with the MS 2003 server no. of users.
    Any other suggestions for SYSTEM Tablespace to be free.
    If you can get me your mail id then I can send the
    attachment of alert.log file as [email protected] bounces back.
    Thanks
    Ajaz Ahmed

  • Database Hanging

    Hi,
    I have Oracle database 10g my database is hanging and i want to check what locks occur on which table how can i work on this database hanging process.Kindly help...because i am new for this i need good procedure to face this problem..

    If db hangs there could be a lot of possibilities..
    eg. no more space for archive log or datafiles
    Locks on table are different. You can look on alertlog and start investigating..

  • Oracle 8i database hangs for the same amount during JDBC calls.

    Hi all,
    I have a Oracle 8i database on solaris. The database hangs for exactly 61 seconds for a random jdbc call from my Java application, i mean there is no particular pattern or a JDBC call for which the database hangs. If i choose to wait for 61 seconds it returns the results from the database but if i try to access the database with another query(SQL Plus) the first query starts executing immediately. Is that a database configuration problem or some thing else.
    And i also get some exceptions occasionally( Maximum Open Cursors exception). Are these two related?
    Any feedback is appreciated.
    Thanks
    Hugo Victor

    I have uninstalled Oracle 8i.
    Removed all the services from Windows services by deleting corresponding registry entries and deleted the Oracle folder.
    I have reinstalled Oracle 8i, this time i changed the name of global database to oracledb.
    I channged the tnsnames.ora file from DevSuite appropriately as:
    # tnsnames.ora Network Configuration File: C:\DevSuiteHome_1\network\admin\tnsnames.ora
    # Generated by Oracle configuration tools.
    EXTPROC_CONNECTION_DATA =
    (DESCRIPTION =
    (ADDRESS_LIST =
    (ADDRESS = (PROTOCOL = TCP)(HOST = mscserver)(PORT = 1521))
    (CONNECT_DATA =
    (SERVICE_NAME = ORACLEDB)
    Still it can't login using scott/tiger authentication.
    What may be the problem? Please help.
    -Sameer

  • Urgent !! - Hang analysis doesn't work

    Hello Friends,
    I'm using 10.2.0.3 grid control. When I click on performance - > hang analysis it throws up the following erro
    The agent received the following error when getting data for the Hang Analysis display: ld.so.1: oracle: fatal: libodm9.so: open failed: No such file or directory ld.so.1: oracle: fatal: libodm9.so: open failed: No such file or directory ORADEBUG HANGANALYZE could not be run [ORA-12547: TNS:lost contact, ORA-12547: TNS:lost contact] at /oraDBusers/agentn10g/agent10g/sysman/admin/scripts/db/hanganalyze.pl line 76.
    I have spent about 6 hours. deinstalled, reinstalled i don't seem to find the above library that it is complaining about.
    Thanks,
    Madhu

    Paste the install action log

  • What to do when database hang

    Hi all,
    May I know what to do when database hangs after shutdown immediate command perform? Can I use kill process command to stop Oracle process?

    Hello,
    see documentation for details of immediate-clause:
    http://download.oracle.com/docs/cd/B19306_01/server.102/b14231/start.htm#sthref589
    You can do an shutdown abort instead, but then sql is terminated. You can read details here: http://download.oracle.com/docs/cd/B19306_01/server.102/b14231/start.htm#sthref595
    Greets,
    Hannibal

  • On performing continuous delete operation database hangs

    Dear All,
    On performing continuous delete operation , the instance on which delete operation is performed database hangs i.e it does not allows to make new connection and some times on alert log we get ORA-3136 error
    Database version 10.2.0.3 , OS : HP-UXvi3
    Regards

    Refer this thread
    ORA-3136 while performing bulk delete

  • Database Hanging Error

    Hi,
    I am working oracle9i and WIndows 2003 Server.Sometimes my Database is Hanged up and i am not able to logint to the Database using PL/SQL. Then i restarted the DB again . After that the error is resolved. But everyday i am geeting this Database Hanging problem...
    Server and Network are fine.
    Please provide me the solution...
    REgs..

    Hi,
    You can see (Diagnosing Database Hanging Issues (Doc ID: Note:61552.1)) on Oracle Metalink.
    Adith

  • Enterprise Manager,under performance, aclick on hang analysis fails.

    Hi there,
    Am on 10.2.0.3.0
    on my enterprise manager when am logged in as sys, when i get to performance menu and go to hang analysis,
    or top activity, i get a rejection sighting wrong credentials , that i log in with the right credentials, am wondering how this is happening when am already logged in as sys and can access
    all other items with no problems.
    am running on three clustes with 3 instances.
    Some one there to guide me.

    Did you check the entries you made in your preferred credentials?
    Eric

  • Enterprise Manager - Hang Analysis

    I am using Oracle 10g Release 1 . On Enterprise Manager , Hang Analysis I am looking for a way to examine data from the last 40 hours. I had some frozen sessions starting yesterday morning and am searching for possible causes. The system is working fine at the moment. On the hang analysis option on Enterprise Manager there doesn't seem to be a way of examing past data.

    Did you check the entries you made in your preferred credentials?
    Eric

  • Weird : Database Hangs without any message in Alert File

    Hi
    We have production database 10.2.0 , in the last 3 days database database has to be restarted as it hangs without any information in the Alert File.
    No trace file is genrated at that time.
    Only SYS user Can LOGIN to the database.
    I had no other option but to restart
    Database is in NO ARCHIVE MODE
    There is Enough Space in Temporary Tablespace and all other tablespace.
    NO patches were applied to the database
    NO parameters has been modified
    When i check the CPU usage its normal.
    There is Enough Disk Space.
    This is procution and i apreciate Help.
    Thank You

    user6421665 wrote:
    Only SYS user Can LOGIN to the database.
    What is the OS ? Windows --> Did you check virus scan or some other process running during that time ?
    Unix / Linux --> Is sysadmin running a defrag on the disks or some other OS Utility ?
    Do you login as SYS from the server ? Can you login as any other user from the server ?
    Hang == Slow ?
    >
    This is procution and i apreciate Help.
    Production database in NOARCHIVE log mode... is interesting by itself, but that is not a question here.

  • Database hangs after retrieving some records....

    hi,
    I create a two level B+-tree index, for the first level i'm using key as logical database name, for the second level B+-tree i'm using some other field in my data. I am retrieving the records based on logical database name. I am using C++ to implement my index.
    In the following code i'm retrieving records from database. Program is behaving differently for different logical database names. For example in my database i have around 4 lakhs records with logical database name 'A' and around 1 lakh of records with logical database name 'B'. The following code displays all B records but programs hangs after retrieving some A records.
    I'm using PAGE_SIZE=8192, numBuffers=2048, and runnig in Fedora Core3.
    DbEnv myEnv(0);
         myEnv.set_cachesize(0, PAGE_SIZE * numBuffers, 0);
         myEnv.set_data_dir("/home/raviov/new/database");
              try {
                   myEnv.open("./", DB_CREATE | DB_INIT_MPOOL, 0);
              catch (DbException &e) {
                   cerr << "Exception occurred: " << e.what() << endl;
                   exit(1);
              db=new Db(&myEnv,0);
              db->set_pagesize(PAGE_SIZE);     
              db->set_bt_compare(compare_int);
         dbname=itoa(p);
         try
         db->open(NULL,
                   "treedb.db",
                   dbname,
                   DB_BTREE,
                   0,
                   0);
         catch(DbException &e)
              cerr << "Failed to open DB object" << endl;
              exit(1);
         db->cursor(NULL,&cursorp,0);
         key=new Dbt();
         data=new Dbt();
         while(ret=(cursorp->get(key,data,DB_NEXT))==0)
              j=*((int*)key->get_data());
              q=(*((struct tuple*)data->get_data())).startPos;
              r=(*((struct tuple*)data->get_data())).endPos;
              s=(*((struct tuple*)data->get_data())).level;
              cout<<"position : "<<j<<"\t";
              cout<<"start : "<<q<<"\t";
              cout<<"end : "<<r<<"\t";
              cout<<"level : "<<s<<"\n";
         if(ret==DB_NOTFOUND)
              cout<<"no records";
              exit(1);
         if(cursorp!=NULL)
              cursorp->close();
         try
              db->close(0);
         catch(DbException &e)
              cerr << "Failed to close DB object" << endl;
              exit(1);
         myEnv.close(0);

    HI Andrei,
    thank you for giving reply, I'm not using any secondary indecies, subdatabases and not using any threads.
    the following code displays index...
    #include <stdio.h>
    #include <db_cxx.h>
    #include <iostream.h>
    #include <db.h>
    #include <stdlib.h>
    #include <string.h>
    #include <sys/types.h>
    int PAGE_SIZE=8192;
    int numBuffers=2048;
    char *itoa(const int x)
          char buf[100];
          snprintf(buf, sizeof(buf), "%d", x);
           return strdup(buf);
    int compare_int(Db dbp, const Dbt a, const Dbt *b)
      int ai;
      int bi;
      memcpy(&ai,a->get_data(),sizeof(int));
      memcpy(&bi,b->get_data(),sizeof(int));
      return(ai-bi);
    struct tuple
           int startPos;
           int endPos;
           int level;
    main()
         FILE *fp;
         int i,j,k,l,m,n,total=0;
         char dbname[500],filename[500],str [500];
         tuple t;
         Db *db;
         Dbt key,data;
         DbEnv myEnv(0);
         myEnv.set_cachesize(0, PAGE_SIZE * numBuffers, 0);
         try {
                myEnv.open("/home/raviov/Desktop/example", DB_CREATE | DB_INIT_MPOOL, 0);
           catch (DbException &e) {
                  cerr << "Exception occurred: " << e.what() << endl;
                  exit(1);
         for(n=0;n<=84;n++)
              db = new Db(&myEnv, 0);     // Instantiate the Db object
                    db->set_bt_compare(compare_int);
               db->set_pagesize(PAGE_SIZE);
              strcpy(filename,"/home/raviov/Desktop/GTCReport/code/sequence/splitter/tree/");
              strcat(filename,itoa(n));
              fp=fopen(filename,"r");
              if(fp==NULL)
                   cout<<"error in opening the file";
                   exit(0);
              while(fgets (str , 500 , fp)!=NULL)
                   sscanf(str,"%d(%d,%d,%d,%d)",&i,&j,&k,&l,&m);
                   key=new Dbt(&i,sizeof(int));
                   if(total==0)
                        strcpy(dbname,itoa(j));
                   t.startPos=k;
                   t.endPos=l;
                   t.level=m;
                   data=new Dbt(&t,sizeof(t));     
                   if(total==0)
                        try
                             db->open(NULL,
                             "tree.db",
                                       dbname,
                              DB_BTREE,
                                DB_CREATE,
                                0);
                        catch(DbException &e)
                               cerr << "Failed to create DB object" << endl;
                             exit(1);
                        total=99;
                   int ret=db->put(NULL,key,data,DB_NOOVERWRITE);
                   if(ret==DB_KEYEXIST)
                        cout<<"key already exist\n";
                        exit(1);
                   delete key;
                   delete data;
              total=0;
              fclose(fp);
              try
                   db->close(0);
              catch(DbException &e)
                     cerr << "Failed to close DB object" << endl;
                   exit(1);
         myEnv.close(0);
    }The following code retrieves the records from database that we had built above...
    #include <stdio.h>
    #include <db_cxx.h>
    #include <iostream.h>
    #include <db.h>
    #include <stdlib.h>
    #include <string.h>
    #include <sys/types.h>
    int PAGE_SIZE=8192;
    int numBuffers=2048;
    char *itoa(const int x)
          char buf[100];
          snprintf(buf, sizeof(buf), "%d", x);
           return strdup(buf);
    int compare_int(Db dbp, const Dbt a, const Dbt *b)
      int ai;
      int bi;
      memcpy(&ai,a->get_data(),sizeof(int));
      memcpy(&bi,b->get_data(),sizeof(int));
      return(ai-bi);
    struct tuple
           int startPos;
           int endPos;
           int level;
    main()
         FILE *fp;
         int i,j,k,l,m,n,total=0;
         char *dbname;
         char filename[200],str [100];
         tuple t;
         Db *db;
         Dbt key,data;
         Dbc *cursorp=NULL;
         int ret,x=4;
         char *ravi;
         int p=84763;
         int y=2134872;
         int r,s,q,count=0;
         DbEnv myEnv(0);
         myEnv.set_cachesize(0, PAGE_SIZE * numBuffers, 0);
         myEnv.set_data_dir("/home/raviov/new/database");
              try {
                      myEnv.open("./", DB_CREATE | DB_INIT_MPOOL, 0);
               catch (DbException &e) {
                      cerr << "Exception occurred: " << e.what() << endl;
                      exit(1);
              db=new Db(&myEnv,0);
              db->set_pagesize(PAGE_SIZE);     
              db->set_bt_compare(compare_int);
         dbname=itoa(p);
         try
              db->open(NULL,
                    "tree.db",
                     dbname,
                     DB_BTREE,
                     0,
                     0);
         catch(DbException &e)
                cerr << "Failed to open DB object" << endl;
              exit(1);
         db->cursor(NULL,&cursorp,0);
         key=new Dbt();
         data=new Dbt();
         while(ret=(cursorp->get(key,data,DB_NEXT))==0)
              j=*((int*)key->get_data());
              q=(*((struct tuple*)data->get_data())).startPos;
              r=(*((struct tuple*)data->get_data())).endPos;
              s=(*((struct tuple*)data->get_data())).level;
              cout<<"position   : "<<j<<"\t";
              cout<<"start : "<<q<<"\t";
              cout<<"end   : "<<r<<"\t";
              cout<<"level   : "<<s<<"\n";
              delete key;
              delete data;
         if(ret==DB_NOTFOUND)
              cout<<"no records";
              exit(1);
         if(cursorp!=NULL)
              cursorp->close();
         try
              db->close(0);
         catch(DbException &e)
                cerr << "Failed to close DB object" << endl;
              exit(1);
         myEnv.close(0);     
    }

Maybe you are looking for

  • Two different accounts? How do I know if my stuff is backed up?

    My laptop and my iPhone seem to have two different iCloud accounts. The first account (laptop) is the username that I had before I changed my username to the account that is on my iPhone. Since my laptop has my old one, I tried to log out and sign in

  • I want to change the battery on my macbook pro 13" by myself

    Replacing a battery takes 5mn, even on an early 2011 macbook pro. I need my computer everyday and don't want to send it to apple for a simple battery replacement. Do I have to throw it and buy a laptop pc or will apple sell a battery(A 1322) for our

  • Error with the PL/SQL below as  ORA-06550: line 35, column 3

    declare    c pls_integer := 0;    l_detail varchar2(4000);    i pls_integer;    l pls_integer;    l_max_rows   integer; begin l_max_rows := nvl(:P2_ROWS,10); if :P2_SEARCH is null and :P2_LastName is null and :P2_FirstName is null and P2_Course  is n

  • Can we configure more values for Status 2

    Hi every one, can we configure  more values  associated with employment status(status 2)i.e Value 0: Employee no longer with the company Value 1: Employee is a part of the company but inactive Value 2: Employee is a part of the company but retiree Va

  • Creating a CNN, Sky type news ticker

    Hi, I have a three minute video and would like to add a news ticker that will loop once during the three minutes. The ticker text is considerable. Before I used to use Sony Vegas. A video I created using Sony Vegas is at http://youtu.be/bmcXM7fE6s0 (