Cleanup RAC DB

Hi, I'm in middle of RAC instalaltion (2-node cluster) on CRS 10.2.0.4 and DB 10.2.0.4 versions on RHEL 4u6-64-bit. I could successfully install clusterware/DB and upgrade to 10.2.0.4 version. I started creating the Database but the database creation failed due to the mount options for controlfile location was missing notime0. I added that but now i had to delete the files created for the database, whihc failed. When i try to recreate the DB, its failing obvisouly as its finding the same name. I cleaned up the oratab and other locations. My question is when i do the crs start ot stat, the clusterware is still looking and showing offline for the DB which failed creation and its two instances are being tried by CRS. How do i clean up the failed DB so the CRS goes back to the pre-DB creation and don't look for the failed DB nodes?
thanks
Ash

Hi Akon,
Try steps described on this blog http://www.pythian.com/blogs/1044/oracle-silent-mode-part-6-remove-a-node-from-a-102-rac#rminstmanual
regards,
Rodrigo Mufalani
http://mufalani.blogspot.com

Similar Messages

How to cleanup / remove RAC installation (CRS,RDBMS,ASM)

Hi, all
How to remove or cleanup RAC installation after ran OUI deinstall it couldn't finish and get a error... because 1 node of 3 was off (the box).
ACS

Don't forget to mention the Oracle version you're working with.

Not able to install Statspack.

DB 9.2.0.5.0
OS WIn 2003 server
I've gone through the following procedure to generate a statspack report. but, I am not able to insatall statspack, though the same procedure worked in 10g.
1. create a perfstat tablespace
2. run spcreate.sql
SPCUSR complete. Please check spcusr.lis for any errors.
SQL>
SQL> --
SQL> -- Build the tables and synonyms
SQL> connect perfstat/&&perfstat_password
ERROR:
ORA-12560: TNS:protocol adapter error
Warning: You are no longer connected to ORACLE.
SQL> @@spctab
SQL> Rem
SQL> Rem $Header: spctab.sql 16-apr-2002.14:54:46 vbarrier Exp $
SQL> Rem
SQL> Rem spctab.sql
SQL> Rem
SQL> Rem Copyright (c) 1999, 2002, Oracle Corporation. All rights reserved.
SQL> Rem
SQL> Rem NAME
SQL> Rem spctab.sql
SQL> Rem
SQL> Rem DESCRIPTION
SQL> Rem SQL*PLUS command file to create tables to hold
SQL> Rem start and end "snapshot" statistical information
SQL> Rem
SQL> Rem NOTES
SQL> Rem Should be run as STATSPACK user, PERFSTAT
SQL> Rem
SQL> Rem MODIFIED (MM/DD/YY)
SQL> Rem vbarrier 03/20/02 - 2143634
SQL> Rem vbarrier 03/05/02 - Segment Statistics
SQL> Rem cdialeri 02/07/02 - 2218573
SQL> Rem cdialeri 01/30/02 - 2184717
SQL> Rem cdialeri 01/11/02 - 9.2 - features 2
SQL> Rem cdialeri 11/30/01 - 9.2 - features 1
SQL> Rem cdialeri 04/22/01 - Undostat changes
SQL> Rem cdialeri 03/02/01 - 9.0
SQL> Rem cdialeri 09/12/00 - sp_1404195
SQL> Rem cdialeri 04/07/00 - 1261813
SQL> Rem cdialeri 03/20/00 - Support for purge
SQL> Rem cdialeri 02/16/00 - 1191805
SQL> Rem cdialeri 01/26/00 - 1169401
SQL> Rem cdialeri 11/01/99 - Enhance, 1059172
SQL> Rem cmlim 07/17/97 - Added STATS$SQLAREA to store top sql stmts
SQL> Rem gwood 10/16/95 - Version to run as sys without using many vi
SQL> Rem cellis.uk 11/15/89 - Created
SQL> Rem
SQL>
SQL> set showmode off echo off;
If this script is automatically called from spcreate (which is
the supported method), all STATSPACK segments will be created in
the PERFSTAT user's default tablespace.
Using perfstat tablespace to store Statspack objects
... Creating STATS$SNAPSHOT_ID Sequence
SP2-0640: Not connected
SP2-0640: Not connected
... Creating STATS$... tables
SP2-0640: Not connected
SP2-0640: Not connected
SP2-0640: Not connected
SP2-0640: Not connected
SP2-0640: Not connected
SP2-0640: Not connected
SP2-0640: Not connected
SP2-0640: Not connected
NOTE:
SPCTAB complete. Please check spctab.lis for any errors.
SQL> -- Create the statistics Package
SQL> @@spcpkg
SQL> Rem
SQL> Rem $Header: spcpkg.sql 17-apr-2002.16:59:10 vbarrier Exp $
SQL> Rem
SQL> Rem spcpkg.sql
SQL> Rem
SQL> Rem Copyright (c) 1999, 2002, Oracle Corporation. All rights reserved.
SQL> Rem
SQL> Rem NAME
SQL> Rem spcpkg.sql
SQL> Rem
SQL> Rem DESCRIPTION
SQL> Rem SQL*PLUS command file to create statistics package
SQL> Rem
SQL> Rem NOTES
SQL> Rem Must be run as the STATSPACK owner, PERFSTAT
SQL> Rem
SQL> Rem MODIFIED (MM/DD/YY)
SQL> Rem vbarrier 03/20/02 - 2184504
SQL> Rem spommere 03/19/02 - 2274095
SQL> Rem vbarrier 03/05/02 - Segment Statistics
SQL> Rem spommere 02/14/02 - cleanup RAC stats that are no longer needed
SQL> Rem spommere 02/08/02 - 2212357
SQL> Rem cdialeri 02/07/02 - 2218573
SQL> Rem cdialeri 01/30/02 - 2184717
SQL> Rem cdialeri 01/09/02 - 9.2 - features 2
SQL> Rem cdialeri 11/30/01 - 9.2 - features 1
SQL> Rem hbergh 08/23/01 - 1940915: use substrb on sql_text
SQL> Rem cdialeri 04/26/01 - 9.0
SQL> Rem cdialeri 09/12/00 - sp_1404195
SQL> Rem cdialeri 04/07/00 - 1261813
SQL> Rem cdialeri 03/28/00 - sp_purge
SQL> Rem cdialeri 02/16/00 - 1191805
SQL> Rem cdialeri 11/01/99 - Enhance, 1059172
SQL> Rem cgervasi 06/16/98 - Remove references to wrqs
SQL> Rem cmlim 07/30/97 - Modified system events
SQL> Rem gwood.uk 02/30/94 - Modified
SQL> Rem densor.uk 03/31/93 - Modified
SQL> Rem cellis.uk 11/15/89 - Created
SQL> Rem
SQL>
SQL> set echo off;
Creating Package STATSPACK...
SP2-0640: Not connected
SP2-0640: Not connected
SP2-0641: "SHOW ERRORS" requires connection to server
Creating Package Body STATSPACK...
SP2-0640: Not connected
SP2-0640: Not connected
SP2-0641: "SHOW ERRORS" requires connection to server
NOTE:
SPCPKG complete. Please check spcpkg.lis for any errors.
hare krishna
Alok

Hi Alok
I have encountered this before
In the script there will be a comnnection link at the spcreate sql..
Rem
Rem $Header: spcreate.sql 16-apr-2002.11:22:55 vbarrier Exp $
Rem
Rem spcreate.sql
Rem
Rem Copyright (c) 1999, 2002, Oracle Corporation. All rights reserved.
Rem
Rem    NAME
Rem      spcreate.sql - Statistics Create
Rem
Rem    DESCRIPTION
Rem      SQL*PLUS command file which creates the STATSPACK user,
Rem      tables and package for the performance diagnostic tool STATSPACK
Rem
Rem    NOTES
Rem      Note the script connects INTERNAL and so must be run from
Rem      an account which is able to connect internal.
Rem
Rem    MODIFIED   (MM/DD/YY)
Rem    cdialeri    02/16/00 - 1191805
Rem    cdialeri    12/06/99 - 1103031
Rem    cdialeri    08/13/99 - Created
Rem
-- Create PERFSTAT user and required privileges
@@spcusr
-- Build the tables and synonyms
connect perfstat/&&perfstat_password@MARAN
@@spctab
-- Create the statistics Package
@@spcpkgadd the @servicename and run the script
It will work
Message was edited by:
Maran Viswarayar

Lots of ORA- in my alertPROD.log

Hi,
EBS 11i (11.5.10.2)
RHEL 4.6
I browsed our database alert logs and I see lots of following:
1)
ORA-01555 caused by SQL statement below (Query Duration=1286790742 sec, SCN: 0x056d.6f011aeb):
Mon Oct 11 17:52:22 2010
SELECT TO_CHAR(REQUEST_DATE,'MM/DD/YYYY') FROM APPS.OE_ORDER_HEADERS_ALL WHERE TO_CHAR(ORDER_NUMBER) = :B1
2)
ORA-1652: unable to extend temp segment by 212992 in tablespace SYSTEM
3)
ORA-07445: exception encountered: core dump [ptmak()+106] [SIGSEGV] [Address not mapped to object] [0x8] [] []
Mon Oct 18 17:27:57 2010
4)
WARNING:Oracle instance running on a system with low open file descriptor
limit. Tune your system to increase this limit to avoid
severe performance degradation.
Have you seen this in your alert logs too?
How do you resolve this errors please....
Thanks a lot,
Ms K

Hi hussein/Helios
My DB is 9.2.0.6 for EBS 11i
I am now installing Statspack, using the command
SQL> connect / as sysdba
SQL> @?/rdbms/admin/spcreate
NOTE:
SPCTAB complete. Please check spctab.lis for any errors.
SQL> -- Create the statistics Package
SQL> @@spcpkg
SQL> Rem
SQL> Rem $Header: spcpkg.sql 10-may-2004.15:32:48 cdialeri Exp $
SQL> Rem
SQL> Rem spcpkg.sql
SQL> Rem
SQL> Rem Copyright (c) 1999, 2004, Oracle. All rights reserved.
SQL> Rem
SQL> Rem    NAME
SQL> Rem         spcpkg.sql
SQL> Rem
SQL> Rem    DESCRIPTION
SQL> Rem         SQL*PLUS command file to create statistics package
SQL> Rem
SQL> Rem    NOTES
SQL> Rem         Must be run as the STATSPACK owner, PERFSTAT
SQL> Rem
SQL> Rem    MODIFIED   (MM/DD/YY)
SQL> Rem    cdialeri    05/10/04 - 3566569
SQL> Rem    vbarrier    03/20/02 - 2184504
SQL> Rem    spommere    03/19/02 - 2274095
SQL> Rem    vbarrier    03/05/02 - Segment Statistics
SQL> Rem    spommere    02/14/02 - cleanup RAC stats that are no longer needed
SQL> Rem    spommere    02/08/02 - 2212357
SQL> Rem    cdialeri    02/07/02 - 2218573
SQL> Rem    cdialeri    01/30/02 - 2184717
SQL> Rem    cdialeri    01/09/02 - 9.2 - features 2
SQL> Rem    cdialeri    11/30/01 - 9.2 - features 1
SQL> Rem    hbergh      08/23/01 - 1940915: use substrb on sql_text
SQL> Rem    cdialeri    04/26/01 - 9.0
SQL> Rem    cdialeri    09/12/00 - sp_1404195
SQL> Rem    cdialeri    04/07/00 - 1261813
SQL> Rem    cdialeri    03/28/00 - sp_purge
SQL> Rem    cdialeri    02/16/00 - 1191805
SQL> Rem    cdialeri    11/01/99 - Enhance, 1059172
SQL> Rem    cgervasi    06/16/98 - Remove references to wrqs
SQL> Rem    cmlim       07/30/97 - Modified system events
SQL> Rem    gwood.uk    02/30/94 - Modified
SQL> Rem    densor.uk   03/31/93 - Modified
SQL> Rem    cellis.uk   11/15/89 - Created
SQL> Rem
SQL>
SQL> set echo off;
Creating Package STATSPACK...
Package created.
No errors.
Creating Package Body STATSPACK...
Package body created.
No errors.
NOTE:
SPCPKG complete. Please check spcpkg.lis for any errors.
SQL>I got those logs above ... Is the installation of STATSPACK completed? It left hanging on SQL> prompt.
Thanks,

Rac node failure crs cleanup failing

I have a three node rac database, 10.2.0.4 running on Windows server 2008. I lost a hard drive on one of the servers and it corrupted the mirror disk as well so I am having to rebuild. I am going through these procedures, RAC on Windows: How to Cleanup When A Node Has Been Disconnected or The OS Rebuilt (Doc ID 742737.1) and am running into a problem once I tried to delete the listener and then on to crs to delete the nodeapps for node3.
For the listener, I go into netca and the option to delete a listener is grayed out. When I run crs_stat I can still see the ora.node3.lsnr there. Does this mean that I just need to update tnsnames.ora or is there another place the information would be held? I hate to manually delete because I am afraid I won't get it cleaned out from everywhere. Any idea why that option would not be there?
My second issue is when I run this:
srvctl stop nodeapps -n node3 The nodeapps stop doesn't return any output and then when I try to remove nodeapps it gives me PRKO-2112: Some or all node applications are not removed successfully on node.
I have searched metalink for that error with no success as the document I found also says that you must stop nodeapps. I have already deleted the node from the db and asm and updated the appropriate inventory. I just need to finish the listener and crs and update the inventory for crs. Also, I noticed that the vip for the failed node was reassigned to node2 and I show that it has been released when i run cluvfy to check. Would crs give me errors on this if that was not the case?
I appreciate any help or guidance!

Wanted to post a follow up in case any others are interested in the results...
I had tried to add the listener back to one of the remaining nodes .ora file and then delete but that didn't work. Also, remove nodeapps continued to throw an error that it could not stop the listener or vip for the failed node.
After a few days of reading I make a decision to just unregister the abandoned services from crs. I made sure to backup the OCRCONFIG before I ran crs_unregister and was able to successfully remove the listener and vip services from the failed node.
This eliminated my issue with netca, the node did not show up there anymore. I then went on to remove nodeapps and it failed saying it could not find the resource vip. I then ran olsnodes -n and used crssetup to remove the node entirely. Everything showed removed and I went and updated the crs inventory to finish.
All looks good and now I am working to rebuild and add the node back in.

APEX Application behaviour in a RAC setup

Hi
Caveat first: I'm pretty new to Oracle RAC and just looking into it as an option. We have an APEX application currently running in Oracle 11gR2 single node currently and are considering HA for this.
My question is: What would be the expected behaviour seen by a User of an APEX application, in the event of a node failure, when running with an OHS / RAC configuration? Will they get "transparent fail-over" and see nothing or will they see an error?
I appreciate I could post in the APEX forum, but feel that is probably more of a development forum and possibly someone here has had to look at things at this level.
I have read what I think may be the definitive reference for this:
http://www.oracle.com/technetwork/developer-tools/apex/learnmore/apex-rac-wp-133532.pdf
but while it covers most of what I want I don't believe I have found an answer to my question
This states:
"The Transparent Application Failover (TAF) feature of Oracle Net Services is a runtime failover for high-availability environments. It enables client applications to automatically reconnect to the database if the connection fails and, optionally, resume a SELECT statement that was in progress. The reconnection happens automatically from within the Oracle Call Interface (OCI) library. For applications that do insert, update or delete transactions, the application must trap the error when the failure occurs, rollback the transaction, and then resubmit. If the application is not written to be TAF aware, the session will get disconnected."
However (as I understand it) APEX runs in the database and would fail with the database, it isn't a typical "client application" connecting to Oracle via a TAF aware connection pool - it is essentially a large pl/sql package and TAF only covers SELECT statements not packages.
May be I'm over-reading this and it's simpler than that: APEX/Mod_plsql might just handle it?
- APEX User/HTTP session state is stored in database APEX: Understanding session state which is available on other nodes
- Mod_plsql in OHS can detect the error returning and reissues the request to good server and APEX on that instance can retrieve Users/HTTP state and process the request (APEX/RAC doc states mod_plsql can see an error from database and cleanup connection up and form a new connection, but not that it will retry the request for the client into other APEX/DB node).
I'm really just after a (transparent/non-transparent) statement based on experience, but an outline of how the components behave would be useful.
Thanks in advance
Dave

Hi
Any chance of getting that link outside of Metalink? - I'm trying to get our customer support id, but no luck at present.
I'm aware that APEX can run with RAC (as per the link I posted) - I'm really after next level info around behaviour in that environment.
Thanks
Dave

Need suggestion regarding drop RAC database of 11g

Hi All,
I have 11gR2 RAC environment setup. Database required recovery and I have no backup, based on support engineer guidance I restore the database using some hidden parameters though its a corrupted db so I need to rebuild. For rebuild purpose I took expdp of entire database and then drop it. I had drop through "dbca" , db was open and I run "dbca" and select drop database option, at one time it showing me warning/error ORA-01586:database must be mounted EXCLUSIVE and not open for this operation but after that drop was run perfectly.
but when I check the ASM disk status , it still show as it is , I mean no change in their free_space column, I also verified through asmcmd command and all the Datafiles are still there.
I want to cleanup asm disk which are still fillup with dbfs and I thought that would be cleared by dropping database but no my assumption was wrong.
can any one suggest me what to do if I want to make free space in asm disk which was allocated for dbf files?
Thanks in advance.

Hi Forstmann,
Please find both the commands output below:
ASMCMD> lsdg
State Type Rebal Sector Block AU Total_MB Free_MB Req_mir_free_MB Usable_file_MB Offline_disks Voting_files Name
MOUNTED NORMAL N 512 4096 1048576 14004 13208 0 6604 0 N ASM_FOR_CRS/
MOUNTED NORMAL N 512 4096 1048576 75069 19755 18440 657 0 N ORADATA/
MOUNTED NORMAL N 512 4096 1048576 200060 199773 74 99849 0 N ORADATA_PROD/
MOUNTED EXTERN N 512 4096 1048576 99998 48689 0 48689 0 N ORAFLASH_AREA/
MOUNTED NORMAL N 512 4096 1048576 106013 105823 0 52911 0 N TEST/
ASMCMD> find + *
+ASM_FOR_CRS/
+ASM_FOR_CRS/RWCORA-cluster/
+ASM_FOR_CRS/RWCORA-cluster/ASMPARAMETERFILE/
+ASM_FOR_CRS/RWCORA-cluster/ASMPARAMETERFILE/REGISTRY.253.725970931
+ASM_FOR_CRS/RWCORA-cluster/OCRFILE/
+ASM_FOR_CRS/RWCORA-cluster/OCRFILE/REGISTRY.255.725970933
+ORADATA/
+ORADATA/ASM/
+ORADATA/ASM/CONTROLFILE/
+ORADATA/ASM/CONTROLFILE/ctlbkp-23FEB.334.743838771
+ORADATA/ASM/DATAFILE/
+ORADATA/ASM/DATAFILE/SYSAUX.260.726054695.bkp.336.746443183
+ORADATA/ASM/DATAFILE/SYSTEM.259.726054677.bkp.335.746443047
+ORADATA/ASM/DATAFILE/UNDOTBS1.261.726054711.bkp.338.746443879
+ORADATA/ASM/DATAFILE/UNDOTBS2.263.726054721.bkp.339.746444613
+ORADATA/ASM/DATAFILE/USERS.264.726054727.bkp.337.746443729
+ORADATA/ASM/PARAMETERFILE/
+ORADATA/ASM/PARAMETERFILE/spfile.267.726056723.orig.324.731752409
+ORADATA/ASM/PARAMETERFILE/spfileracdb_08DEC.ora.333.737181947
+ORADATA/ASM/TEMPFILE/
+ORADATA/ASM/TEMPFILE/TEMP.262.726054713.bkp.340.746444645
+ORADATA/CMDOC
+ORADATA/CMDOC702
+ORADATA/CMDOC702RAC
+ORADATA/DM_CMDOC702RAC_INDEX
+ORADATA/DM_CMDOC702_INDEX
+ORADATA/DM_CMDOC_INDEX
+ORADATA/RACDB/
+ORADATA/RACDB/CONTROLFILE/
+ORADATA/RACDB/CONTROLFILE/Current.256.726054673
+ORADATA/RACDB/DATAFILE/
+ORADATA/RACDB/DATAFILE/JISPCARS.dbf
+ORADATA/RACDB/ONLINELOG/
+ORADATA/RACDB/ONLINELOG/group_1.257.746533235
+ORADATA/RACDB/ONLINELOG/group_2.258.746533237
+ORADATA/RACDB/ONLINELOG/group_3.265.746533239
+ORADATA/RACDB/ONLINELOG/group_4.266.746533241
+ORADATA/RACDB/PARAMETERFILE/
+ORADATA/RACDB/PARAMETERFILE/spfile.267.726056723.orig
+ORADATA/RACDB/PARAMETERFILE/spfile.267.737185455
+ORADATA/RACDB/PARAMETERFILE/spfile.325.731754029
+ORADATA/RACDB/PARAMETERFILE/spfileracdb.ora
+ORADATA/RACDB/TEMPFILE/
+ORADATA/RACDB/TEMPFILE/TEMP.262.726054713
+ORADATA/RACDB/TEMPFILE/TEMP.262.726054713.bkp
+ORADATA/RACDB/TEMPFILE/TEMPS1.341.746954959
+ORADATA/RACDB/TEMPFILE/temps1
+ORADATA/RACDB/ctlbkp-23FEB
+ORADATA/RACDB/spfileracdb.ora
+ORADATA/RACDB/spfileracdb_08DEC.ora
+ORADATA_PROD/
+ORAFLASH_AREA/
+TEST/
total 68 dbfs.

Error in installing Oracle Software while adding node in RAC

Hi All
Oracle Version: 11.1.0.6.0
Linux
In the RAC environment, I am in the process of adding a node to the existing rac environment.
I have succssfully installed cluster software in the new node. all gsd, ons and vip services are running perfectly for new node in the existing and new nodes.
Next step, I started installing oracle software by executing the script $ORACLE_HOME/oui/bin/addNode.sh.
Here after specifying the the new node information (public host, private host and vip details) and when I pressed next,
OUI is giving following error - "Error occured while retrieving node numbers of the existing nodes. Please check if clusterware home is properly configured".
I ran olsnodes -n, it is giving proper node number.
Can somebody help me in identifying the problem and solution.
Thanks

I am hitting the same issue, my problem was when I added the new node initially it fails because of the hint8-priv1. I followed the link below to remove and cleanup the last install. olsnode show only my existing nodes NOT including the new node.
http://download-west.oracle.com/docs/cd/B12037_01/rac.101/b10765/adddel.htm
then I ran
./addNode.sh -silent CLUSTER_NEW_NODES={hint-8} CLUSTER_NEW_PRIVATE_NODE_NAMES={hint8-priv1} CLUSTER_NEW_VIRTUAL_HOSTNAMES={hint8-vip}
I am getting this when use command line:
Error ocurred while retrieving node numbers of the existing nodes. Please check if clusterware home is properly configured.
SEVERE:Error ocurred while retrieving node numbers of the existing nodes. Please check if clusterware home is properly configured.
also run addNode.sh via GUI, the new node name still listed and can't continue.
BTW, adding new node should support online, shutdown is not required

How unhealthy is this RAC?

Here's is the contents of v$system_event..
Is this
EVENT     TOTAL_WAITS     TIME_WAITED     AVERAGE_WAIT
enq: TX - index contention     40564851     214701526      5.29
enq: TX - row lock contention     188846     12454614     65.95
enq: SQ - contention     141971     70568     0.5
cause for concern?
EVENT     TOTAL_WAITS     TIME_WAITED     AVERAGE_WAIT
SQL*Net message to client     6015051449     607254     0
SQL*Net message from client     6015048542     178177969892     29.62
gcs remote message     2948555287     2633481757     0.89
CGS wait for IPC msg     1517805027     634397     0
db file sequential read     1500615188     816364485     0.54
ges remote message     1247679701     1407300224     1.13
gc cr multi block request     778432813     9913464     0.01
gc current block 2-way     747852637     38030616     0.05
db file scattered read     709428365     460939295     0.65
rdbms ipc message     708473316     37650068633     53.14
gc buffer busy acquire     671285134     1033621285     1.54
PX Deq: reap credit     667784615     484449     0
gcs log flush sync     592376026     171712257     0.29
gc cr block 2-way     530861847     19607062     0.04
library cache pin     437937120     15126237     0.03
log file sync     379523272     797193932     2.1
DIAG idle wait     359607166     2822108755     7.85
log file parallel write     351225436     259263769     0.74
LNS ASYNC end of log     350170653     1398410516     3.99
LNS wait on SENDREQ     321652621     3209301     0.01
PX qref latch     297396661     94308     0
read by other session     289140108     148440270     0.51
buffer deadlock     163505781     983055     0.01
gc current block busy     119223825     467716658     3.92
PX Deq: Table Q Normal     117332841     23574867     0.2
ksxr poll remote instances     110480324     90333     0
buffer busy waits     106938153     19933900     0.19
direct path read     93429599     108427028     1.16
SQL*Net more data from client     86471785     23026529     0.27
gc current grant busy     84978157     28215346     0.33
control file sequential read     82646297     23694583     0.29
PX Deq Credit: send blkd     78641669     9569299     0.12
latch: cache buffers chains     74218671     690277     0.01
gc current grant 2-way     72557796     1920419     0.03
library cache: mutex X     71106697     75993     0
DFS lock handle     70722498     2716407     0.04
gc cr grant 2-way     64558237     1633004     0.03
PX Deq: Execution Msg     61706261     314222076     5.09
gc cr block busy     61469863     119850802     1.95
library cache lock     52428649     3773354     0.07
PX Deq: Slave Session Stats     48040224     1886805     0.04
db file parallel read     46415188     118467902     2.55
IPC send completion sync     46250594     965101     0.02
enq: TX - index contention     40564851     214701526     5.29
PX Deq: Execute Reply     39689685     17243721     0.43
gc buffer busy release     36976909     242714774     6.56
SQL*Net more data to client     36627952     44167     0
PX Deq: Msg Fragment     30501244     343397     0.01
rdbms ipc reply     29725302     1352370     0.05
RMAN backup & recovery I/O     28824547     37722662     1.31
reliable message     27892263     3082134     0.11
PX Idle Wait     27356097     4651277341     170.03
ASM file metadata operation     25098749     8850323     0.35
gc object scan     22705857     7485     0
db file parallel write     19896252     52152606     2.62
latch: ges resource hash list     19336183     427451     0.02
enq: PS - contention     19143961     707455     0.04
PX Deq: Parse Reply     19093356     895799     0.05
gc cr disk read     17816846     448909     0.03
ASM background timer     16101806     1383957874     85.95
PX Deq: Slave Join Frag     16044789     233149     0.01
wait for unread message on broadcast channel     15056320     1413552546     93.88
cursor: mutex X     13435193     24140     0
KJC: Wait for msg sends to complete     13268497     11397     0
PX Deq: Signal ACK RSG     13214824     101941     0.01
KSV master wait     13206286     4235645     0.32
direct path read temp     12617694     5487608     0.43
PX Deq Credit: need buffer     11675868     879967     0.08
row cache lock     11536185     398216     0.03
PX Deq Credit: Session Stats     9480862     78910     0.01
SQL*Net message to dblink     9312894     1538     0
SQL*Net message from dblink     9312894     6279631     0.67
control file parallel write     7760982     11854435     1.53
pmon timer     7558889     1412576090     186.88
PX Deq: Join ACK     7548816     498931     0.07
gc current multi block request     6035173     155898     0.03
PING     5706961     1413230267     247.63
enq: XR - database force logging     4662671     198813     0.04
class slave wait     4561877     7097429006     1555.81
Streams AQ: waiting for messages in the queue     4495828     1543411682     343.3
SQL*Net more data from dblink     3696582     444575     0.12
LGWR wait for redo copy     3655353     17840     0
log file sequential read     3387305     6610414     1.95
Log archive I/O     2990486     276772     0.09
SQL*Net break/reset to client     2971976     2385935     0.8
direct path write temp     2839390     2522114     0.89
Space Manager: slave idle wait     2827526     1412987186     499.73
latch: shared pool     2808517     298150     0.11
latch: gc element     2421717     24688     0.01
SGA: MMAN sleep for component shrink     2336447     2458094     1.05
latch: enqueue hash chains     2279645     15435     0.01
latch free     2089418     78732     0.04
gc current split     2044784     1864009     0.91
PX Deq: Signal ACK EXT     1976164     19263     0.01
enq: FB - contention     1473469     61036     0.04
cursor: pin S wait on X     1313129     1464789     1.12
SQL*Net more data to dblink     1232891     986     0
Streams AQ: RAC qmn coordinator idle wait     1211300     788     0
enq: HW - contention     1175390     2077008     1.77
latch: session allocation     1167768     21883     0.02
Streams AQ: qmn coordinator idle wait     1144699     1412546634     1233.99
Streams AQ: qmn slave idle wait     1031585     2227183681     2158.99
lock deadlock retry     962937     4698     0
enq: CF - contention     956154     609647     0.64
latch: cache buffers lru chain     902764     37552     0.04
latch: object queue header operation     817911     27717     0.03
global enqueue expand wait     768633     654105     0.85
Data file init write     756191     329758     0.44
latch: gcs resource hash     647021     4147     0.01
local write wait     603007     286191     0.47
latch: row cache objects     599358     6453     0.01
ges lmd/lmses to freeze in rcfg - mrcvr     481759     156345     0.32
shared server idle wait     471190     1413238589     2999.3
enq: RF - DG Broker Current File ID     469833     23209     0.05
smon timer     432383     1411851085     3265.28
SGA: allocation forcing component growth     363333     379008     1.04
gc current retry     341104     1121252     3.29
enq: RF - synch: DG Broker metadata     319143     588290     1.84
enq: PG - contention     313659     14830     0.05
enq: TT - contention     260134     11207172     43.08
enq: KO - fast object checkpoint     236745     820808     3.47
dispatcher timer     236637     1413242481     5972.2
direct path write     231382     191008     0.83
cursor: pin S     229011     394     0
Streams AQ: waiting for time management or cleanup tasks     199981     1413148548     7066.41
enq: TX - row lock contention     188846     12454614     65.95
enq: TX - allocate ITL entry     153703     54252     0.35
enq: SQ - contention     141971     70568     0.5
ksdxexeother     141885     56     0
latch: redo allocation     138912     1858     0.01
recovery area: computing applied logs     126415     45925     0.36
gc current block congested     126318     21768     0.17
resmgr:cpu quantum     123074     151384     1.23
jobq slave wait     120678     35574221     294.79
Datapump dump file I/O     90431     9127     0.1
ges inquiry response     89402     4041     0.05
os thread startup     83809     222586     2.66
cr request retry     80062     71896     0.9
PX Deq: Table Q Sample     79665     133402     1.67
gc cr block congested     79026     14792     0.19
gc cr failure     77521     25019     0.32
enq: WF - contention     73983     825388     11.16
enq: TQ - TM contention     72871     3319     0.05
lock escalate retry     65714     1574     0.02
buffer exterminate     59775     64919     1.09
fbar timer     47136     1413183353     29980.98
log file switch completion     46911     452097     9.64
recovery area: computing obsolete files     45699     8547     0.19
enq: US - contention     40401     8805     0.22
enq: TM - contention     39149     5435032     138.83
library cache load lock     36311     382575     10.54
kjbdrmcvtq lmon drm quiesce: ping completion     31668     47443     1.5
enq: TD - KTF dump entries     31468     1424     0.05
enq: RO - fast object reuse     28422     31772     1.12
parallel recovery slave wait for change     27558     3163     0.11
name-service call wait     23694     181533     7.66
control file single write     22375     1624     0.07
kksfbc child completion     21239     106926     5.03
PX Deq: Table Q qref     19325     245     0.01
enq: TX - contention     18805     113253     6.02
latch: messages     17203     181     0.01
enq: RS - prevent file delete     16913     1013     0.06
enq: RS - prevent aging list update     15682     642     0.04
PX Deq: Table Q Get Keys     14322     42935     3
gc current grant congested     14292     2192     0.15
cursor: mutex S     13285     8     0
log file single write     13164     5371     0.41
latch: undo global data     12649     178     0.01
kksfbc research     11894     12680     1.07
parallel recovery slave idle wait     11193     5872     0.52
wait list latch free     11026     11794     1.07
enq: CT - state     11001     417     0.04
latch: checkpoint queue latch     10526     132     0.01
enq: PE - contention     10506     1139     0.11
ARCH wait on SENDREQ     9957     216480     21.74
gc cr grant congested     9465     1584     0.17
wait for scn ack     9377     3155     0.34
enq: TA - contention     8856     324     0.04
log buffer space     8777     89323     10.18
enq: TK - Auto Task Serialization     8542     343     0.04
enq: DR - contention     7842     323     0.04
process diagnostic dump     7707     2072     0.27
JOX Jit Process Sleep     7612     11286431     1482.72
enq: TC - contention     7357     340817     46.33
ges global resource directory to be frozen     7140     12299     1.72
enq: CO - master slave det     6850     312     0.05
enq: JS - job run lock - synchronize     6704     397     0.06
gcs drm freeze in enter server mode     6542     40742     6.23
enq: TS - contention     5959     89332     14.99
ARCH wait for archivelog lock     5600     36     0.01
PX Nsq: PQ load info query     5377     104798     19.49
db file single write     5373     3452     0.64
gc remaster     5315     50625     9.52
latch: parallel query alloc buffer     4939     1906     0.39
enq: TO - contention     4799     143     0.03
enq: AF - task serialization     4395     161     0.04
enq: PI - contention     4251     163     0.04
ges2 LMON to wake up LMD - mrcvr     4210     28     0.01
enq: DL - contention     3889     239     0.06
kjctssqmg: quick message send wait     3408     22     0.01
LNS wait on DETACH     3275     741     0.23
ksfd: async disk IO     3274     1     0
LNS wait on ATTACH     3273     51940     15.87
ARCH wait on DETACH     3231     714     0.22
ARCH wait on ATTACH     3226     43238     13.4
enq: BR - file shrink     2787     116     0.04
write complete waits     2631     1070     0.41
enq: MD - contention     2596     67     0.03
enq: WL - contention     2198     266518     121.25
single-task message     2098     25896     12.34
enq: OD - Serializing DDLs     2054     66     0.03
resmgr:internal state change     2001     14735     7.36
ARCH wait on c/f tx acquire 2     1751     175230     100.07
enq: WR - contention     1636     69     0.04
latch: cache buffer handles     1610     29     0.02
statement suspended, wait error to be cleared     1497     22626     15.11
Streams AQ: qmn coordinator waiting for slave to start     1214     678966     559.28
enq: PD - contention     1182     33     0.03
JS kgl get object wait     1096     4922     4.49
undo segment extension     1070     10065     9.41
PL/SQL lock timer     949     8739819     9209.5
enq: AE - lock     937     28     0.03
LGWR-LNS wait on channel     832     913     1.1
ges DFS hang analysis phase 2 acks     816     495     0.61
latch: redo writing     729     9     0.01
gc quiesce     665     564     0.85
enq: JS - queue lock     482     2111     4.38
PX Deq: Test for credit     442     13     0.03
enq: SS - contention     386     274     0.71
recovery area: computing dropped files     328     1400     4.27
recovery area: computing backed up files     328     496     1.51
ksdxexeotherwait     279     10592     37.97
log switch/archive     250     137570     550.28
gc domain validation     223     39964     179.21
auto-sqltune: wait graph update     195     96514     494.95
wait for a undo record     170     1214     7.14
parallel recovery coord send blocked     168     4     0.02
enq: JS - wdw op     168     3741     22.27
enq: KT - contention     165     5     0.03
switch logfile command     156     6290     40.32
gcs resource directory to be unfrozen     149     12839     86.17
Data Guard Broker Wait     139     10906     78.46
enq: SK - contention     129     4     0.03
enq: JS - job recov lock     128     4     0.03
gc cr block lost     125     6772     54.17
virtual circuit wait     122     3     0.03
ges LMON to get to FTDONE      100     187     1.87
enq: CU - contention     80     242     3.02
enq: JQ - contention     78     7     0.09
cursor: pin X     73     83     1.14
parallel recovery coord wait for reply     70     510     7.29
PX Deq: Txn Recovery Start     67     3436     51.29
SQL*Net break/reset to dblink     60     0     0
gc current block lost     57     2869     50.33
ges LMD suspend for testing event     51     709     13.89
inactive session     46     4550     98.91
recovery read     45     5     0.11
JS kill job wait     41     3548     86.53
enq: AS - service activation     40     1     0.03
enq: TL - contention     35     2     0.05
enq: UL - contention     34     524     15.42
gcs enter server mode     33     1559     47.24
wait for stopper event to be increased     30     218     7.27
enq: TQ - DDL contention     24     300     12.52
enq: MR - contention     21     1     0.03
ges reconfiguration to start     20     54     2.72
ges enter server mode     20     502     25.08
buffer latch     18     1337     74.26
enq: SR - contention     18     1     0.05
Streams: RAC waiting for inter instance ack     18     3748     208.21
enq: PR - contention     17     46     2.72
kupp process wait     16     166     10.39
checkpoint completed     15     3678     245.19
PX Deque wait     14     68     4.87
enq: BF - allocation contention     14     1     0.08
enq: XL - fault extent map     14     51     3.66
enq: FU - contention     14     17     1.18
enq: TH - metric threshold evaluation     13     114     8.78
enq: MW - contention     12     0     0.04
enq: DD - contention     10     0     0.04
process terminate     8     41     5.08
ges cgs registration     8     151     18.9
buffer resize     7     0     0
ktm: instance recovery     7     698     99.66
LNS wait on LGWR     6     0     0
ASM background starting     6     381     63.43
gc cr block 3-way     5     0     0.08
enq: PV - syncstart     5     9     1.74
Global transaction acquire instance locks     4     4     1.09
enq: RS - read alert level     4     0     0.04
LGWR wait on LNS     3     0     0
gc recovery     3     540     179.85
Streams AQ: enqueue blocked on low memory     3     544     181.2
DBWR range invalidation sync     3     17     5.83
enq: DM - contention     3     0     0.03
enq: RF - FSFO Observer Heartbeat     3     0     0.03
enq: JS - q mem clnup lck     3     0     0
DG Broker configuration file I/O     2     0     0
enq: RC - Result Cache: Contention     2     493     246.6
enq: KM - contention     2     0     0.03
enq: RT - contention     2     0     0.04
instance state change     2     0     0.12
kkdlgon     2     10     5.11
enq: TQ - INI contention     2     292     146.07
enq: JS - contention     2     0     0
ARCH wait for netserver start     1     400     400.02
log file switch (checkpoint incomplete)     1     3     3.44
JS coord start wait     1     50     50.09
ges lmd and pmon to attach     1     1     1.26
wait for tmc2 to complete     1     3     3.03
control file heartbeat     1     400     400.02
enq: SW - contention     1     0     0.04
enq: PW - perwarm status in dbw0     1     0     0.09
enq: FS - contention     1     0     0.04
enq: XR - quiesce database     1     0     0.04
enq: RS - write alert level     1     0     0.02
enq: CN - race with init     1     0     0.03
enq: FE - contention     1     4     3.77
Wait for shrink lock2     1     10     10.03
enq: IA - contention     1     0     0.02
enq: RF - atomicity     1     0     0.05
enq: RF - synchronization: aifo master     1     0     0.02
enq: RF - RF - Database Automatic Disable     1     0     0.06
enq: WP - contention     1     0     0.02
enq: TB - SQL Tuning Base Cache Load     1     0     0.05
enq: JS - evt notify     1     0     0.02Edited by: steffi on Mar 20, 2011 12:21 AM
Edited by: steffi on Mar 20, 2011 8:18 AM
Edited by: steffi on Mar 20, 2011 8:19 AM

Text can be formatted by tagging the beginning and end of the block of text with the code ta
\Formatted text goes here.
\Example:
This is formatted.When cutting and pasting text such as execution plans, excerpts from AWR reports, etc, it will maintain spacing and formatting, and make for much easier reading.
As to the content, well, dumping the contents of v$system_event is pretty close to useless.
As to the first three events you listed, 'enq: TX - index contention', 'enq: TX - row lock contention', 'enq: SQ - contention', well, all of those are easily tunable.
First, for 'enq: SQ - contention', check your sequences. Do you have any NOCACHE sequences? Or sequences with small caches?
As for 'enq: TX - row lock contention', well that's fairly self-explanatory. You have multiple sessions trying to lock the same row in the same table at the same time.
Last, 'enq: TX - index contention', that's non-row level contention on an index. For example, if you have a unique index, insert a row w/ column value 1, but don't commit, then try to insert that same value from another session.
But, before you do any of that, I think you need to clearly understand where the bottlenecks are. Try taking some AWR snapshots, about 5 minutes apart, when you're having performance problems. Look at the AWR report for that 5 minute period. In particular, look at your Top 5 timed events.
Hope that helps,
-Mark

Uninstall Oracle 11gr2 RAC database in grid infrastructure

Hi all,
After several attempt to install my Oracle database RAC with grid infrastructure, i want now to do a fresh installation as i have attempted 3 times and now i have all the procedure on installing the database and RAC.
Actually i have installed it correctly but now i want to cleanup my server and remove all oracle installation directory and do a fresh installation.
My question is what is the procedure to uninstall an Oracle RAC database and Clusterware with grid infrastucture and cleanup oracle base installation.
The architecture is:
GRID and clusterware: Oracle grid 11gR2
Database: Oracle database 11gR2
Database and grid storage: ASM
OS: linux centos 6
Thank you.
Raluce.

The deinstallation of Oracle GI could be not so easy thing to do, because it contains many components one should be aware of. The proper deinstall is important because it will safe you from many issues with next install on these servers
In general we need to be sure that:
1. all sowftware stopped properly
2. removed from oraInventory
3. binaries removed
4. /etc/oracle cleared
5. ocr and votes cleared using dd
6. /etc/oratab updated
7. .profile updated
8. init.d files in /etc/ cleard
Usually its recommended to use deconfigure scripts, if they fails for some reason, the manual procedure should be followed.
How to Deconfigure/Reconfigure(Rebuild OCR) or Deinstall Grid Infrastructure [ID 1377349.1]
How to Deinstall Oracle Clusterware Home Manually [ID 1364419.1]
As general recommendation its good idea to save your crs configuration for future reference.
Regards
Ed Rudans
http://erudans.blogspot.com

EBS R12 RAC Clusterware Installation.

Hi hussein/helios
The Oracle support has installed a RAC on our EBS Prod server but it was a POC(proof of concept) only. So the name was TEST RAC instance.
Now, we want to install the final config setup named PROD following the docs he made on the TEST POC. We will also
reuse all the files, directories, and raw devices used in TEST as PROD. Meaning all the traces of files and home dirs of TEST will be renamed to PROD, and change ownerships from oratest to oraprod.
First, we run deinstall of all the TEST install software like the CRS, ASM. We also delete the traces of TEST on inittab and oraInventory.
Then we now installed first the CRS_PROD. But to our surprise the CRS installation failed , it seems that it still
can see the "test" setup :(
Failure at SlfAccess(/u02/oracle/oratest/product/11.1.0/crs) with code -1Note the we already deleted the above directory since it belongs to the old setup, while the existing is:
/u02/oracle/oratprod/product/11.1.0/crs
My question is, how do you completely cleanup or deinstall all the previous traces of CRS?
Thanks a lot,
Ms K

Hi;
Please check below link which could be helps you
http://www.idevelopment.info/data/Oracle/DBA_tips/Oracle10gRAC/CLUSTER_65.shtml
http://www.appsdba.com/blog/?p=155
http://appsfusion.blogspot.com/2007/11/steps-to-create-ocr-and-voting-disk-on.html
How to Restore CRS after accidentally run localconfig on RAC system [ID 747415.1
As i said before its better to move your thread OCR forum site
Regard
Helios

Rac node crash

Hi,
I am using 2 node rac enviroment.I am doing some tests for availability.When i poweroff one node of cluster the other node of cluster reboot itself. The voting disk is online. I think this is caused from network heartbeat of private network. So i wonder when one node of cluster goes down , does the other node have to reboot itself or not ?
Thanks.

Hi,
This is 10gr1 rac.I could not find alertlog.But taken log from ocssd logs from both nodes.I have rebooted node2 and node1 reboot itself.
Node2
CLSS-3001: local node number 2, master node number 1
2012-05-22 08:45:57.247 [773] >TRACE:   clssgmClientConnectMsg: Connect from con(111cdebb0), proc(111ce38b0) pid()
2012-05-22 08:45:57.247 [773] >TRACE:   clssgmClientConnectMsg: Connect from con(111ce1650), proc(111ce3b70) pid()
2012-05-22 08:47:20.865 [773] >TRACE:   clsc_receive: (111ce0e10) Connection failed, transport error (507, 0, 0)
2012-05-22 08:47:20.865 [773] >TRACE:   clscreceive: (111ce1650) Physical connection (111ce0e10) not active, rc 11
2012-05-22 08:47:20.865 [773] >TRACE:   clscreceive: (111ce3fb0) Physical connection (111ce0e10) not active, rc 11
2012-05-22 08:47:20.865 [773] >TRACE:   clssgmDeleteClientListener: cleanup for proc(111ce3b70) con(111ce1650) pid()
2012-05-22 08:47:20.879 [1287] >TRACE:   clscsendx: (111ce3fb0) Connection not active
2012-05-22 08:47:20.934 [773] >TRACE:   clsc_receive: (111cde370) Connection failed, transport error (507, 0, 0)
2012-05-22 08:47:20.934 [773] >TRACE:   clscreceive: (111cdebb0) Physical connection (111cde370) not active, rc 11
2012-05-22 08:47:20.934 [773] >TRACE:   clssgmDeleteClientListener: cleanup for proc(111ce38b0) con(111cdebb0) pid()
2012-05-22 08:47:51.100 [773] >TRACE:   clssgmClientConnectMsg: Connect from con(111cdebb0), proc(111ce0e10) pid()
Node1
CLSS-3001: local node number 1, master node number 1
2012-05-22 08:47:51.112 [516] >TRACE:   clsc_receive: (111ce6c50) Connection failed, transport error (507, 0, 0)
2012-05-22 08:47:51.112 [516] >WARNING: clssnmeventhndlr: Receive failure with node 2, rc=11
2012-05-22 08:47:51.112 [1287] >TRACE:   clsc_receive: (111bd8ab0) Connection failed, transport error (507, 0, 0)
2012-05-22 08:47:51.157 [1544] >WARNING: clssnmPollingThread: Eviction started for node 2, flags 0x0001, state 3, wt4c 0
2012-05-22 08:47:56.157 [1544] >TRACE:   clssnmDoSyncUpdate: Initiating sync 3
2012-05-22 08:47:56.157 [516] >TRACE:   clssnmHandleSync: Acknowledging sync: src[1] seq[9] sync[3]
2012-05-22 08:47:56.203 [1] >USER:    NMEVENT_SUSPEND [00][00][00][02]
2012-05-22 08:48:00.165 [1544] >TRACE:   clssnmEvict: Evicting node 2, birth 2, death 0, killme 1
2012-05-22 08:48:00.165 [1544] >TRACE:   clssnmWaitOnEvictions: Waiting for node 2 to die, missed HB 0 of 22503
2012-05-22 08:48:01.165 [1544] >TRACE:   clssnmWaitOnEvictions: Waiting for node 2 to die, missed HB 1 of 22503
2012-05-22 08:48:02.166 [1544] >TRACE:   clssnmWaitOnEvictions: Waiting for node 2 to die, missed HB 2 of 22503While node2 rebooting node1 has begun to reboot . Below log is OS log from node1
LABEL:          CORE_DUMP
IDENTIFIER:     C69F5C9B
Date/Time:       Tue May 22 08:48:05 2012
Sequence Number: 26098
Machine Id:      00C2F7704C00
Node Id:         kopstest1
Class:           S
Type:            PERM
Resource Name:   SYSPROC
Description
SOFTWARE PROGRAM ABNORMALLY TERMINATED
Probable Causes
SOFTWARE PROGRAM
User Causes
USER GENERATED SIGNAL
        Recommended Actions
        CORRECT THEN RETRY
Failure Causes
SOFTWARE PROGRAM
        Recommended Actions
        RERUN THE APPLICATION PROGRAM
        IF PROBLEM PERSISTS THEN DO THE FOLLOWING
        CONTACT APPROPRIATE SERVICE REPRESENTATIVE
Detail Data
SIGNAL NUMBER
           6
USER'S PROCESS ID:
                909402
FILE SYSTEM SERIAL NUMBER
          12
INODE NUMBER
      109027
CORE FILE NAME
/oracle/product/10.1.0/crs/css/init/core
PROGRAM NAME
ocssd.bin
STACK EXECUTION DISABLED
           0
COME FROM ADDRESS REGISTER
PROCESSOR ID
hw_fru_id: N/A
hw_cpu_id: N/A
ADDITIONAL INFORMATION
pthread_k 88
Symptom Data
REPORTABLE
1
INTERNAL ERROR
0
SYMPTOM CODE
PCSS/SPI2 FLDS/ocssd.bin SIG/6 FLDS/pthread_k VALU/88Thanks.
Edited by: ecer on May 21, 2012 11:11 PM

OVM 3.0.3 unable to Import RAC Template

This could be just a minor issue, but I need assistance to troubleshoot.
A) Problem: Server error message: Template import error: [Errno 14] curl#7 - "couldn't connect to host"
B) Environment:
OVS 3.0.3 fresh install
- hostname OVServer & ip address = 192.168.1.222
- Downloaded RAC templates: OVM_EL5U5_X86_64_11201RAC_PVM-1of2.tgz & OVM_EL5U5_X86_64_11201RAC_PVM-2of2.tgz
- unzipped templates in /OVS/seed_pool
- Storage: 2 Hard drives: 1 TB & 80 GB - The 80GB is used for the Repositories at this time.
- I have created 5 Virtual Disks disks as per RAC11201-OVM-template-TestConfiguration.pdf
OVM 3.0.3 fresh install
- hostname: OVManager & ip address = 192.168.1.223
Doubts: Wile trying to import the Template, I was not sure of what to enter for "Template URL". I tied different entries but they all failed including:
https://192.168.1.222//OVS/seed_pool/OVM_EL5U5_X86_64_11201RAC_PVM-1of2.tgz
https://192.168.1.222//OVS/seed_pool/OVM_EL5U5_X86_64_11201RAC_PVM-2of2.tgz
where 192.168.1.222 is the ip address of the OVS.
C) Error details:
Job Construction Phase
begin()
Appended operation 'Import Virtual Machine Template' to object 'cfgFile_0004fb0000140000d751156e96401489'.
Appended operation 'Virtual Machine Cfg File Configure' to object 'cfgFile_0004fb0000140000d751156e96401489'.
commit()
Completed Step: COMMIT
Objects and Operations
Object (CREATED): [VirtualMachineCfgFile] cfgFile_0004fb0000140000d751156e96401489
Operation: Import Virtual Machine Template
Operation: Virtual Machine Cfg File Configure
Object (CREATED): [VirtualMachineTemplate] 0004fb0000140000d751156e96401489 (OVM_EL5U5_X86_64_11201RAC_PVM-1of2.tgz)
Job Running Phase at 00:41 on Wed, Jan 4, 2012
Job Participants: [53:4d:43:49:00:02:2b:90:25:00:2b:90:25:00:f1:d7 (OVServer)]
Actioner
Starting operation 'Import Virtual Machine Template' on object 'cfgFile_0004fb0000140000d751156e96401489'
Job Internal Error (Operation)com.oracle.ovm.mgr.api.exception.JobFailureException: OVMAPI_5001E Job "1325655693252" Failed due to failure "(01/04/2012 12:41:35:964 AM)
Async operation failed on server: OVServer.
Object: cfgFile_0004fb0000140000d751156e96401489,
PID: 2211,
Server error message: Template import error: [Errno 14] curl#7 - "couldn't connect to host"
" on server "53:4d:43:49:00:02:2b:90:25:00:2b:90:25:00:f1:d7" associated with object "cfgFile_0004fb0000140000d751156e96401489"
Wed Jan 04 00:41:36 EST 2012
at com.oracle.ovm.mgr.api.job.JobEngine.checkJobStatus(JobEngine.java:602)
at com.oracle.ovm.mgr.api.job.JobEngine.objectAsyncStart(JobEngine.java:580)
at com.oracle.ovm.mgr.api.job.JobEngine.objectAsyncStart(JobEngine.java:564)
at com.oracle.ovm.mgr.op.virtual.VirtualMachineTemplateImport.action(VirtualMachineTemplateImport.java:68)
at com.oracle.ovm.mgr.api.job.JobEngine.operationActioner(JobEngine.java:193)
at com.oracle.ovm.mgr.api.job.JobEngine.objectActioner(JobEngine.java:264)
at com.oracle.ovm.mgr.api.job.InternalJobDbImpl.objectCommitter(InternalJobDbImpl.java:1090)
at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:247)
at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:207)
at com.oracle.odof.core.BasicWork.invokeMethod(BasicWork.java:136)
at com.oracle.odof.command.InvokeMethodCommand.process(InvokeMethodCommand.java:100)
at com.oracle.odof.core.BasicWork.processCommand(BasicWork.java:81)
at com.oracle.odof.core.TransactionManager.processCommand(TransactionManager.java:751)
at com.oracle.odof.core.WorkflowManager.processCommand(WorkflowManager.java:401)
at com.oracle.odof.core.WorkflowManager.processWork(WorkflowManager.java:459)
at com.oracle.odof.io.AbstractClient.run(AbstractClient.java:42)
at java.lang.Thread.run(Thread.java:662)
FailedOperationCleanup
Starting failed operation 'Import Virtual Machine Template' cleanup on object 'cfgFile_0004fb0000140000d751156e96401489'
Complete rollback operation 'Import Virtual Machine Template' completed with direction=cfgFile_0004fb0000140000d751156e96401489
Rollbacker
Objects To Be Rolled Back
Object (CREATED): [VirtualMachineCfgFile] cfgFile_0004fb0000140000d751156e96401489
Object (CREATED): [VirtualMachineTemplate] 0004fb0000140000d751156e96401489 (OVM_EL5U5_X86_64_11201RAC_PVM-1of2.tgz)
Write Methods Invoked
Class=InternalJobDbImpl vessel_id=3428 method=addTransactionIdentifier accessLevel=6
Class=VirtualMachineTemplateDbImpl vessel_id=3446 method=setName accessLevel=6
Class=VirtualMachineTemplateDbImpl vessel_id=3446 method=setFoundryContext accessLevel=6
Class=VirtualMachineTemplateDbImpl vessel_id=3446 method=onPersistableCreate accessLevel=6
Class=VirtualMachineTemplateDbImpl vessel_id=3446 method=setLifecycleState accessLevel=6
Class=VirtualMachineTemplateDbImpl vessel_id=3446 method=setRollbackLifecycleState accessLevel=6
Class=VirtualMachineCfgFileDbImpl vessel_id=3451 method=setName accessLevel=6
Class=VirtualMachineCfgFileDbImpl vessel_id=3451 method=setFoundryContext accessLevel=6
Class=VirtualMachineCfgFileDbImpl vessel_id=3451 method=onPersistableCreate accessLevel=6
Class=VirtualMachineCfgFileDbImpl vessel_id=3451 method=setLifecycleState accessLevel=6
Class=VirtualMachineCfgFileDbImpl vessel_id=3451 method=setRollbackLifecycleState accessLevel=6
Class=VirtualMachineCfgFileDbImpl vessel_id=3451 method=setJobAsset accessLevel=6
Class=VirtualMachineTemplateDbImpl vessel_id=3446 method=setVirtualMachineCfgFile accessLevel=6
Class=VirtualMachineCfgFileDbImpl vessel_id=3451 method=setVirtualMachine accessLevel=6
Class=VirtualMachineCfgFileDbImpl vessel_id=3451 method=importVirtualMachineTemplate accessLevel=6
Class=VirtualMachineTemplateDbImpl vessel_id=3446 method=setSimpleName accessLevel=6
Class=VirtualMachineCfgFileDbImpl vessel_id=3451 method=setAsset accessLevel=6
Class=VirtualMachineCfgFileDbImpl vessel_id=3451 method=configure accessLevel=6
Class=VirtualMachineTemplateDbImpl vessel_id=3446 method=setOrigin accessLevel=6
Class=VirtualMachineTemplateDbImpl vessel_id=3446 method=setDescription accessLevel=6
Class=VirtualMachineCfgFileDbImpl vessel_id=3451 method=setAsset accessLevel=6
Class=VirtualMachineCfgFileDbImpl vessel_id=3451 method=configure accessLevel=6
Class=InternalJobDbImpl vessel_id=3428 method=setCompletedStep accessLevel=6
Class=InternalJobDbImpl vessel_id=3428 method=setAssociatedHandles accessLevel=6
Class=InternalJobDbImpl vessel_id=3428 method=setFailedOperation accessLevel=6
Class=VirtualMachineCfgFileDbImpl vessel_id=3451 method=nextJobOperation accessLevel=6
Class=VirtualMachineTemplateDbImpl vessel_id=3446 method=nextJobOperation accessLevel=6
Completed Step: ROLLBACK
Job Failed Commit due to OVMAPI_5001E Job "1325655693252" Failed due to failure "(01/04/2012 12:41:35:964 AM)
Async operation failed on server: OVServer.
Object: cfgFile_0004fb0000140000d751156e96401489,
PID: 2211,
Server error message: Template import error: [Errno 14] curl#7 - "couldn't connect to host"
" on server "53:4d:43:49:00:02:2b:90:25:00:2b:90:25:00:f1:d7" associated with object "cfgFile_0004fb0000140000d751156e96401489"
Wed Jan 04 00:41:36 EST 2012
com.oracle.ovm.mgr.api.exception.JobFailureException: OVMAPI_5001E Job "1325655693252" Failed due to failure "(01/04/2012 12:41:35:964 AM)
Async operation failed on server: OVServer.
Object: cfgFile_0004fb0000140000d751156e96401489,
PID: 2211,
Server error message: Template import error: [Errno 14] curl#7 - "couldn't connect to host"
" on server "53:4d:43:49:00:02:2b:90:25:00:2b:90:25:00:f1:d7" associated with object "cfgFile_0004fb0000140000d751156e96401489"
Wed Jan 04 00:41:36 EST 2012
at com.oracle.ovm.mgr.api.job.JobEngine.checkJobStatus(JobEngine.java:602)
at com.oracle.ovm.mgr.api.job.JobEngine.objectAsyncStart(JobEngine.java:580)
at com.oracle.ovm.mgr.api.job.JobEngine.objectAsyncStart(JobEngine.java:564)
at com.oracle.ovm.mgr.op.virtual.VirtualMachineTemplateImport.action(VirtualMachineTemplateImport.java:68)
at com.oracle.ovm.mgr.api.job.JobEngine.operationActioner(JobEngine.java:193)
at com.oracle.ovm.mgr.api.job.JobEngine.objectActioner(JobEngine.java:264)
at com.oracle.ovm.mgr.api.job.InternalJobDbImpl.objectCommitter(InternalJobDbImpl.java:1090)
at sun.reflect.GeneratedMethodAccessor1097.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:247)
at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:207)
at com.oracle.odof.core.BasicWork.invokeMethod(BasicWork.java:136)
at com.oracle.odof.command.InvokeMethodCommand.process(InvokeMethodCommand.java:100)
at com.oracle.odof.core.BasicWork.processCommand(BasicWork.java:81)
at com.oracle.odof.core.TransactionManager.processCommand(TransactionManager.java:751)
at com.oracle.odof.core.WorkflowManager.processCommand(WorkflowManager.java:401)
at com.oracle.odof.core.WorkflowManager.processWork(WorkflowManager.java:459)
at com.oracle.odof.io.AbstractClient.run(AbstractClient.java:42)
at java.lang.Thread.run(Thread.java:662)
End of Job
Why can't I connect to the host - "couldn't connect to host"?
Is my URL wrong?
In addition to the above, Is the location of my templates correct?
Thanks

Thanks. I didn't see that fact in the docs.
I moved the templates into my VM Manager box (192.168.1.223) into a new folder /ora_templates which has chmod 777.
192.168.1.223 is a Virtual Machine.
I tried testing the URL http://192.168.1.223/ora_templates in my windows browser and it failed with "HTTP Error 403" & "The server has denied your browser request"
I tried the Template import again using the following url and it failed again with similar errors:
http://192.168.1.223/ora_templates/OVM_EL5U5_X86_64_11201RAC_PVM-1of2.tgz
http://192.168.1.223/ora_templates/OVM_EL5U5_X86_64_11201RAC_PVM-2of2.tgz
Error Details:
Job Construction Phase
begin()
Appended operation 'Import Virtual Machine Template' to object 'cfgFile_0004fb00001400007ce6220db5c76163'.
Appended operation 'Virtual Machine Cfg File Configure' to object 'cfgFile_0004fb00001400007ce6220db5c76163'.
commit()
Completed Step: COMMIT
Objects and Operations
Object (CREATED): [VirtualMachineCfgFile] cfgFile_0004fb00001400007ce6220db5c76163
Operation: Import Virtual Machine Template
Operation: Virtual Machine Cfg File Configure
Object (CREATED): [VirtualMachineTemplate] 0004fb00001400007ce6220db5c76163 (OVM_EL5U5_X86_64_11201RAC_PVM-1of2.tgz)
Job Running Phase at 00:02 on Thu, Jan 5, 2012
Job Participants: [53:4d:43:49:00:02:2b:90:25:00:2b:90:25:00:f1:d7 (OVServer)]
Actioner
Starting operation 'Import Virtual Machine Template' on object 'cfgFile_0004fb00001400007ce6220db5c76163'
Job Internal Error (Operation)com.oracle.ovm.mgr.api.exception.JobFailureException: OVMAPI_5001E Job "1325739720365" Failed due to failure "(01/05/2012 12:02:04:344 AM)
Async operation failed on server: OVServer.
Object: cfgFile_0004fb00001400007ce6220db5c76163,
PID: 15086,
Server error message: Template import error: [Errno 14] curl#7 - "couldn't connect to host"
" on server "53:4d:43:49:00:02:2b:90:25:00:2b:90:25:00:f1:d7" associated with object "cfgFile_0004fb00001400007ce6220db5c76163"
Thu Jan 05 00:02:04 EST 2012
at com.oracle.ovm.mgr.api.job.JobEngine.checkJobStatus(JobEngine.java:602)
at com.oracle.ovm.mgr.api.job.JobEngine.objectAsyncStart(JobEngine.java:580)
at com.oracle.ovm.mgr.api.job.JobEngine.objectAsyncStart(JobEngine.java:564)
at com.oracle.ovm.mgr.op.virtual.VirtualMachineTemplateImport.action(VirtualMachineTemplateImport.java:68)
at com.oracle.ovm.mgr.api.job.JobEngine.operationActioner(JobEngine.java:193)
at com.oracle.ovm.mgr.api.job.JobEngine.objectActioner(JobEngine.java:264)
at com.oracle.ovm.mgr.api.job.InternalJobDbImpl.objectCommitter(InternalJobDbImpl.java:1090)
at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:247)
at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:207)
at com.oracle.odof.core.BasicWork.invokeMethod(BasicWork.java:136)
at com.oracle.odof.command.InvokeMethodCommand.process(InvokeMethodCommand.java:100)
at com.oracle.odof.core.BasicWork.processCommand(BasicWork.java:81)
at com.oracle.odof.core.TransactionManager.processCommand(TransactionManager.java:751)
at com.oracle.odof.core.WorkflowManager.processCommand(WorkflowManager.java:401)
at com.oracle.odof.core.WorkflowManager.processWork(WorkflowManager.java:459)
at com.oracle.odof.io.AbstractClient.run(AbstractClient.java:42)
at java.lang.Thread.run(Thread.java:662)
Questions:
1. How do I test access to these files from a browse?
2. If I have to ftp, what will the entry look like.
3. ftp doesn't seem to be installed on the OVM Server - how do I install it or activate if it there?
4. What is the solution to my problem above?
Trying to get these templates working has been pretty time consuming much more than described and expected.
Thanks for your help.
MAC

Steps to remove entrie Oracle 10gR2 (10.2.0.4) RAC on linux

Hi Gurus,
We have 10gR2 RAC (10.2.0.4) is running on RHEL5.5 on development servers. I would like to set up 11gR2(11.2.0.3) RAC on these development servers. Could you please let me know the steps to cleanup existing 10g RAC on these 2 node RAC.
My existing 10g RAC details are as follows
No of nodes 2
<RDBMS_HOME> for oracle binaries
<ASM_HOME> for AMS binaries
<CLUSTER_HOME> for clusterware home
ASM for storage
Thanks
Masoom

A simple Google search turns up this documentation source: [http://download.oracle.com/docs/cd/B28359_01/rac.111/b28255/votocr.htm#BGBGDHFG|http://download.oracle.com/docs/cd/B28359_01/rac.111/b28255/votocr.htm#BGBGDHFG]
You have to stop the node applications and the CRS stack, then use SRVCTL to change the public IP addresses and OIFCFG to change the private ones. The doc provides detailed steps.

Non RAC to RAC

Hi,
I need to convert a Non-rac to RAC. Can somebody help me with note or doc for doing this ?
Thx

1. Set the init parameter cluster_database=false (or remove it since false is the default). You'll have to set it in the parameter file since it is not dynamic.
2. Shutdown all instances. Start the one that you want to survive.
3. Cleanup as necessary: remove extra UNDO, REDO files
4. If you want to remove clusterware, that's another whole question. There's a ML article on how to remove clusterware.
5. If your database uses ASM, you'll need to do #1 for the ASM instances as well.
That's high-level, but I haven't seen a complete paper/procedure for this published anywhere yet.

Cleanup RAC DB

Similar Messages

Maybe you are looking for