Oracle RAC instance related

We have Oracle Databases 11g Enterprise Edition Release 11.2.0.1.0 - 64bit Production running on Linux x86 64-bit
It is a two instance RAC running on servers, let's say - node1 and node2 . We are using ASM
Node 1 has an ASM instance ASM1 and Node 2 has an ASM instance ASM2.
There are 3 11g rdbms databases running on these nodes.
Instances db11,db21,db31 are running on node 1 and corresponding RAC instances db12,db22,db32 are running on node 2
The listeners are configured exactly the same on both nodes.
On Node 2, when I do
[oracle@node2 admin]$ lsnrctl status
LSNRCTL for Linux: Version 11.2.0.1.0 - Production on 19-NOV-2010 14:34:34
Copyright (c) 1991, 2009, Oracle. All rights reserved.
Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=IPC)(KEY=LISTENER)))
STATUS of the LISTENER
Alias                     LISTENER
Version                   TNSLSNR for Linux: Version 11.2.0.1.0 - Production
Start Date                15-NOV-2010 13:33:49
Uptime                    4 days 1 hr. 0 min. 44 sec
Trace Level               off
Security                  ON: Local OS Authentication
SNMP                      OFF
Listener Parameter File   /opt/oracle/product/11.2.0/grid/network/admin/listener.ora
Listener Log File         /opt/app/oracle/diag/tnslsnr/node2/listener/alert/log.xml
Listening Endpoints Summary...
(DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=LISTENER)))
(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=10.10.7.42)(PORT=1521)))
(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=10.10.7.52)(PORT=1521)))
Services Summary...
Service "+ASM" has 1 instance(s).
Instance "+ASM2", status READY, has 1 handler(s) for this service...
Service "db1" has 2 instance(s).
Instance "db11", status READY, has 1 handler(s) for this service...
Instance "db12", status READY, has 2 handler(s) for this service...
Service "db2" has 2 instance(s).
Instance "db21", status READY, has 1 handler(s) for this service...
Instance "db22", status READY, has 2 handler(s) for this service...
Service "db3" has 2 instance(s).
Instance "db31", status READY, has 1 handler(s) for this service...
Instance "db32", status READY, has 2 handler(s) for this service...
The command completed successfullyThe above looks good which is what should be the case.
Now, if I try doing the same on node 1 (and this is where I am concerned)
[oracle@node1 admin]$ lsnrctl status
LSNRCTL for Linux: Version 11.2.0.1.0 - Production on 19-NOV-2010 14:41:45
Copyright (c) 1991, 2009, Oracle. All rights reserved.
Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=IPC)(KEY=LISTENER)))
STATUS of the LISTENER
Alias                     LISTENER
Version                   TNSLSNR for Linux: Version 11.2.0.1.0 - Production
Start Date                19-NOV-2010 03:20:44
Uptime                    0 days 11 hr. 21 min. 1 sec
Trace Level               off
Security                  ON: Local OS Authentication
SNMP                      OFF
Listener Parameter File   /opt/oracle/product/11.2.0/grid/network/admin/listener.ora
Listener Log File         /opt/app/oracle/diag/tnslsnr/node1/listener/alert/log.xml
Listening Endpoints Summary...
(DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=LISTENER)))
(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=10.10.7.41)(PORT=1521)))
(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=10.10.7.51)(PORT=1521)))
Services Summary...
Service "+ASM" has 1 instance(s).
Instance "+ASM1", status READY, has 1 handler(s) for this service...
Service "db1" has 1 instance(s).
Instance "db11", status READY, has 2 handler(s) for this service...
Service "db2" has 1 instance(s).
Instance "db21", status READY, has 2 handler(s) for this service...
Service "db3" has 1 instance(s).
Instance "db31", status READY, has 2 handler(s) for this service...
The command completed successfullyThe node 1 does not seem to report the fact that each of these 3 databases have 2 instances and also does not list its other instances besides the ones running on it. Any ideas or suggestions as to where to look?

One problem is here
The listeners are configured exactly the same on both nodes.This is incorrect, as the listeners need to have different names.
They need to have different names as listener_node1 is the remote_listener for node 2 and vice versa.
The correct set up is:
The listener name is node dependent.
The listener definition
listener_<node>=(host=)(protocol=)(port=)
is included in tnsnames.ora
the remote_listener parameter is set to the listener of the other node.
Using hardcoded IPs in listener.ora and tnsnames.ora is a bad idea.
Not sure why you don't ask this question in the RAC forum.
Sybrand Bakker
Senior Oracle DBA

Similar Messages

Oracle RAC 10g and drivers

Hello,
I'm configuring a WLP cluster to use a Oracle 10g RAC instance. [url http://e-generation.beasys.com/wlp/docs92/db/oracle.html]This document seems like a very good starting point. My question is: the document only specifies using Oracle Thin drivers when creating datasource for Oracle RAC. Is there any issue in using the Bea Oracle jdbc drivers to connect to an Oracle Rac instance?
Thanks
Luciano

Luciano Fiandesio wrote:
Hello,
I'm configuring a WLP cluster to use a Oracle 10g RAC instance.
[url http://e-generation.beasys.com/wlp/docs92/db/oracle.html]
This document seems like a very good starting point. Myquestion is: the document only specifies using Oracle Thin
drivers when creating datasource for Oracle RAC. Is there any
issue in using the Bea Oracle jdbc drivers to connect to an
Oracle Rac instance?
Thanks
Luciano
Theoretically, there should be zero problem using the BEA-branded
Oracle driver to talk to RAC. Our recommended configuration for
MultiPools, and RAC use the driver in only the simplest way.
However, for complete and least-confusing support, you should
use the thin driver for RAC. This is because the RAC testing is so
involved that we chose only to test one driver, the thin driver,
so WLP is only certified with the thin driver. However, I would
expect the BEA driver to work as well or better.
Joe

Oracle RAC interfaces

HI,
While ORACLE 10g R2 CRS Installation , why should we give public interface information.
I understand we give private interface information because it should be used for Inter instance communication.
But why do we need to specify public interface.Is this poublic interface gonna be used for Cache fusion at all?
Thanks
Pramod

When one server goes down, the other server will take up both virtual IPs on the public interface, ensuring there is no delay in failover.
The interface type indicates the purpose for which the network is configured. The supported interface types are:
Public—An interface that can be used for communication with components external to Oracle RAC instances, such as Oracle Net and Virtual Internet Protocol (VIP) addresses.
Cluster_interconnect—A private interface used for the cluster interconnect to provide interinstance or Cache Fusion communication.
If you set the interface type to cluster_interconnect, it affects instances as they start up and changes do not take effect until you restart the instances.

ASM instances on 2 node Oracle RAC 10g r2 on Red Hat 4 u1

Hi all
I'm experiencing a problem in configuring diskgroups under +ASM instances on a two node Oracle RAC.
I followed the official guide and also official documents from metalink site, but i'm stuck with the visibility of asm disks.
I created fake disks on nfs with Netapp certified storage binding them to block device with the usual trick "losetup /dev/loopX /nfs/disk1 " ,
run "oracleasm createdisk DISKX /dev/loopX" on one node and
"oracleasm scandisks" on the other one.
With "oracleasm listdisks" i can see the disks at OS level in both nodes , but , when i try to create and mount diskgroup in the ASM instances , on the instance on which i create the diskgroup all is well, but the other one doesn't see the disks at all, and diskgroup mount fails with :
ERROR: no PST quorum in group 1: required 2, found 0
Tue Sep 20 16:22:32 2005
NOTE: cache dismounting group 1/0x6F88595E (DG1)
NOTE: dbwr not being msg'd to dismount
ERROR: diskgroup DG1 was not mounted
any help would be appreciated
thanks a lot.
Antonello

I'm having this same problem. Did you ever find a solution?

Max Connections to Oracle 10g RAC Instance

Hi,
We have IBM Websphere Application Server on one unix server which is connecting to a Oracle 10g RAC Instance which is on a different unix server through JDBC connection. They(Application Team) are doing stress tests, please tell me how many connections can a Oracle Database Allow simultaneously. They tested for 2000 connections simultaneously. It failed...This is urgent, they are on call please reply ASAP
Thanks,

Use Oracle Shared Server (the default is Dedicated Server),
I've shown JBOSS app developers how a little Dell desktop (256MB RAM, 1.7Ghz P4) can accept a 1000 app server connections (using Shared Server). While a big HP-UX server failed to accept 200 connections (using Dedicated Server).
Oracle RAC should easily be able to handle 100+ concurrent client OLTP connections per RAC node using Shared Server - assuming that each of the concurrent sessions are actually doing stuff and not just sitting there idle. (if 99.99% of the connections are idle, then even a little desktop Oracle database can handle well over a 1000)
Bottomline - you must have your primary focus and expertise on Oracle. On the the data tier. Not the app tier. The data tier is what scales the app tier. Not vice versa. Adding an extra app tier server does not enable the data tier platform to suddenly be able to run more SQLs, The app tier does not scale the data tier.
If you want performance and scalability, you must get the data tier right. App tier be damned as that is neither here nor there ito performance and scalability. This means Oracle expertise is required first and foremost.

Oracle single instance rac installation

hi,
do I need minimum of 2 servers in order to carry out 11.2 oracle single instance RAC installation ? or that its possible with one node? and then can add more nodes in future?
is there any article/documents on this?
thanks

http://download.oracle.com/docs/cd/E11882_01/rac.112/e10718/clonerac.htm#CHDGFHCA
"The cloning process assumes that you successfully installed an Oracle Clusterware home and an Oracle home with Oracle RAC on at least one node. In addition, all root scripts must have run successfully on the node from which you are extending your cluster database."
also
http://download.oracle.com/docs/cd/E11882_01/rac.112/e10717/intro.htm#BEIHEJGD
"Overview of Cloning and Extending Oracle Clusterware in Grid Environments
Cloning nodes is the preferred method of creating new clusters. The cloning process copies Oracle Clusterware software images to other nodes that have similar hardware and software. Use cloning to quickly create several clusters of the same configuration. Before using cloning, you must install an Oracle Clusterware home successfully on at least one node using the instructions in your platform-specific Oracle Clusterware installation guide.
For new installations, or if you must install on only one cluster, Oracle recommends that you use the automated and interactive installation methods, such as Oracle Universal Installer or the Provisioning Pack feature of Oracle Enterprise Manager. These methods perform installation checks to ensure a successful installation. To add or delete Oracle Clusterware from nodes in the cluster, use the addNode.sh and rootcrs.pl scripts."
it doesnt directly say its possible and I cant find full documentation on how to go about it ... but seem like its possible ?
Edited by: user9198889 on Feb 24, 2010 5:01 PM

Help required of creating Oracle RAC database having multiple instances

Hello Guys!
I want to create one database having 2 instances on Oracle 11g R1
I have 2 nodes in this Oracle RAC
Database name will be Val
and instance name on node 1 will be val1 and instance name on node 2 will be val2
Raw storage will be used, that is, without using automatic storage management or an Oracle Cluster File System
Can anyone help how to do that ?
Thanks

Hi,
Wonderful example thanks for the link.You're welcome :)
I'm little confused which option to choose in shared storage option of DBCA
i.e. Cluster File System or Raw devices?I suggest you to use ASM for you database storage requirement and select raw devices. oracle recommends to use ASM storage for sahred database.
thanks,
X A H E E R

Does Oracle RAC improve performance over single-instance db

Hallo!i am running a single instance db that seems to be overwhelmed by workload as CPU is constantly 100% utilized despite code and instance tuning and users persistently complain of system slowness.
It is our plan to uprgade to more powerful servers but would also like to implement load balancing to have 2 instances and 2 servers share workload via RAC solution.
Some of peers state that RAC mostly a High Availability solution to enable continuity should one instance fail.I would like to know if db performance would improve on RAC as load balancing between 2 or more instances occurs in a RAC setup.
Thanks.

4joey1 wrote:
Some of peers state that RAC mostly a High Availability solution to enable continuity should one instance fail.I would like to know if db performance would improve on RAC as load balancing between 2 or more instances occurs in a RAC setup.RAC also provides scalability - as you have additional servers and database instances to deal with the workload. It is not a mere high availability and redundancy architecture.
BTW, a 2 node RAC is IMO not a "real" RAC... We also have one of those and I regret not insisting that it be, at minimum, a 4 node RAC - from both a redundancy/availability and performance/scalability viewpoints.
Simple example - on our bigger RAC we are dealing with about 37000+ row inserts every second (3+ billion rows per day). This workload is only handled (on smallish dual 2-core CPU servers) due to RAC. No way that a single one of these servers would alone (as a single non-RAC instance) be capable to deal with that workload.
So yeah - RAC is most definitely also about performance and scalability too. (but that does not mean it's a magic wand for solving your performance problems either).

Encountered ora-29701 during Sun Cluster for Oracle RAC 9.2.0.7 startup (UR

Hi all,
Need some help from all out there
In our Sun Cluster 3.1 Data Service for Oracle RAC 9.2.0.7 (Solaris 9) configuration, my team had encountered
ora-29701 *Unable to connect to Cluster Manager*
during the startup of the Oracle RAC database instances on the Oracle RAC Server resources.
We tried the attached workaround by Oracle. This workaround works well for the 1^st time but it doesnt work anymore when the server is rebooted.
Kindly help me to check whether anyone encounter the same problem as the above and able to resolve. Thanks.
Bug No. 4262155
Filed 25-MAR-2005 Updated 11-APR-2005
Product Oracle Server - Enterprise Edition Product Version 9.2.0.6.0
Platform Linux x86
Platform Version 2.4.21-9.0.1
Database Version 9.2.0.6.0
Affects Platforms Port-Specific
Severity Severe Loss of Service
Status Not a Bug. To Filer
Base Bug N/A
Fixed in Product Version No Data
Problem statement:
ORA-29701 DURING DATABASE CREATION AFTER APPLYING 9.2.0.6 PATCHSET
*** 03/25/05 07:32 am ***
TAR:
PROBLEM:
Customer applied 9.2.0.6 patchset over 9.2.0.4 patchset.
While creating the database, customer receives following error:
ORA-29701: unable to connect to Cluster Manager
However, if customer goes from 9.2.0.4 -> 9.2.0.5 -> 9.2.0.6, the problem does not occur.
DIAGNOSTIC ANALYSIS:
It seems that the problem is with libskgxn9.so shared library.
For 9.2.0.4 -> 9.2.0.5 -> 9.2.0.6, the install log shows the following:
installActions2005-03-22_03-44-42PM.log:,
[libskgxn9.so->%ORACLE_HOME%/lib/libskgxn9.so 7933 plats=1=>[46]langs=1=> en,fr,ar,bn,pt_BR,bg,fr_CA,ca,hr,cs,da,nl,ar_EG,en_GB,et,fi,de,el,iw,hu,is,in, it,ja,ko,es,lv,lt,ms,es_MX,no,pl,pt,ro,ru,zh_CN,sk,sl,es_ES,sv,th,zh_TW, tr,uk,vi]]
installActions2005-03-22_04-13-03PM.log:, [libcmdll.so ->%ORACLE_HOME%/lib/libskgxn9.so 64274 plats=1=>[46] langs=-554696704=>[en]]
For 9.2.0.4 -> 9.2.0.6, install log shows:
installActions2005-03-22_04-13-03PM.log:, [libcmdll.so ->%ORACLE_HOME%/lib/libskgxn9.so 64274 plats=1=>[46] langs=-554696704=>[en]] does not exist.
This means that while patching from 9.2.0.4 -> 9.2.0.5, Installer copies the libcmdll.so library into libskgxn9.so, while patching from 9.2.0.4 -> 9.2.0.6 does not.
ORACM is located in /app/oracle/ORACM which is different than ORACLE_HOME in customer's environment.
WORKAROUND:
Customer is using the following workaround:
cd $ORACLE_HOME/rdbms/lib make -f ins_rdbms.mk rac_on ioracle ipc_udp
RELATED BUGS:
Bug 4169291

Check if following MOS note helps.
Series of ORA-7445 Errors After Applying 9.2.0.7.0 Patchset to 9.2.0.6.0 Database (Doc ID 373375.1)

UTL_FILE_DIR and Oracle RAC

Gentlemen,
I have installed with sucess an environment with Oracle RAC with two servers. The application that runs in these servers, has batch procedures, and it has to save flat files during these processes. When the processes run with both servers something goes wrong and the system stops. It doesn´t happen when I shutdown one of the servers. I tried to setup directories in the storage system (external disks) but it wasn´t allowed too.
Does anyone know how can I setup utl_file_dir to work with both servers? Is there another parameter to set?
Thank you all,
Best regards.
Roberto Cavalcante
Brazil

VCFS/CFS/NFS are just filesystems and UTL_FILE does file i/o. If you have sessions in multiple instances that write to the same file on a cluster filesystem, the outcome would be unpredictable. Most likely, the last person to write to the file would have their changes saved. Oracle DB (RAC or not) doesn't handle any concurrency related to UTL_FILE writes. So, even if you were in the same instance (non-RAC) and had two sessions writing to the same file, I don't think you'd get the desired results :). The file would likely be a mess and you'd have to use different file names or control concurrency yourself manually.

Oracle RAC performance Suddenly terminates on one of the two node cluster

I have a strange problem that happens frequently from time to time when My M400 Machine which is a part of two node RAC cluster goes down suddenly
I tried so many times to understand what's the cause behind that but when I read the logs there are so many messages related to the Oracle RAC which I don't have any experience or knowledge about so I hope I can find here any one who can explain to me these log messages knowing that they are always the same
Jun 18 08:30:00 kfc-rac1 sendmail[17709]: [ID 702911 mail.crit] My unqualified host name (kfc-rac1) unknown; sleeping for retry
Jun 18 08:31:00 kfc-rac1 sendmail[17709]: [ID 702911 mail.alert] unable to qualify my own domain name (kfc-rac1) -- using short name
Jun 18 11:44:15 kfc-rac1 iscsi: [ID 454097 kern.notice] NOTICE: unrecognized ioctl 0x403
Jun 18 11:44:15 kfc-rac1 scsi: [ID 243001 kern.warning] WARNING: /pseudo/fcp@0 (fcp0):
Jun 18 11:44:15 kfc-rac1 Invalid ioctl opcode = 0x403
Jun 18 17:09:41 kfc-rac1 Cluster.RGM.global.rgmd: [ID 224900 daemon.notice] launching method <bin/rac_udlm_monitor_stop> for resource <rac-udlm-rs>, resource group <rac-fw-rg>, node <kfc-rac1
, timeout <300> secondsJun 18 17:09:41 kfc-rac1 Cluster.RGM.global.rgmd: [ID 224900 daemon.notice] launching method <bin/rac_framework_monitor_stop> for resource <rac-fw-rs>, resource group <rac-fw-rg>, node <kfc-r
ac1>, timeout <3600> seconds
Jun 18 17:09:41 kfc-rac1 Cluster.RGM.global.rgmd: [ID 224900 daemon.notice] launching method <bin/rac_svm_monitor_stop> for resource <rac-svm-rs>, resource group <rac-fw-rg>, node <kfc-rac1>,
timeout <300> seconds
Jun 18 17:09:41 kfc-rac1 Cluster.RGM.global.rgmd: [ID 224900 daemon.notice] launching method <scal_dg_monitor_stop> for resource <scal-racdg-rs>, resource group <scal-racdg-rg>, node <kfc-rac
1>, timeout <300> seconds
Jun 18 17:09:41 kfc-rac1 Cluster.RGM.global.rgmd: [ID 224900 daemon.notice] launching method <scal_mountpoint_monitor_stop> for resource <racfs-mntpnt-rs>, resource group <racfs-mntpnt-rg>, n
ode <kfc-rac1>, timeout <300> seconds
Jun 18 17:09:41 kfc-rac1 Cluster.RGM.global.rgmd: [ID 515159 daemon.notice] method <bin/rac_framework_monitor_stop> completed successfully for resource <rac-fw-rs>, resource group <rac-fw-rg>
, node <kfc-rac1>, time used: 0% of timeout <3600 seconds>
Jun 18 17:09:41 kfc-rac1 Cluster.RGM.global.rgmd: [ID 515159 daemon.notice] method <bin/rac_udlm_monitor_stop> completed successfully for resource <rac-udlm-rs>, resource group <rac-fw-rg>, n
ode <kfc-rac1>, time used: 0% of timeout <300 seconds>
Jun 18 17:09:41 kfc-rac1 Cluster.RGM.global.rgmd: [ID 515159 daemon.notice] method <bin/rac_svm_monitor_stop> completed successfully for resource <rac-svm-rs>, resource group <rac-fw-rg>, nod
e <kfc-rac1>, time used: 0% of timeout <300 seconds>
Jun 18 17:09:41 kfc-rac1 Cluster.RGM.global.rgmd: [ID 224900 daemon.notice] launching method <bin/rac_udlm_stop> for resource <rac-udlm-rs>, resource group <rac-fw-rg>, node <kfc-rac1>, timeo
ut <300> seconds
Jun 18 17:09:41 kfc-rac1 Cluster.RGM.global.rgmd: [ID 515159 daemon.notice] method <scal_dg_monitor_stop> completed successfully for resource <scal-racdg-rs>, resource group <scal-racdg-rg>,
node <kfc-rac1>, time used: 0% of timeout <300 seconds>
Jun 18 17:09:41 kfc-rac1 Cluster.RGM.global.rgmd: [ID 515159 daemon.notice] method <scal_mountpoint_monitor_stop> completed successfully for resource <racfs-mntpnt-rs>, resource group <racfs-
mntpnt-rg>, node <kfc-rac1>, time used: 0% of timeout <300 seconds>
Jun 18 17:09:41 kfc-rac1 Cluster.RGM.global.rgmd: [ID 224900 daemon.notice] launching method <scal_mountpoint_postnet_stop> for resource <racfs-mntpnt-rs>, resource group <racfs-mntpnt-rg>, n
ode <kfc-rac1>, timeout <300> seconds
Jun 18 17:09:41 kfc-rac1 SC[SUNW.rac_udlm.rac_udlm_stop]: [ID 854390 daemon.notice] Resource state of rac-udlm-rs is changed to offline. Note that RAC framework will not be stopped by STOP me
thod.
Jun 18 17:09:41 kfc-rac1 Cluster.RGM.global.rgmd: [ID 515159 daemon.notice] method <bin/rac_udlm_stop> completed successfully for resource <rac-udlm-rs>, resource group <rac-fw-rg>, node <kfc
-rac1>, time used: 0% of timeout <300 seconds>
Jun 18 17:09:42 kfc-rac1 samfs: [ID 320134 kern.notice] NOTICE: SAM-QFS: racfs: Initiated unmount filesystem: vers 2
Jun 18 17:09:43 kfc-rac1 samfs: [ID 522083 kern.notice] NOTICE: SAM-QFS: racfs: Completed unmount filesystem: vers 2
Jun 18 17:09:43 kfc-rac1 Cluster.RGM.global.rgmd: [ID 515159 daemon.notice] method <scal_mountpoint_postnet_stop> completed successfully for resource <racfs-mntpnt-rs>, resource group <racfs-
mntpnt-rg>, node <kfc-rac1>, time used: 0% of timeout <300 seconds>
Jun 18 17:09:43 kfc-rac1 Cluster.RGM.global.rgmd: [ID 224900 daemon.notice] launching method <scal_dg_postnet_stop> for resource <scal-racdg-rs>, resource group <scal-racdg-rg>, node <kfc-rac
1>, timeout <300> seconds
Jun 18 17:09:43 kfc-rac1 Cluster.RGM.global.rgmd: [ID 515159 daemon.notice] method <scal_dg_postnet_stop> completed successfully for resource <scal-racdg-rs>, resource group <scal-racdg-rg>,
node <kfc-rac1>, time used: 0% of timeout <300 seconds>
Jun 18 17:09:43 kfc-rac1 Cluster.RGM.global.rgmd: [ID 224900 daemon.notice] launching method <bin/rac_svm_stop> for resource <rac-svm-rs>, resource group <rac-fw-rg>, node <kfc-rac1>, timeout
<300> seconds
Jun 18 17:09:43 kfc-rac1 SC[SUNW.rac_svm.rac_svm_stop]: [ID 854390 daemon.notice] Resource state of rac-svm-rs is changed to offline. Note that RAC framework will not be stopped by STOP metho
d.
Jun 18 17:09:43 kfc-rac1 Cluster.RGM.global.rgmd: [ID 515159 daemon.notice] method <bin/rac_svm_stop> completed successfully for resource <rac-svm-rs>, resource group <rac-fw-rg>, node <kfc-r
ac1>, time used: 0% of timeout <300 seconds>
Jun 18 17:09:43 kfc-rac1 Cluster.RGM.global.rgmd: [ID 224900 daemon.notice] launching method <bin/rac_framework_stop> for resource <rac-fw-rs>, resource group <rac-fw-rg>, node <kfc-rac1>, ti
meout <300> seconds
Jun 18 17:09:43 kfc-rac1 SC[SUNW.rac_framework.rac_framework_stop]: [ID 854390 daemon.notice] Resource state of rac-fw-rs is changed to offline. Note that RAC framework will not be stopped by
STOP method.
Jun 18 17:09:43 kfc-rac1 Cluster.RGM.global.rgmd: [ID 515159 daemon.notice] method <bin/rac_framework_stop> completed successfully for resource <rac-fw-rs>, resource group <rac-fw-rg>, node <
kfc-rac1>, time used: 0% of timeout <300 seconds>
Jun 18 17:09:44 kfc-rac1 root: [ID 702911 user.error] Oracle CRSD 3932 set to stop
Jun 18 17:09:44 kfc-rac1 root: [ID 702911 user.error] Oracle CRSD 3932 shutdown completed
Jun 18 17:09:44 kfc-rac1 root: [ID 702911 user.error] Oracle EVMD set to stop
Jun 18 17:09:44 kfc-rac1 root: [ID 702911 user.error] Oracle CSSD being stopped
Jun 18 17:09:45 kfc-rac1 xntpd[980]: [ID 866926 daemon.notice] xntpd exiting on signal 15
Jun 18 17:09:45 kfc-rac1 ip: [ID 646971 kern.notice] ip_create_dl: hw addr length = 0
Jun 18 17:09:45 kfc-rac1 pppd[516]: [ID 702911 daemon.notice] Connection terminated.
Jun 18 17:09:47 kfc-rac1 pppd[9462]: [ID 860527 daemon.notice] pppd 2.4.0b1 (Sun Microsystems, Inc.) started by root, uid 0
Jun 18 17:09:47 kfc-rac1 pppd[9462]: [ID 702911 daemon.notice] Connect: sppp0 <--> /dev/dm2s0
Jun 18 17:09:47 kfc-rac1 rpc.metamedd: [ID 702911 daemon.error] Terminated
Jun 18 17:09:48 kfc-rac1 inetd[482]: [ID 702911 daemon.warning] inetd_offline method for instance svc:/network/rpc/scrcmd:default is unspecified. Taking default action: kill.
Jun 18 17:09:48 kfc-rac1 inetd[482]: [ID 702911 daemon.warning] inetd_offline method for instance svc:/network/rpc/metacld:default is unspecified. Taking default action: kill.
Jun 18 17:09:49 kfc-rac1 inetd[482]: [ID 702911 daemon.warning] inetd_offline method for instance svc:/network/rpc/scadmd:default is unspecified. Taking default action: kill.
Jun 18 17:09:50 kfc-rac1 pppd[9462]: [ID 702911 daemon.notice] local IP address 192.168.224.2
Jun 18 17:09:50 kfc-rac1 pppd[9462]: [ID 702911 daemon.notice] remote IP address 192.168.224.1
Jun 18 17:09:50 kfc-rac1 cl_eventlogd[1554]: [ID 247336 daemon.error] Going down on signal 15.
Jun 18 17:09:52 kfc-rac1 ip: [ID 372019 kern.error] ipsec_check_inbound_policy: Policy Failure for the incoming packet (not secure); Source 192.168.224.001, Destination 192.168.224.002.
*Jun 18 17:09:56 kfc-rac1 ip: [ID 646971 kern.notice] ip_create_dl: hw addr length = 0*
*Jun 18 17:09:56 kfc-rac1 pppd[9462]: [ID 702911 daemon.notice] Connection terminated.*
*Jun 18 17:09:56 kfc-rac1 Cluster.PNM: [ID 226280 daemon.notice] PNM daemon exiting.*
*Jun 18 17:09:57 kfc-rac1 pseudo: [ID 129642 kern.info] pseudo-device: tod0*
*Jun 18 17:09:57 kfc-rac1 genunix: [ID 936769 kern.info] tod0 is /pseudo/tod@0*
*Jun 18 17:09:57 kfc-rac1 pseudo: [ID 129642 kern.info] pseudo-device: pm0*
*Jun 18 17:09:57 kfc-rac1 genunix: [ID 936769 kern.info] pm0 is /pseudo/pm@0*
*Jun 18 17:09:57 kfc-rac1 rpc.metad: [ID 702911 daemon.error] Terminated*
Jun 18 17:10:01 kfc-rac1 syslogd: going down on signal 15
*Jun 18 17:10:07 kfc-rac1 rpcbind: [ID 564983 daemon.error] rpcbind terminating on signal.*
*Jun 18 17:10:32 kfc-rac1 Cluster.RGM.fed: [ID 831843 daemon.notice] SCSLM thread WARNING pools facility is disabled*
*Jun 18 17:10:40 kfc-rac1 genunix: [ID 672855 kern.notice] syncing file systems...*
*Jun 18 17:10:40 kfc-rac1 genunix: [ID 904073 kern.notice] done*
Jun 19 14:20:12 kfc-rac1 genunix: [ID 540533 kern.notice] ^MSunOS Release 5.10 Version Generic_141444-09 64-bit
Jun 19 14:20:12 kfc-rac1 genunix: [ID 943908 kern.notice] Copyright 1983-2009 Sun Microsystems, Inc. All rights reserved.
Jun 19 14:20:12 kfc-rac1 Use is subject to license terms.
Jun 19 14:20:12 kfc-rac1 genunix: [ID 678236 kern.info] Ethernet address = 0:21:28:2:21:b2
Thanks in advance for all of you
your response is highly appreciated

Hi I have checked the interconnect between the two nodes and it's as follow
ifconfig -a
lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1
inet 127.0.0.1 netmask ff000000
bge0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2
inet 10.1.100.126 netmask ffffff00 broadcast 10.1.100.255
groupname sc_ipmp0
ether 0:14:4f:3a:6c:19
bge0:1: flags=9040843<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVER> mtu 1500 index 2
inet 10.1.100.127 netmask ffffff00 broadcast 10.1.100.255
bge0:2: flags=1040843<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4> mtu 1500 index 2
inet 10.1.100.140 netmask ffffff00 broadcast 10.1.100.255
bge1: flags=1008843<UP,BROADCAST,RUNNING,MULTICAST,PRIVATE,IPv4> mtu 1500 index 6
inet 172.16.0.129 netmask ffffff80 broadcast 172.16.0.255
ether 0:14:4f:3a:6c:1a
nxge0: flags=9040843<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVER> mtu 1500 index 3
inet 10.1.100.128 netmask ffffff00 broadcast 10.1.100.255
groupname sc_ipmp0
ether 0:21:28:d:c9:8e
nxge1: flags=1008843<UP,BROADCAST,RUNNING,MULTICAST,PRIVATE,IPv4> mtu 1500 index 5
inet 172.16.1.1 netmask ffffff80 broadcast 172.16.1.127
ether 0:21:28:d:c9:8f
e1000g1: flags=1008843<UP,BROADCAST,RUNNING,MULTICAST,PRIVATE,IPv4> mtu 1500 index 4
inet 172.16.1.129 netmask ffffff80 broadcast 172.16.1.255
ether 0:15:17:81:15:c3
clprivnet0: flags=1009843<UP,BROADCAST,RUNNING,MULTICAST,MULTI_BCAST,PRIVATE,IPv4> mtu 1500 index 7
inet 172.16.4.1 netmask fffffe00 broadcast 172.16.5.255
ether 0:0:0:0:0:1
sppp0: flags=10010008d1<UP,POINTOPOINT,RUNNING,NOARP,MULTICAST,IPv4,FIXEDMTU> mtu 1500 index 8
inet 192.168.224.2 --> 192.168.224.1 netmask ffffff00
ether 0:0:0:0:0:0
root@kfc-rac1 #
and it's direct attached between both nodes interfaces
back to back
and about the status of the hba cards here is it as well
fcinfo hba-port -l
HBA Port WWN: 2100001b3284c042
OS Device Name: /dev/cfg/c1
Manufacturer: QLogic Corp.
Model: 375-3355-02
Firmware Version: 05.01.00
FCode/BIOS Version: BIOS: 1.24; fcode: 1.24; EFI: 1.8;
Serial Number: 0402R00-0844647023
Driver Name: qlc
Driver Version: 20090519-2.31
Type: N-port
State: online
Supported Speeds: 1Gb 2Gb 4Gb
Current Speed: 4Gb
Node WWN: 2000001b3284c042
Link Error Statistics:
Link Failure Count: 0
Loss of Sync Count: 0
Loss of Signal Count: 0
Primitive Seq Protocol Error Count: 0
Invalid Tx Word Count: 0
Invalid CRC Count: 0
HBA Port WWN: 2100001b321c462b
OS Device Name: /dev/cfg/c2
Manufacturer: QLogic Corp.
Model: 375-3355-02
Firmware Version: 05.01.00
FCode/BIOS Version: BIOS: 1.24; fcode: 1.24; EFI: 1.8;
Serial Number: 0402R00-0844646557
Driver Name: qlc
Driver Version: 20090519-2.31
Type: N-port
State: online
Supported Speeds: 1Gb 2Gb 4Gb
Current Speed: 4Gb
Node WWN: 2000001b321c462b
Link Error Statistics:
Link Failure Count: 0
Loss of Sync Count: 0
Loss of Signal Count: 0
Primitive Seq Protocol Error Count: 0
Invalid Tx Word Count: 0
Invalid CRC Count: 0
HBA Port WWN: 2100001b32934b3c
OS Device Name: /dev/cfg/c3
Manufacturer: QLogic Corp.
Model: 375-3294-01
Firmware Version: 05.01.00
FCode/BIOS Version: BIOS: 2.2; fcode: 2.1; EFI: 2.0;
Serial Number: 0402R00-0947745866
Driver Name: qlc
Driver Version: 20090519-2.31
Type: N-port
State: online
Supported Speeds: 1Gb 2Gb 4Gb
Current Speed: 4Gb
Node WWN: 2000001b32934b3c
Link Error Statistics:
Link Failure Count: 0
Loss of Sync Count: 0
Loss of Signal Count: 0
Primitive Seq Protocol Error Count: 0
Invalid Tx Word Count: 0
Invalid CRC Count: 0
HBA Port WWN: 2101001b32b34b3c
OS Device Name: /dev/cfg/c4
Manufacturer: QLogic Corp.
Model: 375-3294-01
Firmware Version: 05.01.00
FCode/BIOS Version: BIOS: 2.2; fcode: 2.1; EFI: 2.0;
Serial Number: 0402R00-0947745866
Driver Name: qlc
Driver Version: 20090519-2.31
Type: unknown
State: offline
Supported Speeds: 1Gb 2Gb 4Gb
Current Speed: not established
Node WWN: 2001001b32b34b3c
Link Error Statistics:
Link Failure Count: 0
Loss of Sync Count: 0
Loss of Signal Count: 0
Primitive Seq Protocol Error Count: 0
Invalid Tx Word Count: 0
Invalid CRC Count: 0
root@kfc-rac1 #
In addition here is the ocssd log file as well
http://www.4shared.com/file/Txl9DqLW/log_25155156.html?
you'll find on the lines for the dates in which this issue happens
look at 2012-06-09
2012-06-18
2012-06-21
you'll see something related to the voting disk
it suddenly becomes unavailable which causes the problem
thanks a lot for your help
I'm waiting for your recommendation
hope these logs gives more look for the problem
Thanks in advance :)

Oracle RAC Course Institues in India (Hyderabad)

Hi
I am interested in doing the Oracle RAC course , can someone suggest me any good institutes in India that are offering the course specially in hyderabad .
thanks
Sri

DB Experts offers Best training for Oracle RAC 11g R2 at Hyderabad
Oracle RAC 11g R2 Contents
Introduction
     Introduction of clusters
     Types of cluster
     Components of a cluster
     Hardware requirements
     Define Oracle RAC
     List the advantages of using RAC
     RAC architecture and components
     SGA structures
     Background processes
     RAC internals and overview of cache fusion
Oracle 11g R2 RAC Installation (RHEL 5)
     Operating system preparation for RAC
     Introduction to network hardware
     Private and public network configuration
     Pre installation configurations
     Configuring DNS for SCAN
     Attaching shared storage
     Preparing attached shared storage for ASM Diskgroups
     Installing Grid Infrastructure
     Configuring ASM
Parameters required to configure ASM
Understanding SCAN
     Installation of 11g R2 RDBMS software
RAC Database Creation
     Storage options for RAC Database
     Understanding Parameters
     Creating Disk groups for Database, FRA and archive log
     Creating RAC Database using DBCA
     Creating RAC Database manually
     Policy managed and admin managed databases
Administering RAC and DB
     Using ASMCA and ASMCMD
     ASM administration
     ASM Disk groups metadata backups
     Configuring ACFS
Enabling and Disabling CRS
     Startup and Shutdown of CRS
     Enabling and Disabling Databases
     Startup and Shutdown of Database
     Startup and shutdown of Instances
     Important log files and monitoring
     checking various events
     Management of OCR and Voting disks
     Adding more OCR and Votingdisks
     Backup of OCR and voting disks
     Restoring OCR and voting disks
     Changing locations of OCR and voting Disks
     missing ocr scenario
     changing SCAN IP and port numbers
     Creating and droping redo log threads
     modifying parameters
Managing Services and TAF, Load Balancing
     Introductions to services
     Creating different services
     Startup and shutdown of services
     Understanding and Configuring TAF
     Configuring Oracle Net services
     Configuring Load Balancing
     relocating services
Managing Backup and Recovery
     Configuring Archivelog
     Configuring FRA
     Configuring RMAN
     Configuring Recovery Catalog
     Performing Hot Backups uing RMAN
     Recovery Using RMAN
     Some RMAN recovery scenarios
Database Cloning
     Cloning RAC DB to Single Instance DB using RMAN
     Cloning Single Instance DB to RAC DB using DBCA
Adding and Deleting Nodes
     Adding a new node to RAC
     preparing a node
     preinstallation configuration
     extending clusterware
     extending oracle home
     adding asm and DB instances
     Deleting a node from RAC
     Removing instance
     deinstallation of oracle
     removing crs services
     deinstalling clusterware
Patching and Upgradation
     Applying patches
     Upgrading CRS from 10g to 11g
     Upgrading DB from 10g to 11g
Installing and configuring CHM
Migrating Non RAC DB to RAC DB
     Assuming non RAC DB using File System and
     RAC DB using ASM storage
Some important cache fusion related views
Faculty : Mr Basavraju
Contact : 9866465379
address : F 204, KVR Enclave, Ameerpet
Hyderabad

SAP - Oracle RAC - Linux - VMware

Hi,
we would like to install SAP on a virtual environment (VMware) on Linux RedHat platform.
For the database, Oracle RAC is certified with OCFS2 (1.6) only with Oracle Linux and not RedHat Linux 5.
Can we intstall Oracle RAC database on 2 nodes with Oracle Linux and OCFS2 and other SAP components on Redhat Linux 5 ?
Are there any issues with SAP matrix compatibility ?
Thank you!
Dan

Hi Audun,
Thank you for the links.
I can see the ASM support is new (2.2.2011)
On Note 527843 from 25.01.2011 - Oracle RAC support in the SAP environment
Using raw devices and ASM (Automatic Storage Management) is not supported in the SAP environment. In the case of ASM, there is an exception regarding the Oracle Cluster Registry (OCR) and the voting disks
and
on Note 1550133 - Oracle Automatic Storage Management (ASM) from 2 Feb 2011 they support ASM but with restrictions:
ACFS is required for RAC installations with ASM
To use ASM now, you have to migrate your database to ASM manually
Some important functions of SAP BR*Tools like backup, restore, recovery, tablespace and datafile management are not supported at the moment
Support for tablespace and datafile management is planned for Q2/2011
Also ASM instance will introduce another layer in the system with its problems, failures and resources consumption. I see on the Oracle Metalink alot of problems related to ASM. From the marketin point of view is ok, let's oracle do everything but I think if you will keep it simple it will be better.
I didn't figure out if you can directly install Oracle 11.2.0.2, they say "Allows direct SAP system installations with Oracle 11.2"
We have a vBlock hardware with flash disks as FAST Cache so I believe with 1TB of cache IO will not be a problem.
Thank you!
Dan

Row cache wait in Oracle RAC

Hi,
Our application is running on Oracle RAC. During certain time of the day, the applcation responds very slowly. At these times, it is observed that the row cache waits are very high. We have even tried altering the sys.AUDSES$ sequence and changing it cache size to 10000 from default 20, but this did not help.
Can anyone suggest a solution for this problem? And why this problem occurs?

Hi,
it looks like your problem is related to the fact that you do not cache sequences (this is a well know RAC tuning topic).
Oracle introduced sequences (wrong name, definitely) to generate unique numbers, not to actually support a time sequence of events, or to preserve an order or to have ascending sequences of numbers with no gaps.
Ordering a sequence of events is a serialization process that should not be implemented by a sequence.
Now, if you do not cash sequences, in RAC the lock (enqueue) on the sequence (that is required when you ask for the next set of values) is a global resource on which inter instance contention occurs.
Furthermore, in case the application has a high volume of inserts, having nocache sequences leads to inter instance index block contention.
Oracle says that the default cache value of 20 for sequences is inappropriate in most case of RAC implementations and it is frequent to have caches of 1000 values or more. You need to test what is your ideal value.
Now it is up to you to decide between:
- keep things as they are and have a non scalable RAC installation
- find a way to cache sequences without harming the application assumptions.
Hope it helps,
Regards,
Corrado

OIM 9.1.0.2 - Weblogic JDBC Multi Data Sources for Oracle RAC

Does OIM OIM 9.1.0.2 BP07 support Weblogic JDBC Multi Data Sources (Services>JDBC>Multi Data Sources) for Oracle RAC instead of inserting the "Oracle RAC JDBC URL" on JDBC Data Sources for xlDS and xlXADS (Services>JDBC>Data Sources> xlDS|xlXADS > Connection Poll> URL) ?
If yes, is there are any other modifications that need to be made on OIM, or just changing the data sources?

Yes, it's supported. You install against one instance directly of the Rac Server. Then you update the config.xml file and the jdbc resource in your weblogic server with the full rac address. It is documented for installation against RAC. http://docs.oracle.com/cd/E14049_01/doc.9101/e14047/database.htm#insertedID2
-Kevin

Oracle RAC instance related

Similar Messages

Maybe you are looking for